Diversification: Don’t Get Misled by these Charts

Investment Diversification: Don't Get Misled by These Charts

Diversification is an important component of any investing plan.  It assists you in limiting your risk either to a single asset class or a single security within an asset class.  However, I have seen a couple of graphs from which you could form the wrong conclusions about diversification.  In this post, I show you the charts, identify the wrong conclusion that could be drawn from them, and explain and illustrate the correct conclusion.

Diversification Fallacy #1: A Combination of Stocks and Bonds Provides a Higher Return than just Stocks

I first saw a chart[1] in a post on Schwab’s website a couple of years ago.  It is the first graph on this page.  It was prepared in 2018 and compares the cumulative return on the S&P 500 and a portfolio that is 60% stocks (as measured by the S&P 500), 35% bonds and 5% cash from 2000 to 2017.  I’m not sure why Schwab chose to use an 18-year period for this chart, other than the beginning of the time period corresponds to the turn of the century.  The portfolio is re-balanced annually.  In that chart, the total return on the re-balanced portfolio is slightly higher than the S&P 500 (167% versus 158% or 5.6% vs 5.4% per year).

My Version of Chart

Because I can’t include the Schwab chart here, I created a chart (shown below) that shows a similar result for the same time period.  It compares the cumulative returns on the S&P 500 with those of a portfolio of 60% stocks and 40% 20-year US Treasuries (using an approximation I derived for older years).  The mixed portfolio is re-balanced annually, similar to the calculations in the Schwab chart.

Cumulative returns on S&P 500 and mixed portfolio
Cumulative Returns

In this graph, the ratio of the value of the S&P 500 at each year end to its value on December 31, 1999 is shown in purple.  The blue line shows the corresponding ratios for the portfolio of 60% stocks and 40% bonds.  The S&P 500 never makes up the losses it experienced in the first few years of this 18-year time period.

Incorrect Inference about Diversification

At first glance, these charts appear to imply that you can earn more if you hold a 60%/35%/5% mix of stocks, bonds and cash (or 60% stocks/40% bonds) than if you invest in just the S&P 500.  That conclusion confused me, as bonds tend to have total returns that are lower than stocks over the long run and cash has close to no return.   If you re-balance your portfolio annually, as assumed in the graph, your total return in each year will be 60% times the return on stocks plus 35% times the return on bonds plus 5% times the return on cash.  Since the returns on bonds and cash are less than the return on stocks, I was sure that the weighted average of the returns would have to be less than the return on stocks alone.

The Reality

It wasn’t until recently that I figured out why the chart looks the way it does.  The analysis was performed in 2018, so it used the most recent complete 18-year period available.

Historical Perspective

It turns out that period was a rarity in recent history – it was one of only three 18-year periods in which bonds had a higher total return than stocks when considering all such periods from the one starting in 1975 to the one starting in 2002!  If we go back all the way to 1962, the mixed portfolio had higher returns in about a third of the 18-year periods.  The chart below illustrates this point.

18-Year Cumulative Returns for period starting in 1963

Each pair of bars corresponds to an 18-year period (the time period in the Schwab chart) starting in the year shown.  The bar on the left in each pair shows the estimated cumulative 18-year return on a portfolio of 60% stocks[2] and 40% bonds[3] that is re-balanced annually.  The bar on the right shows the corresponding return on the S&P 500 during each period.  As you can see, in most recent years, the right bar (100% stocks) has a higher return than the left bar (60% stocks and 40% bonds).  In older years, the left bar tends to be higher.

How to Use this Information

If your investment goal is to maximize your return without regard to risk, a portfolio with 100% stocks will better meet that objective more than two-thirds of the time when considering 18-year periods and an even higher percentage of the time if you consider only more recent experience.  If interest rates increase substantially at some point in the future, you might look at the longer time period for deciding whether to add bonds to your portfolio, as interest rates were higher and rose in many of the years from 1962 to 1980.  But you’ll want to wait until interest rates are a fair amount higher than their current levels before those years are relevant to your decision-making.

If, however, you want to reduce volatility, adding bonds (or other asset classes) to your portfolio can help.  My post on diversification for investments provides several illustrations about how the addition of bonds to your portfolio reduces risk, but also reduces your total return.  As you consider using other asset classes to reduce volatility, you will need to consider your time horizon for your investments.  As indicated in the chart above, there have been no 18-year periods in the time covered by the analysis in which the S&P 500 had less than a 3% annualized return or 59% compounded return.

Fallacy #2: Diversification in Rank Order Matters

When I first saw this chart from Callan[4], I thought it was very impressed with how it illustrated the benefits of diversification.

Callan Periodic Table of Investment Returns

The font is small so your probably can’t read the words and numbers, but the visual impact is terrific.  Each column is a calendar year.  Each color corresponds to a different index.  The rows correspond to the order of the returns on each index in each calendar year, with the top row showing the index with the highest return; the bottom, the lowest return.

The indices by color (in the order they appear in the first column) are:

  • Rust: S&P 500 Growth
  • Olive green: S&P 500
  • Grey: MSCI (Morgan Stanley Capital International Index) World ex US
  • Dark blue: S&P 500 Value
  • Light green: Bloomberg Barclays Aggregate US Bond Index
  • Medium blue: Bloomberg Barclays High Yield Bond Index
  • Mustard: Russel 2000 Growth
  • Brown: Russell 2000
  • Light blue: Russell 2000 Value
  • Orange: MSCI Emerging Markets

Incorrect Inference about Diversification

At first glance, it appears that there is a lot of diversification among these asset classes, as the colored boxes move up and down on the chart from year to year.

The Reality

It wasn’t until I plotted the returns (using roughly the same colors) on a line chart that the true lack of diversification became apparent.

Annual returns from 1997 to 2017 for funds in Callan chart

Even though the order of the indices changes, as shown in the Callan chart, most of them actually move substantially in sync.  For example, the MSCI Emerging Markets Index moves all over the Callan chart not because it adds diversification but because its returns are much more volatile.  In 14 of the 20 years in the Callan chart, the MSCI Emerging Markets Index is either at the top or the bottom.  It moves in the same direction as most of the other indices, it just makes bigger moves.

Correlations

The goal of adding new asset classes to your portfolio is to increase diversificationAsset classes are diversifying when they have negative or even small positive correlation.  I provide a detailed explanation of correlation and diversification in this post.  The chart below shows the correlations between each pair of indices in the Callan chart.

Correlations between funds in Callan chart

High positive correlations are highlighted in red (as that means they aren’t diversifying).  Medium positive correlations are shown in yellow and small positive and negative correlations (the ones we are seeking) are in green.

It becomes quickly apparent that the only asset class that is diversifying over this time period is US bonds (Bloomberg Barclays (BB) Aggregate US Bond Index).  If you look at the line graph above, I have made the line for the Bloomberg Barclays Aggregate US Bond Index a bit thicker than the others to help you see its lack of correlation with the other investment classes.

Different Insights

While I found the diversification message misleading in this chart, I still found value in the data itself.

Investment-Grade Bonds add Diversification

First, as discussed above, the diversification benefit of investment-grade bonds relative to all of the stock indices is quite evident.  Interestingly, high-yield bonds are highly correlated with stocks, so don’t add diversification.

Asset Classes Show Risk-Reward Balance

Second, I calculated the average annual return and the standard deviations of those returns.  As shown in the chart below, the different indices are spread widely along the spectrum that balances risk and reward.

Average and standard deviations of returns on funds in Callan chart

Specifically, the Bloomberg Barclay Aggregate US Bond Index is in the lower left corner indicating it has a lower average return than all of the other asset classes over this time period but also has the lowest risk as measured by the standard deviation of the annual returns.  By comparison, the MSCI Emerging Markets Index has both the highest annual average return and the highest risk, as it is in the upper right corner of the chart.  All of the other indices fall in the middle on both average return and risk.

Selecting Asset Classes for Your Portfolio

As you are choosing the asset classes in which you want to invest, you need to consider all three of average annual return, risk and diversification benefits.  For example, if you have a very long time horizon and can tolerate the ups and downs of the returns, the historical data indicates that investing primarily in the Emerging Markets index would maximize your return.

If you have a shorter time horizon or are less able to watch the value of your investments go up and down, you might want to invest in something with a lower return, such as one of the stock indices.  If you have even lower risk tolerance or a shorter time horizon, you might want to add something like the Aggregate US Bond Index to your portfolio.  It is important to recognize, though, that adding the less volatile asset classes to your portfolio, even if they are diversifying, will lower the expected annual return on your portfolio at the same time it is lowering your risk.

Caution about Using Past Findings in the Future

In closing, I caution you that the time period covered by the Callan charts corresponds to a time period during which interest rates were relatively low and generally decreasing.  During the time period from 1997 to 2017, the highest yield on the 10-year US Treasury on a year-ending date was 6.7% at the end of 1999.    It decreased to 1.7% at the end of 2014 and increased very slightly to 2.7% by the end of 2017.  By comparison, it hit a high of 12.7% at the end of 1981 and is currently (August 2020) below 1%.  Neither extreme is covered by this time period.

The relatively stable returns on the Bloomberg Barclay Aggregate US Bond Fund Index may be more representative of the time period included in the analysis and may understate the overall volatility of that index over a longer period of time.  Similarly, the other indices may behave differently in other interest rate environments.

I suggest using the information in this post to enhance your understanding of the returns, volatility and diversification benefit of the different asset classes.  You’ll want to supplement this information with your views on future economic environments before making any investment decisions.

[1] I am not able to include the chart directly in this post as I am not willing to accept the conditions that would be required by Schwab to get its permission.

[2] As measured by the S&P 500.

[3] As measured by the iShares 20+ Year Treasury Bond Fund ETF starting in 2002 and my approximation of those returns for prior years.  I note that the Schwab chart uses the Bloomberg Barclays U.S. Aggregate Bond Index for bonds and the FTSE Treasury Bill 3 Month Index for cash.  I don’t not have access to that information.  Because I used a government bond index that tends to provide lower returns than a corporate bond index, I used 40% weight to bonds and ignored the cash component.

[4] Used with permission.  https://www.callan.com/?s=2017+Periodic+Table.  August 8, 2020.

Investment Diversification Reduces Risk

Investment Diversification

Investment diversification is an important tool that many investors used to reduce risk. Last week, I explained diversification and how it is related to correlation.   In this post, I’ll illustrate different ways you can use investment diversification and provide illustrations of its benefits.

Investment Diversification: Key Take-Aways

Here are some key take-aways about investment diversification.

  • Diversification reduces risk, but does not change the average return of a portfolio. The average return will always be the weighted average of the returns on the financial instruments in the portfolio, where the weights are the relative amounts of each instrument owned.  Be careful to remember this point.  It is important and some charts on diversification can be misleading.
  • The smaller the correlation among financial instruments (all the way down to -100%), the greater the benefit of diversification. Check out last week’s post for more about this point.
  • Diversification can be accomplished by investing in more than one asset class, more than one company within an asset class or for long periods of time. One of the easiest ways to become diversified across companies is to purchase a mutual fund or exchange traded fund.  Funds that focus on one industry will be less diversified than funds that includes companies from more than one industry.
  • Diversification reduces risk, but doesn’t prevent losses. If all of the financial instruments in a portfolio go down in value, the total portfolio value will decrease.  Also, if one financial instrument loses a lot of value, the loss may more than offset any gains in other instruments in the portfolio.
  • A diversification strategy can be very risky if you purchase something without the necessary expertise to select it or without understanding all of the costs of ownership.

I’ll explain these points in more detail in the rest of the post.

Diversification and Returns

The purpose of diversification is to reduce riskIt has no impact on return.  The total return of any combination of financial instruments will always be the weighted average of the returns on the individual financial instruments, where the weights are the amounts of each instrument you own.  For example, if you own $3,000 of a financial instrument with a return of 5% and $7,000 of a different financial instrument with a return of 15%, your total return will be 12% (={$3,000 x 5% + $7,000 x 15%}/{$3,000+$7,000} = {$150 + $1,050}/$10,000 = $1,200/$10,000).  Similarly, two instruments that both return 10% will have a combined return of 10% regardless of how correlated they are, even -100% correlation.

Investment Diversification among Asset Classes

When investing, many people diversify their portfolios by investing in different asset classes. The most common of these approaches is to allocate part of their portfolio to stocks or equity mutual funds and part to bonds or bond mutual funds.  These allocation approaches require that the portfolio be re-balanced on a regular basis to maintain the target asset allocation.

Correlation between Stocks and Bonds

Two very common asset classes for personal investment are bonds and stocks. Click here to learn more about bonds, including a comparison between stocks and bonds.  Click here to learn more about stocks.

The Theory

The prices of stocks and bonds sometimes move in the same direction and sometimes move in opposite directions.  In good economies, companies make a lot of money and interest rates are often low.  When companies make money, their stock prices tend to increase.  When interest rates are low, bond prices are high.[1]  So, in good economies, we often see stock and bond prices move in the same direction.

However, from 1977 through 1981, bond prices went down while stocks went up.  At the time, the economy was coming out of a recession (which means stock prices started out low and then rose), but inflation increased. When inflation increases, interest rates tend to also increase and bond prices go down. [2]

Correlation of S&P 500 and Interest Rates

Over the past 40 years, interest rates have generally decreased (meaning bond prices went up) and stock markets increased in more years than not, as shown in the graph below.

10 Year Treasury vs. S&P 500

The blue line shows the amount of money you would have each year if you invested $100 in the S&P 500 in 1980.  The green line shows the interest rate on the 10-year US treasury note, with the scale being on the right side of the graph.  Because bond prices go up when interest rates go down, we anticipate that there will be positive correlation between stock and bond prices over this period. If we looked at a longer time period, the correlation would still be positive, but not quite as high because, as mentioned above, there were periods when bond prices went down and stock prices increased.

Historical Correlation of Stocks and Bonds

I will use annual returns on the S&P 500 and the Fidelity Investment Grade Bond Fund to illustrate the correlation between stocks and bonds.  The graph below is a scatter plot of the annual returns on these two financial instruments from 1980 through 2018.  The returns on the bond fund are shown on the x axis; the returns on the S&P 500, the y axis.  Over this time period, the correlation between the returns on these two financial instruments is 43%.  This correlation is close to the +50% correlation illustrated in one of the scatter plots in last week’s post.  Not surprisingly, this graph looks somewhat similar to the +50% correlation graph in that post.

Scatter plot of returns on Bonds vs. Stocks

Stock and Bond Returns and Volatility

Recall that diversification is the reduction of risk, in this case, by owning both stocks and bonds.  The table below sets the baseline from which I will measure the diversification benefit.  It summarizes the average returns and standard deviations of the annual returns on the S&P 500 (a measure of stock returns) and a bond fund (an approximation of bond returns) from 1980 to 2018.  The bond fund has a lower return and less volatility, as shown by the lower average and standard deviation, than the S&P 500.

Bond FundS&P 500
Average0.6%0.8%
Standard Deviation1.6%4.3%

 

Diversification Benefit from Stocks and Bonds

The graph below is a box & whisker plot showing the volatility of each of these financial instruments separately (the boxes on the far left and far right) and portfolios containing different combinations of them.  (See my post on risk for an explanation of how to read this chart.)

Annual Returns for different asset allocations

In this graph, the boxes represent the 25th to the 75th percentiles.  The whiskers correspond to the 5th to 95th percentiles.  As the portfolios have increasing amounts of stocks, the total return and volatility increase.

Diversification Benefit from Stocks and Bonds – A Different Perspective

These results can also be shown on a scatter plot, as shown in the graph below.  In this case, the x or horizontal axis shows the average return for each portfolio.  The y or vertical axis shows the percentage of the time that the return was negative. (See my post on making financial decisions for an explanation of optimal choices.)

Optimal frontier chart of different stock & bond portfolios

There are three pairs of portfolios that have the same percentage of years with a negative return, but the one with more stocks in each pair has a higher return.  For example, about 24% of the time the portfolios with 30% and 50% invested in bonds had negative returns.  The 30% bond portfolio returned 8.9% on average, whereas the 50% bond portfolio returned 8.5% on average.   Therefore, the portfolio with 30% bonds is preferred over the one with 50% bonds using these metrics because it has the same probability of a negative return but a higher average return.

How to Pick your Mix Between Stocks and Bonds

The choice of mix between stocks and bonds depends on how much return you need to earn to meet your financial goals and how much volatility you are willing to tolerate.  A goal of maximizing return without regard to risk is consistent with one of the portfolios with no bonds or only a very small percentage of them.  At the other extreme, a portfolio with a high percentage (possibly as much as 100%) of bonds is consistent with a goal of minimizing the chance of losing money in any one year.  The options in the middle are consistent with objectives that combine attaining a higher return and reducing risk.

Another approach is to use one of the asset allocations advocated by others.  There is the three-fund portfolio, the four-fund portfolio, the Swensen portfolio and the All Seasons portfolio, among others.

Other Asset Classes

There are many other asset classes that can be used for investment diversification.  Some people prefer tangible assets, such as gold, real estate, mineral rights (including oil and gas) or fine art, while others use a wider variety of financial instruments, such as options or futures.  When considering tangible assets, it is important to consider not only the possible appreciation in value but also the costs of owning them which can significantly reduce your total return.  Examples of costs of ownership include storage for gold and maintenance, insurance and property taxes for real estate.  All of the alternate investments I’ve mentioned, other than gold, also require expertise to increase the likelihood of getting appreciation from your investment.  Not everyone can identify the next Picasso!

Investment Diversification across Companies within an Asset Class

One of the most common applications of diversification is to invest in more than one company’s stock. It is even better if the companies are spread across different industries.  The greatest benefit from diversification is gained by investing in companies with low or negative correlation.  Common factors often drive the stock price changes for companies within a single industry, so they tend to show fairly high positive correlation.

Diversification across industries is so important that Jim Cramer has a segment on his show, Mad Money, called “Am I Diversified?”  In it, callers tell him the five companies in which they own the most stock and he tells them whether they are diversified based on the industries in which the companies fall.

To illustrate the benefits of diversification across companies, I have chosen five companies that are part of the Dow Jones Industrial Average (an index commonly used to measure stock market performance composed of 30 very large companies). These companies and their industries are:

American Express (AXP)Financial Services
Apple (AAPL)Technology
Boeing (BA)Industrial
Disney (DIS)Consumer Discretionary
Home Depot (HD)Consumer Staples

 

Correlation Between Companies

The graph below shows the correlations in the annual prices changes across these companies.

Correlations among Am Ex, Apple, Boeing & Disney

The highest correlations are between American Express and each of Boeing and Disney (both between 50% and 55%).  The lowest correlation is between Apple and Boeing (about 10%).

The graph below shows a box & whisker plot of the annual returns of these companies’ stocks.

Annual Returns for Am Ex, Apple, Boeing, Disney and Home Depot

All of the companies have about a 25% chance (the bottom of the box) of having a negative return in one year.  That is, if you owned any one of these stocks for one calendar year between 1983 and 2018, you had a 25% chance that you would have lost money on your investment.

Adding Companies Reduces Risk

The graph below shows a box & whisker chart showing how your volatility and risk would have been reduced if you had owned just Apple and then added equal amounts of the other stocks successively until, in the far-right box, you owned all five stocks.

Annual returns starting with Apple and adding individual stocks successively

The distance between the tops and bottoms of the whiskers get smaller as each stock is added to the mix. If you had owned equal amounts of all five stocks for any one calendar year in this time period, you would have lost money in 19% of the years instead of 25%.  The 25th percentile (bottom of the box) increases from between -5% and 0% for each stock individually to +14% if you owned all five stocks.  That is, 75% of the time, your return would have been greater than +14% if you had owned all 5 stocks.

As always, I remind you that past returns are not necessarily indicative of future returns. I used these five companies’ stocks for illustration and do not intend to imply that I recommend buying them (or not).

Investment Diversification Doesn’t Prevent Losses

The above illustration makes investing look great!  Wouldn’t it be nice if 75% of the time you could earn a return of at least 14% just by purchasing five stocks in different industries?  That result was lucky on my part.  I looked at the list of companies in the Dow Jones Industrial Average and picked the first five in alphabetical order that I thought were well known and in different industries.  It turns out that, over the time period from 1983 through 2018, all of those stocks did very well.  Their average annual returns ranged from 19% (Disney) to 40% (Apple).  The Dow Jones Industrial Average, by comparison, had an average return of 10%.  That means that most of the other stocks in the Average had a much lower return.

Being diversified won’t prevent losses, but it reduces them when one company experiences significant financial trouble or goes bankrupt.  Here’s a recent example.

Pacific Gas and Electric

Pacific Gas and Electric (PG&E) is a California utility that conservative investors have bought for many, many years.  I’ve added it to the box & whisker plot of the companies above in the graph below.

Annual returns on 5 stocks plus PG&E

PG&E’s average return (10%) is lower than the other five stocks and about equal to the Dow Jones Industrial Average.  Its volatility is similar to Boeing and Disney as shown by the height of its box and spread of it whiskers being similar to those of the other two stocks.

However, on the day I am writing this post, PG&E declared bankruptcy.  PG&E has been accused of starting a number of large wildfires in California as the result of allegedly poor maintenance of its power lines and insufficient trimming of trees near them.  Here is a plot of its daily stock price over the past 12 months.

PG&E stock price history

In the year ending January 26, 2019, PG&E’s stock price decreased by 72%.  From its high in early November 2018 to its low in January 2019, it dropped by 87%.

How to Reduce the Impact of Another PG&E

Although diversification can’t completely protect you from such large losses, it can reduce their impact especially if you are invested in companies in different industries.   If the only company in which you owned stock was PG&E, you would have lost 72% of your savings in one year.  If, on the other hand, you had owned an equal amount of a  second stock that performed the same as the Dow Jones Industrial Average over the same time period (-6%), you would have lost 39%.  The graph below shows how much you would have lost for different numbers of other companies in your portfolio.

Total return of PG&E plus varying numbers of other stocks

This graph shows how quickly the adverse impact of one stock can be offset by including other companies in a portfolio.  In a portfolio of five stocks (PG&E and four others that performed the same as the Dow), the 72% loss is reduced to about a 20% loss.  With 20 stocks, the loss is reduced to 10% (not much worse than the -6% for the Dow Jones Industrial Average).

Investment Diversification Over Time

Another way to benefit from diversification is to own financial instruments for a long time. In all of the examples above, I illustrated the risk of holding financial instruments for one year at a time. Many financial instruments have ups and downs, but tend to generally follow an upward trend.  The volatility and risk of the average annual return of these instruments will decrease the longer they are held.

20-Year Illustration

For illustration of the diversification benefit of time, I have used returns on the S&P 500. The graph below shows the volatility of the average annual return on the S&P 500 for various time periods ranging from one to twenty years.

S&P 500 Annualized Returns over different time periods

To create the “20 Years” box and whiskers in this graph, I started by identifying all 20-year periods starting from 1950 through the one starting in 1997.  I calculated the average annual return for each 20-year period.  I then determined the percentiles needed to create this graph.  The values for the shorter time periods were calculated in the same manner.

The average return over all years is about 8.8%.  Because we are using data from 1950 to 2018 for all of these calculations, the average doesn’t change.

The benefits of long-term investing are clear from this graph.  There were no 20-year periods that had a negative return, whereas the one-year return was negative 25% of the time.

More Complicated Example

My post about whether Chris should pay off his mortgage provides a bit more complicated application of the same concepts. In that case, Chris puts money into the account for five years and then withdraws it for either the next five years or the next 21 years. The longer he invests, the more likely he is to be better off investing instead of paying off his mortgage.

A Caution about Individual Stocks

As a reminder, it is important to remember that this concept applies well to financial measures such as mutual funds, exchange-traded funds and indexes.  It also applies to the financial instruments of many companies, but not all.  If a company starts a downward trend, especially if it is on the way to bankruptcy, it will show a negative return no matter how long you own it.  If you choose to own stocks of individual companies, you will want to monitor their underlying financial performance (a topic for a future post) and news about them to minimize the chance that you continue to own them through a permanent downward trend.


[1]The price of a bond is the present value of the future interest and principal payments using the interest rate on the date the calculation is performed.  That is, each payment is divided by (1+today’s interest rate)(time until payment is made). Because the denominator gets bigger as the interest rate goes up, the present value of each payment goes down.    I’ll talk more about this in a future post on bonds.

[2]An explanation of the link between inflation and interest rates is quite complicated.  I’ll write about it at some point in the future.  For now, I’ll just observe that they tend to increase at the same time.

What is Diversification and How Does it Work?

What is Diversification and How Does It Work?

One of the key concepts used by many successful investors is diversification.  In this post, I’ll define diversification and explain how it works conceptually.  I explain different ways you can diversify your investments and provide illustrations of its benefits in this post.

What is Diversification?

Diversification is the reduction of risk (defined in my post a couple of weeks ago) through investing in a larger number of financial instruments.  It is based on the concept of the Law of Large Numbers in statistics. That “Law” says that the more times you observe the outcome of a random process, the closer the results are likely to exhibit their true properties.

Coin Flip Illustration

For example, if you flip a fair coin twice, there are four sets of possible results:

 

First flipSecond flip
HeadsHeads
HeadsTails
TailsHeads
TailsTails

Estimating the True Probability of Heads

The true probability of getting heads is 50%.  In two rows (i.e., two possible results), there is one heads and one tails.  These two results correspond to the true probability of a 50% chance of getting heads.  The other two possible results show that heads appears either 0% or 100% of the time.

If you repeatedly flip the coin 100 times, you will see heads between 40% and 60% of the time in 96% of the sets of 100 flips.  Increasing the number of flips to 1,000 times per set, you will see heads between 46.8% and 53.2% of the time in 96% of the sets.  Because the range from 40% to 60% with 100 flips is wider than the range of 46.8% to 53.2% with 1,000 flips, you can see that the range around the 50% true probability gets smaller as the number of flips increases.  This narrowing of the range is the result of the Law of Large Numbers.

Following this example, the observed result from only one flip of the coin would not be diversified. That is, our estimate of the possible results from a coin flip would be dependent on only one observation – equivalent to having all of our eggs in one basket.  By flipping the coin many times, we are adding diversification to our observations and narrowing the difference between the observed percentage of times we see heads as compared to the true probability (50%).   Next week, I’ll apply this concept to investing where, instead of narrowing the range around the true probability, we will narrow the volatility of our portfolio by investing in more than one financial instrument.

What is Correlation?

As discussed below, the diversification benefit depends on how much correlation there is between the random variables (or financial instruments). Before I get to that, I’ll give you an introduction to correlation.

Correlation is a measure of the extent to which two variables move proportionally in the same direction. In the coin toss example above, each flip was independent of every other flip.

0% Correlation

When variables are independent, we say they are uncorrelated or have 0% correlation. The graph below shows two variables that have 0% correlation.

Scatter plot of 0% correlation

In this graph, there is no pattern that relates the value on the x-axis (the horizontal one) with the value on the y-axis (the vertical one) that holds true across all the points.

100% Correlation

If two random variables always move proportionally and in the same direction, they are said to have +100% correlation.  For example, two variables that are 100% correlated are the amount of interest you will earn in a savings account and the account balance.  If they move proportionally but in the opposite direction, they have -100% correlation.  Two variables that have -100% correlation are how much you spend at the mall and how much money you have left for savings or other purchases.

The two charts below show variables that have 100% and -100% correlation.

Scatter plot of 100% correlation

Scatter plot of -100% correlation

In these graphs, the points fall on a line because the y values are all proportional to the x values. With 100% correlation, the line goes up, whereas the line goes down with -100% correlation.  In the 100% correlation graph, the x and y values are equal; in the -100% graph, the y values equal one minus the x values. 100% correlation exists with any constant proportion.  For example, if all of the y values were all one half or twice the x values, there would still be 100% correlation.

50% Correlation

The graphs below give you a sense for what 50% and -50% correlation look like.

Scatter plot of 50% correlation

Scatter plot of -50% correlation

The points in these graphs don’t align as clearly as the points in the 100% and -100% graphs, but aren’t as randomly scattered as in the 0% graph.  In the 50% correlation graph, the points generally fall in an upward band with no points in the lower right and upper left corners.  Similarly, in the -50% correlation graph, the pattern of the points is generally downward, with no points in the upper right or lower left corners.

How Correlation Impacts Diversification

The amount of correlation between two random variables determines the amount of diversification benefit.  The table below shows 20 possible outcomes of a random variable.  All outcomes are equally likely.

The average of these observation is 55 and the standard deviation is 27.  This standard deviation is measures the volatility with no diversification and will be used as a benchmark when this variable is combined with other variables.

+100% Correlation

If I have two random variables with the same properties and they are 100% correlation, the outcomes would be:

Remember that 100% correlation means that the variables move proportionally in the same direction.  If I take the average of the outcomes for Variable 1 and Variable 2 for each observation, I would get results that are the same as the original variable.  As a result, the process defined by the average of Variable 1 and Variable 2 is the same as the original variable’s process.  There is no reduction in the standard deviation (our measure of risk), so there is no diversification when variables have +100% correlation.

-100% Correlation

If I have a third random variable with the same properties but the correlation with Variable 1 is -100%, the outcomes and averages by observation would be:

The average of the averages is 0 and so is the standard deviation!  By taking two variables that have ‑100% correlation, all volatility has been eliminated.

0% Correlation

If I have a fourth random variable with the same properties but it is uncorrelated with Variable 1, the outcomes and averages by observation would be:

The average of the averages is 54 and the standard deviation is 17.  By taking two variables that are uncorrelated, the standard deviation has been reduced from 27 to 17.

Other Correlations

The standard deviation of the average of the two variables increases as the correlation increases.  When the variables have between -100% and 0% correlation, the standard deviation will be between 0 and 17. If the correlation is between 0% and +100%, the standard deviation will be between 17 and 27.  This relationship isn’t quite linear, but is close.  The graph below shows how the standard deviation changes with correlation using random variables with these characteristics.

Standard Deviation vs. Correlation

Key Take-Aways

Here are the key take-aways from this post.

  • Correlation measures the extent to which two random processes move proportionally and in the same direction. Positive values of correlation indicate that the processes move in the same direction; negative values, the opposite direction.
  • The lower the correlation between two variables, the greater the reduction in volatility and risk. At 100% correlation, there is no reduction in risk.  At -100% correlation, all risk is eliminated.
  • Diversification is the reduction in volatility and risk generated by combining two or more variables that have less than 100% correlation.

Financial Risk: An Introduction

Introduction to Risk

Understanding financial risk is key to making sound decisions.  Many people don’t have a good grasp on what risk means, particularly in a financial context, so I will focus this post on financial risk.  While I don’t provide any specific practical suggestions in this post, I believe that understanding risk is fundamental to financial literacy. So, in this post, I define financial risk, identify some ways to measure it and provide different types of graphs to illustrate it.  In this post, I provide insights on how to make financial decisions in the context of risk and reward.

Financial Risk

Risk is the possibility that something bad will happen.  Examples of bad things that have financial implications include:

  • Fire destroys your home.
  • You are injured in a car accident and can’t work.
  • The value of an investment goes down.
  • You spend too much or make a poor financial decision so don’t have enough money to meet your financial goals or commitments.

By comparison, volatility refers the possibility that something will deviate from its expected or average value, including both good and bad results.  For example, if you own an S&P 500 index fund, risk would focus on how often and by how much the value of the fund goes down.  Volatility focuses on how often and by how much the value of the fund goes both up and down.

Measures of Risk and Volatility

Most measures of risk have some element of probability associated with them.  A probability is a percentage or the equivalent fraction that falls between 0% and 100% (i.e., between 0 and 1).  It represents the ratio of the number of times that the outcome meets some criteria to the number of possible outcomes.

Probability – Simple Example

Let’s start with some simple, non-financial probabilities.  A coin has two possible outcomes – heads and tails.  When flipping a fair coin, it is equally likely that the result will be heads or tails.

  • The probability of getting heads on one flip is 50%, derived as one result being heads divided by two possible choices.
  • The probability of getting two heads both times on two flips is 25%. There are four possible results, as follows:

First flip

Second flip

Heads

Heads

Heads

Tails

Tails

Heads

Tails

Tails

 

There is one result (the first row) in which there are two heads.  The probability of getting two heads is therefore one result meeting our criterion divided by four possible results or 25%.

  • The probability of getting one heads and one tails on two flips is 50%. There are two rows in the table that have one heads and one tails.  Dividing the count of two results meeting our criterion by the four possible results gives us a 50% probability.

Probability – Applied to S&P 500 Returns

We can now extend this concept to a financial measure.  I downloaded the month-ending values of the S&P 500 from Yahoo Finance from 1951 through 2018.  I calculated the annual change in the index in each year to derive 68 years of returns.  Although the past is seldom a perfect predictor of the future, we can use it as a model of what might happen.  So, when I say there is a certain probability that the S&P 500 return will meet some criteria, I am using short hand for saying that it happened that percentage of the time in the period from 1951 through 2018.

The bar chart below shows the number of years in which the S&P 500 return fell into certain ranges.

Histogram of Annual S&P 500 Returns 1950-2018

We can use this information to calculate the probabilities of certain results, as follows:

  • There are 28 years in which the return was less than the average of the returns over that time period (8.4%). We can therefore calculate that there is a 43% probability that the S&P 500 will return less than 8.4% in any one year by taking the 28 years in which it fell below its average and dividing by the total number of years for which we have data (68).
  • There are 18 years in which the return was negative over that time period (2 of which fell in the -1.6% to +8.4% range). We can therefore calculate that there is a 29% probability that an investor in the S&P 500 will lose money in any one year by taking 18 years in which the return was negative and dividing by the total number of years for which we have data (68).

More Complicated Metric

Sometimes people are not only interested in how often a bad result happens but also how bad it will be when it is worse than that.  For example, you might want to know the average amount you will lose in a year in which there is a loss.  Using the information above about the S&P 500, we would select only the 18 years in which return on the S&P 500 was negative and take the average of those returns. In this case, the average is -11%. With this metric, you now know that there is a 29% probability that an investment in the S&P 500 will lose money in a year and that, in those years, you will lose 11% on average. This metric is a richer metric than probability, but is also much harder to grasp so I won’t spend a lot of time on it.

Standard Deviation

Another metric commonly associated with risk is the standard deviation.   While standard deviation is a very common metric, it doesn’t actually measures risk. It measure volatility because the calculation of standard deviation includes both good and bad results, not just bad ones.  For processes that have symmetric results (more on that in a minute), such as the S&P 500 returns graphed above, you can learn a bit about the distribution just based on the standard deviation.

  • Roughly 2/3 of the possible results will fall in the range defined by the average minus one standard deviation up to the average plus one standard deviation
  • Roughly 96% of the possible results will fall in the range defined by the average minus two standard deviations to the average plus to standard deviations.

As such, something with a higher standard deviation has a higher probability of being below a fixed threshold than one with a lower standard deviation.  For example, we might be looking at two investments both with average returns of 5%.  One might have a standard deviation of 2.5% and the other a standard deviation of 5%. The second one has about a 16% probability of having a negative return as compared to only a 2% probability for the first one.

Pictures of Risk

There are many ways to illustrate risk graphically.  The bar chart of the S&P 500 shown above is one example.

Line Graphs

The data can also be presented in a line graph.  A line graph is essentially the same as a bar chart except there is a point on the line rather than a bar corresponding to ranges of possible results. The line graph below shows the annual returns for the S&P 500.  The ranges I used in this chart are narrower than the ones I used in creating the bar chart, so the graph is bumpier.

Line graph of Annual S&P 500 Returns from 1950 to 2018

In this graph, I also changed the counts of the outcomes on the y-axis (the vertical one) to percentages or probabilities. A graph of the probabilities of possible results is called a probability density function or pdf. (Just in case you were curious.)

Symmetric and Skewed Distributions on a Line Graph

I mentioned earlier that some processes have symmetric results.  If both sides of the line chart are identical, then it is symmetric. The S&P 500 graph above isn’t quite symmetric, but it is close.  Relative to the mean of 8.4%, the possible results extend further to the left (in the downward direction) than to the right (in the upward direction).  That is, the worst observed result was -40% or 48 percentage points worse than the average.  The best observed result was +45% or 37 percentage points better than the average.

Processes that are not symmetrical are called skewed.  In extreme cases, one side of the graph is very tall and doesn’t go very far.  The other side of the graph has a long skinny “tail.” Examples of processes that are skewed are (1) winning the lottery and (2) damage to your house.

Graph of Skewed and Symmetric Risks

The green line in the graph above represents a symmetric distribution with an average of 0.  You can see that it is the same on both the right and left sides of the y-axis.

The blue line represents the change in your financial position if you play the lottery.  There is a very high probability you won’t win anything ($0 change to your financial position after you’ve already bought your ticket).  The probability you will win a small amount is small and the probability you will win a lot is tiny.  This distribution is skewed and the long tail goes to the right.

The red line illustrates the change in your financial position due to possible damage to your home before considering insurance.  There is a high chance you won’t have any damage ($0 change to your financial position). The probability you will have a small loss is small and the probability you will have a large loss (but less than the value of your $100,000 home) is tiny.  Interestingly, there is a larger probability of having a total loss than of have a large loss because, at some point, the damage because so expensive to repair that it is cheaper to replace the whole house.  This distribution is skewed and the long tail goes to the left.

For processes that have skewed results, the rules of thumb about standard deviations don’t apply, so looking at probabilities and average losses below a threshold are more informative.

Comparing Risk

I’ve downloaded monthly returns from Yahoo Finance for four possible equity investments:

  • S&P 500
  • NASDAQ composite
  • Boeing
  • Apple

Because Apple went public in early 1981, I used returns from 1981 through 2018.  I’ll use these monthly returns to demonstrate several ways of illustrating and comparing the risk of different investment options.

Tables

Some people prefer to look at the numbers.  The chart below shows five statistics that measure the volatility or risk of the five equity investments.

S&P 500

Nasdaq

Boeing

Apple

Standard Deviation

3.3%4.6%6.4%18.8%

25th percentile

-1.8%-2.6%-4.5%-5.1%

Interquartile range

5.3%7.0%12.0%15.3%

Average loss when negative

-3.0%-4.7%-6.7%-8.6%

 

As indicated above, standard deviation is a measure of volatility.  The least volatile investment is the S&P 500 index.  The S&P 500 index is the weighted average of the prices of 500 large companies.  Larger companies tend to have less volatility.  Also, the large number of companies in the index adds diversification which also reduces volatility.  I’ll have a post about diversification in a few weeks.

The Nasdaq composite is the weighted average of the prices of all of the companies that trade on the Nasdaq exchange.  Although the companies that trade on the Nasdaq tend to be smaller and more volatile, there are over 3,300 of them so the index is fairly diversified. Nonetheless, the Nasdaq has a higher standard deviation than the S&P 500.

Boeing is a fairly large company, but looking at its stock alone offers no diversification (because you need two things, in this case companies, to create diversification). Therefore, its stock price has a higher standard deviation than either of the indices.  Apple, though a large company, has been a fast growing company so has had even more volatility in its stock price than Boeing.  It has the highest standard deviation of the four investments in the table.

The 25thpercentile (below which 25% of the monthly returns fall) is a measure of risk. We can see that this risk measure shows that these investments fall in the same order looking at this risk metric as when measuring volatility using standard deviation.

I’ve also shown the interquartile range.  It is calculated as the difference between the 75thand 25thpercentiles.  That is, the 75thpercentile is the value above which 25% of the monthly returns fall.  Therefore, the middle 50% (half) of the observations fall in the interquartile range. It is also a measure of volatility that tracks fairly closely with the standard deviation for processes that aren’t highly skewed.

The last two metrics are the probability that the return is less than 0% and the average return when it falls below zero.  Interestingly, Boeing stock has a lower probability of have a negative return in a month than the Nasdaq!  It turns out that Boeing’s average monthly return is enough higher than the Nasdaq’s (6.4% versus 4.6%) to offset the higher volatility (as measured by both the standard deviation and interquartile range).

Line Graphs

The figure below illustrates the monthly returns for the four investment options using a line graph.

Line graphs of monthly returns on S&P 500, Nasdaq and two stocks - 1981 to 2018

Consistent with the information in the tables above, we can see the following:

  • The S&P 500 (red line) has the least risk. The peak in the middle of the chart is the highest and the plot is narrower than that of any of the other options.
  • The Nasdaq composite (purple line) has the next lowest risk. Its peak is only slightly lower than that of the S&P 500.  The tails are a little wider than the S&P 500.
  • Boeing (blue line) is next in order. The general shape of the Boeing plot is similar to those of the S&P 500 and Nasdaq composite, but is lower in the middle and wider in the tails.
  • Apple (green line) is the most risky. It barely has a peak in its plot and has some points that are far from the middle of the graph.

Box & Whisker Plots

A box & whisker plot has less information than a line graph, but is less busy than a line graph so many people find it easier to interpret quickly.  The box & whisker plot of the monthly stock returns is shown below.

Box and whisker plot of monthly returns on S&P 500, NASDAQ and two stocks

The green rectangles are the “boxes” and the lines extending above and below the boxes are the “whiskers.” In this box & whisker plot, 5% of the monthly returns for each option fall below the bottom of each whisker and 5% fall above the top of the upper whisker.  Alternately, 95% of the returns were below the top of the upper whisker. As such, 90% of the monthly returns fell in the range defined by the whiskers.

Similarly, 25% of the monthly returns for each investment fell below the bottom of each box.  75% of the monthly returns are less than the top of each box.  Alternately, 25% of the returns were above the top of the box.  As such, 50% of the monthly returns fell in the range defined by the boxes.  The boxes correspond to the interquartile range I mentioned above.

The risk of each option can be seen by comparing the height of the boxes and whiskers.  We see the same characteristics as were described for the line chart.

Spectral Plots

A spectral plot focuses solely on risk, not volatility.  A spectral plot of the monthly returns on the four investments is shown below.

Spectral plot of monthly returns on S&P 500, NASDAQ and two stocks from 1981 to 2018

The legend shows whole numbers.  These numbers represent how frequently or seldom something will happen in months. In this case, the yellow-green boxes (corresponding to 5 in the legend) show the loss you would have every five months. Every five months corresponds to 20% of the time, so I took the 20th percentile values and plotted the negative of them (since the chart shows the percentage you will lose).  The bright red boxes (corresponding to 100 in the legend) show the percentage loss you would have every 100 months or at the 1st percentile.

It is clear that the S&P 500 has the least risk and Apple has the most risk of the four investments.  Boeing and the Nasdaq have very similar risk, with Boeing very slightly riskier.

Closing Thoughts

To be clear, I don’t anticipate that many of you will be able to create charts that look like these. I hope that by providing these examples, you’ll be able to understand any articles or graphics you read that address risk.

It is also important, in mmking financial decisions, to understand of the nature of the volatility involved.  Is it skewed like that of damage to your house?  Or, is it somewhat symmetric and short-tailed like the S&P 500?  Or somewhere in between?  If you have a good understanding of the nature of the risk involved, you’ll make a better decision.   I’ll talk more about risk and making financial decisions in my next post.