Forecasting the Stock Market


Major Evidence Against Forecasting

There is plenty of research supporting the idea that predicting the stock market is a waste of time. Probably the most famous critic is Princeton University Professor Burton Malkiel, author of the best-selling finance classic A Random Walk Down Wall Street, which was first published in 1973. Malkiel argues that the stock market is simply too random for any investor to make predictions allowing for superior returns in the long run (unless that investor takes on more risk).

One example from his 2011 edition shows that in the 1970s, the top 20 equity funds had a compound annual return of 19.0% vs. the average of 10.4% for all funds. Unfortunately, these fund managers could not replicate the results in the next decade. In the 1980s, those same “winning” managers had annual returns of 11.1% vs. 11.7% for all funds. The same turn of events happened in the 1980s and 1990s. The top 20 funds of the 1980s had annual returns of 18.0% vs. 14.1% for the S&P 500. In the 1990s, those same 20 funds had annual returns of 13.7% vs. 14.9% for the S&P 500. The funds which were originally beating the index lost to it in the next period, suggesting the winning decade was just luck.

Malkiel presents solid evidence that the stock market is impossible to predict because very few can consistently do better than average.

Evidence From the Vanguard Study

Vanguard published a research paper on the topic of stock returns called Forecasting Stock Returns: What Signals Matter and What Do They Say Now? They analyzed 16 popular metrics, studying market data from 1926 through 2011 to determine the predictive abilities of these allegedly useful tools. Below are their three main findings.

First Major Finding

The first takeaway is that stocks are basically unpredictable in the short term. The analysts used one-year periods of returns and applied the 16 metrics. For all of them, the predictive power was close to zero. Nothing was effective.

Second Major Finding

The second critical point is that even for long-term periods, most of the popular methods had very weak predictive power. For example, the researchers purposely included rainfall (which should have zero predictive power) to compare against actual metrics. Rainfall was able to predict 6% of the variance of 10-year stock returns, which beat eight of the supposedly legitimate predictive tools.

Third Major Finding

The third key conclusion is that P/E ratios have modest predictive ability for the long-term. Finishing in first place was Yale University professor Robert Shiller’s P/E10 ratio (also called the cyclically adjusted P/E ratio or Shiller CAPE). To calculate it, Shiller uses the S&P 500 P/E ratio and divides by the inflation-adjusted earnings of the prior 10 years. This tool was able to predict 43% of the variance of stock returns in an ensuing ten-year period. In second place was the P/E1 ratio, which is similar, but divides by the earnings of the previous year. This tool had a predictive power of 38%.

Trading in Accordance With the P/E10

Professor Shiller recommends putting less money into stocks when the ratio is high, and more into stocks when the ratio is low. You can find the ratio on his website, with data going back to 1871 to determine when to invest more in stocks. Alternatively, you can invest in Shiller’s mutual fund (ticker symbol: CAPE), which focuses solely in industries with a low ratio.

For other personal finance topics by this author, visit Additional stock investing strategies can be found at his post