McLean and Pontiff 2016
“Does Academic Research Destroy Stock Return Predictability?” by R. David McLean and Jeffrey Pontiff (The Journal of Finance, 2016, 71(1), 5–32) is the empirical anchor of the vault’s Backtest-to-Live Performance Gap. The authors replicate 97 variables that peer-reviewed finance, accounting and economics journals had shown to predict the cross-section of stock returns, then track each predictor’s long-short hedge-portfolio return across three windows: the original in-sample period, an out-of-sample extension from the end of that sample to the publication date, and a post-publication window running to 2013.
The headline result: average gross monthly hedge-portfolio return is 0.58% in-sample, 0.40% out-of-sample (26% lower), and 0.26% post-publication (58% lower). The 26% out-of-sample decline measures statistical bias — the lucky sample window ending — while the further drop to 58% implies a ~32% decline attributable to post-publication trading by investors who learned the anomaly. Decay is largest three to four years after publication, consistent with the time it takes investors to devise and implement exploiting strategies; trading volume and short interest in the relevant stocks rise post-publication, and predictor portfolio returns become more correlated, both signs of coordinated arbitrage. Decay is worst for predictors with higher in-sample returns, higher in-sample statistical confidence, and signals built only from technical (price and volume) data.
For this vault the paper carries two lessons. First, it is direct evidence that published profitability systematically overstates future profitability — the authors conclude investors should expect “less than half the gross return reported in published studies,” and less again after trading frictions the study itself does not deduct. Second, the technical-signal finding is pointed: many Markov and regime-switching trading rules are exactly the price/volume-only, high-in-sample-return predictors that decayed most. The paper therefore earns a negative profitability grade — not because its methodology is weak (it is a careful, replicable study) but because its finding about profitability is that published edges erode. It contradicts the implicit assumption behind every backtest-only Markov claim in this vault.
Connections
- Backtest-to-Live Performance Gap — supports, supplies the headline 26%/58% decay measurement, source: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2156623
- Data-Snooping Bias — supports, post-sample decline is the multiple-testing prediction, source: https://www.cxoadvisory.com/big-ideas/effects-of-market-adaptation/
- Overfitting in Quantitative Trading — supports, in-sample-only edges decay out-of-sample, source: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2156623
- Out-of-Sample Backtesting — reports_underperformance, even honest OOS returns are 26% below in-sample, source: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2156623
- Chen and Zimmermann 2022 — relates, larger replication confirming and re-interpreting the decay, source: https://www.openassetpricing.com/
- Replication Crisis in Quantitative Finance — relates, 12 of 97 predictors miss claimed significance on replication, source: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2156623
McLean and Pontiff 2016 [supports] Backtest-to-Live Performance Gap McLean and Pontiff 2016 [supports] Data-Snooping Bias McLean and Pontiff 2016 [contradicts] Out-of-Sample Backtesting
Sources
- McLean, R. D., & Pontiff, J. (2016). “Does Academic Research Destroy Stock Return Predictability?” The Journal of Finance, 71(1), 5–32. DOI 10.1111/jofi.12365. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2156623 — https://onlinelibrary.wiley.com/doi/abs/10.1111/jofi.12365
- CXO Advisory summary, “Effects of In-sample Bias and Market Adaptation on Stock Anomalies.” https://www.cxoadvisory.com/big-ideas/effects-of-market-adaptation/