SignalTrace

Why Do Regime-Switching Models Forecast So Badly?

John Montgomery4 min read

Dacco and Satchell 1999

“Why Do Regime-Switching Models Forecast So Badly?” by Robert Dacco and Steve Satchell appeared in the Journal of Forecasting (vol. 18, issue 1, 1999, pp. 1-16), having circulated since 1995 as Birkbeck College Discussion Paper FE-7/95. It is the canonical analytical explanation of a pattern that had become an embarrassment for the non-linear time-series literature: regime-switching and other non-linear models routinely deliver excellent in-sample fit to exchange-rate and asset-return data, yet are usually beaten by a trivial random walk (or random walk with drift) in genuine out-of-sample forecasting. The paper does not run a backtest — its contribution is a theorem.

Working with a deliberately simple two-regime specification, the segmented trend model, Dacco & Satchell derive the mean squared forecast error of the true regime-switching model as a function of how accurately the forecaster can classify the current regime. The striking result is that the true model can have higher MSE than a random walk. The mechanism is regime-classification error, not model misspecification. Even when the data-generating process genuinely is a regime-switching model, and even when the econometrician knows that model exactly, producing a forecast still requires assigning the world to a current regime — and that assignment is noisy in real time. The authors show that only a small misclassification rate is enough to lose the entire advantage of knowing the correct model specification. Knowing the right model buys you nothing if you cannot reliably identify which state you are in.

This is the theoretical spine of the vault’s central scepticism. It cleanly separates two things that profitability claims routinely conflate: a model’s descriptive power (regime-switching models are excellent at explaining volatility clustering and fat tails in hindsight) and its predictive power (which collapses once you must classify the regime forward in time without lookahead). The result is why the vault treats the Markov Regime-Switching Model as primarily descriptive econometrics with a defensible risk-filtering trading use, rather than as a directional alpha source. It is the formal counterpart of the Real-Time Regime Identification Lag and Regime Misclassification failure modes, and it explains why smoothed in-sample regime labels flatter strategies that cannot be traded — see Lookahead Bias from Smoothed Regime Estimates.

The Dacco & Satchell theorem is cited throughout the regime-switching asset-allocation literature as the reason switching strategies underperform after costs. Bulla et al. 2010 cite it directly — “even a small number of wrong regime forecasts is sufficient to lose any advantage of a superior model” — and engineer their out-of-sample design (daily data, Viterbi paths, median filtering) specifically to limit the damage; their tiny after-cost excess returns are exactly what the theorem predicts. Hess (2006) found regime-forecast CAPM strategies had no advantage over a single-state benchmark after transaction costs, attributing the failure to inaccurate regime forecasts and noisy parameter estimates. The point recurs in foreign-exchange work such as Marsh (2000). Because the result is analytical, it is fully reproducible from the paper and is not subject to data-snooping; this is negative evidence on profitability in the most rigorous form available — a proof, not a backtest.

Profitability grade — negative. The paper is not a profitability claim; it is the formal demonstration that the forecasting basis of regime-switching trading is fragile. It shows underperformance against a trivial benchmark arising structurally from real-time classification error. Within the vault’s grading scheme this is graded negative: it is robust, reproducible evidence that the directional-forecasting use of these models fails, which is precisely why their legitimate use is confined to risk classification rather than alpha.

Dacco and Satchell 1999 [contradicts] Markov Regime-Switching Model Dacco and Satchell 1999 [supports] Regime Misclassification Dacco and Satchell 1999 [causes] Real-Time Regime Identification Lag Regime Misclassification [contradicts] Regime-Based Asset Allocation

Connections

Sources

Graph View

Backlinks