Replication Crisis in Quantitative Finance
The replication crisis in quantitative finance is the field-level problem that a large share of published trading-strategy results cannot be independently re-run and confirmed by another team. The causes are concrete: papers rarely release code, train on private or non-standard datasets, omit key hyperparameters and model architectures, and report only winning configurations from an undisclosed search. A positive backtest that no one else can reproduce is a claim, not a verified finding — so a literature with low reproducibility cannot accumulate trustworthy knowledge even when each individual paper looks rigorous.
It appears in this vault because Millea 2021, surveying 152 deep-RL-trading papers, documents exactly this deficit, and because independent replication is the missing top tier of the vault’s profitability-grading rubric: a result can only be graded strong if it has been independently replicated. The crisis is the mechanism that converts the field’s many positive backtests into un-aggregable, un-checkable claims, and it compounds Overfitting in Quantitative Trading and Data-Snooping Bias — without code and disclosed settings, the lucky-seed and tuned-on-the-test-set failure modes cannot be audited.
Connections
- Millea 2021 — replication_missing, source: https://www.mdpi.com/2306-5729/6/11/119
- Sun Wang An 2021 — relates, source: https://arxiv.org/abs/2109.13851
- Overfitting in Quantitative Trading — relates, source: https://www.mdpi.com/2306-5729/6/11/119
- Data-Snooping Bias — relates, source: https://www.mdpi.com/2306-5729/6/11/119
- Out-of-Sample Backtesting — relates, source: https://www.mdpi.com/2306-5729/6/11/119
Replication Crisis in Quantitative Finance [opposes] Out-of-Sample Backtesting Overfitting in Quantitative Trading [causes] Replication Crisis in Quantitative Finance