FinRL Framework and Contests

FinRL is an open-source deep reinforcement learning framework for quantitative trading, first released by Liu, Yang, Gao & Wang (arXiv:2111.09395) and maintained by the AI4Finance Foundation. It provides a three-layer pipeline — market environments, DRL algorithm implementations (PPO, DDPG, SAC, TD3, A2C), and application templates — that turns historical and live market data into Gym-style training environments. Its companion library FinRL-Meta adds an automated data-curation pipeline producing hundreds of standardised environments. FinRL appears in this vault because it is the de facto reference implementation of the Markov Decision Process Trading Model and Reinforcement Learning Trading Policy approach, and because its 2023-2025 FinRL Contests are the most serious recent attempt to put deep-RL trading on a reproducible, benchmarked footing.

The FinRL Contests (ACM ICAIF 2023, 2024; IEEE IDS 2025) attracted 200 participants from over 100 institutions across 22 countries, running tasks that include multi-asset Dow Jones stock trading, second-level single-asset Bitcoin limit-order-book trading, and — newest — LLM-engineered trading signals (FinRL-DeepSeek, RLMF). Every task is explicitly formulated as a Markov Decision Process: a state vector of balance, prices, holdings and engineered features; a buy/sell/hold action; a reward equal to the change in total asset value, optionally Sharpe- or risk-adjusted. The contests’ methodological contributions are real and directly attack the reproducibility deficit: open-source starter kits, GPU-parallel environments, a uniform evaluation platform with controlled transaction costs, and — crucially — withheld out-of-sample evaluation data (either the most recent ~15% of the series with encrypted timestamps, or data collected after the submission deadline) to block lookahead leakage.

FinRL Framework and Contests [part-of] Reinforcement Learning Trading Policy FinRL Framework and Contests [supports] Replication Crisis in Quantitative Finance FinRL Framework and Contests [relates] Markov Decision Process Trading Model

The contest results, read honestly, reproduce this vault’s verdict rather than overturn it. In FinRL Contest 2023’s data-centric stock-trading task, the winning agents beat the Dow Jones index on Sharpe ratio and maximum drawdown in the pre-deadline window — but in the post-deadline out-of-sample window, two of the three winners posted negative cumulative returns and negative Sharpe ratios, and all three underperformed the index on raw return; the organisers’ own summary concedes “their generalization to new, unseen market conditions remains a challenge.” The 2025 FinRL-DeepSeek stock-trading task produced headline cumulative returns of 190-340% over 2019-2023 — but paired with maximum drawdowns of -28% to -92%, far worse than the S&P 500’s -34% over the same span; a -92% drawdown is a near-total wipeout, not a tradeable result. The contest papers themselves name “policy instability” — performance swinging wildly with hyperparameters, random seeds and market noise — as a central unsolved problem, which is why the contests push ensemble methods as a mitigation.

FinRL Framework and Contests [reports_underperformance] S&P 500 FinRL Framework and Contests [supports] Non-Stationarity Overfitting in Quantitative Trading [contradicts] FinRL Framework and Contests

The honest grade: FinRL is confirmed as valuable benchmarking and reproducibility infrastructure — it gives the field standardised tasks, shared environments and genuine out-of-sample protocols, which is real progress. But it provides no evidence of substantiated, live, after-cost tradeable alpha. Every contest result is a backtest against withheld historical data, not a record of disclosed live capital; the organisers explicitly note that even “paper trading” is a separate, rarer setting. FinRL belongs in the same evidence tier as the rest of the deep-RL trading literature surveyed in Millea 2021 and Sun Wang An 2021: a research and engineering platform, not a profitable trading system. A FinRL Contest leaderboard ranking is a hypothesis, not a result.

Connections

• Reinforcement Learning Trading Policy — proposes_model, source: https://arxiv.org/abs/2111.09395
• Markov Decision Process Trading Model — relates, source: https://arxiv.org/html/2504.02281v3
• Recent Developments 2024-2025 — part-of, source: https://arxiv.org/html/2504.02281v3
• Replication Crisis in Quantitative Finance — replication_available, source: https://arxiv.org/html/2504.02281v3
• Non-Stationarity — suffers_overfitting_risk, source: https://arxiv.org/html/2504.02281v3
• Overfitting in Quantitative Trading — suffers_overfitting_risk, source: https://arxiv.org/html/2504.02281v3
• S&P 500 — reports_underperformance, source: https://arxiv.org/html/2504.02281v3
• Cryptocurrency Market — trades_market, source: https://arxiv.org/html/2504.02281v3
• Live Regime-Model Evidence Gap — lacks_live_evidence, source: https://arxiv.org/html/2504.02281v3
• Millea 2021 — relates, source: https://arxiv.org/html/2504.02281v3

Sources

• Wang, K., Holzer, N., Xia, Z., Cao, Y., Gao, J., Walid, A., Xiao, K. & Liu, X.-Y. (2025) “FinRL Contests: Benchmarking Data-driven Financial Reinforcement Learning Agents”, arXiv:2504.02281 — https://arxiv.org/html/2504.02281v3
• Liu, X.-Y., Yang, H., Gao, J. & Wang, C. D. (2021) “FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance”, arXiv:2111.09395 — https://arxiv.org/abs/2111.09395
• Wang et al. (2025) “FinRL Contests: Data-Driven Financial Reinforcement Learning Agents for Stock and Crypto Trading”, Artificial Intelligence for Engineering (Wiley) — https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/aie2.12004
• AI4Finance-Foundation FinRL repository — https://github.com/AI4Finance-Foundation/FinRL