Backtests aren't prophecies · understanding overfitting

Backtesting simulates how a strategy would have performed on historical data. It's an essential research tool — but a good backtest doesn't mean a strategy will work in the future. The most common trap is overfitting.

What is overfitting?

Overfitting happens when a strategy or model fits the noise in historical data instead of genuinely repeatable patterns. It's like memorizing past exam answers without learning how to solve problems — it fails on new questions.

Common causes

• Too many parameters that can “explain” any historical path.
• Repeatedly tuning parameters until the backtest looks best.
• Testing on a single market regime or a single time period.
• Ignoring trading costs and slippage.

Common ways to test for it

• Out-of-sample testing: hold back data that never touched tuning for final validation.
• Walk-forward validation: train and test repeatedly across time windows.
• Regime checks: see how the strategy behaves in trending, ranging and shock markets.
• Include costs: model fees and slippage.

How to read a backtest

A backtest is most valuable for rejecting bad ideas, not proving good ones. It helps rule out strategies that clearly don't work, but any strategy in a live market still needs risk protection and ongoing monitoring.

This article explains a method and is not investment advice.