How Many Trades Do You Need Before a Backtest Means Anything?

Usually more than you have, and the count of trades is the wrong thing to count. What matters is how much independent evidence the trades carry. On my shipped five-minute gold system, 1,643 backtest trades carry the statistical weight of about 512 independent observations, because a few large winners dominate the result. I judge a sample by that smaller number, and I write down how many I need before I look.

MasterVP · Systematic Trading Journey

Written by the founder of KenKem, a software engineer of twenty years who traded gold by hand before building the research stack behind MasterVP. This is knowledge sharing from my own research, not a recommendation. Every figure below is a backtest or research measurement on real tick data, from my release 1.26 write up of 15 September 2026, and none of it is a live result.

Why is the raw number of trades misleading?

Because the trades are not equally informative.

My system is a breakout system. Most trades end at a capped stop or get scratched near break-even, and a small number run far enough to pay for everything else. On the shipped five-minute system, the best 30 of 1,643 trades add up to about three quarters of the whole result. One trade that reaches its full target pays for about five full losses.

That shape has a statistical consequence. The distribution has fat tails, with excess kurtosis of about 4.4, and fat tails mean a sample is noisier than its length suggests. Adjusted for that, the 1,643 trades are worth about 512 independent observations. Every significance test I run uses the adjusted number, not the headline one. A count you cannot defend is decoration.

What is a minimum track record length?

It is the answer to a question most trading pitches never ask: given how big the edge looks and how noisy it is, how many observations do you need before you can be reasonably confident it is above zero?

The smaller and noisier the edge, the longer the record has to be. The test only means something if you compare it with the effective sample, not the raw one.

I learned that the uncomfortable way. An earlier version of my own notes compared a raw count of over 1,400 trades against a required minimum of about 192 and called it "seven times over". In release 1.26 the comparison is made against the effective sample. The one setting I adopted on a pre-registered pass mark, a cost filter, was judged on 428 effective trades against 281 required. That still clears the bar, by roughly one and a half times, not seven. The honest margin is the thinner one, and it is the one I quote now.

Why can a big sample still miss a small improvement?

Because the noise sets a floor under what you can detect.

With roughly 500 to 1,000 effective observations per system, the smallest improvement I can reliably detect is larger than the whole per-trade edge. Put plainly, if a tweak to an entry or an exit makes the system slightly better, my sample is too small to tell that apart from luck.

My own research record says the same thing. Of 77 studies that refined an entry or an exit, I adopted none. Of 13 studies about cost, venue and when to stand aside, I adopted 4. The small refinements were never going to be visible. The structural levers were big enough to measure. So the program now tests large structural levers only, each with its pass mark written first.

Does it matter how many things you tried?

More than almost anything else.

If you test enough variations, one of them will look excellent by chance. Across the work behind MasterVP I tried 7,990 configurations and ran 158 pre-declared experiments. Three systems shipped. The best result from a search that wide has to be judged against what luck alone would produce from a search that wide.

I use two tools for that. The first, a deflated Sharpe ratio, discounts the best result for the number of attempts. I dropped it as a headline check, because I worked out that once a sample is large enough it reads near perfect for almost any candidate, which makes it decoration rather than a test. The second, the probability of backtest overfitting, asks how often the best configuration in-sample turns out below the median out of sample. A coin flip scores 50%. The shipped cost filter scores 27%. The best cell of a parameter grid, by contrast, scored above 50% in 8 of 9 cases.

That is why I ship the centre of a stable plateau, not the single best cell. Re-tested on data they were not chosen on, the shipped configurations land between the 75th and 87th percentile of their own search. Good, not the best. That is what a result that was not memorised tends to look like.

For the same reason I never quote a ratio measured only inside the conditions the system was built for. A number measured only in good weather describes a world that does not stay.

Should the result hold up in separate periods?

Yes, and it should be allowed to fail in some of them.

Since 2021 there are eleven half-year blocks. Three of them lost money, and none lost more than 1.5%. A record with no losing half-year should make you suspicious. A record where one hot stretch drags dead years over the line should make you more suspicious.

The test I trust most for this is a random-entry comparison. I built 1,000 fake versions of each system, identical in every way except the moment of entry, and wrote the pass mark down first: p below 0.05 on both periods. On 2024 to 2026 none of the 1,000 beat the real rule. On 2021 to 2023, 299 did, p = 0.30, which misses the bar. I publish that panel next to the passes, and it is why the system stands aside in that quieter kind of market. I wrote about how that test works in how do you turn a trading pattern into a rule you can test.

One more honest detail. The five-minute system took only 160 trades on 2021 to 2023 with its cost filter active, and that is too thin to test at all. Not every question has a sample big enough to answer it, and saying so is part of the result.

Are 567 start dates 567 experiments?

No, and it is an easy mistake to make.

I simulated a prop-firm evaluation from every one of 567 possible start days, at 0.25% risk per trade with a 5% daily and 10% overall loss limit. None failed, and the worst approach to the failure line came to 8.3% drawdown against the 10% limit. That sounds like a large sample. It is not one. The 567 starts overlap and resample one history, so they measure how much the outcome depends on when you begin. They say nothing new about the future.

What does a big enough sample still not tell you?

Whether the next year looks like the last ones.

Clearing a minimum track record says the past result is probably not noise. That is a much smaller claim than most pages make. Markets change in ways no past sample contained. The worst drawdown in the release 1.26 backtest was 11.2%, and I expect live trading to find costs and conditions the backtest did not. That is why the live account has its stand-down rules written before its data arrives, and why I watch it in months, not days.

Frequently asked questions

How many trades do you need for a backtest to be statistically significant? It depends on how large and how noisy the edge is. A minimum track record length calculation gives the number. On my five-minute gold system the requirement was 281 effective trades and the sample held 428, a backtest measurement, not a live result.

What is effective sample size in trading? It is the number of independent observations your trades are worth after adjusting for fat tails. When a few large trades dominate, the effective sample is much smaller than the trade count. My 1,643 trades counted as about 512.

What is a minimum track record length? A statistic that estimates how many observations you need before you can be confident an edge is above zero, given its size and its noise. Compare it with your effective sample, not your raw count.

Why does testing many strategies make a good backtest less trustworthy? Because the best of many attempts is partly selected for luck. The wider the search, the higher the bar the winner must clear. Tools like the deflated Sharpe ratio and the probability of backtest overfitting adjust for it.

What is the probability of backtest overfitting? It measures how often the configuration that looked best in-sample ranks below the median on data it was not chosen on. 50% is a coin flip. My shipped cost filter scored 27%.

Can a backtest with thousands of trades still be overfit? Yes. A large sample makes noise smaller, not the search narrower. If you picked the best of thousands of settings, the count of trades does not undo that.

Is a losing year in a backtest a bad sign? Not by itself. A record with no losing period at all is usually the bigger warning. Three of my eleven half-year blocks since 2021 lost money, the worst by 1.5%.

If you want to see where this work ended up, MasterVP is my own intraday method built on public volume profile theory. It ships as the Master Volume Profiler indicator on TradingView and MetaTrader 5, which shows the read and leaves the order with you, and as Master Volume Sniper on MQL5 market, an Expert Advisor that executes the rules and risk settings you configure. How many trades did your last backtest have, and how many of them were really independent?

---

This article was composed from the sample-size and statistics posts of the MasterVP build log series. Educational purpose only. Not financial advice. Figures cited are backtest and research results on real tick data, and past performance does not guarantee future results. Every trader remains responsible for their own risk settings and decisions.