How Do I Keep My Trading Research Honest?

I keep my trading research honest with process, not willpower. Explicit rules I can debug, a validation chain where each gate removes a different kind of false confidence, written records so my memory cannot quietly edit the past, and a habit of reviewing losers as carefully as winners. None of that predicts the market. All of it slows me down, which is the point.

What follows is the working version of those habits, including the numbers they have produced and the boundaries they have found.

Why do I write rules explicit enough to debug?

Because a rule I cannot trace is a rule I cannot fix.

Clean rules are easier to test, easier to explain, and easier to debug. I prefer explicit logic over vague intuition wherever I can get it, not because intuition is useless, but because explicit rules can be improved and a feeling cannot.

Clean rules are also what make an edge autopsy possible. When a bad trade happens, I want to know which rule allowed it and what data triggered it. Master Volume Profile is built that way, with coded rules and named inputs, precisely so a bad trade traces back to a rule rather than to a mood.

What does a validation chain actually give me?

Confidence without illusion. That is the whole return on it.

It does not promise success and it does not guarantee live results. What it does is separate structure that repeats from luck that happened once. In the Dquants engine I developed, the chain runs in a fixed order: realistic costs first, then a sensitivity plateau where I accept stable regions of parameter space and reject lone peaks, then walk-forward validation judged by the worst fold rather than the pooled number, then Monte Carlo on shuffled per-trade returns, then an overfitting gate, and only then a confirmation run in MetaTrader 5.

Each gate removes a different illusion. Costs kill the fantasy version of the fill. The plateau test kills the parameter I only found by searching hard enough. The worst fold kills the average that one kind year was carrying.

The one gold configuration that has been all the way through produced 1,423 trades on roughly 160 million real broker ticks, against a statistical minimum of 192 trades needed before the math will take a result seriously, with a headline profit factor near 1.42 (source: my own XAU M5 KPI scorecard in the Dquants research repo, dated 2026-07). Every one of those figures is a backtest, not a live track record.

The number I actually plan around is the unflattering one. At the 5th percentile of the Monte Carlo runs, that profit factor falls to about 1.11, and the honest drawdown expectation is roughly 24 percent at the median and 38 percent at the 95th percentile. So the research notes say to size for a 30 to 40 percent peak, not for a smooth line. Anyone who plans for the average is planning to be surprised at the worst possible moment.

Why do I never skip the awkward questions?

The awkward questions are usually the important ones.

Is this sample big enough. Is this robust across regimes. Is this just a lucky pocket of history. Is this still alive after costs. They are annoying to ask when a curve already looks good, which is exactly when they matter.

The one I ask most often is the simplest: is this edge still there after realistic execution? I answer it by stressing spread and slippage upward in the engine and checking whether expectancy is still positive on the far side.

Sometimes the answer is no, and that is useful too. In 2024, a high-cost regime the edge was never built to handle, a live-fill test bled badly. That result stays in my documents on purpose. A boundary I know about in advance is a completely different experience from one that ambushes me with money on the line.

What counts as an early signal, and what does not?

An early signal means something is worth watching carefully. It does not mean the journey is over.

Promising is fine. Proven is better. An early signal stays useful exactly as long as I keep it in the right category, and it becomes a liability the moment I promote it to evidence. That distinction is what stops a decent research lead from turning into a premature lock.

I am still hopeful, and hopeful is not the same as certain. The process keeps getting better, which improves the odds. Odds are the only thing on offer here.

Why do I write everything down, including what failed?

Because memory is unreliable, and trading is unusually brutal on memory.

If I do not record the assumptions, the decisions, and the parity gaps as they happen, I will remember the more comfortable version of the truth later. Documentation is part of the research method, not administrative overhead.

That is why negative results are treated as permanent capital in my stack. Every falsified idea is recorded with its evidence and is not re-chased without a genuinely new structural reason, a new data tier or a new clock, rather than the same idea with different parameters.

Structured review is the same habit pointed at individual trades: logs, screenshots, metrics, context. Without it, it is far too easy to remember only the winners. Reviewing maximum adverse excursion on the losing trades is where most of my next fixes have come from, because that is what tells a normal statistical loss apart from a process error.

What am I actually trying to avoid?

Three things: overconfidence, overfitting, and overpromising. Each of them can undo months of work quickly.

So the skepticism sits in the process rather than in my mood on a given day. There is an overfitting gate and a realistic cost model in the engine, and on the execution side the MetaTrader 5 Expert Advisor only ever acts on the risk settings the user configures in advance, including a volatility veto and a maximum daily risk limit. No single layer is sufficient. Each one removes a specific way of fooling myself.

Underneath all of it is one question I keep asking: is this a signal, or a story I am telling myself? Clean thinking is a trading skill, because a story can survive an entire evening of work while producing no new information at all.

Why does small progress still count?

Because durable work is built out of it. Some weeks the progress is one better decision, one cleaner test, one more honest rejection. Thin progress becomes meaningful once it repeats often enough.

The work feels difficult because it is difficult, and difficulty is usually the sign that I am finally looking at the right problem. The hardest one so far was moving off a bar-level engine onto a pure tick engine, so the tests stopped lying to me about fills that never could have happened.

That is also why I keep a long-term view. One good month says very little. I want to see whether a system holds up across regimes, cost levels, and long boring stretches, because that is a far better test of quality than a single sharp sample.

Frequently asked questions

What is a validation chain in systematic trading? It is a fixed, ordered sequence of tests an idea must survive before it is trusted. Each gate is built to kill a different failure mode rather than to confirm the result. Mine runs costs, sensitivity plateau, walk-forward, Monte Carlo, an overfitting gate, and a final MetaTrader 5 confirmation run.

How many trades does a backtest need before it means anything? Enough that luck averages out, and the threshold should be computed rather than guessed. The minimum track record length depends on the strategy's own return distribution, so a noisier strategy needs a larger sample to say the same thing. For my gold configuration that minimum was 192 trades and the validated sample was 1,423, roughly seven times the floor (backtest, 2026-07 scorecard).

Does Master Volume Profile have a live track record? No, and I will not imply otherwise. Everything published so far is research: backtests and a MetaTrader 5 confirmation run on real broker ticks, not audited live results. Treat every figure I quote as historical evidence about a hypothesis, not as a prediction.

What is an overfitting gate? It is a statistical check asking whether the winner of a parameter search still looks good once you account for how many candidates were tried. Searching hard enough will always produce something impressive, so the gate needs the real search context, including the number of trials. In my stack a deflated Sharpe ratio of 0.95 or above passes, 0.90 to 0.95 is a warning I have to state out loud, and below 0.90 fails.

Why judge walk-forward by the worst fold instead of the average? Because you have to live through the worst fold, not the average one. A pooled number can hide a period where the strategy was badly out of step, carried by one unusually kind stretch of history. It also matches how a drawdown actually arrives, all at once rather than spread evenly.

What drawdown do I plan for in my own research? The 95th percentile, not the median. A benign split-window backtest of my gold configuration can show something near 10 percent, which is exactly the kind of number that gets people into trouble. The Monte Carlo distribution puts the honest expectation nearer 24 percent at the median and 38 percent at the 95th percentile, so the notes say to size for a 30 to 40 percent peak.

Why review losing trades rather than winners? Because losers carry more information per trade. A winner says the setup worked once; a loser says where the rule was fragile and whether the loss was normal variance or a process error. Maximum adverse excursion on losing trades is the single most useful field in my logs.

Does any of this remove risk? No. It removes some specific ways of being wrong, which is a much smaller claim. Trading involves risk including the risk of loss, every figure above is historical, and no amount of validation turns a hypothesis into a certainty.

---

Written by KenKem, a software engineer and founder of twenty years, learning quantitative trading in the open and publishing the process, rejections included.

This article was composed from posts 37 to 48 of the KenKem build-log series. Educational purpose only. Not financial advice. All figures cited are backtest results, not a live track record. Past performance does not guarantee future results.