Why Most of My Trading Ideas Get Rejected
Β· #research-method #systematic-trading #backtesting #trading-psychology #volume-profile
I did not start building a research stack because I wanted another trading idea. I started because I was tired of inconsistent decisions.
About 20 years in software engineering, about 8 years of real trading with real money, and the combination made one thing obvious: if the process is weak, the outcome will be weak too. So the thing I actually set out to build was not a strategy. It was a process that could tell me, honestly and repeatedly, when I was wrong. Most of what that process produces is rejection. That turned out to be the point.
Why start with rejection instead of discovery?
The real problem was never finding entries. Entries are easy to find and easy to fall in love with. The problem was trusting a process under pressure, when emotion is loudest and the temptation to override the plan is strongest.
That gave me the first thing I would call a genuine quant observation: if my behavior changes under stress, the edge may be weaker than it looked. An edge that only survives when I am calm is not an edge, it is a mood.
Excitement is not a strategy. So I moved the burden off willpower and onto structure. Not to remove human judgment, just to stop bad judgment from running the show. Discretion without structure becomes noise. Structure without discipline also fails. You need both, and the structure has to come first.
What made me distrust my own data first?
Before I could test anything, I had to answer a set of questions that sound too basic to matter, and matter enormously.
What does this data actually represent? What is a quote tick? What is real exchange volume? What is only a proxy standing in for something I cannot see?
In trading data, the wrong field can manufacture a fake edge that looks completely convincing. The right field, correctly interpreted, can save months of wasted work. This is why the engine I built, Dquants, works from real tick data and keeps quote ticks and real exchange volume as separate fields. A proxy never gets quietly promoted to the real thing.
If the data is weak, the conclusion is weak, no matter how sophisticated the analysis sitting on top of it. Truth first, then everything else.
What does an idea have to survive to count as evidence?
I do not want opinions dressed up as facts. So an idea has to survive variation before I take it seriously:
- different sessions
- different years
- different cost assumptions
- different market regimes
That is the first robustness filter, and it is deliberately unkind. In Dquants an idea runs out of sample and walk forward across different years and sessions before it earns any of my attention. If it cannot survive that, I treat it as weak. That is not pessimism, it is just the standard.
The loop itself is not glamorous. Test, measure, reject, refine, test again. Most days the answer is no. That is not the loop failing, that is the loop doing its job. The goal is not to be right quickly. It is to be right for the right reason.
Why is a strong backtest still not enough?
A backtest is a checkpoint, not a finish line. Three things kept turning promising results into nothing:
If the result ignores real trading costs, it is fantasy. In short-duration work especially, spread, slippage, commission and session effects are not a footnote on the strategy, they are part of the signal. If a setup only works before costs, it does not really work.
If the result depends on a narrow parameter pocket, it is fragile. A lone peak on a heatmap is usually noise wearing a costume.
If the result disappears out of sample, it was never strong enough to begin with.
So backtest strength is necessary and nowhere near sufficient. In Dquants a result only counts once it has survived out of sample and walk forward with commission, spread and slippage already subtracted on real tick data.
Why keep a record of the ideas that failed?
This is the part I underestimated. The hardest thing in research is rejecting an idea you wanted to work, and the second hardest is keeping it rejected.
Without a record, a failed hypothesis comes back six months later wearing a slightly different name, and you spend the same weeks proving the same thing. So the engine keeps a registry of rejected hypotheses on record. A failed idea stays failed.
Every rejection is a small protection against self-deception. In quant work that is not a setback, it is a form of risk management. I trust a system more when it is willing to say no, and I trust a set of results more when I can see what was thrown away to get there.
Why build the infrastructure at all?
Good tools do not create edge. They make honesty easier, which is a different and more useful thing.
I wanted a workflow that could store, test, compare and replay ideas the same way every time. Research is easier when the data is organized. Validation is stronger when the metrics are consistent. Trust grows when a result can be reproduced on demand.
The observation that pushed me to actually build it: if you cannot replay a result cleanly, you probably do not understand it well enough yet. That is the whole reason Dquants exists as a Python and C++ engine over DuckDB rather than a folder of one-off scripts.
Where does market structure fit?
Working on Master Volume Profile, my own volume profile approach aimed at short-duration setups on M3 and M5, forced a better class of question. Instead of asking whether price is going up or down, I started asking where the market has accepted value, where it is being rejected, and what the auction context around a move looks like.
That framing is more useful than reading candle shapes, because structure often explains why a move continues or fails. But the framing itself proves nothing. A structural idea has to go through exactly the same gauntlet as any other, and plenty of them have not made it through. Context makes a hypothesis worth testing. It does not exempt it from being tested.
What is the honest takeaway?
The output of this process is mostly the word no, and I have made peace with that. A method that tells me the truth is worth more than one that makes me feel good for a week.
If you are an engineer looking at markets, my honest expectation is that most of your ideas will not survive contact with clean data and real costs. That is not the process failing you. That is the only version of the process that is worth having.
This article was composed from posts 01 to 12 of the KenKem build-log series. Educational purpose only. Not financial advice. Past performance does not guarantee future results.