Records, testing and the hindsight problem

By the end of this lesson

Why reviewing past charts systematically misleads you
What to record, and what to compute from the record
How to test an idea without curve-fitting it to death

The one thing

Every chart of the past looks obvious, because you can see what happened next. That single fact invalidates most self-assessment in technical analysis — including, especially, the confident feeling that you would have taken the trade.

The three ways hindsight distorts

Hindsight
How it operates: The pattern is obvious once the outcome is visible
Countermeasure: Bar-replay: hide the future and decide bar by bar
Survivorship
How it operates: You study charts of stocks that did well and exist today
Countermeasure: Test on a universe as of the historical date, delistings included
Selection
How it operates: You remember the setups that worked and forget the identical ones that didn't
Countermeasure: Log every setup you identify, including the ones you skipped

Bias	How it operates	Countermeasure
Hindsight	The pattern is obvious once the outcome is visible	Bar-replay: hide the future and decide bar by bar
Survivorship	You study charts of stocks that did well and exist today	Test on a universe as of the historical date, delistings included
Selection	You remember the setups that worked and forget the identical ones that didn't	Log every setup you identify, including the ones you skipped

The third is the most corrosive in day-to-day practice. Memory keeps the trades that confirm the method and quietly discards the rest — so an unrecorded practitioner's confidence in their edge rises regardless of whether the edge exists.

What to record

The journal has to be written before the outcome is known, or it records your rationalisation rather than your reasoning. Everything below except the last two lines should be filled in at entry.

A minimum viable trade record

Date, instrument, direction
Why it earns its place: The basics
Setup type
Why it earns its place: So you can compute expectancy per setup type and drop the ones that don't work
Entry, stop, target, size
Why it earns its place: Lets you verify you sized correctly, not just that you traded
Planned R
Why it earns its place: The reward-to-risk you believed you had at entry
Regime at entry
Why it earns its place: So you can measure the regime effect in your own record
Why — in one sentence
Why it earns its place: The thesis. Written before, it cannot be edited afterwards.
A snapshot of the chart
Why it earns its place: What you actually saw, not what you remember seeing
Exit and realised R
Why it earns its place: Filled at exit
Did I follow the plan? Yes/no
Why it earns its place: The single most valuable field in the journal

Field	Why it earns its place
Date, instrument, direction	The basics
Setup type	So you can compute expectancy per setup type and drop the ones that don't work
Entry, stop, target, size	Lets you verify you sized correctly, not just that you traded
Planned R	The reward-to-risk you believed you had at entry
Regime at entry	So you can measure the regime effect in your own record
Why — in one sentence	The thesis. Written before, it cannot be edited afterwards.
A snapshot of the chart	What you actually saw, not what you remember seeing
Exit and realised R	Filled at exit
Did I follow the plan? Yes/no	The single most valuable field in the journal

Worked exampleWhat the record reveals that memory cannot

A hundred logged trades, sorted afterwards. Each cut reveals something no amount of reflection would have produced.

All trades: +0.18R average
Plan followed: +0.41R average
Plan not followed: −0.52R average
Breakout setups: +0.46R
Counter-trend setups: −0.38R
Taken in constructive regime: +0.55R
Taken in hostile regime: −0.44R

Three specific, actionable findings from one spreadsheet:

1. Deviating from the plan costs ~0.9R per occurrence.

2. Counter-trend setups are a net loss — stop taking them.

3. The regime filter is worth ~1R per trade.

None of these would emerge from memory, because memory keeps

the counter-trend trade that worked spectacularly and drops

the eleven that didn't.

So → The journal is not a diary — it is a dataset. Its purpose is to be sorted and counted, and that is only possible if the fields are consistent and filled in before the outcome is known.

Testing an idea without fooling yourself

1Write the rules down completely first. If a rule requires judgement, it cannot be tested — and it also cannot be followed consistently, which is a separate problem worth discovering early.
2Use a universe as of the historical date. Testing today's index constituents over ten years builds the answer into the question: those companies were selected for having survived and performed.
3Account for costs realistically — brokerage, STT, slippage on entry and exit. Strategies with many small edges die entirely on costs, and this is where most published results quietly cheat.
4Hold back data. Develop on one period, test on another the rules have never seen. If it works only on the development period, you fitted noise.
5Check robustness, not optimality. If a 50-day average works and 45 and 55 don't, you have found an artefact. Real effects are broad and shallow, not narrow and spectacular.
6Look at the drawdown, not just the return. A method with a 65% maximum drawdown is untradeable by a human regardless of its long-run return, because nobody continues executing through it.

Worth knowing

Why signals get validated against the same code that ships

A backtest that reimplements the live logic in a separate script will drift from it — a threshold changes in one place and not the other, and the validated signal is no longer the shipped signal.

The engine behind this site avoids that by keeping the accumulation logic in a single pure module that both the live scanner and the backtest import. The signal that was validated is byte-for-byte the signal that ships. It's an unglamorous discipline and it removes an entire category of quiet self-deception.

Where people lose money

Sample size

Twelve trades tell you nothing. With a 40% win rate, runs of six losses and six wins both occur regularly by chance alone, and either will feel like evidence about your method.

Meaningful conclusions about expectancy generally need at least a hundred trades, and more if the win rate is low. Until then you are measuring variance. The practical implication: judge yourself on process adherence, which is measurable immediately, rather than on returns, which are not.

The review routine

☐Log every trade at entry, including the one-sentence thesis.
☐Log setups you identified but skipped — the counterfactual matters.
☐Monthly: compute expectancy overall, by setup type, and by regime.
☐Monthly: compute the plan-followed versus plan-broken split.
☐Quarterly: drop the setup type with the worst expectancy over a meaningful sample.
☐Never judge a method on fewer than a hundred trades.

In one screen

1Hindsight makes past charts obvious and self-assessment unreliable.
2The journal must be written before outcomes are known, and treated as a dataset.
3Sort by setup type, regime and plan-adherence — those cuts produce the findings.
4Test on out-of-sample data, with realistic costs, checking robustness rather than optimality.
5Fewer than a hundred trades measures variance, not edge.

Check yourself

Nothing is scored or stored — the explanation after each answer is the point.

1/2

Why is "did I follow my plan?" arguably the most valuable field in a trade journal?

2/2

A strategy tested on the current Nifty 500 constituents over the last decade shows excellent returns. What's the main problem?

Next lesson →

Five episodes, read technically

← Previous

Regime: when any of this works

← All lessons