How to test a trading strategy change without moving the goalposts

Write version A before inventing version B

Define a baseline with objective entry conditions, eligible instruments, session window, stop, target, exit rule and costs. Give it a version identifier and keep its original journal. If you cannot reproduce A without verbal interpretation, a comparison with B will measure changing judgement as much as the filter you wanted to test.

For this exercise B adds just one fixed eligibility filter; size, execution model and exits stay unchanged. The example does not recommend a particular filter. Record the motivation before evaluation, such as excluding a precisely defined condition, and state which result would make you reject the change. “Looks better” is not an acceptance criterion.

Compare the same opportunities, including the skips

Use six baseline candidates, numbered 1–6. Version A’s invented gross outcomes are +2, −1, −1, +2, −1 and −1 R: zero R overall. Suppose B accepts only candidates 1, 2 and 4 with the same fills and exits, producing +2, −1 and +2R; mark 3, 5 and 6 as skipped, not missing.

B totals +3R and averages +1R per executed trade. It averages +0.5R per baseline candidate because only three of six opportunities were executed. State both denominators. A filter can raise per-trade expectancy while reducing participation, so comparing only win percentages hides how frequently the revised rule actually acts.

Trading expectancy: a worked example beyond win rate

Apply costs and inspect the reason for the difference

With an invented cost of 0.10R per executed trade, A pays 0.6R and ends at −0.6R; B pays 0.3R and ends at +2.7R. A skip has zero trading cost in this simplified model. Do not charge B for six executions when it made three, or exempt A’s losing trades from expenses.

These totals were constructed to be easy to verify. They do not demonstrate that B has an edge. The change appears favourable here partly because the example excludes three losing candidates. If the filter was selected after those outcomes were known, this table is development evidence only. It cannot serve as an independent evaluation of the change.

Reserve later dates before adjusting the rule

Split the research process chronologically. Use one preselected period to identify problems and define the candidate filter, then freeze it before opening a later evaluation period. Keep warm-up data separate from the future observations used to assess the rule. Do not tune B after looking at the evaluation results and continue calling that same period unseen.

An evaluation period becomes development data once you use it to select another variant. Keep a log of every version tried, including rejected ones. Testing many thresholds and reporting only the winner increases the opportunity to select a chance pattern; an impressive best result is less informative without the search history.

Testing one setup across ten replay sessions: keep the whole journal

Audit differences before choosing a winner

For each candidate, record whether each version was eligible, the exact skip reason, entry and exit assumptions, gross R and cost. If both versions take a trade but their fills differ, identify why. A changed start date, missing candle or altered position size is not evidence that the eligibility filter improved anything.

Compare drawdown, losing sequences, participation and performance across separate periods alongside totals. If a change works only on one date or depends on an unresolved both-touched candle, treat that as a limitation requiring further investigation. Avoid setting a universal pass threshold from these six invented opportunities; their role is to teach bookkeeping, not statistical validation.

Stop loss and take profit in the same candle: what a backtest can know

Keep the original record and the decision

Finish with an experiment note: baseline version, proposed change, development dates, reserved dates, cost assumptions, all attempted variants and the decision to retain, reject or investigate. If results disappoint, preserve the failure. It is more useful than silently replacing B with C while presenting the combined story as one planned test.

The downloadable candidate worksheet keeps all six opportunities visible and verifies the cost arithmetic. Reuse its structure for a real replay study, with dates selected before outcomes are inspected. A defensible improvement is one you can explain and reproduce on independent evidence; a higher score on the chart you already studied is only a reason to investigate.

Forex backtesting journal template for a spreadsheet