Test without looking ahead

A backtest applies stated rules to historical observations with a declared execution and cost model. It can reveal unclear instructions, arithmetic mistakes and how a rule would be classified under those assumptions. It is not a record of orders actually filled unless the data really come from those orders. A chart-only simulation cannot manufacture missing bid/ask, liquidity or sequence evidence.

Forward testing records a frozen rule on observations that arrive after the rule is dated. A demo platform can add order-status and fill records, but demo fills and emotions can differ from live conditions. Live trading is not required for this beginner exercise. Keep historical simulation, forward observation and demo rehearsal clearly labelled.

The purpose is to build evidence with limits, not to make a beautiful chart prove a future income. A valid test can reveal that the rule is unsuitable, too costly, difficult to execute or still uncertain. Those are useful findings rather than failures to be hidden.

Systematic testing has developed through many methods and contributors; it has no single universal founder. Research on backtest overfitting, including work by David Bailey, Jonathan Borwein, Marcos López de Prado and Qiji Jim Zhu, examines how selecting strategies from many trials can make historical success misleading. You do not need the paper’s advanced mathematics to understand the beginner problem: searching more possibilities can produce a flattering winner by chance.

Regulators such as NFA highlight limitations of hypothetical performance, including hindsight, missing liquidity effects and the absence of actual financial risk. These references explain why a simulated record must be labelled and its assumptions shown. They do not establish that one disclaimer or one reserved sample makes a method safe or profitable.

Keep a development log of tried rules, filters, costs and samples. The selected version may be clearly written and still have been chosen using favourable noise. Avoid calling a tuned backtest a prediction. Later evidence should be reported separately with the same honesty about limitations.

In the USD/JPY teaching example, a US-based student uses January–June to develop a rule and keeps July–September unseen. Freeze the rule, costs, measurements and exclusions before opening the later period. A later sample is reserved information; it is not necessarily statistically independent of the earlier market conditions.

Decide how many observations are reserved and why before looking at their outcomes. Ensure required indicator warm-up uses only earlier data. If an outcome needs later bars, avoid using a development case whose label reaches into the reserved period to choose the rule without accounting for that overlap. State the boundary and handling of positions still open there.

Repeatedly retesting a changed rule on the same “unseen” dates does not keep them unseen. If July caused a revision, July influenced development. Date a new version and reserve genuinely later or unviewed observations. Keep the old evaluation result even when it is disappointing.

At each simulated decision, ask when every input became available. A daily-close condition cannot justify entry at the same day’s morning price. A swing requiring later bars cannot be treated as confirmed earlier. Revised economic data cannot replace the initially published figure in a rule that depended on what was known at release.

Store both the chart/event timestamp and the availability timestamp when they differ. Indicators that revise prior marks require particular care. A final historical marker can be useful for description without being a usable earlier entry signal. Missing availability information makes a claimed decision unresolved, not automatically valid.

Do not let whole-sample calculations leak into earlier decisions. A threshold chosen from all later volatility, a future maximum used to scale earlier values or selecting the best pair after viewing the evaluation period uses information the earlier rule did not possess. Document how each reference is formed from permitted inputs.

Identify the product, provider, quote side, timeframe, timezone, date range and file version. Check missing or duplicated timestamps, stale quotes, abnormal values, session gaps and changes in definitions. Save the original input and a dated cleaning log. Deleting inconvenient observations after seeing their losses creates another selection problem.

If only midpoint bars exist, a separate spread model may create a hypothetical executable-price series, but it does not create historical observed spreads. Label that model and show sensitivity to wider costs. Avoid treating daily high/low as an order book or a guarantee that a given quantity could trade.

A longer file is not automatically a better file. Quality, relevance and consistent definitions matter. A large dataset with future information or incorrect quote sides can produce a precise-looking but misleading score. Resolve input limitations before presenting performance as if execution were known.

A manual replay hides later observations until the rule comparison is recorded. For each eligible date, save the prior reference, completed trigger, next permitted quote, gates and decision. Then reveal the observations needed to classify the outcome. The sequence is more important than how quickly you complete the exercise.

For an automated test, the same requirements become explicit data indexing and decision timestamps. A script can repeat an incorrect assumption faster than a person, so compare a few worked cases with the manual record. Use cases involving equality, missing data, gaps and rejected gates, not only an easy winning example.

Keep the original rule version, all decisions and corrections. If a previously ambiguous definition is clarified using a viewed outcome, report the change as development work. Do not quietly alter old decisions and describe the entire record as though the clarified rule existed from the beginning.

A simulation needs instructions about what counts as an entry or exit fill. Name the order type, quote side, allowed price/time, size and handling of missing liquidity or sequence. “Enter at the signal close” may be impossible when the completed close is known only after that price has passed. A next-quote assumption is more explicit but still an assumption.

In a conventional long simulation, use ask entry and bid exit. In a short, use bid entry and ask exit. If the record already uses these sides, do not subtract the same spread again as a separate fee. A bid-only chart-based entry requires a different conversion or spread model, clearly named.

A simulation may compare a baseline fill model with adverse slippage scenarios. Keep the scenario results separate; the favourable one is not “actual,” and the adverse one is not a guaranteed worst case. For demo rehearsal, preserve actual order status and fills alongside the intended model so differences are visible.

An OHLC bar contains open, high, low and close, not the order of every movement inside it. Suppose a simulated EUR/USD long has stop 1.0980 and target 1.1040. A later bar has high 1.1050 and low 1.0970. Both levels were visited in the price series, but those four values alone cannot establish which came first after entry.

Choose the policy before inspecting which choice makes performance look better. A green closing candle does not prove target-first. Dropping every ambiguous case can improve the remaining sample artificially, so report how many cases were affected and how the policy changes the ledger.

Even finer price data may not establish executable fills for your size or provider. It can resolve some ordering questions while leaving liquidity or trigger rules uncertain. Write which uncertainty has been reduced and which remains, rather than claiming that tick data remove every limitation.

Use the entry/exit price convention consistently. With actual or modelled ask/bid fills, the price result already reflects crossing those quotes. Add separately charged commission, financing and conversion costs once. If a midpoint model deducts a spread allowance instead, explain that different convention. Mixing the two can charge the same spread twice.

The third row is a worse exit than the second, adding five adverse pips. The fixed US$0.20 fee is invented for teaching and financing is assumed zero in this table. Real schedules can vary by size, day, product and provider. A credit, when genuinely part of the product, should also be recorded with its sign and source.

If quote currency differs from account currency, record the conversion convention and observation time. Conversion can change the reported cash outcome. Keep taxes or other costs separate if not modelled, and disclose the scope rather than calling an incomplete price result an all-cost profit.

The study needs a denominator for each metric. All eligible chart candidates, accepted execution opportunities and completed filled cases are different counts. Retain rejected candidates and their reasons. A skipped quote gate is not automatically a completed trade with zero profit, and a missing outcome is not automatically a confirmed loss or win.

The counts reconcile: 20 candidates become 13 assumed entries after seven skips; those entries divide into eleven resolved and two unresolved outcomes. A performance statistic based only on eleven needs that denominator stated, and the unresolved cases need a disclosed treatment. Do not describe eleven as every opportunity without mentioning the others.

Include rule breaches and operational failures in the appropriate forward/demo record rather than hiding them from a tidy strategy report. You may analyse rule-following cases separately, but show the full account or study record and the exclusion definition. A sub-record is not the whole experience.

A handful of cases is fragile evidence. More relevant observations can help show variation, but more data cannot fix biased selection, look-ahead, inconsistent costs or a changing product. There is no universal number of trades after which a strategy becomes proven. The question depends on variability, dependence, conditions, effects being studied and data quality.

Overlapping positions may share the same news shock, and several pairs may share USD exposure. One hundred such outcomes are not necessarily one hundred independent experiments. Record the sampling frequency, overlap, currency concentration and regimes covered. A thousand one-minute observations do not automatically provide more independent evidence than a smaller set at another horizon.

Use uncertainty language appropriate to the record: “under these rules and assumptions, this period produced these outcomes.” Avoid announcing a precise future win probability from a tiny sample. Advanced statistical evaluation requires assumptions and methods beyond counting wins. The beginner’s responsibility is complete definitions, a full denominator and honest limits.

A comparison period can contain very different volatility or costs. Report those differences rather than silently deleting the difficult period. If the method is only intended for certain conditions, those conditions must be identifiable using available information and defined before the reserved evaluation, not labelled after the losses.

Date the rule and the forward observation plan before new data arrive. Use the same eligibility, inputs, gates, outcome definition and ledger as far as possible. Save misses, skipped days, platform states and practical interruptions. The Japan after-work routine from Level 8 can support this exercise without requiring a trade each day.

For the US-based USD/JPY learner, forward observation begins after the written historical/evaluation plan is locked. Save source/local times and price sides. A demo order can be used for control rehearsal if the policy allows it, but a demo accepted fill is not proof that a live account would receive the same price or size.

If you discover a bug, missing input or unsafe assumption, pause new actions and preserve the existing record. Correct the issue with a dated note and new version as appropriate. “Frozen” does not mean continuing a known faulty plan; it means avoiding silent retrospective changes and being honest about which evidence belongs to which version.

Robustness asks whether the conclusion depends on one convenient assumption or narrow choice. You can predeclare tests with somewhat wider spreads, additional fees, delayed entries or nearby parameter settings, then report all results. The goal is to expose sensitivity, not select whichever variation wins and erase the rest.

A walk-forward design can repeatedly use earlier information for development and a later window for evaluation under a plan fixed beforehand. Every update must respect the chronology and label overlap. It is not a licence to keep inspecting the same later data until a profitable rule appears. Advanced search and statistical safeguards require careful methodology.

Record the number of versions tried, all parameter ranges and the source of each revision. A reserved sample can still mislead if it is repeatedly reused or selected because it looked favourable. Keep a simple starting rule and a clear explanation of why each condition exists. Stop a study with an uncertain or negative result when that is what the evidence supports.