Paper Trading Is a Software Test, Not a Track Record

A good stretch in a paper account gets read as proof a strategy works. What paper trading actually validates is narrower, more useful, and has almost nothing to do with whether the idea behind it is any good.

The question paper trading doesn’t answer

“How’s the paper account doing?” sounds like a reasonable way to check on a strategy. It’s also the wrong question, because it assumes paper trading is measuring something it isn’t built to measure.

A backtest answers “would this have made money on history, assuming the code and the data are both correct.” Paper trading answers a completely different question: “does this code do what I believe it does, running against a live, unpredictable feed, with nobody standing by to fix it by hand.” That’s a statement about the software. It says nothing about the strategy’s edge, because a well-designed strategy and a strategy with no edge at all look identical in a paper account if the surrounding code has the same bugs.

What it actually catches

None of the following show up in a backtest, because a backtest replays a clean, already-known sequence of bars and assumes every order fills exactly as intended. Paper trading is the first place a system meets a feed it can’t rewind.

Partial fills the code doesn’t expect. A backtest fills an order or it doesn’t — there’s no in-between state to model. A live (or paper) broker connection can hand back a partial fill, and code that silently treats “partially filled” the same as “filled” will misreport its own position from that point forward.

Decision-time data that doesn’t match backtest-time data. A backtest computes an indicator on a bar that has already fully closed. A running system has to decide what “the current bar” means while that bar is still forming, and a small mismatch between the bar an indicator was computed on and the bar a decision was made on is invisible until something is actually running in real time.

Recovery after a restart. A backtest never crashes mid-run. A live process does — a deploy, a dropped connection, a machine reboot — and what matters is whether it comes back knowing what it already did. A restart that quietly forgets an open position, or resubmits an order that already went out, is a bug a backtest structurally cannot expose.

Calendar reality. Holidays, early closes, exchange halts, daylight-saving transitions on a schedule — a backtest’s index is already clean of all of it. The first time a system meets a half-day session or a DST rollover is usually the first time it’s actually run past one.

Why it’s easy to over-read anyway

A paper account produces a P&L curve, and a P&L curve is very easy to interpret as evidence either way. A good stretch feels like confirmation. A bad stretch feels like disproof. Neither conclusion is usually justified, for two reasons that compound each other.

First, the sample is small by construction. Most paper stages run for weeks, not years, which is nowhere near enough trades to distinguish a real edge from noise — the same problem that makes a five-trade winning streak meaningless in a backtest doesn’t go away just because the trades happened in real time.

Second, paper fills tend to be optimistic. Without real order-book queue position, real slippage, or the market impact of an actual order, a paper fill often happens at a friendlier price than a live one would. That flatters the P&L curve in a way that has nothing to do with whether the code is correct — which means a system can pass every software check that matters and still show a paper P&L that overstates what it would do with real orders in the market.

Reading a paper P&L as a verdict on the strategy inherits both problems at once. Reading it as a verdict on the software avoids both, because “did it behave correctly” doesn’t need a large sample or realistic fill prices to answer.

What a paper stage should actually be scored on

If P&L isn’t the right scoreboard, something else has to be. A more useful set of pass/fail questions:

• Did every signal result in exactly one order, or a logged, specific reason for no trade — never silence?
• After a forced restart mid-session, did the system recover the correct open positions and order state, rather than assuming a clean slate?
• Has it now run through at least one real market interruption — a holiday, a halt, a dropped connection — and behaved the way the design says it should?
• Is every skipped or rejected signal attributable to a specific, named condition rather than “didn’t fire” with no further explanation?

A system that passes all four with a flat or slightly negative paper P&L has told you it’s probably safe to trust with the next question — whether the strategy has edge — which is a separate piece of work with its own evidence standard, not something paper trading was ever going to answer on its own.

What this generalises to

Worth auditing your own paper stage for:

• If someone asked “what does this paper account actually prove,” would the honest answer be about the code or about the market view?
• Has the system been forced to restart mid-position at least once, on purpose, to see what it does?
• Are paper fills modeled with any friction at all, or are they always at the exact requested price?
• If the paper P&L were deleted and only the logs remained, could you still tell whether the stage passed?

The moment a paper stage is treated as a track record instead of a test, every subsequent decision inherits a confidence the data was never able to support.