The line we added to keep answers independent made them biased

Someone put our methodology page through an unsympathetic review this week and came back with a specific accusation: our question bank decides which moments become questions by looking at what happened next. A trader standing at 10:35 cannot know that the next fifteen minutes will produce a move of 1–3× ATR. A SwipeTA player, implicitly, can — because if the chart is in front of them, it already passed that test.

That is correct, and we will come back to what it means. But checking it turned up something we did not expect, which is that we had been wrong about our own numbers for two weeks, and our first attempt to fix it was wrong too.

One thing to rule out before anything else, because the title invites the wrong reading: no answer in the bank was ever computed incorrectly. The labels are right; every question's outcome is what price actually did. What went wrong is upstream of that — which moments survived into the bank at all. This is a story about sample composition, not about arithmetic.

The sentence that started it#

We publish a figure that gets quoted more than anything else we have measured: fading the last 5-minute candle is right 60.67% of the time. And this site used to defend it like this:

It is not noise, and it is not an artifact of our selection — it is there in the raw candidate pool before any filtering.

That sentence was written from intuition. Nobody measured it. Eight days later we shipped a horizon sweep that scanned 140,319 unfiltered decision points and found the same rule sitting at 50.0% — and nobody reconciled the two, because the new script was checked against the article it was written for and not against the article it happened to falsify.

So the first thing to say plainly: the rule is a coin. Across 366,606 unfiltered decision points on the exact seven symbols and date range the 60.67% came from, it is right 49.78% of the time.

The obvious suspect is innocent#

The natural next move is to blame the magnitude filter. It is the step that conditions on the future, it is the one an outside reviewer would point at, and it is the one we pointed at ourselves in a second article.

Apply the 1–3× ATR band to every eligible bar and the rule scores 49.31%.

Half a point. The filter that looks like the whole story is worth almost nothing on its own.

Where the thirteen points actually are#

The jump happens at the next line of the generator, and that line is not a filter. It is this:

after accepting a decision point, skip forward one horizon before accepting another

Its purpose is written in the code beside it — outcome windows must not overlap, or two questions in the bank are decided by the same future bars. That reasoning is correct, and we would still enforce non-overlapping outcome windows. What we would not do again is assume this implementation of it is neutral: taking the first member of each run is one way to enforce disjointness, and picking a random member, or staggering the offset, are others that do not select on position. It is the same instinct behind keeping answers disjoint everywhere else in the pipeline.

Here is what nobody checked. Qualifying bars do not arrive one at a time. Neighbouring decision points share most of an outcome window, so when one bar's next fifteen minutes lands in the band, its neighbours usually do too. They come in runs.

A skip-forward rule applied to a run keeps the first bar and discards the rest.

So the rule does exactly what it was designed to do — no two questions share outcome bars — while also, silently, changing what kind of moment ends up in the bank. Not a random point inside a qualifying move, but its onset. Run the identical scan with that one rule switched off and the figure falls from 63.00% back to 49.31%.

We have not established why the onset of a qualifying run should favour a contrarian call. Having now been wrong about this twice in one day, we would rather leave it open than guess a third time.

The general form#

This is the part worth taking away, because it has nothing to do with candles.

De-duplication is sampling. Any time you remove overlapping observations — one trade per day per symbol, one signal per event, one row per user session, drop near-duplicates before training — you are not tidying the data. You are choosing which member of each cluster survives, and clusters are rarely arranged so that the choice is neutral.

What made it invisible here is that the step had a correct stated purpose. Nobody audits a line whose comment explains a real statistical problem it really does solve. The filter above it got scrutinised for years, in public, by us and by outside reviewers, because it obviously conditions on the future. The line below it was housekeeping.

The obvious selection step was worth half a point. The invisible one was worth thirteen.

The warm-up mistake described below is the same theme in a different costume: requiring fourteen bars of history before asking a question is not a neutral technical detail, it is a rule that deletes the opening period. In a data pipeline, preprocessing is rarely just preprocessing.

We got it wrong twice, in public, on the same day#

This piece would be dishonest without the process that produced it, so:

First wrong answer, inherited. The original sentence claimed the rate was not a selection artifact. It was written before the measurement existed, and it survived because our lint gate checks that numbers have sources — and "this is not a selection effect" contains no number.

Second wrong answer, ours, this morning. We shipped a correction blaming the per-session cap: the bank keeps one setup per symbol per session, which is a real selection step and sounds like a good culprit. We inferred that from the pool size — 6,083 rows against about 6,160 symbol-sessions — instead of opening the generator. When we did open it, the rule turned out to keep the day's most decisive setup, and reconstructing that stage properly moves the rate down, from 63.0% to 59.3%. The stage we blamed was working in the opposite direction.

There was a second defect in that attempt worth naming, because it is a trap anyone reconstructing a pipeline will meet. Our scan started at bar 14 to let a 14-bar ATR warm up. The generator starts at bar 5 and computes ATR from whatever history exists. A warm-up requirement silently deleted the entire opening period, which is 19% of the published bank and, as we measured last week, the part of the session that behaves least like the rest.

Three attempts, two of them published. The measurement that settled it took about forty seconds to run.

What we changed#

• The methodology page no longer says the band marks where information exists. It is a question-quality filter. A small outcome is not the same thing as an unpredictable one, which is the distinction we had already drawn for time of day and had not applied to our own filter.
• It no longer says answers are independent. Disjoint outcome windows remove one dependence. Intraday returns still share market-wide factors and time-of-day structure, so the sample-size figures we quote are the optimistic end of a range.
• The 60.67% figure carries a reference line at 49.78%. Charts travel without their articles.
• Our research directory has a new rule: when a new script measures anything an existing article asserts, grep the articles, not just the previous output. That is the failure that let the first sentence stand.

And the accusation itself#

It stands, and it is the right frame for reading anything we publish about our own bank.

Every question in SwipeTA passed a test applied after the fact. That makes the bank good at asking "given a moment where something happened, which way?" — and silent on "was this a moment worth acting on at all?", which is a large part of the job. A baseline measured inside the bank describes the bank. We presented one as though it described the tape, and that was ours to fix rather than theirs to catch.

The review was right about the conditioning and wrong about what produced the thirteen-point jump — wrong in exactly the place we were, by assuming the conditioning happens where the conditioning is visible.

SwipeTA is a training game and a simulation: no real money, no broker, and it does not provide investment advice. The parameters behind our measurements are on the methodology page.