Thirty-One Signals, Six Bets · part 9 of 10

§ 6

Limitations

  • One decade, one regime. September 2016 to September 2026 contains a historic bull market and two short drawdowns. A momentum book that fails here might not fail in a different regime, and a short leg that loses money here is being asked to fight the trend of the whole sample.
  • Survivorship is controlled, not eliminated. Equal-weighting the same universe removes the bias from the comparison. It does not make the universes representative — the construction premia of +150 and +226 points show how far from representative they are.
  • Daily bars only. No intraday data, and no market-on-close or market-on-open execution. The overnight/intraday decomposition we measured (SPY: +9.10%/yr overnight against +3.98% intraday, Sharpe 0.81 vs 0.35) is deliberately not wired into the harness, because capturing it needs execution the daily-bar engine cannot represent.
  • Spreads are quoted, current, and delayed. Measured in 2026 and applied to a nine-year history.
  • Thirty-one signals is not "all signals." It is 31 price-derived signals. The breadth result is about that class, and the paper's conclusion is scoped to it.
  • The IC study is in-sample across the whole decade. That is deliberate — it is a decision about whether to keep building, not a strategy — but it means the null is a null on the most favourable possible reading. In-sample failure is dispositive in a way in-sample success never is.
  • One implementation, one author. No independent replication.
§ 7

What would change the answer, and why it cannot be tested

The only thing that moves this result is genuinely orthogonal information — not another transformation of the same OHLCV series. The obvious candidate is text: earnings-call language, guidance changes, filings. Unlike the price path, its arithmetic is not absurd.

Table 10 — Hypothetical: 112 names × 4 earnings events = 448 bets/yr
If text features gave ICIRYears to t = 2
0.020.4222
0.030.6410
0.051.064

Ten years to significance is a different universe from three hundred and five. That is a real difference in kind, and it is the one place an LLM could plausibly contribute — as a source of features, not as a decision maker over the same prices it has already been shown to reproduce.

But it cannot be backtested honestly. A model trained through 2025 already knows what happened after a 2019 earnings call.

This is worth stating precisely because it defeats the specific defence this project relies on everywhere else. Our look-ahead prevention is structural: the strategy holds an object that has no method returning the future. That defence is useless here, because the contamination is not in the data handed to the model — it is in the weights. There is no held-out set, no windowing scheme, and no prompt discipline that removes it. A model can be instructed not to use hindsight; it cannot be made not to have it.

So the only valid test of the text path is forward, on paper, over years, on data that must be licensed. That is not an argument that it will not work. It is the price of finding out, and it should be paid knowingly rather than discovered after a backtest that was never capable of being honest.