New to these reports? Start here
- Dotted-underlined words have a plain-English definition — hover or tap them. Every term is also on the glossary page.
- "Null" means we found nothing, not that something broke. Most reports here are negative results, on purpose — knowing an idea doesn't work is the point.
- Two questions get asked separately. First, is the effect real? Second, is it already priced into the betting odds? An effect can be completely real and still useless to bet on.
- A "calibration" row is a self-check. It runs the same method on something already known to be true. If that fails, the whole report is unreliable — so it's reported alongside the findings.
- If a confidence interval includes zero, the real effect might be nothing at all, so no claim gets made.
Tennis consensus-stacking evaluation — REJECT (2026-07-04)
A full historical backtest was possible: [redacted]::tennis_matches (81,622 matches 2010-2026 with embedded closing odds: market average, Pinnacle, best price) + tennis_match_elo (point-in-time pre-match EloElo ratingA rating system, originally from chess, that moves a team up or down based on results and the strength of the opponent., through 2024-11-17). Joined usable set: 69,090 matches.
Experiment
Orientation randomized on match id (base rate 0.5000, no leakage). Features: devigged consensus (avg close) + Elo prob + elo_diff + prior-match counts + surface/tour/best-of-5. Walk-forwardwalk-forwardEvaluating week by week using only what was knowable before each week, mimicking how the model would actually have been used at the time. by year 2013-2024: 56,201 OOS predictions.
| OOS n=56,201 | stacked | raw consensus |
|---|---|---|
| accuracy | 0.6880 | 0.6875 (tie/noise) |
| log losslog lossA score for probability forecasts that punishes confident wrong answers harshly. Lower is better. | 0.5809 | 0.5803 |
Log loss gate FAIL (consensus won 8/12 years). Consensus wins every slice: ATP, WTA, hard, clay, grass — including WTA-grass (the live scanner's niche: model 0.5945 vs consensus 0.5925, n=3,105), consistent with the scanner's own finding that price-vs-Pinnacle line-shopping, not probability modeling, carried that niche. Flat ROIreturn on investmentProfit as a percentage of the money wagered. +2% means $2 profit per $100 bet. at Pinnacle close: -4.3% -> -14.9% as min_ev rises; at BEST price -0.65% -> -3.5% (the near-breakeven is line-shopping value, not model value).
Forward capture plan: moot
unified_odds tennis capture (June 2026, ~93 closing games) would need ~2 seasons to reach 1/6 of what the archive already answered with 56k OOS matches. No capture campaign justified for stacking.
⚠ Operational side-finding (unrelated to stacking)
tennis_match_elo AND the live scanner's tennis_elo ratings both end
2024-11-17 — the tennis_elo_v1 scanner is running on ~19-month-stale ratings
during the current grass season. Flagged to the coordinator.
Reproduce: run_experiment.py; raw numbers in experiment_output.json.