New to these reports? Start here
- Dotted-underlined words have a plain-English definition — hover or tap them. Every term is also on the glossary page.
- "Null" means we found nothing, not that something broke. Most reports here are negative results, on purpose — knowing an idea doesn't work is the point.
- Two questions get asked separately. First, is the effect real? Second, is it already priced into the betting odds? An effect can be completely real and still useless to bet on.
- A "calibration" row is a self-check. It runs the same method on something already known to be true. If that fails, the whole report is unreliable — so it's reported alongside the findings.
- If a confidence interval includes zero, the real effect might be nothing at all, so no claim gets made.
NFL moneyline Brier: what was left to improve (2026-10-21)
Question: can the NFL win probability's Brier score get better with the data we have?
Market-side options, 2010-2025, walk-forward (market_side_walkforward.py)
Fit on prior seasons, scored on the next. 4,276 games, ties dropped. Brier, lower is better.
| Market probability | Brier | vs devigged moneyline |
|---|---|---|
| Moneyline, proportional devigdevigRemoving the bookmaker's cut from odds to recover the market's actual implied probability. | 0.21110 | — |
| Moneyline, power devig | 0.21105 | −0.00004 (se 0.00009) |
| Moneyline, logit recalibrated | 0.21111 | +0.00002 (se 0.00012) |
| Spread → Φ(spread/σ) | 0.21181 | +0.00072 (se 0.00034) |
| Spread, σ grows with total | 0.21182 | +0.00073 |
| Spread → logistic | 0.21107 | −0.00002 |
| Average of moneyline and Φ(spread/σ) | 0.21125 | +0.00015 |
The blend's market side was Φ(spread/σ): the worst option here, about 2 standard errors behind the moneyline. Changing the devig method, letting σ scale with the total, or recalibrating made no difference.
Same fitted model, both anchors, held-out seasons (anchor_same_fit.py <season>)
Production trainer, 5 prior seasons, test on the next. Same XGB fit is scored both ways.
Run 1: weight chosen on shuffled 5-fold (the old method). The weight came out at 0.25–0.45.
| Season | Moneyline alone | Spread anchor, model σ (old) | Moneyline anchor, blend σ |
|---|---|---|---|
| 2025 | .2104 | .2084 | .2081 |
| 2024 | .2010 | .2049 | .2025 |
| 2023 | .2186 | .2200 | .2210 |
| 2022 | .2088 | .2109 | .2107 |
| mean | .2097 | .2110 | .2106 |
The old blend lost to the plain moneyline in 3 of 4 seasons. 2025 was the season quoted as "beats the market", and it was the exception. Shuffled folds let each fold train on games from the same season it is predicting, so the folds rate the model better than it does on a new season.
Run 2: weight chosen on season-ahead folds (fit on earlier seasons, predict the next). The weight came out at 0.15 in all four seasons.
| Season | Moneyline alone | Spread anchor | Moneyline anchor (shipped) |
|---|---|---|---|
| 2025 | .2104 | .2100 | .2097 |
| 2024 | .2010 | .2042 | .2017 |
| 2023 | .2186 | .2171 | .2182 |
| 2022 | .2088 | .2073 | .2076 |
| mean | .2097 | .2096 | .2093 |
Shipped: old .2110 → new .2093, a 0.0018 Brier improvement. - Almost all of it comes from the weight. - The moneyline anchor adds ~0.0004 at w = 0.15, and its sign varies by season. - The model is now level with the market: −0.0004, beating it in 3 of 4 seasons. That is within noise. - Production retrain 2026-10-21: w = 0.10, blend σ 12.70, 2025 test Brier 0.2093 vs market 0.2108.
σ mismatch, fixed
Live predictions used to convert the blended margin with the model's own unblended σ.
The weight had been chosen with σ refit for the blend. Now σ is fitted with the weight and stored as blend_sigma.
At w = 0, the served probability equals the devigged moneyline exactly.
Measurement
nfl_predictionsis rewritten every run. For a played game, nflverse's lines are the CLOSE, so those rows describe close-informed predictions.- 2024 rows are also in-samplein-sampleMeasured on the same data used to build or tune the idea. Nearly always looks better than reality.: training covers 2020–2024.
scripts/nfl_model_clv.pygraded those rows, so "model agrees with where the line went" was partly true by construction.- Its 2025/2026 output rows were deleted.
- New
nfl_prediction_snapshotstable: written only before kickoff, never rewritten. The CLVclosing line valueWhether you got a better price than the market settled at. Widely used as a faster signal of skill than profit, which takes ages to measure. script now reads only this table, starting with the 2026 games kicking off from Oct 22.
Not worth doing (tested above or earlier)
- Devig variants.
- σ as a function of the total.
- Moneyline recalibration.
- More features: five iterations were nullnull resultA test that found nothing. "Null" is the starting assumption that there is no real effect; a "null result" means the data gave us no reason to abandon that assumption. It does not mean the data was missing or the test failed to run. (see memory
nfl-joint-model-2026-09).
Further Brier gains would need information the market doesn't have.