New to these reports? Start here
- Dotted-underlined words have a plain-English definition — hover or tap them. Every term is also on the glossary page.
- "Null" means we found nothing, not that something broke. Most reports here are negative results, on purpose — knowing an idea doesn't work is the point.
- Two questions get asked separately. First, is the effect real? Second, is it already priced into the betting odds? An effect can be completely real and still useless to bet on.
- A "calibration" row is a self-check. It runs the same method on something already known to be true. If that fails, the whole report is unreliable — so it's reported alongside the findings.
- If a confidence interval includes zero, the real effect might be nothing at all, so no claim gets made.
Kalshi daily-high temperature markets: not beatable with free forecasts
2026-10-11. Seven cities (NYC, Chicago, Miami, Austin, LA, Philadelphia,
Denver), 2024-10-01 to 2026-10-09: 29,860 settled bucket markets, about 4,500
city-days, hourly bid/ask from Kalshi's public API (free; settled history and
candles are kept). Scripts: scripts/odds/kalshi_weather_backfill.py,
scripts/odds/nbm_station_forecasts.py, backtests/kalshi_weather/analysis.py
(results in results.json, analysis_output.txt).
Setup
Quotes at two times, each against a forecast published before the quote:
- eve: 18:00 local the day before.
- morn: 09:00 local on the day.
Two forecasts:
- NBM: the National Blend of Models station forecast for the settlement
station, from Iowa State's IEM archive. It uses the latest run published at
least 2 h before the quote, with NBM's own spread (
xnd). - Open-Meteo: the gridded previous-run forecast.
BiasbiasWhether the misses lean consistently one way. A projection can have a good average error size but still be biased if it is almost always too high. Bias is often the more fixable problem. and spread are fitted walk-forwardwalk-forwardEvaluating week by week using only what was knowable before each week, mimicking how the model would actually have been used at the time. per city from earlier settled days. Settlement check: the stored high reproduces all 23,139 results.
Results
| eve | morn | |
|---|---|---|
| Log losslog lossA score for probability forecasts that punishes confident wrong answers harshly. Lower is better., Kalshi mid | 0.343 | 0.301 |
| Log loss, NBM | 0.387 | 0.383 |
| Log loss, Open-Meteo | 0.457 | 0.430 |
| NBM betting, ≥5¢ edge after fee (per contract) | −1.3¢ (SE 0.4) | −1.8¢ (SE 0.3) |
| Blend fitted yr 1, yr-2 log loss gain | 0.0005 (SE 0.0008) | 0.0004 (SE 0.0005) |
| Blend bets yr 2, any edge | −0.05¢ (SE 0.3) | −0.4¢ (SE 0.3) |
| Buy No on every bucket < 5¢ | +0.15¢ (SE 0.17) | +0.31¢ (SE 0.18) |
- The market beats both forecasts in every city, at both times. NBM's day-before error is about 2.8°F (NYC), too wide for 2°F buckets. The market's ~45% buckets win ~45% of the time, so traders use sharper inputs: high-resolution short-range models and live station readings.
- NBM still carries some information the price lacks (z=8.4 eve, 4.6 morn). But it is too little to clear a 2–3¢ spread plus the fee. Blended with the price on year 1, it gave no year-2 gain.
- Longshots are slightly overpriced. Buckets quoted under 5¢ won 1.3% (eve) and 0.8% (morn) against mids of 2.4¢ and 1.6¢. Selling them (buying No at 1 − bid) earns about 0.3¢ per contract, with p≈0.08 in the morning. The eve year 1 was negative (−0.85¢). This is not significant, the size available at the bid is unknown, and it ties up about 98¢ per contract. Not worth building.
Not tested
- Intraday trading on live station observations (METAR/ASOS during the day). That is where weather-market edges are usually claimed. It is a speed game, and hourly candles can't test it.
- Making markets (resting orders). Candles carry no fills.
Verdict: as a taker using free forecasts, Kalshi weather markets are not soft.