Glossary
Plain-English definitions for the terms used across these reports. Each of these is also explained inline the first time it appears in a document — look for the dotted underline.
95% CI
95% confidence interval
The range the true value is plausibly in. If this range includes zero, we cannot rule out that the real effect is nothing at all.
bias
Whether the misses lean consistently one way. A projection can have a good average error size but still be biased if it is almost always too high. Bias is often the more fixable problem.
bootstrap
Re-running a calculation on thousands of resampled versions of the data to see how much the answer wobbles. The spread of those answers becomes the confidence interval.
calibration
A deliberate sanity check on the method itself: run it on something already known to be true. If it fails to detect the known thing, the method is broken and its other results mean nothing.
closing line
The final odds right before a game starts. It reflects everything the betting market knows, which makes it the hardest benchmark to beat.
CLV
closing line value
Whether you got a better price than the market settled at. Widely used as a faster signal of skill than profit, which takes ages to measure.
confidence interval
A range of values the true answer is plausibly in. If it includes zero, the effect might genuinely be zero, so we cannot claim it is real.
correlation
How closely two things move together, from -1 (opposite) through 0 (unrelated) to +1 (in lockstep). It does not by itself mean one causes the other.
CPOE
completion percentage over expected
How often a quarterback completed passes compared to what was expected given the difficulty of the throws he attempted. It credits accuracy rather than rewarding a QB for only attempting easy throws.
decisive null
A null result we are confident in, because the same method demonstrably CAN find a real effect when one exists (see calibration). Distinct from simply not having enough data.
devig
Removing the bookmaker's cut from odds to recover the market's actual implied probability.
ECR
expert consensus ranking
FantasyPros' averaged ranking across many fantasy analysts. A strong benchmark, and hard to beat.
Elo
Elo rating
A rating system, originally from chess, that moves a team up or down based on results and the strength of the opponent.
EPA
expected points added
How much a play changed the number of points its team should expect to score on that drive. A better measure of a play than raw yards.
expected value
The average result if you could repeat a bet or decision endlessly. Positive expected value means it pays off on average, even though any single instance can lose.
FDR
false discovery rate
A method for handling many simultaneous tests, controlling what share of your "discoveries" are expected to be flukes.
fixed effects
A design where each subject is compared against themselves, which cancels out their permanent differences. Here it means comparing a player to his own other seasons rather than to other players.
holdout
Data deliberately set aside and never looked at while developing an idea, then used once at the end as a fair test. Peeking at it first would defeat the purpose.
in-sample
Measured on the same data used to build or tune the idea. Nearly always looks better than reality.
Kelly
Kelly criterion
A formula for how much to stake given your edge. Full Kelly maximizes long-run growth but swings wildly; most people use a fraction of it.
log loss
A score for probability forecasts that punishes confident wrong answers harshly. Lower is better.
look-ahead
look-ahead bias
Accidentally using information that was not available at the time. It makes predictions look brilliant and is the single most common way a backtest fools you.
MAE
mean absolute error
Average size of the miss, ignoring direction. If a projection is off by 3 one week and -5 the next, the MAE is 4. Lower is better.
market error
How far the actual result landed from what the betting market predicted. If this is unrelated to some factor, the market already accounted for that factor.
monotone
Consistently moving one direction as the input increases. A real dose-response effect should be monotone; if more of the cause does not mean more of the effect, the pattern is suspect.
multiple comparisons
Testing many ideas at once. Test twenty things and one will usually look significant by luck alone, so results need a stricter bar.
null
null result
A test that found nothing. "Null" is the starting assumption that there is no real effect; a "null result" means the data gave us no reason to abandon that assumption. It does not mean the data was missing or the test failed to run.
null result
A test that found nothing real. The starting assumption is "there is no effect here", and a null result means nothing in the data argued against it.
out-of-sample
Tested on data that was not used to build or tune the idea. This is the honest test; results on the data you built with are almost always flattering.
p-value
The probability of seeing a result this strong purely by chance if nothing real were going on. Small = surprising under "nothing is happening". Below 0.05 is the usual convention, but it is a convention, not proof.
percentage points
An absolute difference between two percentages. Going from 50% to 53% is 3 percentage points, not 3%.
permutation
permutation test
Randomly shuffling the labels thousands of times to see what results pure chance produces, then checking whether the real result stands out from that.
PPR
points per reception
A fantasy scoring format that awards a point for every catch, which raises the value of high-volume receivers.
regression to the mean
Extreme results tend to be followed by more ordinary ones, purely by chance. Mistaking this for a real decline is a classic error.
ROI
return on investment
Profit as a percentage of the money wagered. +2% means $2 profit per $100 bet.
season-block bootstrap
A bootstrap that resamples whole SEASONS rather than individual games, because games within a season are related. Ignoring that makes results look more certain than they are.
selection bias
When the group being studied was not chosen neutrally, so it differs from the population in ways that distort the answer.
Sharpe
Sharpe ratio
Return relative to how much it bounced around. Higher means smoother returns for the same profit.
Spearman
Spearman correlation
A measure of whether two rankings agree, from -1 (opposite) through 0 (unrelated) to +1 (identical). Cares about order, not exact values.
split-half
split-half reliability
A repeatability test: split the data in two (here, odd vs even seasons) and check whether the same units score similarly in both halves. If a trait does not repeat, it was noise, not a trait.
Stage A
The first question: is the effect real at all? Measured against what actually happened, ignoring betting markets.
Stage B
The second question: is the effect already reflected in the betting odds? An effect can be completely real and still worthless to bet, because the price already includes it.
standard deviation
A measure of how spread out values are. Expressing an effect "per standard deviation" makes different quantities comparable on the same scale.
survivorship bias
Studying only the things that stuck around. Looking at current players ignores everyone who washed out, which flatters the results.
tradeable floor
The minimum edge needed to overcome the bookmaker's cut. Anything smaller is real but unprofitable.
underpowered
Not enough data to detect an effect even if it is really there. An underpowered null means "we could not tell", which is very different from "there is nothing there".
vig
The bookmaker's built-in cut. It is why a coin-flip bet at -110 needs you to win about 52.4% of the time just to break even.
walk-forward
Evaluating week by week using only what was knowable before each week, mimicking how the model would actually have been used at the time.