MissedTakes scoring methodology

Status: APPROVED by Ben, 2026-07-31. This is the build contract for the grading engine (E6) and the source text for the public /methodology page. Changing the math after predictions have been published means regrading every one of them, so amendments go through Ben, not through a code review.

1. What we are actually measuring

Two numbers, published side by side, answering different questions.

Accuracy — how often a pundit is right. Plain, honest, instantly understood. It is also, on its own, a bad measure of skill.

MissedTakes Score — how much better a pundit does than a naive strategy. This is the number that separates insight from going along with the crowd.

The reason we need both: some predictions are nearly free. Home teams win more often than they lose, and last season's best teams are usually good again. A pundit who mechanically picks the home team, or who reprints last year's standings as this year's rankings, will post a respectable accuracy while demonstrating no knowledge whatsoever. A pundit making genuinely sharp contrarian calls may score lower and be far more useful to listen to. Publishing accuracy alone would rank the second pundit below the first, which is precisely the misleading conclusion we exist to correct.

So: accuracy is the headline, skill is the story. Both appear on every pundit page, always with the sample size next to them.

The baseline

For each prediction type we define a naive baseline — what a person with no knowledge beyond public information would predict.

The MissedTakes Score is then, per prediction, how much better or worse the pundit did than the baseline did on that same prediction, normalized so that 0 means "no better than naive" and positive means genuine skill. A pundit who only ever repeats the baseline lands at approximately zero however high their raw accuracy climbs. That is the correct and interesting result.

Decided (Ben, 2026-07-31): no betting odds. Lines would be the sharpest measure of "what everyone already thought", but they cost money to license and associate MissedTakes with sportsbooks, which is not the company we keep. The baselines above use only public results from prior seasons.

This makes the baseline weaker, and we should be honest about the consequence: it is easier to beat than a betting line, so MissedTakes Scores will skew slightly positive across the board. That is acceptable — the number's job is to separate pundits from each other and to expose chalk-pickers, and a last-season baseline does both. It is not a claim to have beaten the market.

2. Grading by prediction type

Every grader returns a score in 0–100 for accuracy, plus a skill delta versus the baseline. Both are stored per prediction so they can be recomputed if this methodology changes.

2.1 Game outcome picks

The simplest case, and the only binary one.

• Correct winner → 100. Wrong → 0.
• Ties: if the game ends in a tie, every pick is graded 50. Rare enough not to distort, common enough in the NFL that ignoring it would be wrong.

Skill delta: measured against the baseline, which without odds data is the home team. Correctly picking a road team earns more than correctly picking the home team, and a pundit who only ever picks the home side converges on a skill score of zero no matter how high their raw accuracy climbs. That is the whole point of the second number.

Once we have enough graded seasons, this baseline can be sharpened using our own data — last season's standings are a better predictor than home advantage alone — without ever touching a betting line.

2.2 Score predictions

Graded on two axes, because a score prediction contains two claims.

Winner (50% weight): as above.

Margin (50% weight): how close the predicted margin came to the actual. Credit decays with the error, reaching zero at 21 points — three scores, the point at which a prediction has stopped being informative.

margin_score = max(0, 100 × (1 − |predicted_margin − actual_margin| / 21))

A predicted 24–20 (margin 4) against an actual 27–17 (margin 10) scores 100 × (1 − 6/21) ≈ 71 on margin, and 100 on winner, for 86 overall. That feels right: they called it correctly and were reasonably close.

Decided (Ben, 2026-07-31): margin only, not raw scores. Predicting 24–20 when the game finishes 31–27 is an excellent read — same margin, right winner — and should not be penalised for missing the absolute totals. A pundit who nails both is captured anyway, since nailing the raw scores necessarily nails the margin.

2.3 Power rankings

The subtlest one, because a ranking is a whole ordering rather than a single claim.

Graded on rank correlation between the pundit's ordering and the end-of-season standings — how well the shape of their ordering matches reality, rather than how many teams they placed exactly right. Nobody places 32 teams exactly; that is not the skill being tested.

Concretely, Spearman's rank correlation, rescaled from its natural −1…+1 range to 0–100, so 50 means "no better than random ordering" and 100 means perfect.

Three practical rules:

• Partial lists. Plenty of pundits publish a top 10 rather than all 32. We grade the teams they actually ranked, against those teams' relative finishing order. A correct top 10 is a real achievement and gets full credit for what it claimed.
• Ties in the standings are resolved by the league's own tiebreakers, since that is what the standings page shows.
• Mid-season rankings are graded against the final standings, but tagged with the week they were published. A Week 2 ranking and a Week 16 ranking are not comparable achievements, and pundit pages should let you filter.

Decided (Ben, 2026-07-31): earlier predictions are worth more. Calling a team's season in July is genuinely harder than describing in December what has already largely happened, and without this the optimal strategy is to publish nothing until the answer is obvious.

Every prediction carries a difficulty multiplier based on how much of the season remained when it was published:

difficulty = 1 + (weeks_remaining / total_weeks)

A preseason call (18 of 18 weeks left) is worth 2.0×; a Week 9 call about 1.5×; a Week 17 call about 1.06×. The multiplier scales the skill contribution, not the raw accuracy — accuracy stays a plain honest percentage of how often someone was right, and the MissedTakes Score is where boldness and timing are rewarded.

This applies to every season-long prediction type, not just rankings: win totals, playoff berths, and Super Bowl picks made in July should all outrank the same call made in December. Ben's framing: a perfect-season team being tipped for the Super Bowl in Week 17 is not an impressive call.

2.4 Season win totals

• Graded on distance from the team's actual win count, with credit reaching zero at 5 games of error.

score = max(0, 100 × (1 − |predicted_wins − actual_wins| / 5))

Predicting 11 wins for a team that finishes 10–7 scores 80. Predicting 11 for a team that finishes 4–13 scores 0. Five games is deliberately tight — win total predictions are made with full offseason information and a full season to be right in.

2.5 Playoff and Super Bowl picks

Added to the POC (Ben, 2026-07-31 — see §4 for why this one and nothing else).

Two closely related claims, both binary and both published constantly during the preseason we are about to ingest.

• Playoff berth. "These teams make the playoffs." Graded per team named: 100 if that team made it, 0 if not. A pundit naming 14 teams gets 14 graded predictions, which is a healthy sample from one article.
• Super Bowl pick. 100 for the winner, 50 for picking a losing finalist — reaching the Super Bowl is most of the call, and grading it as a total miss would be unfair to a genuinely good read.

Both take the §2.3 difficulty multiplier, which matters more here than anywhere: picking the eventual champion in July is a real call, and picking them in January is barely a prediction at all.

2.6 Draft position (mock drafts)

Added by amendment, Ben, 2026-08-01, and revised the same day after we looked at what the first version produced.

A mock draft is graded twice, because two different questions are worth answering and one number cannot answer both.

Accuracy — how close was each pick. Per pick, on its own. Credit halves every sixteen picks you are out:

accuracy = 100 × 0.5 ^ (|predicted pick − actual pick| / 16)

This is the number a reader wants: you had him at 12, he went 23. It is what appears next to each pick.

Amended by Ben, 2026-08-02, replacing a straight line that reached zero at one full round. The line put 29% of all graded picks (102 of 355) on exactly zero and scored a pick 32 places out identically to one 128 places out — two predictions that are not remotely equally wrong. The curve was chosen over simply widening the line to two rounds because it agrees with the old rule where the old rule was trusted — 100 for an exact hit, 50 at sixteen picks out — and differs only in the tail. Widening instead would have moved a twenty-pick miss from 37.5 to 68.8, loosening the standard for ordinary misses in order to fix a problem that only exists at the extremes.

Measured against the archive before adopting it: the Round 1 average moves 74.2 → 73.1, scores landing on zero fall from 102 to 4, and the ranking of pundits is unchanged — so this changes the published number rather than who is ahead. A score now approaches zero without arriving: 128 picks out is 0.4, because it is a worse answer than 32 out and the number should say so.

Skill — did they get the shape of the draft right. Scored across the whole mock with the same rank correlation as power rankings (§2.3), against a naive strategy that shuffled the names, which scores 50.

Every pick in one mock therefore carries the same skill figure, because getting a draft's order right is one achievement, not thirty-two of them. That is also consistent with §3: the article is the unit of evidence about a pundit.

A player who was mocked and then went undrafted is not graded, and is left out of the ordering. That is a real, wrong prediction, but "undrafted" has no position in the real sequence to be measured against, and scoring it as the last pick would be inventing a fact. A mock with only one gradable pick gets an accuracy figure and no skill figure — there is no order to have got right, and publishing a skill number for it would assert something we did not measure.

Why not a per-pick baseline. A per-pick skill score needs a naive guess for one named player, and every candidate is either third-party — consensus big boards and mock aggregates, already rejected for fantasy sleepers in §4 — or useless. The first version of this section used the midpoint of the draft (pick 129). That scores zero against every pick inside the first two rounds, which is where essentially all published mocks live, so skill = accuracy − 0 = accuracy: the MissedTakes Score silently became a copy of the accuracy figure, and mock drafters outranked every other kind of pundit on any mixed leaderboard. Recorded here because the rejected version is part of why the current one is shaped as it is.

Draft claims do not take the §2.3 difficulty multiplier. It asks how much season was left to be wrong in, and a draft is a single event rather than a season; these grade at 1.0.

2.7 Player season statistics

Added by amendment, Ben, 2026-08-01.

"He finishes with 1,400 receiving yards." Graded on relative error, because one absolute tolerance cannot serve both receiving yards and sacks:

scale    = max(|actual|, |predicted|, 1)
accuracy = 100 × (1 − (|predicted − actual| / scale) / 0.5)

Credit reaches zero at 50% out. Scaling by the larger of the two values keeps the score symmetric and does not divide by zero when a player is hurt in Week 1 and finishes on nothing.

Regular season only, never summed with the postseason. A fantasy year ends before the playoffs, and everyone making these claims means the 17 games. Quietly adding January would inflate every actual and mark honest predictions wrong. fantasy_points means standard scoring, not PPR (Ben, 2026-07-31).

Baseline: what he did last season, regressed halfway toward his position's average. The same shape as the win-total baseline in §2.4, deliberately — this is the philosophy §1 already commits to rather than a second one invented for players. A player with no prior season is a rookie, and the positional average is the whole of what public information said about him. Positional averages are computed over players who actually posted a figure, since including every third-stringer's zero would describe an "average quarterback" that no real starter resembles.

2.8 Positional finishes and ranked player lists

Added by amendment, Ben, 2026-08-01.

• Positional finish — "he finishes as WR5." Graded on distance in places, credit reaching zero at 12 places, roughly the startable tier these claims are actually made in. Baseline: he finishes where he finished last season; a player with no prior season is graded against the bottom of the startable tier, which is what an unknown at that position is worth to anyone drafting — QB12, RB24, WR36, TE12, the same startable tiers used for fantasy sleepers in §4. Positions outside that group fall back to 24, which is an acknowledged imprecision rather than a considered number: these rankings are ordered by fantasy points, which say little about a cornerback.
• Ranked player lists (fantasy draft rankings, top-100 lists) — graded with the same rank correlation as §2.3, within position. Ben, 2026-08-01: no statistic ranks a left tackle against a cornerback, so a mixed list is scored as its positional sub-orderings and the cross-position ordering is not scored at all. Each position's score is weighted by how many players it contributed, so a ten-deep receiver ordering counts for more than an incidental pair of tight ends.

Baseline for any ranked list: 50, not 0. The rank-correlation score maps onto 0–100 as ((rho + 1) / 2) × 100, so a coin-flip ordering already scores 50. A baseline of zero would credit a pundit for the half of the scale that random guessing earns for free.

3. Aggregation

A pundit's headline accuracy is the mean of their graded prediction scores, computed overall, per sport, and per prediction type.

Time weighting. Not in the POC, per your answer. The engine stores a timestamp on every grade so that a decay function can be switched on later without regrading anything. When we do turn it on, my recommendation is a two-season half-life: last season matters most, three years ago barely.

Score at time of prediction. Every published opinion records the pundit's accuracy as it stood when they made the call, alongside their current score. Without this, an old post silently rewrites itself as the pundit's record changes, and the archive stops being trustworthy.

The three levels, and which one is the evidence

Amended by Ben, 2026-08-01. Originally a pundit's record was the mean of their graded predictions. That counted a 32-pick mock draft as 32 independent judgments and rated its author "established" off a single article — precisely what the sample thresholds below exist to prevent.

Every article carries equal weight, however many takes are in it. A 91-item list and a one-line take are each a single act of judgment: one moment, one set of information, one chain of reasoning. Counting the takes inside them as separate observations would treat a pundit's longest article as their most tested opinion, and would make publishing longer lists the cheapest way to move up a leaderboard.

Stated the way a reader would say it: "Rich Eisen averaged 76% on power rankings this season." Not 76% of his individual placements — 76% across the articles.

Small samples

We publish scores at any sample size, per your answer — with an explicit callout. The thresholds count ARTICLES (Ben, 2026-08-01).

• Fewer than 10 graded articles in a scope: show the number, but label it prominently as provisional, and exclude the pundit from leaderboards for that scope.
• 10–29: show without the warning, still flagged as a limited sample on the pundit page.
• 30+: treated as an established record.

Where two records are otherwise level, the one built from more graded takes ranks higher (Ben, 2026-08-01). A longer article genuinely does tell us more about someone, even though it remains one article of evidence about them.

The thresholds are judgment calls, not statistics — 30 is the conventional point where a mean starts to stabilize. What matters is that a pundit with three lucky calls never appears to be the most accurate person in football, and that neither does one with a single long list.

Clarity is article-weighted too, on the same reasoning: it is a statement about how a person writes, so one enormous vague list should not define someone who is otherwise precise. The minimum sample still counts claims, because having enough evidence to publish a rate is a different question from how the rate is weighted.

4. Other prediction types worth grading later

You asked whether we are missing frequently published, reconcilable opinions. Here is the survey, ordered by how cleanly each resolves.

Strong candidates — unambiguous resolution:

Good candidates — need a definition first:

Weak candidates — I would not grade these:

• Injury return timelines — the honest answer is usually medical and private, and being wrong is not a pundit failing.
• "Team X has a good culture" — unfalsifiable.
• Anything about a player's character or attitude — even where checkable, grading it invites exactly the tone we said we would avoid.

The last group matters editorially, not just technically. The line I would draw: we grade claims about outcomes, never claims about people.

5. What this means for the tone

The methodology is the product's credibility, so it is public in full, and every graded prediction links to the source and shows its own arithmetic. A pundit who disagrees with a grade should be able to see exactly how it was computed and tell us we got it wrong — the corrections channel in E10 exists for that.

Two things we will not do:

• No leaderboard of "worst" pundits as a headline feature. The data will support it; presenting it that way is the cheap version of this idea. Rank by accuracy ascending if a user asks for it, but do not build the product around humiliation.
• No grading of predictions a pundit did not actually make. Paraphrase is where this kind of site loses its integrity. If a claim is too vague to grade, we do not grade it.

Sign-off

Approved by Ben, 2026-07-31.

Five prediction types are therefore in scope for the POC: power rankings, game outcomes, score predictions, season win totals, and playoff/Super Bowl picks.

Amendment 1 — scored scope, approved by Ben 2026-08-01

We were capturing seventeen kinds of prediction and permitted to score six, which left 336 of the 337 claims held at the time unscorable and the public site with nothing to show until January 2027.

Six further types are therefore in scope: draft position, player season statistics, positional finishes, fantasy draft rankings and top-player lists. The remaining captured types stay unscored until they have both a grader and a baseline written here.

Amendment 2 — the unit of a pundit's record, approved by Ben 2026-08-01

Amendment 3 — baselines signed off, Ben 2026-08-01

Every baseline now in the code has been reviewed. There are no unsigned numbers left in the scoring.

The positional-finish rookie rule was changed at sign-off: it had been a flat 24 for every position, which held a rookie quarterback to a generous baseline since twelve quarterbacks start rather than twenty-four.

Amendments after this point require Ben's sign-off and a regrade, since the published archive is computed from these rules. 5. Anything in §4 you want pulled forward into the POC.