Outcomes per unit, and the faction tables' terms (T-571)

3 Oct 2026 (UTC). Step 1 of design/tier-plan.md's queue: the outcome yardsticks (plan section 5, items 2 and 3; design/matrix.md sections 6.3 and 9). This is a report only. Nothing in it reaches the site.

Credit. The tournament lists come from grimstat-corpus at commit 570b8c6. The compilation is published under CC BY 4.0, and the lists were published by their players and organisers on MiniHeadQuarters (miniheadquarters.com). We didn't read the corpus's listhammer/ folder. Everything below is our own counts and averages. It holds no list text and no player or event names.

In one paragraph

From the corpus's last three months (July to September 2026) we placed 382 lists from 23 events across 33 armies. Every list has a final placing and none has a game record: the corpus's top-level files carry final placings only, with no round-by-round results and no win-loss records. We matched unit names to our stubs with the app's own list importer. 6,337 of 7,340 unit lines matched (86%), and every matched name is in docs/data/index.json. We set aside 92 lists because fewer than 85% of their units matched; most were exported from the GW app in French. A unit's lift is the mean finishing percentile of its army's lists that include it, minus the mean of the lists that don't, and it is noisy. Of the 418 units with three or more lists on each side, 33 (7.9%) have a 95% interval clear of zero, where chance alone would give about 21 (5%). There is a signal, but at today's volume it is only strong enough to test the engine across many units at once. It can't grade a single unit. Nobody lets a machine read their faction-versus-faction tables: no candidate's terms allow it:

• Stat Check has no terms and names AI crawlers to keep out in its robots.txt.
• 40kstats (Tabletop Battles) and Goonhammer answer machines with a bot check.
• Games Workshop's Metawatch forbids copying outright.
• Listhammer's tables carry no licence.

All of them are listed in tools/sources.json, switched off. Jordan decides whether to ask.

What was loaded

node tools/outcomes.js --corpus build/grimstat-corpus --out reports/tiers/2026-10-03-outcomes.json runs in about a second. The corpus is cloned at run time (a shallow clone of https://github.com/N041M/grimstat-corpus) into build/grimstat-corpus/, which git ignores like every file under build/.

• Sources read: only the sources switched on in tools/sources.json, which today means minihq (the corpus's top-level files). They are read through tools/sources/index.js, the same reader build_lists.js uses.
• Window: the latest three months the corpus holds (2026-07, 2026-08, 2026-09), the same window docs/data/inclusion.json and design/matrix.md 6.3 use. --months changes it. One event on 4 July is named as a farewell to the old edition and may have been played under it. It is kept and flagged below.
• Placing percentile: 1 for first and 0 for last, over the event's field. The field is whichever is larger: the number of lists the corpus holds for the event, or the highest placing seen. The corpus holds nearly every player of these events: of the 35 events it carries, the list count matches the top placing in 25, and is at most two short in the other 10.
• Lists kept: a list is kept when three things hold:
  • its army is recognised (all were);
  • 85% or more of its units match a datasheet, the same threshold build_lists.js uses;
  • its own unit costs add up to 1,500 points or more (--min-points). This left out 39 lists from 1,000-point events, because squads are judged at 2,000.
• Per unit: a unit is keyed by its name in the army, or by faction|name for an allied datasheet. A chapter's Space Marine datasheets count under the chapter's own name. Every Adeptus Astartes chapter is also pooled into one "every chapter" group (pools in the JSON), because no single chapter has more than 22 lists.
• The interval: Student's t at 95%, with the two sides' variance pooled. Pooling stops three lists that happen to agree from claiming a narrow interval. Two things aren't corrected for: lists of one army in one event aren't independent, and many units are tested at once. Read the intervals as a guide.
• Win rate: whenever a source gives game records (record: { wins, losses, draws } in the source layer's shape), the tool computes the win rate with and without each unit, with its own interval. No source does today, so every win-rate field is null and gameResults says why.

Coverage

Lists per army:

Unmatched names: 1,003 unit lines across 767 distinct names. The JSON's unmatched lists them all, by army, with counts. About four in five come from the GW app's French export ("Ligne 1 : 20 Carnivores Kroot", "Héros épiques 2 : …"), which the importer doesn't read. The rest are spellings and allies the importer misses:

• "Myphitic Blight-haulers" (11 lines);
• "Nurglings" and "Beasts of Nurgle" in Chaos lists;
• "10 Skitarii Vanguards" (a count before the name);
• "Wartrakk";
• "Callidus Assassin";
• "Inquisitor Draxus";
• "Sisters of Battle Immolator" in Imperial Knights lists.

Teaching roster.js these forms would bring back up to 92 lists, about a quarter more data. That is a later card. The prefix forms and counts are a parsing change; French names would need a name table of our own.

Game results: what exists

• The corpus's top-level files (CC BY 4.0, read): final placings only. No rounds, no opponents and no win-loss records.
• The corpus's listhammer/ folder (not read, outside the licence): its README says the folder holds:
  • a win-loss record for each list;
  • for Grand Tournament lists, each round's result and score and the opponent's faction;
  • every player's record at each event.

That is exactly what both yardsticks need: the win rate with and without a unit, and faction-versus-faction results we would compute ourselves. It waits on Listhammer's OK (T-411; the permission email is on T-418). tools/outcomes.js will read records as soon as a source that has them is switched on.

The sanity view: top and bottom 10 by placing lift

These tables show units with three or more lists on each side, ranked by lift. The percentiles and lifts are in percentile points, where a list's finish runs from 0 to 100. The samples are small: Tyranids and Orks have only 23 and 16 lists in all.

Tyranids: 23 lists, mean percentile 54; 31 units with 3+ lists each side

Top 10

Bottom 10

Space Marines (every chapter pooled): 73 lists, mean percentile 48; 90 units with 3+ lists each side

Top 10

Bottom 10

Orks: 16 lists, mean percentile 40; 21 units with 3+ lists each side

Top 10

Bottom 10

Reading it

• What looks right: The Swarmlord's lift and Flash Gitz's both have intervals clear of zero, and both units are prominent in the current competitive picture.
• Builds, not single units. Lift often belongs to a whole build rather than one unit. Logan Grimnar and Wolf Guard Terminators share exactly the same lists, and Arjac Rockfist and Wolf Scouts ride with them, so the pooled Space Marine top 10 is mostly "Space Wolves did well". The bottom 10 is mostly "Blood Angels did badly" (Commander Dante, Chaplain with Jump Pack, Vanguard Veterans with Jump Packs). The pool mixes chapter with unit: the per-chapter rows in armies are the cleaner read once there are enough lists.
• Zoanthropes at the bottom (16 lists with, 7 without) says the lists without them did unusually well. That is a property of seven lists, not a verdict on the unit. Tier-engine rounds that rank Zoanthropes high are not contradicted at this sample size: the interval reaches zero.
• What the engine can use: two things suit the stop rule and the fitter (plan sections 6 and 10, step 5):
  • in aggregate, the rank correlation between a letter and its lift across hundreds of units, weighted by the smaller side's count;
  • a held-out check: do units the engine grades S/A show positive lift more often than chance?

A single unit's lift should not move its letter.

Faction-versus-faction tables: the terms

Each site was checked on 2 Oct with tools/sources/check.js (its robots.txt and the pages about terms, one request a second under our named agent). statcheck, goonhammer and metawatch were added to its list of platforms, and Stat Check's dashboard, Tabletop Battles and Tableau Public were checked by hand. Nothing was scraped and nothing was stored except our notes.

Verdict: none allows it today. Jordan's 30 Sep rule already treats these tables as a cross-check only (T-332: nothing of theirs on the site, not even a link). That rule still holds.

In what form a yardstick could be stored, if a source agrees:

1. Never their table. Their numbers, even reformatted, stay out of the repository and out of docs/.
2. Our computed agreement only. From design/matrix.md 9, we would keep the ρ between our predicted pair margins and their pair win rates (over pairs with 30+ games, weighted by games), the Brier score against an army-only baseline, and the number of pairs. Their numbers would be read at run time into memory or build/, which git ignores, and thrown away. This needs the source's yes for a machine to read it, even though nothing of theirs is kept.
3. Better: compute faction-versus-faction ourselves from game records a source licenses (Listhammer's games through the corpus, once agreed). Then the yardstick is entirely ours: our counts of games between armies, credited, with no one's table involved. This is the recommendation.

Until one of these is open, the army-against-army yardstick can't run. The per-unit outcomes above can.

Files

• tools/outcomes.js (new): the tool. Its parsing and arithmetic are exported for tests/unit/outcomes.test.js.
• reports/tiers/2026-10-03-outcomes.json: per army and unit, with against without. pools holds the Adeptus Astartes chapters pooled, unmatched the names that didn't match, coverage the counts above.
• tools/sources.json: statcheck, ttbattles and metawatch added, switched off, with their terms in our own words, plus a line on Listhammer's /stats.
• tools/sources/check.js: the three new platforms added to its list.

Doubts

• The 4 July event named as a farewell to the old edition (26 lists in the corpus) may have been played under 10th-edition rules. It is kept, because the corpus doesn't say which edition an event used.
• An event's size is what the corpus holds. Where an organiser published only some lists, the bottom of the field is missing, and percentiles are a little low for everyone there.
• Lists of one army at one event share a meta, so the intervals assume more independence than there is.

For the supervisor

Ready to paste into design/decisions.md:

3 Oct: T-571 outcomes per unit and the faction tables' terms (queue step 1, yardsticks)

• What changed: tools/outcomes.js reads every placed list from the sources switched on (grimstat-corpus's top-level files, CC BY 4.0; listhammer/ not read) and writes, per army and unit, lists with and without the unit, the mean finishing percentile each side, the lift with a 95% interval (Student's t, pooled variance), and win rates whenever a source gives game records. Report: reports/tiers/2026-10-03-outcomes.md and .json.
• Numbers: 382 lists, 23 events, 33 armies (July to September); 86% of unit lines matched, 0 outside our stubs; 92 lists set aside under 85% matched (mostly French exports) and 39 under 1,500 points. Of 418 units with 3+ lists each side, 33 (7.9%) have an interval clear of zero, against about 21 (5%) by chance.
• Game results: none. The licensed files carry final placings only. Round-by-round games exist only in listhammer/, which waits on Listhammer's OK.
• Faction tables: none allows a machine read (Stat Check: no terms, AI crawlers named out; 40kstats and Goonhammer: bot check; Metawatch: no copying; Listhammer: no licence). All are listed off in tools/sources.json.
• Decided alone: the latest 3 months as the window; lists under 1,500 points dropped; the field size taken as the larger of lists held and top placing; Student's pooled-variance t in place of Welch (Welch gave intervals far too narrow for 3-list sides); chapters pooled as an extra view, not in place of the per-chapter rows.
• Use: a yardstick in aggregate only (the correlation of letters with lift, weighted by the smaller side; the S/A hit rate on positive lift). A single unit's lift should never move its letter. If a source agrees, the army-against-army yardstick stores our agreement numbers only, never their table; computing it ourselves from licensed game records is preferred.
• Next: teach roster.js the French export and the missing spellings (up to 92 more lists); Jordan's call on asking Stat Check and Tabletop Battles; the Listhammer reply (T-418).

Ready to paste as a scoring.md changelog line:

• 3 Oct 2026 (T-571): outcome yardstick added, not used in scoring: tools/outcomes.js, the placing lift per unit (mean finishing percentile of the army's lists with it minus without, Student's t 95% interval, pooled variance) and win rate with and without when game records exist (none today); reports/tiers/2026-10-03-outcomes. No published number changes.