Every datasheet variable in the diagnosis (T-636, 4 Oct)

Asked by Jordan's colleague: "any variable we haven't added to be added to the list. Can you confirm that all unit profile items, weapons, etc." This card audits every field the unit records carry (build/units.json at BSData 374f505, the 16 armies with a ledger), adds what the diagnosis didn't read, and re-runs T-632's Tyranid diagnosis and T-635's pooled one on the full list. A report: no ledger number, page default, data file, benchmark or frozen input changed. Our own numbers; unit and keyword names only, never rules text.

Commands: node tools/diagnose.js (Tyranids, 10 × 5 folds, about 14 minutes now), node tools/diagnose.js --pooled (16 armies, about 32 minutes on 4 threads), and --base-vars on either for the first 144 columns alone. Files: 2026-10-04-variables-tyranid-features.csv with its dictionary -tyranid-features.md (341 columns, the old five lists' letters; -variables-headline-tyranid-features.csv the same with the headline lists' letters); 2026-10-04-variables-diagnosis-tables.md and .json (Tyranids, the old five lists); 2026-10-04-variables-headline-diagnosis-tables.md and .json (Tyranids, the step-24 headline lists); 2026-10-04-variables-pooled-diagnosis-tables.md and .json (16 armies). The 10-03 and the other 10-04 files are kept.

The short answer

• Yes, everything on the datasheet is in now. Every field of every unit record is either read by a variable or left out for a stated reason (the coverage table below). The list went from 144 to 341 variables: 197 added in five new families:
  • keywords (93): every unit keyword that varies within some army, Vehicle, Walker, Mounted, Aircraft, Transport and Dedicated Transport each on its own, Harvester and Endless Multitude among them;
  • size (17): the size range, every points step's ends, points per model at each end, the discount for size, and the different statlines inside a unit;
  • loadout (22): weapons and profiles on the datasheet and carried, the wargear choices and limits, the best and worst numbers of any weapon carried;
  • weaponkw (29): the weapon keywords the first list missed (Rapid Fire, Lance, Hazardous, One Shot, Close-Quarters, Cleave, Harpooned, Hive Defences and five rarer ones), the values of Sustained Hits, Melta, Rapid Fire, Blast and Cleave, and Anti-X as eight classes with their thresholds;
  • rules (36): the values BSData gives Deadly Demise, Scouts and Firing Deck, Firing Deck, Support and Super-Heavy Walker as rules, the army rules that some units of an army lack, counts of rules and abilities, the effects files' entries by status, who can lead or support the unit, transport capacity.
• What is still left out, and why:
  • names and identifiers (the unit's id, its link id, model names, weapon names, ability names);
  • free text (what an ability says; the effects files carry what it does, in our words);
  • faction and allegiance keywords (Faction, Chaos, Nurgle, Khorne, Tzeentch, Slaanesh, Aeldari, Great Devourer), because they're the same on every graded unit of each army, so they can't order two units of one army;
  • anything no graded unit has: the Legends and Crucible tags, pointsApprox, 13 keywords, two weapon keywords (Conversion and Reverberating Summons, never in a graded unit's best loadout), Anti-Daemon, a few army rules;
  • Lone Operative's distance (given on one datasheet in all 16 armies).
• Did the extra variables help? On Tyranids, no. With units held out (10 × 5 folds, the old five lists both sides), the best model went from 73.3% to 72.6% against the consensus. The ledger as it is stays at 72.9%. The trees went from 68.3% to 67.2%. The neighbours fell from 71.9% to 64.6%: 197 more columns to measure distance on drown the ones that matter. The plain linear model moved from 69.7% to 70.0%. Every change is inside the noise except the neighbours' fall. The step-24 headline lists give the same picture (section 3): the ledger 74.5% both ways, interactions 73.1% → 72.4%, trees 68.6% → 69.2%, neighbours 72.5% → 66.2%.
• Across armies, it looks like yes, but most of it is the Marine twins again.
  • Leaving one army out, the flexible models rise from 58–63% to 62–66% pooled. The interactions model goes from 59.1% to 66.3%; the ledger stays at 60.0%.
  • Most of that rise is the Space Marine family. The new keywords (Tacticus, Gravis, Phobos, Terminator, Captain…) let a model recognise the same datasheets in the four Marine armies. With all four Marine armies left out together, every flexible model scores 54–58% on them, against the ledger's 57%.
  • On the 12 other armies the gain is small: the army mean rises 1.5 to 3 points (interactions 59.5% → 62.1%, trees 60.9% → 62.3%, against the ledger's 59.1%); pooled by pairs, the trees go 64.1% → 63.7% and the ledger is 63.3%. Scrambled letters still give 48–50%.
  • Tyranids from the other armies get no better: 64.8% for the best flexible model, against the ledger's 73.3%.
• Which new variables matter:
  • Points at each end of the size range (ptsAtMax, ptsAtMin) and the size family are the steadiest new signal: ptsAtMax is the trees' second-biggest split over all 16 armies and drops their score most of any variable (1.6 points, 11 armies). Cheaper units are rated higher at the same Value (positive in 9 or 10 armies alone, negative in 1).
  • How many weapons a unit carries (carriedTotal, dsRanged, defaultWeapons) and the best OC of any model (OC_max) come next.
  • On Tyranids: modelProfiles (a unit with a leader model), OC_max, scouts_inches, anti_infantry and transportCapacity are the new ones the interactions model uses most, each under half a point. In the trees, ppmDiscount (points per model at full size against minimum) is the one new variable with a clear effect (0.7).
  • Keyword flags, weapon keywords and rule values add little on their own: their families drop the score by 0 to 2 points, mostly through the Marine keywords.
• The units every model misses are the same as before.
  • Tyranids: Biovores, Tyrannofex, Exocrine, Lictor, Neurolictor, Hormagaunts rated higher than any model gives; Toxicrene, Hierophant, Tervigon, Ranged Warriors, Deathleaper rated lower. Neurogaunts join the second list.
  • Across armies: leaders, psykers and transports again (the Miasmic Malignifier, Librarian in Phobos Armour, the Venom, the Stormraven, the Chaos Rhino, the Navigator), and fast melee units rated lower (Pteraxii Skystalkers, Tzaangor Enlightened, Serberys Sulphurhounds, Maulerfiend).
  • So the datasheet isn't missing a field. What the reviewers see in these units isn't on any datasheet: what a leader or a transport does for another unit, and how a unit plays.

1. The coverage table

Every field of a unit record in build/units.json (docs/bsdata.js makes them), over the 1,076 datasheets of the 16 armies and the 618 graded units. "Graded" is a unit some list of its army grades. A variable that is the same on every graded unit of every army can't order two units of one army, so it is left out with that reason.

The record's fields

Weapon profile fields and keywords

Weapon variables read the weapons the unit's best form carries (its loadout in the ledger), as T-632 did; the datasheet's other options count through dsWeapons, dsProfiles, dsRanged, dsMelee, dsMultiProfile and the choices.

Rules on the datasheet

Keywords

2. Tyranids, the old five lists (before and after on the same code)

The T-632 setting: the consensus over auspex, hivemind, second, maelstrom and astrategas (step-24's --summaries), 10 repeats × 5 folds, units held out, pairs within a fold. "Before" is --base-vars on this code; it gives T-632's numbers exactly (the ledger 72.9%, interactions 73.3%).

The SE of a held-out figure is 0.4 to 1.4 points; the scrambled spread of a paired gain is 7 to 11. The ceiling (each list against the other four) is 88.5%. Interactions against the ledger: −0.4 ± 0.4 (it was +0.4). In sample, the linear model now fits 99% of pairs: more columns, more memorising, no more held out.

Permutation importance, held out, the interactions model (c): all 197 additions shuffled together cost 2.1 ± 0.5 points; by family: rules 1.0, keywords 0.4, size 0.4, weapon keywords 0.0, loadout −0.5. The T-632 families still lead: rule flags 4.4, weapons 3.0, the ledger 2.7. Top new variables: modelProfiles 0.42, OC_max 0.38, scouts_inches 0.32, anti_infantry 0.28, fx_partly 0.27, kwu_Aircraft 0.23, transportCapacity 0.21.

The trees (d): the additions together −0.3 (none helps on balance); ppmDiscount 0.66 is the one new variable among their top five, ptsAtMax 0.22 and abilitiesOwn 0.17 after it.

Missed by every model the same way (held out, mean predicted letter minus consensus): Biovores −3.2, Toxicrene +1.9, Tyrannofex −1.6, Tervigon +1.6, Hierophant +1.6, Lictor −1.4, Exocrine −1.4, Ranged Warriors +1.3, Neurolictor −1.3, Deathleaper +1.2, Neurogaunts +1.1, Hormagaunts −1.1. The same handful as T-632 (Tyrannocyte drops just off the list, Neurogaunts comes on).

3. Tyranids, the step-24 headline lists

Since step 24 (T-640) the default Tyranid lists are the fuller transcripts: auspexFull, hivemindFull, secondFull, maelstrom and astrategas. Run after rebasing onto it, the same way, both sides on these lists (ceiling 90.4%):

The same picture: every move is inside the noise (SE 0.6 to 1.7) except the neighbours' fall, and the ledger stays the best held-out model. Importance for the interactions model: all additions together 1.9 ± 0.6; top new ones rulesWeaponLike 0.31, bestWS 0.21, lfx_rules 0.18, scouts_inches 0.18, anti_infantry 0.17, transportCapacity 0.17. The trees: the additions together −0.7; abilitiesOwn 0.34 the one clear one. Missed by every model: Biovores −3.2, Neurolictor −1.8, Tervigon +1.7, Toxicrene +1.6, Tyrannofex −1.6, Ranged Warriors +1.5, Hierophant +1.4, Exocrine −1.3, Lictor −1.3, Parasite of Mortrex +1.2, Hormagaunts −1.2, Deathleaper +1.2.

4. All 16 armies (T-635's pooled run, before and after)

T-635's setting exactly (Tyranids on the old five lists, as T-635 had them; the pooled run wasn't repeated after step 24). "Before" is --pooled --base-vars on this code and gives T-635's numbers exactly.

Taking the Marine twins out of the reading:

So the headline rise (interactions +7.2 points pooled) is mostly the four Marine armies recognising each other's datasheets through the new keywords; with them scored fairly, no model passes the ledger's 60.0%. On the other 12 armies the flexible models gain 1.4 to 3 points of army mean, the largest in Thousand Sons (interactions 77% against the ledger's 46%), Drukhari and the Agents, and lose in the Adeptus Mechanicus and Genestealer Cults. Against (a1): interactions +4.9 ± 2.9 per army (10 of 16 better), trees +4.0 ± 2.2 (10 of 16); the scrambled spread of that gain is 0.7 to 1.8, but the SE over armies (2 to 3) is the honest yardstick, and the Marine family carries much of it.

Which new variables matter across armies (importance in the left-out army, drop in pooled pairwise, points; armies where it drops by more than 0.5):

• All 197 additions together: 8.7 (interactions, 13 armies) and 7.3 (trees, 13 armies); of the families, size 1.7 and 3.6, keywords 2.3 and −0.1, loadout 1.0 and 2.6, rules 1.6 and 0.3, weapon keywords 0.5 and −0.4.
• One at a time, the trees: ptsAtMax 1.6 (11 armies), ptsAtMin 0.6 (10), dsRanged 0.6 (10), shortRanged 0.5, sizeRatio 0.5, carriedTotal 0.3 (9). The interactions model: kwu_Captain 0.5, ptsAtMax 0.4, ptsAtMin 0.4, kwu_Centurion 0.4, carriedTotal 0.3, OC_max 0.3; Marine keywords lead it.
• Alone, within army (share of consensus pairs ordered the same way; armies positive / negative): carriedTotal 58.5% (8/4), ptsAtMax 57.7% (9/1), OC_max 56.0% (8/4), ptsAtMin 55.7% (10/1), ledByCount 54.6% (6/3), bestWS 53.6% (10/2). For scale, the ledger's Kill line alone is 60%.
• The trees on all 16 armies split on c_kill most (7.4% of the gain), then ptsAtMax (6.0%), melee Kill, OC_max, Score and choiceEntries.
• T-632's candidates with their new variables: scarce answers still first (1.3 in both models), transports now 0.9 for the interactions (9 armies; with capacity and the two transport keywords), arrival 1.0 (8 armies) for the interactions and nothing for the trees.

Missed by every model the same way (leaving the army out): rated higher than any model gives, the Miasmic Malignifier, Suppressor Squad, the Stormraven, the Venom, Biovores, Librarian in Phobos Armour, Skitarii Marshal, the Chaos Rhino, the Navigator; rated lower, Pteraxii Skystalkers, Company Heroes, Tzaangor Enlightened, Heavy Intercessors, Serberys Sulphurhounds, an Iron Priest, the Maulerfiend, the Primaris Psyker, the Hierophant. The under-rated 30 are low on Kill, Value and points (cheap utility: leaders, transports, support); the over-rated 30 are psykers and hard hitters. T-635's pattern, unchanged.

5. Method notes

• Old columns untouched. The first 144 columns keep their keys, order and values; --base-vars reproduces T-632's and T-635's tables exactly. The new ones follow, in five new families so family importance tells them apart, plus a pseudo-family "all T-636 additions". A unit test checks the first 144 keys and order.
• Which loadout. Weapon variables read the weapons of the unit's best solo form (the ledger's ranking row), as T-632 did; the datasheet's full weapon list counts through ds* and the choices.
• Rule values come from BSData, not the record. build/units.json keeps rule names only. tools/rulevals.js reads the values BSData appends to Deadly Demise, Scouts and Firing Deck from the checkout at the frozen commit (374f505; it refuses another commit and checks build/units.json against the lock), and writes tools/rulevals.json: numbers per unit, nothing else. Every such rule on the 16 armies got a value. Transport capacity is T-629's tools/carry.json.
• Which keywords. A unit keyword is in when it is on 2+ datasheets of the 16 armies and varies among some army's graded units, and isn't a faction keyword or a datasheet's or model's own name (a name shared by 5+ datasheets counts as a role and is in). The lists are fixed in the code, so the columns don't move with the data.
• A fix in the old code path: a column constant within a training fold, read through log(1 + x), had an SD of rounding noise (10⁻¹⁶) instead of 0, so a held-out unit's standardised value came out near 10¹⁴ and the kernel model's predictions blew up in one repeat. The transformer now treats an SD under 10⁻¹² as 0 (the pooled path already did). It changes nothing in the 144-column runs (they reproduce exactly).
• Mono signs are +1 only where more is plainly better (Anti-X quality, Sustained/Melta/Rapid Fire values, Scouts and Firing Deck, ledByCount and the like); keywords, counts and points are free (0).
• Doubts. The pooled gain rests on the Marine family; a pooled run with the four Marine armies always held out together would settle it. 197 columns for 52 Tyranids or 618 units is a lot of room to memorise; the ridge and the trees' column sampling hold it, the neighbours don't.