T-635 tables (node tools/diagnose.js --pooled)

From reports/tiers/2026-10-04-pooled-diagnosis-tables.md , rendered when the site is built.

16 armies, 618 graded units, 144 variables; units held out 5 × 5 folds; 2 scrambles.

The armies:

Army	Lists	Units	Graded	Ceiling	Ledger in sample
Tyranids	auspex, hivemind, second, maelstrom, astrategas	52	52	88.5%	73.3%
Space Marines	auspexFull, tobias	84	84	91.0%	60.4%
Adeptus Mechanicus	belisariusPaul	34	32	–	68.0%
Agents of the Imperium	varik	29	26	–	44.8%
Astra Militarum	auspex	72	61	–	66.0%
Blood Angels	grimdarkChronicle	20	19	–	54.0%
Dark Angels	tacticalrock	93	93	–	54.8%
Death Guard	disgustinglyResilient	36	36	–	47.1%
Drukhari	deployOnTheLine	37	24	–	46.1%
Emperor's Children	joys	23	23	–	56.5%
Genestealer Cults	rey, vonCarmian	90	24	84.9%	68.8%
Leagues of Votann	list2025, discussion	22	20	59.3%	61.0%
Orks	spruesandbrews	55	45	–	70.0%
Space Wolves	artOfWar	21	21	–	46.9%
Thousand Sons	redundantRolling	35	28	–	45.6%
World Eaters	redPath, tacticalSugar, exalted	30	30	89.3%	61.5%

The headline (pairwise against each army's consensus; pooled: every within-army pair once; army mean: the 16 armies equal):

Model	Leave one army out: pooled	army mean	Tyranids	Space Marines	vs lists (pooled)	Units held out: pooled (± SE)	Tyranids	Scrambled, left out: pooled (SD)
(a) the ledger as it is	60.0%	57.8%	73.3%	60.4%	62.3%	60.3% ± 0.2	73.2%	49.1% (0.2)
(a1) the ledger re-fitted	59.9%	57.8%	73.2%	60.4%	62.2%	60.1% ± 0.2	72.9%	49.4% (0.2)
(b) linear, every variable	59.1%	59.8%	64.8%	62.1%	58.8%	62.2% ± 0.5	67.8%	50.9% (3.4)
(b2) linear, core 25	58.6%	57.7%	69.5%	58.6%	59.4%	60.9% ± 0.4	68.7%	48.1% (0.8)
(c) interactions (kernel)	59.1%	59.7%	57.7%	63.7%	58.2%	67.4% ± 0.6	68.0%	50.1% (1.8)
(d) boosted trees	61.8%	60.0%	62.0%	62.3%	60.4%	65.9% ± 0.7	66.9%	49.3% (0.0)
(e) neighbours	63.4%	56.3%	57.9%	69.6%	61.0%	65.4% ± 0.5	62.9%	49.9% (2.6)

Ceiling (each list against the mean of the army's other lists; 5 armies with 2+ lists): Tyranids 88.5%, Space Marines 91.0%, Genestealer Cults 84.9%, Leagues of Votann 59.3%, World Eaters 89.3%.

Leave one army out, per army (pairwise against the consensus):

Army	Graded	Ceiling	a	a1	b	b2	c	d	e
Tyranids	52	88.5%	73.3%	73.2%	64.8%	69.5%	57.7%	62.0%	57.9%
Space Marines	84	91.0%	60.4%	60.4%	62.1%	58.6%	63.7%	62.3%	69.6%
Adeptus Mechanicus	32	–	68.0%	68.0%	52.6%	50.6%	48.9%	65.5%	46.7%
Agents of the Imperium	26	–	44.8%	44.8%	52.9%	47.5%	59.6%	53.4%	44.4%
Astra Militarum	61	–	66.0%	66.0%	64.1%	64.6%	66.3%	74.9%	66.0%
Blood Angels	19	–	54.0%	54.0%	71.8%	53.2%	67.7%	54.0%	43.5%
Dark Angels	93	–	54.8%	54.8%	52.9%	54.0%	53.4%	58.2%	68.5%
Death Guard	36	–	47.1%	47.1%	51.5%	51.3%	52.3%	55.9%	50.2%
Drukhari	24	–	46.1%	46.1%	62.6%	55.2%	70.9%	65.7%	60.9%
Emperor's Children	23	–	56.5%	56.5%	61.3%	56.5%	59.7%	52.4%	57.6%
Genestealer Cults	24	84.9%	68.8%	68.8%	50.8%	59.2%	55.0%	51.2%	57.5%
Leagues of Votann	20	59.3%	61.0%	61.0%	62.3%	57.1%	59.7%	63.6%	47.4%
Orks	45	–	70.0%	70.0%	64.9%	70.5%	63.2%	69.1%	61.4%
Space Wolves	21	–	46.9%	46.9%	60.6%	66.9%	56.3%	54.4%	58.1%
Thousand Sons	28	–	45.6%	45.6%	61.2%	53.1%	58.6%	54.0%	55.3%
World Eaters	30	89.3%	61.5%	61.5%	61.3%	54.8%	62.0%	63.0%	55.1%

The Space Marine family left out together (each of the four scored with all four out of the training; the main run's figure in brackets):

Army	a	a1	b	b2	c	d	e
Space Marines	60.4% (60.4%)	60.4% (60.4%)	55.7% (62.1%)	51.9% (58.6%)	56.6% (63.7%)	56.8% (62.3%)	55.5% (69.6%)
Blood Angels	54.0% (54.0%)	54.0% (54.0%)	59.7% (71.8%)	58.1% (53.2%)	62.9% (67.7%)	50.0% (54.0%)	58.1% (43.5%)
Dark Angels	54.8% (54.8%)	54.8% (54.8%)	46.3% (52.9%)	47.1% (54.0%)	47.1% (53.4%)	50.4% (58.2%)	52.4% (68.5%)
Space Wolves	46.9% (46.9%)	46.9% (46.9%)	62.5% (60.6%)	68.8% (66.9%)	56.3% (56.3%)	48.1% (54.4%)	56.3% (58.1%)
pooled over the four	57.1%	57.1%	51.1%	49.9%	51.8%	53.2%	54.0%

Units held out within armies, per army:

Army	a	a1	b	b2	c	d	e
Tyranids	73.2%	72.9%	67.8%	68.7%	68.0%	66.9%	62.9%
Space Marines	61.1%	60.9%	61.5%	58.3%	69.6%	65.9%	70.4%
Adeptus Mechanicus	70.1%	69.8%	52.3%	56.3%	52.1%	64.7%	55.1%
Agents of the Imperium	43.3%	43.4%	51.1%	51.9%	58.9%	44.2%	47.5%
Astra Militarum	66.5%	66.4%	71.5%	74.8%	78.9%	77.5%	74.9%
Blood Angels	50.3%	50.3%	68.8%	68.6%	66.1%	63.4%	65.5%
Dark Angels	54.9%	54.8%	59.0%	56.7%	66.1%	65.2%	67.0%
Death Guard	46.8%	46.6%	52.8%	52.2%	58.0%	57.5%	54.5%
Drukhari	49.1%	49.1%	63.1%	49.7%	70.8%	69.2%	58.1%
Emperor's Children	53.1%	52.3%	60.5%	54.4%	66.5%	56.1%	58.3%
Genestealer Cults	69.7%	69.7%	62.2%	66.2%	58.5%	56.4%	56.2%
Leagues of Votann	65.4%	65.4%	61.3%	56.6%	63.6%	57.8%	52.0%
Orks	67.9%	67.7%	68.0%	69.9%	66.1%	69.4%	57.6%
Space Wolves	49.8%	49.8%	65.5%	67.8%	61.8%	51.7%	40.1%
Thousand Sons	46.5%	46.5%	62.8%	53.8%	68.6%	61.3%	60.3%
World Eaters	61.9%	62.2%	63.3%	60.3%	59.0%	66.5%	58.3%

Paired, leave one army out (per-army gain in points; SE over the 16 armies; armies better; pooled gain; scrambled spread of the army-mean gain against (a1)):

Pair	Mean gain ± SE	Armies better	Pooled gain	Scrambled SD
b − a1	+2.0 ± 2.7	9 of 16	−0.9	0.3
b2 − a1	−0.1 ± 2.1	6 of 16	−1.4	2.6
c − a1	+1.9 ± 3.0	10 of 16	−0.9	0.6
d − a1	+2.2 ± 2.2	10 of 16	+1.9	0.9
e − a1	−1.5 ± 2.8	7 of 16	+3.4	3.1
a1 − a	−0.0 ± 0.0	0 of 16	−0.0	–
b − a	+2.0 ± 2.7	9 of 16	−0.9	–
b2 − a	−0.1 ± 2.1	6 of 16	−1.4	–
c − a	+1.9 ± 3.0	10 of 16	−0.9	–
d − a	+2.2 ± 2.2	10 of 16	+1.9	–
e − a	−1.5 ± 2.8	7 of 16	+3.4	–

Settings chosen, leave one army out (most often):

a: {} ×16
a1: {"lam":10} ×13, {"lam":0.03} ×1, {"lam":0.01} ×1, {"lam":0.1} ×1
b: {"kind":"ridge","lam":0.3} ×9, {"kind":"ridge","lam":1} ×3, {"kind":"lasso","lam":0.01} ×1, {"kind":"ridge","lam":3} ×1, {"kind":"lasso","lam":0.02} ×1
b2: {"kind":"ridge","lam":3} ×8, {"kind":"ridge","lam":0.003} ×4, {"kind":"ridge","lam":1} ×2, {"kind":"ridge","lam":0.3} ×2
c: {"lam":0.0003} ×11, {"lam":0.03} ×2, {"lam":1} ×1, {"lam":0.003} ×1, {"lam":0.01} ×1
d: {"depth":3,"colsample":1,"trees":400} ×9, {"depth":3,"colsample":1,"trees":25} ×3, {"depth":2,"colsample":1,"trees":25} ×3, {"depth":3,"colsample":1,"trees":200} ×1
e: {"k":20} ×5, {"k":8} ×5, {"k":5} ×3, {"k":30} ×2, {"k":12} ×1

Does pooling help Tyranids? Held-out pairwise on Tyranid units against their consensus:

Model	Tyranids alone (their own folds)	From the other 15 armies only	The other 15 plus their own folds
(a) the ledger as it is	73.2%	73.3%	73.2%
(a1) the ledger re-fitted	71.5%	73.2%	72.9%
(b) linear, every variable	67.2%	64.8%	67.8%
(b2) linear, core 25	66.9%	69.5%	68.7%
(c) interactions (kernel)	65.4%	57.7%	68.0%
(d) boosted trees	66.0%	62.0%	66.9%
(e) neighbours	61.6%	57.9%	62.9%

The learning curve (trained on m random armies at the model's most chosen setting, scored on the army left out; pooled / army mean):

Model	3	6	9	12	15
e	57.1% / 54.1%	59.8% / 55.9%	56.5% / 55.3%	65.5% / 58.2%	63.4% / 56.3%
a1	59.9% / 57.8%	59.9% / 57.8%	59.9% / 57.8%	59.9% / 57.8%	59.9% / 57.8%
b2	52.4% / 53.3%	59.7% / 58.3%	56.6% / 57.3%	58.0% / 57.1%	58.6% / 57.7%
d	59.5% / 58.1%	61.4% / 58.8%	55.1% / 56.5%	65.5% / 59.4%	61.8% / 60.0%
a	60.0% / 57.8%	60.0% / 57.8%	60.0% / 57.8%	60.0% / 57.8%	60.0% / 57.8%

Permutation importance, model e (left-out army, its fit on the other 15; drop in pooled pairwise, points; armies where it drops by more than 0.5):

Family	Drop	Armies
flags	5.05	10
weapons	3.46	9
datasheet	2.44	11
ledger	2.13	10
killclass	0.61	7
interaction	0.55	7

T-632 candidate	Drop	Armies
1. Threat on arrival	0.53	5
2. Scarce answers (anti-monster and anti-vehicle reach)	2.15	10
3. Objective bodies (Score, OC per point)	0.52	8
4. Character sniping (Precision)	0.15	6
5. Transports	0.22	6
6. Psychic attacks and psykers	-0.14	4
7. Cheap reach: actions and indirect fire	0.44	10
8. Value at the cheapest size	0.97	9
9. Melee Soak	0.08	6

Variable	Family	Drop	Armies
kw_melta	weapons	0.97	6
lr_overwatch	flags	0.83	4
fx_overwatch	flags	0.81	4
has_scouts	flags	0.57	6
OC	datasheet	0.55	6
rangedVeh100	weapons	0.49	4
modelsCheap	datasheet	0.46	6
fx_aura	flags	0.46	7
kw_is_character	flags	0.46	7
pts	datasheet	0.44	11
c_soak	ledger	0.42	7
line_killMelee	ledger	0.41	5
kw_assault	weapons	0.41	5
bestM_D	weapons	0.40	8
ix_WxSave	interaction	0.39	9
lr_buff	flags	0.37	5
models	datasheet	0.37	9
bestR_AP	weapons	0.36	10
damagedBracket	datasheet	0.36	3
ptsCheap	datasheet	0.34	8

Permutation importance, model d (left-out army, its fit on the other 15; drop in pooled pairwise, points; armies where it drops by more than 0.5):

Family	Drop	Armies
ledger	4.79	13
weapons	1.67	8
killclass	1.35	10
datasheet	1.29	10
flags	0.72	13
interaction	0.28	8

T-632 candidate	Drop	Armies
1. Threat on arrival	0.54	7
2. Scarce answers (anti-monster and anti-vehicle reach)	2.02	7
3. Objective bodies (Score, OC per point)	0.81	8
4. Character sniping (Precision)	-0.02	3
5. Transports	-0.20	0
6. Psychic attacks and psykers	0.06	2
7. Cheap reach: actions and indirect fire	0.17	6
8. Value at the cheapest size	-0.18	2
9. Melee Soak	-0.02	0

Variable	Family	Drop	Armies
ptsPerModel	datasheet	2.01	11
c_kill	ledger	1.05	13
c_score	ledger	0.61	8
meleeChaff100	weapons	0.59	7
k6_monster	killclass	0.58	7
has_loneOperative	flags	0.49	6
lands	ledger	0.47	7
line_killMelee	ledger	0.44	5
k6_light_vehicle	killclass	0.43	5
kw_devastating	weapons	0.40	8
fx_cp	flags	0.31	4
c_actions	ledger	0.30	8
multTotal	ledger	0.28	1
kw_blast	weapons	0.28	4
k6_heavy_vehicle	killclass	0.26	6
ix_arrivexMelee	interaction	0.25	4
value	ledger	0.24	3
kw_anti	weapons	0.24	5
kw_is_titanic	flags	0.21	1
kw_assault	weapons	0.17	2

Each variable alone, within army (share of within-army consensus pairs it orders the same way, ties half; armies with Spearman above +0.1 / below −0.1):

Variable	Family	Concordance	Armies +	Armies −
c_kill	ledger	60.1%	11	0
value	ledger	60.0%	10	2
valueCheap	ledger	59.6%	10	1
k6_heavy_infantry	killclass	58.8%	10	0
k6_cavalry_and_beasts	killclass	58.6%	11	1
ix_killxSoak	interaction	58.1%	8	4
c_hold	ledger	57.1%	8	3
lands	ledger	56.7%	8	4
ix_bodiesxOC	interaction	56.7%	9	5
pts	datasheet	56.5%	10	2
M	datasheet	56.4%	7	4
k6_light_vehicle	killclass	56.3%	8	2
rangedMeq100	weapons	56.1%	7	7
OC	datasheet	56.0%	8	4
ptsCheap	datasheet	55.7%	10	1
rangedVeh100	weapons	55.5%	7	6
rangedChaff100	weapons	55.3%	5	7
line_killRanged	ledger	54.9%	7	7
k6_character	killclass	54.6%	5	3
k6_elite_infantry	killclass	54.5%	6	2
meleeChaff100	weapons	54.5%	7	1
rangedAttacks100	weapons	54.4%	5	8
c_score	ledger	54.3%	8	4
ix_arrivexMelee	interaction	54.3%	9	2
k6_monster	killclass	54.3%	6	3
modelsCheap	datasheet	54.2%	6	3
models	datasheet	54.1%	6	3
c_soak	ledger	54.1%	7	3
arrives	flags	54.1%	9	2
bestM_skill	weapons	53.9%	11	2

T-632's candidates, their variables alone:

1. Threat on arrival: arrives 54.1% (+9/−2); has_deepStrike 52.6% (+7/−3); has_scouts 50.3% (+3/−2); has_infiltrators 51.5% (+6/−2); fx_arrival 50.9% (+6/−2); kw_is_burrowers 50.4% (+1/−0); kw_is_vanguard 50.4% (+1/−0); ix_arrivexMelee 54.3% (+9/−2)
2. Scarce answers (anti-monster and anti-vehicle reach): k6_monster 54.3% (+6/−3); k6_heavy_vehicle 52.3% (+7/−4); k6_light_vehicle 56.3% (+8/−2); rangedVeh100 55.5% (+7/−6); meleeVeh100 52.3% (+8/−2); kw_heavy 51.8% (+5/−3); kw_melta 51.9% (+6/−2); kw_anti 48.5% (+3/−7); scarcity 53.1% (+8/−1)
3. Objective bodies (Score, OC per point): c_score 54.3% (+8/−4); ocPer100 53.7% (+8/−5); OC 56.0% (+8/−4); ix_bodiesxOC 56.7% (+9/−5); kw_is_battleline 51.0% (+8/−4)
4. Character sniping (Precision): kw_precision 49.7% (+7/−4); k6_character 54.6% (+5/−3)
5. Transports: kw_is_transport 48.8% (+1/−6)
6. Psychic attacks and psykers: kw_psychic 51.8% (+6/−3); kw_is_psyker 52.1% (+7/−3)
7. Cheap reach: actions and indirect fire: kw_indirect 49.2% (+2/−3); c_actions 47.2% (+4/−7); ptsCheap 55.7% (+10/−1)
8. Value at the cheapest size: valueCheap 59.6% (+10/−1); ptsCheap 55.7% (+10/−1); modelsCheap 54.2% (+6/−3)
9. Melee Soak: soakFight 49.0% (+2/−8)

The trees on all 16 armies (depth 3, 100 trees; share of split gain; median threshold in SDs within the army):

c_kill: 8.5% of the gain, threshold ~-0.02 SD, 7 splits
ptsPerModel: 7.7% of the gain, threshold ~0.63 SD, 27 splits
c_score: 7.6% of the gain, threshold ~-1.20 SD, 9 splits
line_killMelee: 5.6% of the gain, threshold ~-1.44 SD, 6 splits
kw_is_fly: 3.6% of the gain, threshold ~2.30 SD, 23 splits
kw_is_infantry: 3.2% of the gain, threshold ~0.78 SD, 17 splits
k6_monster: 2.7% of the gain, threshold ~0.56 SD, 12 splits
meleeChaff100: 2.6% of the gain, threshold ~1.27 SD, 9 splits
k6_chaff: 2.5% of the gain, threshold ~-0.01 SD, 13 splits
kw_pistol: 2.2% of the gain, threshold ~1.27 SD, 21 splits
ix_arrivexMelee: 2.1% of the gain, threshold ~-0.68 SD, 12 splits
kw_devastating: 2.0% of the gain, threshold ~1.20 SD, 12 splits
rangedVeh100: 1.7% of the gain, threshold ~0.80 SD, 11 splits
c_actions: 1.7% of the gain, threshold ~-1.52 SD, 13 splits
has_loneOperative: 1.7% of the gain, threshold ~4.53 SD, 11 splits

Interactions (one split under the other, share of gain):

c_kill × c_score: 4.2%
c_kill × ptsPerModel: 3.7%
c_kill × line_killMelee: 2.0%
c_score × k6_chaff: 1.2%
ix_arrivexMelee × kw_is_fly: 1.1%
meleeChaff100 × ptsPerModel: 1.1%
kw_is_fly × ptsPerModel: 1.0%
c_soak × kw_is_infantry: 0.9%
k6_heavy_infantry × kw_blast: 0.9%
kw_is_epicHero × ptsPerModel: 0.9%
kw_pistol × lfx_effects: 0.8%
kw_is_fly × pts: 0.8%

Partial dependence (letters, consensus centred within army; biggest step in SDs within the army):

c_kill: range 0.32; biggest step +0.30 between -0.05 and 0.08
ptsPerModel: range 0.62; biggest step +0.23 between 0.34 and 0.65
c_score: range 0.60; biggest step +0.46 between -3.38 and -0.72
line_killMelee: range 0.47; biggest step +0.47 between -3.72 and -1.11
kw_is_fly: range 0.44; biggest step -0.44 between 2.18 and 2.46
kw_is_infantry: range 0.58; biggest step +0.43 between 1.11 and 1.39
k6_monster: range 0.23; biggest step +0.10 between -0.56 and -0.23
meleeChaff100: range 0.39; biggest step +0.38 between 1.27 and 2.88
k6_chaff: range 0.26; biggest step +0.16 between -0.21 and 0.08
kw_pistol: range 0.50; biggest step -0.29 between 1.83 and 2.15

Missed by every flexible model the same way, leaving the army out (mean predicted letter minus consensus; the ledger's beside it):

Rated higher by the reviewers than any model gives:

Unit	Army	Consensus (letters)	Models	Ledger
Stormraven Gunship	Dark Angels	4 (A)	-4	-4
Miasmic Malignifier	Death Guard	5 (S)	-4	-4
Biovores	Tyranids	5 (S/S/S/S/S)	-3.907	-0.6
Suppressor Squad	Dark Angels	5 (S)	-3.8	-3
Librarian in Phobos Armour	Dark Angels	5 (S)	-3.6	-4
Librarian	Dark Angels	5 (S)	-3.6	-5
Sorcerer	Thousand Sons	5 (S)	-3.6	-4
Jakhals	World Eaters	5 (S/S/S)	-3.533	-2.333
Venom	Drukhari	5 (S)	-3.4	-5
Infernal Master	Thousand Sons	5 (S)	-3.4	-4

Rated lower by the reviewers than every model gives:

Unit	Army	Consensus (letters)	Models	Ledger
Centurion Assault Squad	Dark Angels	0 (F)	+3.6	+4
Tzaangor Enlightened	Thousand Sons	1 (D)	+3.4	+4
Serberys Sulphurhounds	Adeptus Mechanicus	0 (F)	+3	+4
Company Heroes	Dark Angels	0 (F)	+3	+3
Lazarus	Dark Angels	0 (F)	+3	+3
Deathwing Terminator Squad	Dark Angels	1 (D)	+3	0
Maulerfiend	Thousand Sons	2 (C)	+3	+3
Heavy Intercessor Squad	Dark Angels	1 (D)	+2.8	+3
Grey Knights Terminator Squad	Agents of the Imperium	2 (C)	+2.6	+3
Primaris Psyker	Astra Militarum	2 (C)	+2.6	+1

Missed the same way, per army (higher / lower / graded): Tyranids 5/5/52, Space Marines 19/16/84, Adeptus Mechanicus 5/6/32, Agents of the Imperium 4/2/26, Astra Militarum 6/10/61, Blood Angels 1/3/19, Dark Angels 12/15/93, Death Guard 7/6/36, Drukhari 2/3/24, Emperor's Children 5/3/23, Genestealer Cults 5/6/24, Leagues of Votann 3/2/20, Orks 6/4/45, Space Wolves 1/3/21, Thousand Sons 5/6/28, World Eaters 6/8/30

Traits of the 30 most under-rated (mean SD within army): c_kill -0.88, value -0.83, valueCheap -0.81, ix_killxSoak -0.73, ix_durHeavy100 -0.65, scarcity -0.63, k6_heavy_infantry -0.62, c_hold -0.62, pts -0.59, OC -0.57

Traits of the 30 most over-rated: meleeVeh100 0.55, bestM_D 0.55, value 0.49, valueCheap 0.46, c_kill 0.44, line_killMelee 0.42, kw_psychic 0.42, k6_heavy_infantry 0.40, kw_is_psyker 0.40, meleeMeq100 0.39

Run: 1057 s.