The yardstick exam: Gemini at half weight, held out, and the "Gemini weights"

6 Oct, overnight, Pacific (T-731, T-734). tools/yardstick.js, tools/yardexam.js; numbers in build/yardexam/ (git-ignored). Snapshot 8dcd24032d, inputs hash dcd477d8a78c (verified; nothing re-frozen, nothing rebuilt). Discovery only: nothing in the ledger moves and nothing is adopted.

In plain English

1. Gemini at half weight (T-731). The 15 GEMINI AGGREGATE lists for ledger armies (Emperor's Children has none) are read at scoring time and matched to the ledger's units by name, behind a flag. Nothing was re-frozen: the ledger never reads the panels, so a re-freeze would only have moved the inputs hash. Two ways of scoring, both kept:

• The consensus score (cascade2 and everydial, LEDGER_YARDSTICK=half). A pair the human lists agree on counts 1, whatever Gemini says. A pair one human list orders and Gemini orders the same way also counts 1. A pair only Gemini orders counts 0.5. Most armies have one human list, so Gemini only adds pairs among units the human list didn't grade. The numbers barely move: all 16 armies, Track A 61.9 → 61.7, C1 68.3 → 68.3, C5 69.2 → 69.2, #3 69.1 → 69.2. The scrambled letters stay near 49 to 52. The Tyranids lose about 2 points everywhere (Gemini adds 75 two-grader pairs there).
• The exam's own score (the mean of each list's pairwise accuracy, tools/exam.js --gemini 0.5). Here Gemini is one more list at half weight, so its disagreements count. Every setting loses 1 to 2 points: the exam's 15, Track A 59.3 → 58.4, C1 65.8 → 64.1, C5 66.7 → 64.9, #3 66.5 → 64.9. Blood Angels drop 10 points and Astra Militarum 4 to 7 (Gemini disagrees with both the human list and us). Space Wolves gain 5 and Dark Angels 2.
• C5's lead over Track A survives on both: +7.3 → +7.5 on the consensus score, +7.4 → +6.5 on the exam's.

2. Held out (T-734). The question was whether C5 and #3 beat Track A on armies they weren't chosen on. On each fold the best of Track A, C1, C5 and #3 is picked on 10 armies and scored on the other 5.

• The pick beats Track A by +6.6 held out ([+3.5, +9.8]; 13 of 15 armies up). On the half-weight yardstick it is +6.8. The pick is C5 in 6 folds and #3 in 3. Leaving one army out gives +5.8.
• But it adds nothing over C1: +0.0 ± 0.8 (9 armies up, 6 down; 8 and 7 on half weight). The whole held-out gain is C1's. Choosing arrivalKill and the rest on top of C1 doesn't carry to unseen armies.
• C1 itself is only partly held out. Its game-wide dials were chosen by cascade2's clean chain on 6 armies (Tyranids, World Eaters, Orks, Emperor's Children, Astra Militarum, Adeptus Mechanicus). On the other 10, C1's game-wide part alone is +2.7 over Track A, 9 of 10 up (+2.8 on half weight). That is the cleanest out-of-sample win in this work. The rest of C1's gain (+7.6 with each army's own values) comes from army-specific values chosen on each army's own letters, so it is in sample. They were gated against scrambled letters, though.

3. The "Gemini weights" (T-734): is the ledger too damage-heavy? Today Kill is 89% of the targeted lines' total, Soak 3% and scoring 8% (mean over 16 armies of each line's sum ÷ the Kill's). The line weights were solved, with wKill fixed, so the shares hit three targets: (a) 30 / 25 / 20 / 15, with supportMeasured on (its strength solved) and leaderCredit 0.25; (b) 50 / 25 / 25; (c) 70 / 15 / 15. The hits came within 3 points; (a) came out at 33 / 28 / 23 / 16.

• Every target loses to its base. On Track A: (c) −0.0, (b) −0.6, (a) −1.9 on the exam's 15.
• On C5 the losses are larger: (c) −1.8, (b) −3.5, (a) −4.9 against C5.
• The same on the half-weight yardstick. The held-out picks never choose a weights setting.
• (a) helps a few armies: Astra Militarum +3.6, Space Wolves +5.7, Thousand Sons +4.2, World Eaters +4.1. It costs Adeptus Mechanicus −11, Blood Angels −15 and Dark Angels −8.5.
• So moving value from Kill to Soak and scoring by weight doesn't help. The damage-heaviness isn't a weighting problem. If Soak and scoring matter, they need better measurements, not bigger weights. The same lesson as T-662's ceiling scan.

For Jordan to decide:

• (1) Which Gemini score is the yardstick? The consensus rule as written makes Gemini nearly invisible on single-list armies; the exam's list mean lets it disagree. I'd keep the exam's mean as the headline at half weight, with the consensus score beside.
• (2) The held-out result points at C1's game-wide part, not C5 or #3, as the setting that transfers. A held-out re-run of cascade2's game-wide choice by folds would settle C1's size properly. It costs a cascade2 analysis per fold, minutes from the stored records, with no new computes.

The rule, written down

tools/yardstick.js's header, in brief:

• Consensus scorers (cascade2, everydial):
  1. A pair the human 11e lists agree on (the old rule) counts 1, whatever Gemini says, contradicting included.
  2. A pair one human list orders and Gemini orders the same way counts 1. This only arises in armies with two or more human lists, where the old rule wanted a second grader.
  3. A pair only Gemini orders (no human list grades both units) counts 0.5.
  4. Everything else is left out, as before.

Scores are right weight over total weight. The scrambled control moves Gemini's letter with the unit's row.

• The exam's mean: (Σ human lists + 0.5 × Gemini) ÷ (n + 0.5).
• The flags. LEDGER_YARDSTICK=half (cascade2 and everydial write to build/cascade2-half/ and build/everydial-half/) and tools/exam.js --gemini 0.5. Without them, everything scores as before: the exam, C1, C5 and #3 reproduce T-728's and the exam's numbers on every army, to the pair.

How it was run

• node --max-old-space-size=6000 tools/yardexam.js --compute: Track A, C1, C5 and #3 from T-728's factorial rows (build/everydial/fact.rows.json); C1, C5 and #3 game-wide only (army-specific dials at the shared values); two supportMeasured probes; the six weight settings. The strength for (a) was refined by the secant method, 2.67 on Track A and 2.20 on C5. All 16 armies took about 1 minute; nothing heavy.
• node tools/yardexam.js --analyse --boot 1000, then --md.
• Held out: the exam's 15 armies, seeds 1 to 3, each shuffled into three folds of 5. Intervals come from a paired bootstrap of each army's units ([...]) and from resampling armies ({...}).
• Weights: each group's sum over the army's ranked forms (ledger.js contrib) ÷ the Kill's, averaged over the 16 armies. Score, Actions, Hold and Presence take one common factor. Lines outside the targets (Spawn, Battle-shock, Carry, army rules, the post steps) keep their weights. The brief asked for the median unit's share. That median is zero for the sparse lines (support, characters), so the army sums stand in.
• The setting files are build/yardexam/solved.json (the weights) and T-728's rows.

Gemini at half weight, every army (the consensus score)

The exam's own score, before and after

Held out, and the weights test

Held out (the exam's 15 armies; 3 seeds × 3 folds of 5: picked on 10, scored on 5; change over Track A, points; [units bootstrap] {armies resampled}):

Each setting fixed, over Track A on the exam's 15 (in sample: T-728 chose C5 and #3 on these armies; the weights were solved on all 16, not fitted to letters):

C1's game-wide part, on the 10 armies its choice never saw (cascade2's clean chain chose C1 through 6 armies: tyranids, world-eaters, orks, emperors-children, astra-militarum, adeptus-mechanicus): over Track A, human yardstick (half in brackets):

• C1g: +2.7 (+2.8), 9 of 10 up
• C1: +7.6 (+7.9), 10 of 10 up
• C5g: +5.1 (+5.1), 10 of 10 up
• C5: +9.2 (+9.4), 10 of 10 up
• #3g: +6.9 (+7.1), 9 of 10 up
• #3: +10.7 (+11.0), 10 of 10 up

The weights test (shares of Kill + Soak + scoring + support, mean over the 16 armies; then the scores, %, real (scrambled)):

The weights test per army, human yardstick (half in brackets), %:

Follow-up: cascade2's game-wide choice, held out by army

tools/cascadefold.js. It reads cascade2's stored records (no new computes; about 1 second) and uses the same splits as above: the exam's 15 armies, 3 seeds × 3 folds (choose on 10, score on 5), and leave one army out. The Tyranids are always in training and never scored.

There are three ways of choosing, each on the training armies only:

• main: cascade2's main chain (its order, its bars, the top survivor by margin);
• clean: the clean chain (only the armies whose scrambled letters never clear their bar);
• mean: the best mean score.

Training armies keep their own gated values. Held-out armies are scored at the shared values.

What it says:

• cascade2's game-wide choice holds out.
  • The main chain's pick beats Track A by +3.9 {+1.0, +7.0} on the armies it never saw, 12 up and 3 down. Leaving one army out gives +4.6.
  • The clean chain gives +2.1 {+0.5, +3.7}, close to the fixed C1 game-wide part's +2.4.
  • The plain best mean gives +4.4 {+1.6, +7.2}.
• The control:
  • On scrambled letters the chains die at the Tyranids and pick nothing.
  • The best-mean selector, run on scrambled letters, scores −1.2 / −0.6 / −0.3 held out. So the gain is signal, not the selection itself.
• Intervals: these resample armies only, because the records hold counts. They are wider than the unit bootstraps above.
• Stable in every selector and nearly every fold:
  • charEpic 0.5 (except 3 clean folds);
  • aircraftBoard 1;
  • typicalSize 0.25.
• Stay at Track A's value: lands 1.5, β 6.
• Split by selector:
  • main and mean pick smallPiece 0.2, attachChars 1, armyEnabler 1, r.cp 2.88, r.first 1.8, sureFire 0.5 without reserves, and keep r.synapse 2.18.
  • clean picks C1's own kind: smallPiece 0, attachChars 0, r.synapse 1.5, leaderCompPen 1, sureFire 0.25.
  • So the fair-test armies pull one way and the rest the other. The tie-breaker dials are not identified by 10 armies.
• Armies by army:
  • Adeptus Mechanicus (−5 to −6) and Emperor's Children (−0.5 to −4) lose under every selector.
  • Votann (+7 to +17), Thousand Sons (+7 to +15), Orks and Genestealer Cults gain most.

Which dials the folds pick (count of the 9 fold picks with each value; leave-one-out's 15 in brackets; Track A's value first):

Per army, held out (main / clean / mean, over Track A):

space-marines +2.0 / +0.4 / +2.0; adeptus-mechanicus −6.3 / −5.0 / −5.6; agents-of-the-imperium +0.6 / +1.8 / +2.2; astra-militarum +5.2 / −1.0 / +5.5; blood-angels +5.1 / +4.0 / +3.5; dark-angels +2.5 / +2.1 / +1.6; death-guard +1.5 / +3.2 / +2.0; drukhari −1.0 / −2.3 / +0.8; emperors-children −4.4 / −0.5 / −3.0; genestealer-cults +7.7 / +4.5 / +8.0; leagues-of-votann +17.1 / +6.7 / +17.1; orks +8.1 / +3.4 / +8.7; space-wolves +4.3 / +6.0 / +5.0; thousand-sons +14.6 / +7.2 / +14.3; world-eaters +1.8 / +1.4 / +3.2.