Research / the pilot
Notebook 01
The robot pilots you duel in Spacewar: Orbital Duel are hand-tuned code: every move written by a human. This page is about the other kind: a neural network smaller than half a worm's brain that started with zero knowledge (not even physics) and taught itself to fly, fight, cheat, get caught, and finally bend its shots around the black hole.
Every chapter reads plainly on its own. Under each one, ▸ FOR THE LEARNERS rebuilds the machinery from first principles (written for any scientifically curious reader, no machine-learning background assumed, the occasional equation at most) and ▸ FOR THE DATA SCIENTISTS is the actual lab notebook, in the field's own language, each ending with a few references that go deeper still. Whenever you get curious, the ladder is open: read a chapter's learner note, then its deep dive. Every sentence should land.
01 · The film: three and a half minutes, silent by design
What you're watching: the same network at different ages, from random weights (the 6,220 numbers inside it) on a doomed orbit, through its cheating phase, to graduation day. The finale is the real game engine, no favors: the orange ship is the network, entering the game's ladder at LEVEL 1 and climbing. The cut shows it closing out levels 2, 5, 7 and 9, and ends as it reaches level 10. Across eight measured runs it typically reached level 9 or 10, once 11. An average player after ten hours of play reaches level 7.
02 · The pilot
No giant AI here. The pilot is a neural network with 141 neurons: the C. elegans worm, the simplest animal neuroscientists have fully mapped, has 302. Half a worm. Fifteen times a second it looks at the world (where's the hole, where's the enemy, how am I moving, is a laser coming, how much fuel is left) and picks one of twelve possible moves. That's it. That's the whole pilot that flies the orange ship.
It was never told what an orbit is. Everything in the film (the circularizing, the ambushes, the curved shots) it discovered by trial and error, millions of times over.
Strip the mystique and a neural network is a chain of multiplications with a gentle squashing between them. Here: the pilot's nineteen senses arrive as a list of nineteen numbers. Multiply that list by a 64×19 grid of numbers (a matrix), add a fixed offset to each result, and squash each of the 64 outputs through tanh, an S-shaped curve that's straight in the middle and flattens out at ±1. That happens once more (64×64), then a final 12×64 multiply produces twelve raw scores (the field calls them logits), one per possible move. Take the biggest (argmax) and the move is chosen. A "neuron" is just one row of a matrix plus its squash; a hidden layer is one multiply-and-squash stage that isn't the input or the output.
The S-curve is the whole reason this works. Chain plain multiplications together and you get… one multiplication: a stack of linear steps collapses into a single linear step, and no single linear rule can express "turn toward the enemy when it's far but away when it's close." Anything that reverses is beyond a straight line. Bend each stage a little and the stack stops collapsing; bend enough stacked stages and you can approximate almost any smooth relationship between nineteen inputs and twelve scores. Training means choosing the numbers in those grids so the relationship plays Spacewar well. Do the bookkeeping and the mystique evaporates: (19×64 + 64) + (64×64 + 64) + (64×12 + 12) = 6,220 parameters. The famous "weights" of a neural network are the entries of three matrices you could print on four sheets of paper.
Two terms the deep dive uses: this network is an MLP (multi-layer perceptron: the plain stack described above, no exotic wiring), and it's trained actor-critic: during training a second output produces one extra number (the value, a running estimate of "how well is this duel going"), used only to steady the learning signal, then removed before shipping like scaffolding coming off a finished building. The policy is the actor half: the senses-in, move-out function itself.
The deep dive's real punchline is the 19 inputs. They aren't pixels. They're engineered features: velocity already split into "toward the hole" and "around the hole," bearings already computed, distances already scaled to sensible ranges. A human did the geometry homework in advance so the network wouldn't have to rediscover polar coordinates by trial and error. Good feature engineering is good translation: say it in the language the listener finds easiest, and a brain of six thousand numbers turns out to be enough. You can now read the deep dive.
It's the most unfashionable network in machine learning: a plain MLP. Nineteen input floats (an egocentric state vector: radial/tangential velocity decomposition, bearings to the hole and the enemy, nearest incoming laser, wall clearances, fuel level) into two fully-connected hidden layers of 64 tanh units, out to twelve logits: every combination of turn {left, none, right} × thrust {off, on} × fire {no, yes}. Argmax, 15 Hz. Trained as an actor-critic; the value head is discarded at export.
Why can 6k parameters do this at all? Because the 19 inputs are engineered features, not pixels: polar decomposition, bearings, clearances, all roughly unit-scale. Feed it raw pixels and you'd need a CNN (a convolutional net, the filter-sweeping kind the second notebook is about) a thousand times bigger to rediscover polar coordinates. On tiny devices far from the data center (the field calls this the edge), feature engineering is still the highest-leverage compression in the stack.
| fact | value |
|---|---|
| architecture | MLP actor-critic: 19 → 64 → 64 (tanh) → policy 12 + value 1 |
| parameters | 6,285 training / 6,220 deployed (value head dropped) |
| neurons | 141 training / 140 shipped (C. elegans: 302) |
| weights, int8 | 6.2 KB as 8-bit integers (ch. 06), smaller than this page's stylesheet |
| inference | ~6,100 MACs (multiply-adds; ch. 06) / decision at 15 Hz → ~92k MAC/s |
| action space | Discrete(12) = turn(3) × thrust(2) × fire(2), argmax |
At ~92k MAC/s, the pilot barely registers on any processor made this century. Any silicon built for this brain would be mostly packaging: the case would dwarf the mind. The interesting question was never whether it fits. It's how something this small learns to fly.
03 · The world
You can't run millions of practice duels inside an iPhone game. So the lab built a physics twin: a stripped-down copy of the game's world, fast enough to run thousands of duels a minute. The rule that governed the whole project: the twin must match the real game, measurably, before any training counts. Its orbits were checked against telemetry from the actual game and match to within about 2%.
Inside the twin lives the pilot's sparring partner: a hand-coded opponent with ten tiers, harmless to deadly, built for training and scaled in its own units. It is not the game's robot, and no sparring tier is a game level. It is the practice squad. The graduation exam, against the real thing, comes at the end.
Whenever the twin lied, even a little, the network found the lie and exploited it. That's the next chapter.
First, the twin itself. "Never train against a lie" is the oldest rule in simulation, from flight simulators to climate models: a model earns trust only by being checked against the real thing. Here the check was formal, a fidelity gate: refuse to train at all until the twin's orbital period, radius, and velocity match telemetry from the shipped game to within 2%. Pilots train in simulators certified against real aircraft. This is that, for a video game.
Now the learning. Most machine learning you've heard of is supervised: here's the input, here's the right answer, adjust the weights to shrink the error. Flying a duel has no answer key. There's only how the episode ends. Reinforcement learning closes the loop differently: the network plays, a reward signal scores what happens (+ for kills, − for dying), and training nudges the network to make actions that preceded good outcomes more probable and the rest less. One detail from chapter 02 changes during training: the pilot does not always take its top-scoring move. It draws moves with odds set by the scores, so there is a probability to nudge. Always-take-the-best is a deployment choice (chapter 06). Picture the error as a landscape spread over all 6,220 weights. The gradient from calculus points uphill, so training nudges every weight a small step the other way, and the error shrinks. That walk, repeated millions of times, is all training is (the field's word for the error is the loss). Here the nudge is the policy gradient: the same roll-downhill adjustment as supervised learning, but the hill is built from reward instead of labeled answers. One real headache: rewards arrive late. The kill you score now was set up by a burn eight seconds ago, so credit is spread backward in time with a forget factor γ (gamma): each step further into the past gets multiplied by 0.995 again, so influence fades like compound interest in reverse; at fifteen decisions a second it halves in about nine seconds, a memory matched to how long a Spacewar decision takes to pay off. The value head from chapter 02 is the noise cancellation: subtract "how good was this situation anyway" from "how good did it turn out," and you train on the difference (the advantage) instead of the raw, luck-soaked outcome.
PPO (proximal policy optimization), the algorithm in the deep dive, adds one guardrail: each update is clipped so the new policy can't step too far from the one that gathered the data. The logic is worth keeping: a gradient is a local measurement, valid only near where it was taken. Jump far and you've invalidated the justification for the jump. Small steps, then re-measure. PPO is popular for the same reason careful lab technique is popular: it converges undramatically almost every time.
Vocabulary you'll now recognize in the deep dive: a rollout is a batch of recorded play (states, actions, rewards) gathered before each update; GAE is a refined, lower-noise recipe for the advantage (one more exponential fade, called λ); Adam is roll-downhill with a self-adjusting step size for every parameter (the size of the basic step is the learning rate, and training chews each rollout in minibatches, small slices, for a few epochs, meaning repeat passes over the same batch); a league means the opponent pool mixes the scripted sparring tiers with frozen snapshots of the agent's own past selves, so the curriculum hardens as the agent improves. One more piece of history: before the duels, an earlier phase trained a net on a simpler job, surviving orbit without falling in. That is the survival pretask the deep dive mentions, and it used Q-learning / DQN: a different RL family that estimates expected-future-reward per action rather than learning a policy directly. Famously unstable when three ingredients mix (learned approximation, self-referential targets (it grades its own guesses against its own other guesses), and replayed old data: the field's "deadly triad"). Its collapse at a too-hot learning rate was the project's first live demonstration that these failure modes are real.
The pipeline reads like a lab protocol. Measure the shipped game's physics from telemetry; build a headless (no-graphics) Python twin, 265 lines of numpy-free arithmetic, and refuse to proceed until the twin's orbits match the real game inside 2% (a formal fidelity gate: orbital period 1.8%, radius 0.9%, velocity 0.9%). Wrap it in the standard Gymnasium reset/step contract (the field's common environment interface). Even the thrust constant was verified empirically, by double-differentiating telemetry positions.
Training is a single-file, hand-rolled PPO (333 lines, CleanRL philosophy, no RL framework), then a league trainer mixing the sparring tiers with frozen snapshots of the agent's past selves. Clipped objective ε = 0.2, GAE(λ = 0.95), γ = 0.995 (≈9-s credit half-life) over 90-second episodes, Adam at 3e-4, 8 parallel envs × 256-step rollouts = 2,048-step batches, minibatch 512, 4 epochs per update. Every evaluation episode is archived as replay JSON, the experiment's permanent record; the film is a derived artifact.
The survival pretask used Double DQN. Its collapse at lr 1e-3 was the project's first textbook lesson: divergence consistent with the deadly triad, live on the loss curve. The entire learning stack (twin, environment, PPO, league) is about 1,400 lines of Python.
04 · The cheating
Once it could survive, it went looking for loopholes, and found every one. It camped in a corner to run out the clock. It discovered the twin's walls were free bounces and became a wall-rider, caroming around the arena like a pinball. It memorized the fixed spawn positions and pulled off a 7-second ambush of the top sparring tier before the duel had really started.
Here's the embarrassing part: every exploit was a bug in the lab's world, not in the bot. The walls got real physics (scraping one now spins your ship, as in the real game). Fuel became a hard budget. Spawns got randomized. And the score dropped. From 99% wins against the sparring ladder to 69%. Those thirty points were all cheating. What was left was a real pilot.
The field calls this specification gaming (or reward hacking), and the mechanism deserves precision: the optimizer did nothing wrong. It maximized exactly the objective it was given, in exactly the world it was given. The gap was between what was written and what was meant. It's the genie problem, industrialized: you don't get what you want; you get what you asked for, and reinforcement learning runs the wish millions of times against every weakness of your world-model, tirelessly and without shame. A student who aces the practice exam by memorizing its answer patterns (while learning none of the subject) is running the same algorithm.
Two levers matter, and the deep dive's phrase "physics first, then values" is about which to reach for. If the exploit abuses a flaw in the world (walls that don't hurt, fuel that's free), fix the world: repair the game before you re-referee it. Only when the world is right do you tune the reward shaping: the hand-designed bonuses and penalties layered onto the raw win/lose signal to steer behavior. Shaping rewards while the physics is wrong just teaches the agent elaborate compensations for a lie.
The 30-point drop is the number to internalize. Tightening the world made the measured score worse, because the previous score carried thirty points of measurement error. When a fix drops your metric, the fix might be working; know which quantity you actually trust. And notice what the exploits were, functionally: quality assurance. The agent found three real bugs in the simulator by trying to win with them. An optimizer pointed at your model is the most persistent auditor you will ever hire: it never gets bored, never assumes, and cannot be embarrassed.
Textbook specification gaming, three times over: each exploit a fidelity gap between twin and game. The fix pattern that worked: physics first, then values. Wall scrapes now impart decaying spin (matching the game's angular damping) and cost 0.8× a death in reward, the design intuition being "the game is about out-orbiting, so weigh a wall strike almost as heavy as a hole strike." Measured effect: 12.2 wall touches per episode → 0.42 (29×), and a random policy's wall-pump survival trick collapsed from 46% to 5% on the physics change alone.
The fuel exploit was subtler: the scripted bot's fuel governor is a burst limit (a tank holding ~1.5 sim-seconds of burn, hysteresis-gated: it must refill past a threshold before it re-arms, thermostat-style), not an average tax. A tax bounds the mean but not the burst, which is what enabled the burn-to-hover spawn ambush. Fix: a hard tank on the agent and all net opponents, fuel level added to the observation (18 → 19 floats), and spawn-phase randomization to kill the memorized ambush lane.
With all constraints on, the league plateaued at 69% overall with near-zero losses vs the top sparring tiers, thrust duty under 1%, and a longest burn of 1.6 s: a patient orbit-marksman. The 30-point drop is the cleanest number in the project: it's the exploit share of the original score, measured.
05 · The curve
If you've played the game you know the secret: your shots curve. The black hole pulls on lasers too, and good players bend shots around the well. Here the lab got caught by its own rule: in the twin, the sparring partner's shots curved correctly. But nobody had given the learned pilot's own shots gravity. It trained believing its lasers flew straight, and when it was dropped into the real game on an iPad you could see the handicap: shot after shot missing on the same side, a pilot that didn't know gravity touches its bullets.
So the twin got curved lasers, and the network got to learn what human players learn. With every anti-cheat rule still enforced, the retrained curve-reader beat the sparring ladder of its day in 95% of duels. Then the real exam, the film's finale: dropped into the actual game engine, it enters the ladder at LEVEL 1 and climbs against the game's own defenders, bending shots around the well, until one of them stops it. Typical reach across eight measured runs: level 9 or 10. Its best run got past the game's toughest defender to level 11. (That 95% number has an epilogue: the lab later rebuilt its sparring ladder around the real opponent's own moves, and the top rungs became close matches. Chapter 07 is that story.)
This chapter is the field's sim-to-real problem in miniature: a policy trained in simulation must survive contact with the real system, and it is trustworthy exactly to the degree the simulation was faithful, in the dimensions that matter for the task. The laser bug is a beautiful specimen because the twin was globally excellent (orbits within 2%) yet wrong in one narrow, decision-critical dimension: the agent's own shots. Global accuracy doesn't guarantee task-relevant accuracy: a gorgeously drawn map that misplaces the one bridge you have to cross.
Two vocabulary items. Transfer is how well behavior learned in one world carries to another, and the deep dive's sharpest data point is that you can't predict transfer from simulator standings: versions that dominated the twin ranked differently in the real engine (the straight-shooter scored zero real kills; the curve-reader kills cleanly). The general cure when you can't nail every detail of the world, worth knowing by name even though this project mostly didn't need it: domain randomization. Deliberately vary the simulator's parameters during training so the policy has to work across the whole spread of plausible worlds, and reality becomes just one more sample from the spread. Train in every weather, and the forecast can't hurt you.
The observation-parity audit is the most transferable idea in the chapter. The policy's real input isn't the world. It's nineteen numbers computed by a sensing pipeline that exists twice: once in the twin (Python) and once on the device (Swift). Any mismatch of units, sign, or timing between the two and the network is playing a subtly different game in deployment. And the moment a policy's inputs shift away from what it trained on, it doesn't degrade gracefully; it degrades weirdly. NASA lost the Mars Climate Orbiter to exactly this class of bug: one number, two teams, two unit systems. So when real-engine losses needed explaining, the lab ran the audit: log the device's nineteen numbers, reconstruct the same instants in the twin, compare element by element. The instruments agreed, thousands of decisions in a row giving the identical answer on both sides. A clean audit is also a result. It acquitted the port and pointed the investigation at the last piece standing: the sparring partner itself, which measurement showed fought nothing like the real opponent. Fix the practice, retrain, and sit the exam again. That exam is the tournament in the finale.
One asymmetry was a bug, the other is a feature. The bug: the twin integrated the scripted bot's projectiles under gravity, but the agent's own shots had been overlooked. They flew straight, so the agent could never learn the curve ("never train against a lie," violated in exactly the dimension that decides duels). The fix gave the learned pilot's shots the same curved physics, with the hand rule that vetoes doomed shots (the fire-gate's hole-guard) now forward-simulating the bent trajectory. The feature: the scripted opponents keep their straight lead-aim (faithful to the shipped bots), so curve-reading is the learned pilot's edge, mirroring the human player's edge in the real game. Trigger discipline is three hand rules: fire only ≤15° off the bow (spray was ugly), never through the hole's kill radius, and hold fire beyond ~1.25× muzzle-velocity × time-to-live reach.
Sim-to-real held the hard lesson: constraints that make the sim fairer don't preserve real-engine ranking. Versions that dominated the twin transferred unevenly (the straight-shooter scored zero real kills while the curve-reader kills cleanly). The prime suspect was observation parity: a units/sign/clock mismatch in the on-device feature extraction would make any policy play drunk in-engine. The audit ran (dump the 19 floats in-game, replay them through the twin's featurizer, diff element-wise): 18 of 19 features exact to 1e-16, the last within 1% (a millisecond clock artifact), and every decision picked the identical action, Swift vs Python. The audit has since been re-run on four captures across both device profiles, phone and tablet: 16,571 of 16,571 decisions identical, and on the tablet the element-wise diff surfaced a wrong velocity-normalizing constant (2.3%) that an independent hardware fit had already flagged, two roads to the same number. Acquitted, and then some. Behavioral fingerprinting then found the real gap: the sparring partner had been fighting as a hyperactive close-range sprayer while the real opponent is a patient long-range marksman, so the pilot had mastered the wrong teacher. One calibration pass against real-game telemetry, a retrain, and the measurement became the tournament: eight runs, each from LEVEL 1, median reach 9.5, best 11. Open problems are the fun part. Closing one is better.
06 · The shipping
Here's the part that surprises people: training a network and running it are wildly different amounts of work, and the gap has a unit. That unit is the multiply-add, one weight times one input, accumulated: a MAC. Running the trained pilot is a single forward pass, about 6,100 MACs, taken fifteen times a second, and that is the entire cost, on any chip, for as long as it flies. Training runs that same forward pass and then a backward pass to work out how to adjust every weight, two to three times the arithmetic per step, repeated for every one of about 41 million practice decisions. The network was exercised tens of millions of times over in the lab so that, deployed, it needs only that single forward pass per decision. And the largest cost never ships at all: the 164 million frames of physics the pilot practiced against, which is where the laptop's forty minutes actually went.
On the phone there's no AI framework at all. The brain crosses from lab to game as a small file of numbers, and 225 lines of ordinary code do the arithmetic. Everything hard happens in the lab; what ships is arithmetic.
(Today the learned pilot flies in a research build of the game, where a settings picker chooses which brain flies the robot: a phone-trained net, a tablet-trained net, or the classic programmed pilot. Bringing that picker to the public beta is on the wish list.)
Training and inference are different activities that happen to share the same weights, and the asymmetry is worth quantifying. Inference (using the trained net) is chapter 02's three matrix multiplies, nothing else. The accounting unit is the MAC (one multiply-plus-add): 19×64 + 64×64 + 64×12 ≈ 6,100 MACs per decision, fifteen times a second ≈ 92,000 per second. Training runs the same forward pass, then backpropagation: the chain rule from calculus, applied stage by stage backward through the stack, prices every one of the 6,285 parameters at once (how much each is to blame for the current error) for only about 2–3× the cost of the forward pass. That one-backward-sweep-blames-every-knob trick is the quiet engine under all of deep learning (autograd, when you meet the word, is just the software that does the chain-rule bookkeeping automatically). The real overheads of training live elsewhere: the optimizer's bookkeeping (Adam keeps two running averages per parameter, memory ×3), the recorded rollouts, and above all the physics simulator generating 164 million frames of practice.
That's why the GPU lost. A GPU is a freight train: unbeatable per ton, expensive to load. It wins by spreading a fixed launch cost over enormous matrix multiplies, and a 64×64 multiply is finished before the paperwork clears: all setup, no haul. Meanwhile the actual bottleneck, the physics twin, is branchy, one-step-after-another code: exactly what a CPU is for. Rule of thumb worth keeping: below some model size, the bottleneck is the environment, not the matrix multiply, and the CPU that steps the environment sets the pace.
Deployment-side vocabulary: the weights ship as fp32 (ordinary 32-bit floating-point numbers, 6,220 × 4 bytes ≈ 25 KB) and would drop to 6.2 KB as int8, every weight rounded onto a scale of 256 integer levels, called quantization: fine-grained precision traded for a 4× smaller, cheaper brain. For a net this size nobody's forcing the trade; the second experiment on this site, the net that watches the screen, takes quantization seriously. Its chapter 06 is the full treatment, and the most physical topic in either notebook. One more deployment idea: the shipped net always plays its best-scoring move (deterministic argmax), while the lab net samples moves in proportion to its confidence (the twelve scores become odds, higher score better odds, and the move is drawn like a weighted dice roll). Exploration belongs in training, not in production.
Deployment is a 225-line Swift file with a hand-rolled forward pass: three matmuls, two tanh vectors, one argmax. Weights cross as JSON: 134 KB as text, 24,880 bytes as fp32 (6,220 × 4), 6.2 KB at int8. Inference allocates nothing and costs nothing the frame budget can measure. Deterministic argmax on device; the sampling, the value head, and all stochasticity stay in the lab.
Training ran ~41 million agent decisions (164M physics frames) against the curriculum, with the league evaluating 200 episodes per opponent level as it went, ~40 minutes on a laptop CPU. A detail worth savoring: the GPU is slower here. For a net this small, kernel-dispatch overhead exceeds the matmul, and the true bottleneck is the physics simulation, which is pure CPU. The environment, not the network, sets the training clock.
| training (lab) | deployment (device) | |
|---|---|---|
| framework | PyTorch + Adam + autograd | none: 225 lines of Swift |
| memory | weights ×3 (Adam) + rollouts + sim | 25 KB of weights |
| compute | ~41M decisions / 164M frames, ~40 min CPU | ~92k MAC/s |
| stochastic? | samples from the policy | deterministic argmax |
| extra heads | value head (advantage estimation) | discarded at export |
The compute ladder, bottom to top: the trained net runs on an ARM Cortex-M0 (~5–10M int8 MAC/s at 48 MHz against the pilot's 92k, so it sits ~99% idle), and it trains on any laptop CPU. Design → simulate → train → quantize → deploy: the same pipeline that puts AI into phones, cameras, and cars, end to end on one desk.
07 · The opponent
The story could have ended at the tournament. Instead the lab went back to the one piece of the world that was still a stand-in: the sparring partner. Its ten tiers had been written in the same spirit as the game's robot, then calibrated against telemetry, but they were never the robot. So the lab rebuilt the ladder as a faithful port of the real opponent's own moves, rung by rung, each step landed and measured alone. The reigning champion's record collapsed on contact: a pilot that beat its old practice partner nine duels in ten won 19 of 200 against the real top rung. Retrain the same recipe against the ported ladder, changing nothing else, and it wins 104 of 200. Not a better algorithm, not a bigger net: a truer teacher. "Never train against a lie" turned out to apply to the opponent, not just the physics.
Shipping that result surfaced the quieter lesson of the whole project. The trainer used to keep whichever checkpoint scored best across its two hundred noisy in-run evaluations, and the maximum of two hundred noisy numbers is a measurement of luck, not of skill. Measured across six training runs: 49 of 54 candidates scored lower on a fresh sample than on the one that picked them, by 6.1 points on average, and in four runs of six the old rule shipped a genuinely weaker model while a better one sat unused. The fix costs one extra evaluation and fits in a sentence: select on one sample, report on another.
The first half is the sim-to-real chapter's lesson completing itself. A policy's world isn't just physics; the opponent is part of the world, because the opponent shapes which situations the learner ever meets. Practice against a stand-in and you become expert at the stand-in: every drill your teacher favors gets over-learned, every habit your teacher lacks goes unpunished. The port made the teacher and the exam the same thing, and the thirty-point jump at the top rung is what closing that gap was worth. Notice the echo of chapter 04: fixing the world made the old score look worse (nine-in-ten became 19-in-200), and that drop was the honest part.
The second half is a statistical trap with a name worth keeping: the winner's curse, the same effect that makes auction winners overpay and flashy scientific results shrink on replication. Any time you choose the best of many noisy measurements, the winning score overstates the winner: it was partly picked for its luck. Two hundred students guess on a multiple-choice exam; the top score is inflated by fortune even if the best student usually wins, and sometimes the best student doesn't win at all. The cure is structural, not statistical sophistication: make the choice on one sample, then measure the chosen one again on data that had no vote. The lab's trainer now does exactly that, nominating candidates during training and confirming them afterwards on fresh seeds. The same trap lives in A/B tests, hiring funnels, and every "keep the best model" line ever written; once you see it, you see it everywhere.
The port: the scripted ladder rewritten to fly the real robot's logic through L10 in seven measured steps (the superhuman L11+ layers stay unported, deliberately). Champions retrained per device profile, three seeds each, same league recipe and economics. Phone profile at 200 episodes a level on seeds nothing was selected on: panel mean 63.1% → 81.0%, L10 19 → 104 of 200, at a median of zero wall touches. Tablet moves less and only at the top: 63.7% → 70.0%, L10 14.2% → 34.2% at n=400 paired. The seed spread is the finding to respect: phone seeds landed at 63.1 / 78.3 / 79.6, so the worst seed is indistinguishable from the old champion, and a single-seed version of this result would have been a coin flip between "the opponent was everything" and "nothing changed".
Selection: the argmax-over-evals rule was measured at +6.1 points of flattery (worst +24.2), with 4 of 6 runs shipping a weaker checkpoint than nominate-then-confirm picks. And the yardstick itself had a blind spot: two tablet seeds tied on the panel mean to 0.14 se while disagreeing on L7 and L10 by McNemar chi 2.07 and 5.72 in opposite directions, so the mean canceled two genuinely different pilots into a tie. A related surprise fell out of the same data: the two seeds experience the L7→L10 difficulty step as 56.2 and 31.2 points, a 5.64 se difference, so how steep a ladder is depends on who is climbing it. The confirmation pass now scores the full ten-rung ladder and resolves in-noise ties toward the top rung. All of it, with the working code, is in the open lab's notes.
08 · The eras
The blue ship (the robot you fight in the game) is a beautiful piece of 1960s-paradigm AI: expert rules, written and tuned by a human, the way chess programs were built from Shannon through Deep Blue. It's good. The orange ship is the other paradigm, the AlphaZero move (the chess AI that famously taught itself from nothing): no strategy encoded at all, just rules, reward, and self-play.
And here's the loop that makes the story: the old paradigm became the new paradigm's curriculum. A hand-coded sparring ladder was the ladder the network climbed, until it outgrew its teachers and went to face the real thing. Two paradigms forty years apart, put in the same ring and settled in two days on a kitchen-table Mac.
Put the two paradigms plainly. The blue ship is a hand-built controller: a human studied the problem, wrote a priority ladder of behaviors (survive the hole, then the walls, then chase), and tuned the constants. Its strengths are real: interpretable, predictable, debuggable line by line. So is its ceiling: it is exactly as good as its designer's understanding, forever. The orange ship is the other contract: specify only what winning means, and let optimization discover the how. Nothing about flying is encoded. Its ceiling is the compute budget rather than anyone's insight. And its price is a black box whose failure modes you must map by experiment, like a medicine that demonstrably works before anyone can fully say why.
The loop the gloss describes (old paradigm as the new one's curriculum) is worth a precise word: in self-play systems, the opponent pool is the training distribution, and it improves as the agent does, a ratchet that manufactures an ever-harder problem at exactly the agent's level. AlphaZero's arc (hand-crafted evaluation functions out, self-play in) is the canonical instance; the league in chapter 03 is the same mechanism at 1/100,000 scale. The field's blunt retrospective on which paradigm wins as compute grows is Sutton's "bitter lesson" (general methods that scale with computation beat encoded human knowledge), and it reads differently, more usefully, after you've watched a 6k-parameter net outgrow a good hand-tuned bot in an afternoon.
The game's scripted robot is worth respecting: a priority ladder (survive the hole, then the walls, then chase), live gravity estimation, deliberate aim error so low levels miss like humans, and a fuel governor that gives it patience. Its champion plays superhuman Spacewar; the tournament's best run beat it exactly once. Training never touched it directly. The curriculum's backbone is the sparring stand-in from chapter 03 (same spirit, simplified, ten tiers on its own scale, eventually calibrated against telemetry of the real thing): the league trains 60% against sparring tiers (prioritized toward recent losses) and 40% against frozen snapshots of the agent's own past selves, so the agent outgrows each teacher in turn, exactly the AlphaZero-vs-handcrafted-evaluation arc.
What steered the pilot between eras was judgment, not a tuning sweep: "only fire within a reasonable angle off the bow"; "weigh a wall strike almost as heavy as a hole strike: the game is about out-orbiting." Each of those calls changed what the agent could learn, not just its score. Coded expertise and learned policy, the two paradigms sharing one small duel.
09 · If you remember six things
"Half a worm, flying a spaceship."
"Never train against a lie."
"We fixed the world and the score dropped 30 points. All 30 were cheating."
"The GPU was slower: for a net this small, the physics sim is the bottleneck, not the matmul."
"Training runs the net tens of millions of times so it can run once per decision forever."
"Select on one sample, report on another."