Build on the Winners: How I Tied the Top Time in Hugging Face's MicroDuck Arena Without Training From Scratch

I'm racing a small simulated robot duck in the Pollen Robotics MicroDuck Arena on Hugging Face. I started from the public policies already at the top of the board, jrubiosainz, dashi, and DenRakEiw, and I composed their per-seed joint-offset tables. SPSA and a chained CMA-ES search edited individual seeds. The network weights stayed where those authors left them. On sprint and hill the scored time is jrubiosainz's own, and my seed edits stayed inside the same 0.02 s step. The 0.904 s roller file is their tables.

As of 13:29 CT on Oct 22, sprint-2m still ties the top time at 0.840 s. jrubiosainz entered that time first, so the board lists me 2nd. When I entered roller-sprint-2m at 03:51 CT on Oct 22, it tied the top time at 0.904 s and was listed 2nd, because DenRakEiw had entered that time at 03:09 CT. At 13:10 CT on Oct 22 dashi posted 0.900 s and leads outright. My 0.904 s ties DenRakEiw, which is 3rd per person on the 13:29 CT board. On hill-climb-1m5 my entry is 0.756 s. When I entered it at 23:02 CT on Oct 21, that time tied the top and the board listed me 2nd. By 05:22 CT on Oct 22 the leading time was 0.752 s and the row was 3rd per person. At 13:17 CT on Oct 22 dashi also entered 0.752 s. On the 13:29 CT board my 0.756 s is 4th per person, (0.756 − 0.752 = 0.004 s) behind the leading time.

The Arena scores seeds 1 to 5 in MuJoCo. Posted times move in steps of 0.02 s, so one seed dropping one step moves the five-seed mean by (0.02 / 5 = 0.004 s).

What actually moved the needle

The board only moves when a seed crosses a 0.02 s step. I still entered sprint and hill on jrubiosainz's already-scored times. A few ten-thousandths of a second did not change the board.

What worked

Composing per-seed tables from the public policies. The posted board times come from that composition.

The 0.904 s roller row is jrubiosainz's game-roller-remix6 on seeds 1 to 4, plus DenRakEiw's senate_duck_0904 on seed 5. None of my searched tables are in that file. The five scored seeds are 0.92, 0.90, 0.90, 0.90, and 0.90, and (0.92 + 0.90 + 0.90 + 0.90 + 0.90) / 5 = 0.904. When I entered that row at 03:51 CT on Oct 22, it tied DenRakEiw's leading 0.904 s, and the board listed me 2nd because DenRakEiw had entered at 03:09 CT. At 13:10 CT on Oct 22 dashi posted 0.900 s. On the 13:29 CT board my 0.904 s ties DenRakEiw, which is 3rd per person.

Sprint-2m is jrubiosainz's sprinter-remix4. That policy already scores 0.840 s on its own, with all five scored seeds at 0.84. My seed-1 edit stayed inside the 0.84 step, so the posted time stayed 0.840 s. When I entered it at 23:02 CT on Oct 21, that tied jrubiosainz's leading time and the board listed me 2nd. On the 13:29 CT board the tie is still the top time, and I am still listed 2nd. My earlier sprint row was 0.844 s, which is (0.844 − 0.840 = 0.004 s) slower.

Hill-climb-1m5 is jrubiosainz's hillclimber-remix5. That policy already scores 0.756 s on its own. My seed-3 edit stayed inside the 0.76 step, so the posted time stayed 0.756 s. When I entered it at 23:02 CT on Oct 21, 0.756 s tied the top time and the board listed me 2nd. By 05:22 CT on Oct 22 the leading time was 0.752 s and the row was 3rd per person. At 13:17 CT on Oct 22 dashi entered 0.752 s. On the 13:29 CT board my 0.756 s is 4th per person.

A table search on the one slow seed. The network stayed frozen. I edited the joint-offset table for the seed that was still on the slow step, and each search started from the latest best of that same table.

Roller seed 4 went 0.90281, then 0.90103 at 23:20 CT (gap 0.90103 − 0.90 = 0.00103 s), 0.90059 at 23:32 (gap 0.90059 − 0.90 = 0.00059 s), 0.90048 at 23:48 (gap 0.90048 − 0.90 = 0.00048 s), 0.90039 at 00:25 (gap 0.90039 − 0.90 = 0.00039 s), and 0.90000 at 01:14 CT on Oct 22. That is the 0.90 step. The posted roller time from that seed was 0.908 s, a tie with the leader at that hour, listed 2nd. The 0.904 s row is game-roller-remix6 on seeds 1 to 4 plus DenRakEiw's seed 5. DenRakEiw's senate_duck_0904 matched my 0.908 s entry on seeds 1 to 4, at interpolated crossings 0.91779, 0.89695, 0.89961, and 0.90000, and moved seed 5 from 0.91622 to 0.89994.

Sprint seed 1 moved from 0.83942 to 0.83687. Both numbers sit inside the 0.84 step, so the posted sprint time stayed 0.840 s. Hill seed 3 moved from 0.75963 to 0.75204. Both are inside the 0.76 step, so the posted hill time stayed 0.756 s, the same as jrubiosainz's hillclimber-remix5 on its own. The only 0.74 is seed 1, at 0.73978, and (0.74 + 4×0.76) / 5 = 0.756.

Mixing two searches made the duck slower. I grafted one search's finish rows onto a better mid-race stack and the seed went from 0.90177 to 0.90592, which is (0.90592 − 0.90177 = 0.00415 s) worse. A fresh small search chained from the latest best of one table is what carried seed 4 down to 0.90.

Scoring the real ONNX rollout. A tape replay reported a gain I had to throw out. On hill seed 4 the wrapper said 0.75395 and 0.75426 against a 0.75458 base, which is (0.75458 − 0.75395 = 0.00063 s) and (0.75458 − 0.75426 = 0.00032 s). The same candidate, run as a real ONNX policy, never crossed the line. Another wrapper gain of (0.75461 − 0.75397 = 0.00064 s) came back from the ONNX rollout at 0.78469, which is (0.78469 − 0.75461 = 0.03008 s) slower. I kept a time only after that rollout.

What missed the 0.02 s step

The PPO warm start broke the table. I loaded dashi's public roller network, the one that scores 0.920 s with all five seeds at 0.92. PPO then took a step at learning rate 0.0001. Every actor weight moved by at most 5.6e-4. The same weights, before that update, still scored 0.920 s. After the update the five seeds were 0.96, a fall, 0.94, 0.94, and 0.96. A fall counts as 40 s, so the time was ((0.96 + 40 + 0.94 + 0.94 + 0.96) / 5 = 8.760 s). The first update cost (8.760 − 0.920 = 7.840 s). The next checkpoint was one finish at 1.30 s and four falls, ((1.30 + 4 × 40) / 5 = 32.260 s). The per-seed tables replay fixed, open-loop actions that were tuned against the exact original weights. Once PPO nudged those weights, even by 5.6e-4, the duck's path drifted, the replayed actions stopped fitting, and it fell. That run cost (19.4 / 60 × 0.80 = $0.26) on an L4.

Tape time-shifts. I shifted and swapped recorded action tapes, 67 variants, and all 67 failed.

ES and DIAL-MPC. The first DIAL-MPC pass moved no seed. With the rollout fixed, the largest verified gain was (0.91607 − 0.91578 = 0.00029 s) on roller seed 1, and (0.83687 − 0.83659 = 0.00028 s) on sprint seed 1. Hill seed 4 did not move. Roller seed 1 is still (0.91578 − 0.90 = 0.01578 s) short of the 0.90 step, so that gain is 54 times short of the next 0.02 s step (0.01578 / 0.00029 = 54.4). The posted times stayed 0.904 s and 0.840 s.

Physics knobs and bird-gait knobs. One-knob edits, including neck, head, and cadence, found 0.003 to 0.6 ms per seed. Roller seed 1 moved (0.916074 − 0.916071 = 0.000003 s), which is 0.003 ms. Sprint seed 1 moved (0.836246 − 0.835603 = 0.000643 s), about 0.6 ms. Hill seed 4 moved (0.754576 − 0.754429 = 0.000147 s), which is 0.15 ms. The next step needs 14 to 16 ms. Roller seed 1 is (0.91607 − 0.900 = 0.01607 s) away, about 16 ms. Hill seed 4 is (0.754576 − 0.740 = 0.014576 s) away, about 15 ms. Scored on the Arena's own loop, those knob policies returned 0.752 s on hill, 0.904 s on roller, and 0.840 s on sprint. None of them gained a place.

A new network stayed seconds back

The same afternoon I trained a new roller policy from the start and opened up the skating rewards: faster wheels, more toe-out, a shorter glide, a deeper lean. The best checkpoint finished in 3.084 s. The leading public time that afternoon was 0.924 s, so the new network was (3.084 − 0.924 = 2.160 s) behind. Later checkpoints were 3.368 s, 3.768 s, and 10.576 s. I canceled it. It cost about (68 × 0.013333 ≈ $0.91).

A second run trained a dive task from the start. At step 0 every seed fell, and the score was 40.000 s. Checkpoints 500, 1500, and 2250 still fell on all five seeds. I canceled it after 131.8 minutes. The cost was (131.8 × 0.80 / 60 = $1.76).

How the table search was scored

MuJoCo Warp sets enable_backward=False in 21 of its 31 non-test modules, and the Arena scores on CPU MuJoCo with ONNX Runtime, so the race step has no usable backward pass. I estimated the slope from the race itself, with SPSA. Each table value gets a random sign, Δ = ±1. I push the whole table by +cΔ and run the race, then by −cΔ and run it again.

g ≈ (t(x + cΔ) − t(x − cΔ)) / (2c) · Δ

Two races give a slope for every value. The wider runs moved 468 values together: ((54 − 37 + 1) × 13 = 234) mid-race values, (18 × 10 = 180) body values, and (18 × 3 = 54) head values, and (234 + 180 + 54 = 468).

The 0.01 rad and 0.005 rad radii are separate restarts: 0.01 rad with learning rate 0.002, and 0.005 rad with learning rate 0.001. The stacked table was sharper than those radii. A random step of 0.001 rad made the median finish 0.0058 s worse, and a step of 0.01 rad made the median finish 0.0218 s worse. The line search on that stacked table used 0.0003 rad, 0.0002 rad (2e-4), and 0.0001 rad.

The Hugging Face compute spend, with no jobs running at 10:03 CT, was (94.76 − 1.27 = $93.49). The 0.904 s roller row is a composed table. The Arena scored that entry on its own CPU, so the entry added $0.

Credit to jrubiosainz, DenRakEiw, and dashi. The 0.904 s roller row uses game-roller-remix6 and senate_duck_0904. Sprint uses sprinter-remix4. Hill uses hillclimber-remix5.