133rd of 1,295 in a GPU kernel challenge, on $12.68 of compute
The OpenKernel Challenge asks one thing: make the kernels that serve Muse Spark Open, a mixture-of-experts model running on 8×H200, faster than SGLang's. Five leaderboards, one per piece of a layer. Every entry is timed against SGLang on the same GPU. Your score is the speedup.
I had four days and one rule for myself: a speedup doesn't count until I've tried to break it.
Days
4
Leaderboards
5
Overall rank
133 / 1,295
Rented GPU spend
$12.68
My kernels were built on public baseline solutions; the rank reflects my changes on top of them.
Dark bar: my best entry. Tick: the #1 entry. Rank out of everyone with a scored entry on that board. Provisional: the organizers rerun top entries on held-out data before final positions.
Projections is my best rank and the one I'm least proud of. More on that below. Prefill is my worst: top 55%.
The five boards
| Board | What the kernel computes |
|---|---|
| MoE experts | Top-8 of 256 experts per token. NVFP4 weights, SwiGLU, a norm on each expert's output, weighted sum. |
| Attention | One decode step or an 8-token verify block over a paged FP8 cache. 2,048-token sliding window or full context. |
| Decode layer | The whole decoder layer at decode sizes: projections, attention, router, experts, every norm and residual. |
| Projections | The BF16 matrix multiplies around attention and the experts, plus an FP32 router. |
| Prefill layer | The whole layer at prefill sizes: 753 new tokens on an 80K context, up to 16K-token chunks. |
Decode shapes are tiny: batches of 1, 16 or 32 requests. At that size the GPU spends more time waiting on launches and memory than doing math. Most wins below are about removing waits, not adding FLOPs.
Days one and two: my own kernels
I started from scratch on every board. Agents wrote candidates, I ran them through the gate further down, and the good ones went public. MoE went from 1.41x to 1.63x. Decode from 1.41x to 1.54x. Attention reached 1.60x.
Then stronger open-source kernels showed up. On MoE the gap to mine was 13%.
Two options. Keep polishing my own kernels and stay 13% behind, or rebase onto the best available code and spend the last two days on changes nobody had made yet. I rebased. So to be clear about what's mine: on MoE, attention, decode and prefill, the starting point is open-source. The improvements on top are what I built.
Public leaderboard scores. The middle bar is the starting point I rebased onto; the gap between the middle and dark bars is the part I built on top.
Rebasing wasn't a copy-paste. Several of the faster public kernels were fast partly because they skipped work the rules require. The prefill one kept intermediate values in FP32 where the reference rounds to BF16, and it compiled code inside the timed call. The attention and decode ones needed a BF16 rounding step before the FP8 cache write. Each of those is a disqualification at final review. I fixed them first, then started optimizing.
On projections, nothing I built beat the best open kernel. My best attempt tied it (1.52x against 1.51x, inside the noise) and the next one lost (1.40x). That board runs it unchanged, so the rank there isn't something I'm claiming.
What I changed
| Board | Change | Measured |
|---|---|---|
| Attention | Reworked the full-context verify path | Batch 16: 154.5 → 131.6 µs. Batch 32: 306.9 → 242.9 µs |
| Attention | Sliding-window kernel issues all its loads before any math, then tuned heads per block | 1-token decode: 8.96 → 7.74 µs (1.16x) |
| Attention | Removed a dependency between kernels | +1.8% board |
| MoE | Split-K on the second multiply for small batches | +2.6% board |
| MoE | Second multiply starts tile by tile as each tile of the first finishes (per-tile arrival counters, programmatic dependent launch). The last block to arrive does the final sum, in a fixed order | 1-token decode 1.09x, board +1.6%. Bit-identical across 2,000 replays |
| MoE | Faster routing pass for the large verify batches | +1.05% board |
| Decode | Router moved to a side stream, fused attention kernel ported into the layer, MoE changes ported over | 1.754x → 1.838x |
| Decode | Query kept in BF16 into the attention kernel | Worst error 0.79 → 0.35 of tolerance |
Small numbers. At the top of these boards, a 1% gain is a good day.
The gate
Most of the four days went here, not into kernels. A GPU benchmark lies easily: a stale input, a warm cache, an eager timing that loses under CUDA graphs, a race that happens to pass. So every candidate went through the same steps before it got near the real judge.
- 1
Candidate
One agent, one directory, one idea. No two writers on the same file.
- 2
CPU checks
Shapes, dtypes and protocol checks on a laptop. Free, catches the dumb failures.
- 3
A/B on an H200
My own judge-like harness: CUDA-graph replays, every scored shape plus nearby ones, against the current best.
- 4
Must-fail probe
Break the kernel on purpose (drop a counter reset, inject a delay, poison with NaN). The harness has to go red.
- 5
Soak
2,000 graph replays. Outputs must match bit for bit, run after run.
- 6
Review
A separate agent reads the diff against the rules: precision boundaries, no caching, no work outside the timed call.
- 7
Private check
The official judge on fresh inputs, off the board. Limited per day.
- 8
Submit
Only above the gate, with a confidence interval that clears the current public score.
Step four is the one that earned its keep. The MoE change uses counters that must reset between calls. I built a version with the reset removed and ran it through my own harness. If that version passes, the harness can't see the bug, and every green result before it means nothing. It failed loudly: NaN outputs, relative error 1.34. Only then did I trust the green run on the real version.
The probes needed checking too. On a later version, a reviewer agent pointed out that my remove-the-reset probe could never fail: the same inputs produced the same intermediate values, so stale counters looked correct by accident. I switched to NaN poisoning, which fills the buffers with garbage first. Then the probe failed where it should, 75 of 200 replays wrong on the 1-token shape.
Precision is part of the score
The judge checks every output against a tolerance. After the challenge closes, the organizers rerun top entries on held-out data. So I ran my decode kernel on 51 random seeds and measured the worst case.
Error as a fraction of the judge's tolerance. At 1.0 the run fails. Same speed before and after.
It was using 79% of the allowed error. One unlucky input away from failing a rerun. Keeping the attention query in BF16 brought the worst case to 35%. Speed didn't change. I submitted it anyway: the leaderboard number stayed flat, the risk of losing the whole board dropped.
The flip side happened once. A faster decode candidate failed the official check on one input, about 5% over: 0.0502 against a bound of 0.0476. Re-running until a lucky seed passes is easy and pointless, because the held-out rerun will find it. I dropped it.
What didn't work
| Idea | Result |
|---|---|
| Bulk producer ring for the MoE experts | 0.80x. 24% slower |
| Tensor-core (WGMMA) path for MoE | 1.30x board, below my own 1.63x |
| Bounded L2 weight prefetch in decode | 1.725x vs 1.754x |
| MoE experts kernel reused inside prefill | 650 µs vs 609 µs on the 753-token shape |
| Rewritten projection kernels | 1.52x (a tie) and 1.40x vs 1.51x |
| Remaining launch-overlap edges in decode | +0.07% forecast. Closed |
| Next sliding-window attention variant | +0.19% forecast. Parked |
Most ideas lose. The gate's job is to make them lose cheaply, before they cost a submission.
What it cost
$12.68 of rented H200 time over four days, with a hard daily cap. The official judge allows a fixed number of checks per board per day, so my own harness did the exploring and the judge only saw finalists.
The agents did most of the typing. My job was the gate: deciding what counted as evidence, killing candidates that couldn't prove themselves, and refusing to submit anything I couldn't break first.
What I'd keep
- Compete on the delta. If someone's code is better than yours, start from it and build what's missing. Pride costs leaderboard places.
- Write the test that must fail. A green result from a harness that can't go red is decoration.
- Precision margin is a score. A kernel at 79% of tolerance is one bad input from zero.
- Kill ideas fast. Every row in the failure table died in my harness, not on the public board.
Final standings are provisional until the organizers finish their reruns.