Anthropic shipped what I was building
I'm no longer working on co-train.
The reason is simple. Anthropic shipped Code Mode in Claude Code. It does the core of what co-train was measuring and building on.
What Code Mode does
Normally an agent calls tools one at a time. It asks for a Bash command, reads the result, decides, asks for a file read, reads that result, decides again. Every intermediate result goes back into the context.
In Code Mode, Claude writes a piece of JavaScript that makes the tool calls, and Claude Code runs it in your session. One piece of code can run commands in parallel, pass one result into the next call, and return only the part of the output Claude needs to read. The rest never re-enters the context.
It is experimental and off by default. But it is in the product, built by the company that makes the model.
What co-train was about
co-train started on 20 September. The question: do agents that write a short program to call tools cost less per solved task than agents that call shell commands step by step?
I fixed the test plan in the public repo on 25 September, before the first run, and mirrored it to OSF on 7 October (osf.io/2hfzt). I could not move the goalposts.
In those pre-registered runs, code-calling agents solved at least as many tasks as shell agents, at 6-8x lower cost per solved task. The samples were small. The best run was 14 paired tasks with one rollout each, and the 6x end is a floor.
The more important part is not that number. My first big result was a +64-point win. It was my own broken instrument. I kept digging, found 13 instrument defects in total, and published the negative result on the original hypothesis alongside the cost finding.
I also reached out to 79 founders and CTOs running agents in production. The first reply came from Isotopes AI: in 392 finance-analysis sessions that needed retries, recovering from failures used 38.6% of their credits.
Checking my own premise
Before stopping, I ran one more check on the premise itself. Code-calling solved 14 of 14. Bash solved 12 of 14. At n=14 those are statistically inseparable.
So the cost result stood, and the accuracy edge was not something I could claim. Then the approach shipped as a feature of Claude Code. A startup whose core is "agents should write programs to call tools" now competes with a toggle in the tool everyone already uses.
The market answered the question
I don't read this as bad luck. I spent two months asking whether this approach matters. The model maker building it into its own product is a clear answer: yes, it matters enough to ship.
That is a good outcome for the question. It is just not a company for me.
What I keep
The habit. Agent numbers lie quietly. A +64-point win that was really a broken instrument taught me more than any clean result could. Catching that is the same skill behind my open-source fixes and the Convex security advisory, and it goes with me to whatever I build next.
The code. The co-train repo stays public as eval infrastructure, about 200 commits, with the pre-registration, the negative result and the limits published next to the numbers.
What's next
Two things. I'm looking for an internship at a VC-backed startup while I figure out my next venture. And I'm starting the search for that next company from a blank slate.
More on the internship in the next post.