One email a day: the top topics, each with the angle and the story, before you plan content. The daily report isn't open yet — leave your email and we'll tell you when it opens.
Image: runta.com
Cooling
In 27,280 coding agent runs, 64% of inference spend went to runs that failed
Heat 46 · #36 in AI · Cooling2 reports · 2 sources · First reported Nov 2026 · Updated Nov 20
The story
In an evaluation of 27,280 coding-agent runs, 64.5% of list-price inference spending went to trials that did not pass. The FrontierHarness Eval v1.1, run by Runta with Baseten, consumed 83 billion tokens across 22 harnesses, five open-weight models and 62 tasks, with four attempts at each task. Failures accounted for 52.7% of the wasted spend and timeouts for another 11.8%. The cheapest harness depended on the model: Codex CLI ranked first with GLM-5.3 but only 16th with Kimi-K3. The study also found that failed runs ended with a confident answer 68-82% of the time.
Latest development
same post, same pts/cmt as round 163
How it unfolded
same post, same pts/cmt as round 163
same post, same pts/cmt as round 162
stays active, exactly at the 66.0 line)
snapshot fetch for this item timed out, re-fetched directly from the HN API
studio.aisa.one v1.1 item is an AI-written summary, per AiNews247-mirror precedent not counted as a platform), cross stays 30
search: no independent reporting of the v1.1 finding (thebeat.tech covers v1.0
failed runs end with a confident answer 68-82% of the time
cheapest harness per task changes with the model (Codex CLI 1st with GLM-5.3, 16th with Kimi-K3)
64.5% of list-price inference cost went to trials that did not pass (52.7% failures + 11.8% timeouts)
Runta blog introducing FrontierHarness Eval v1.1 (with Baseten): 83B tokens across 27,280 agent runs, 22 harnesses x 5 open-weight models (GLM-5.3, GLM Flash, DeepSeek V4.1 Flash, Muse Spark Open, Kimi-K3) x 62 tasks x 4 attempts
The angle
64% of the money spent on AI coding agents goes to runs that fail - and the cheapest setup for one model is nearly the priciest for anotherSuggested format: Short video · for Developers using AI coding agents