The story
Researchers report that a 27-billion-parameter open language model can rewrite its own weights by its own reasoning and become better, with no gradients and no training run. Given its weights, its graded answers and the ability to run itself, the model diagnoses its own failure and edits the weights behind it; its own edit makes it 6.6 points more accurate on 490 problems it never saw, rising from 57.2% to 63.8%. The same approach repairs a damaged copy of the model, lifting it from 7% to 70% accuracy on six image benchmarks it never saw, without ever reading the undamaged weights. The ability emerges only with scale: across four sizes of one model family, the 2-billion-parameter model makes no edit and the 4- and 9-billion-parameter models break themselves, while only the 27-billion-parameter model works. The work builds on earlier weight-space programming research, in which a frontier model edited the weights of a much smaller model.Latest development
builds on the weight-space programming line in which a frontier model edited a smaller model.