WTF is Weight-space Programming?
An LLM reads and directly edits another neural network's weights to meet a plain-English goal, with no training run.
Also called: weight space programming, wsp
TL;DR
Normally you change what a model does by training it, i.e. letting gradient descent nudge millions of numbers. Here a frontier LLM instead looks at the raw weights, reasons about what each part does, and rewrites specific numbers by hand, like a programmer patching code. In the first experiments it fixed a small model's habit of looping by changing 0.016% of its weights.
How it works
The authors define it as a general-purpose LLM reading and writing a network's weights directly around a natural-language spec, explicitly not through an optimiser or gradient descent. The editing model gets the target's weights, a task description and a Python sandbox, and may run a small budget of forward passes (0–50 in the experiments) to test its edits. Results included identifying what a tiny policy network had been trained on from its weights alone, writing game-playing networks for MinAtar, and patching repetitive reasoning in Qwen3.5-0.8B: subtracting a repetition-related direction from MLP output projections in two layers lifted MATH-500 from 7.9% to 48.1%. Stated limits: large models don't fit in context, and it worked best on single, localised faults; some tasks failed outright. The authors frame it as an early research signal, not yet a tool.
Origin
- Coined / introduced by
- Takehi, Teoh, Zhang & Khattab (MIT CSAIL)
- When
- 2 Nov 2026
- Story
- Named in the blog post 'Sparks of Weight-Space Programming'. The authors point to recursive self-improvement and model auditing as possible future directions.