MIT CSAIL proposes weight-space programming, GPT-6 Astra lifts Qwen MATH-500 from 7.9 to 48.1 percent
MIT CSAIL had GPT-6 Astra inspect Qwen3.5-0.8B weights and modify about 0.016 percent of parameters, raising MATH-500 from 7.9 to 48.1 percent and IFEval from 21.0 to 62.3 percent.
A set of experiments newly published by the MIT CSAIL team has made it reality for large models to directly modify neural network weights. The team had GPT-6 Astra inspect the internal weights of Qwen3.5-0.8B, locating the key problem causing the model to fall into repetitive reasoning loops, after which Astra modified about 0.016 percent of parameters, raising the MATH-500 score from 7.9 percent to 48.1 percent, GSM8K from 13.8 percent to 67.1 percent, and IFEval, which measures instruction following, from 21.0 percent to 62.3 percent. No retraining and no gradient descent were involved.
Mechanically, Astra found the looping behaviour tied to a particular direction written by the MLP at layers 15 and 19, and subtracted that direction from the 64 output columns of each of those layers, modifying 131,072 weights in total against roughly 800 million weights in the model. The proportion of answers hitting the 32,768 token limit barely improved at 10 and 25 forward computation budgets, but dropped from 93 percent to 33 percent when the budget rose to 50.
Astra also attempted to write neural network weights from scratch, and the networks it produced for three MinAtar games beat PPO models trained for 100 million steps across all three games, scoring 20 times the opponent in Breakout when compared with a PPO model trained for 500 million steps. The team calls this capability weight-space programming.
The research also states its limits: on Gemma 3 270M, which does not fall into loops, the team failed to improve mathematical ability, with MATH-500 gains of no more than about one percentage point, showing that direct weight editing can remove obstacles to a model expressing existing ability but cannot add ability the model never had. The research blog is at rikiyat.github.io/blog/weight-space-programming.