MIT used GPT-6 Astra to edit Qwen weights, lifting math score sixfold

Researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) have reported using OpenAI's GPT-6 Astra to directly edit the weights of a small open-source Qwen model, raising its MATH-500 score from 7.9% to 48.1%, according to VentureBeat. The result matters because it suggests a different route to improving models — not by training on examples, but by having one model inspect and patch another model's internal parameters.

What did the MIT researchers do?

According to VentureBeat, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab described the work in a research blog post published on November 2. They call the approach "weight-space programming." In their experiments, Astra was asked to interpret and alter a target network without using conventional training procedures to calculate the edits.

That is the central change in method. Conventional fine-tuning and other post-training approaches use examples, feedback or rewards to calculate changes to a model's weights, the numerical parameters that govern its behaviour. Retraining can update an existing model or rebuild one from scratch. In the MIT experiments, as reported by VentureBeat, a separate model instead inspected the target network, investigated its behaviour and devised direct changes to its weights.

The researchers tested whether Astra could do this in different settings. In some experiments, weights were supplied as text. In others, Astra was given a controlled Python environment so it could inspect larger networks. VentureBeat quotes Takehi writing on X that, instead of training, models may treat raw weights like readable, editable code, diagnosing failures and patching them directly. The team compared the idea to an AI debugger for neural networks, while noting that this remains a potential workflow rather than an established commercial capability.

How much did the Qwen model's scores change?

The strongest language-model example involved Qwen3.5-0.8B. According to the experiment documentation reported by VentureBeat, Astra's intervention raised its MATH-500 accuracy from 7.9% to 48.1%. The same edited model also improved on other measures under the researchers' evaluation conditions: GSM8K accuracy rose from 13.8% to 67.1%, IFEval instruction-following accuracy rose from 21.0% to 62.3%, and the share of MATH-500 responses reaching the token limit fell from 93.0% to 33.1%.

The patch was small in terms of the number of weights changed. VentureBeat reports that it touched approximately 0.016% of the model's weights, specifically 131,072 weights. The authors attributed the gain to Astra removing a numerical direction associated with repetition from selected components in layers 15 and 19.

Finding that patch, however, required substantial investigation. The strongest attempt allowed 50 evaluation requests, each potentially containing hundreds of examples. VentureBeat reports that the attempt involved 158 conversational turns, 80 Python runs and 20.3 hours, including 4.7 hours waiting through a service outage. Before the intervention, Qwen frequently repeated text until it exhausted the permitted output length. The edit reduced that obstruction and allowed the model to complete more answers using abilities it had already acquired. The experiment therefore demonstrated a form of model repair, rather than establishing that Astra had inserted six times as much mathematical knowledge into Qwen.

How is weight-space programming different from fine-tuning?

Both approaches modify how a model works internally, but they differ in how they determine the modifications. Gradient-based methods, including gradient descent, adjust weights by measuring errors and calculating changes intended to improve performance. The MIT approach asked an already trained language model to examine weights, reason about their function and decide what to change.

Takehi predicted, according to VentureBeat, that neural network weights will soon be natively optimized by language models that deliberately reason about the weights. Outside commenters quoted by VentureBeat offered similarly optimistic views. Yuchen Jin, a technical staffer at Databricks and former co-founder and CTO of Hyperbolic Labs who was unaffiliated with the research, wrote on X that, if the technique became reliable, developers could gain another way to maintain models, investigate unwanted behaviour, modify behaviour and improve performance without conventional fine-tuning or retraining. Voyage AI staff researcher Chandan Singh, formerly of Microsoft AI, suggested the approach could also accelerate development by allowing capable models to modify other models, or eventually themselves.

VentureBeat notes an important limit to those claims. The researchers' approach still required computation and, in the Qwen experiment, practice problems and testing. It did not establish that direct editing would consistently outperform conventional training in speed, cost or effectiveness. Targeted weight edits could potentially reduce the time and computing resources needed for some improvements, but the researchers have not yet established those savings.

There is also a conceptual limit to the software analogy. Individual weights do not necessarily carry the clear meanings of named software functions. Astra brought capabilities acquired through its own prior training, and the researchers tested how it applied those capabilities to another network.

What else did Astra do in the experiments?

The researchers ran additional experiments to test whether Astra could recognise and implement behaviour directly through weights, according to VentureBeat. In one set, Astra identified four game-playing networks as Breakout, Pong, Pac-Man and Freeway despite their matching architectures, inputs and available actions. The researchers gave it neither tools nor a list of candidate games.

In a simulated driving task, a trained controller repeatedly became stuck at a ramp. The team supplied Astra with the controller's weights, a description of the scene and one failed trajectory. Astra returned a patch in one response. Its changes affected 3.5% of the weights and produced a reported 100% success rate.

The researchers also asked Astra to construct networks using simplified games' rules and source code, without allowing it to try the networks before evaluation. In the MinAtar experiments, Astra's Breakout network scored 517.4, compared with 25.3 for a matching network trained through proximal policy optimization, or PPO, for 500 million steps. That roughly 20-fold advantage held for Breakout. Astra trailed the 500-million-step PPO networks in Seaquest and slightly in Asterix, although it beat the 100-million-step baselines in all three games.

These experiments went beyond repairing an existing malfunction: Astra translated some game strategies into numerical networks. VentureBeat notes that the supplied rules and bounded environments left further questions about generalization.

What did the failures and critics show?

The failures limited the case for self-improving AI. A follow-up experiment with Gemma 3 270M produced no established mathematical improvement, according to the authors as reported by VentureBeat. Unlike Qwen in the researchers' setup, Gemma rarely became trapped in prolonged repetition. The account @1a3orn argued on X that this omitted negative result mattered, writing that if there is a clear flaw in a model's reasoning, language models can help edit it out, and questioning whether the work demonstrated stronger underlying reasoning.

A separate experiment exposed another limit. Astra initially failed to distinguish a model trained to agree with users' suggestions from one trained to answer correctly. It succeeded only after the researchers supplied measurements of attention during execution. Those failures showed that access to weights alone did not consistently yield an adequate explanation of behaviour or a successful intervention.

Maxime Fournes, CEO of Pause AI, an advocacy group calling for frontier AI development to stop until it is proven safe and democratically controlled, raised a different objection on X. He argued that successful weight editing could accelerate dangerous progress. VentureBeat notes that Fournes offered a forecast and a policy judgment, and that the experiments did not establish his predicted consequences.

Recursive self-improvement would require a further step: an AI's improvements would help it produce subsequent improvements, potentially strengthening the process across successive cycles. Direct weight editing could provide one route, but the researchers did not demonstrate such a cycle or establish that gains would compound. Earlier research had also explored direct edits to trained models, including the ROME project introduced in research published at NeurIPS 2022, which investigated factual associations in transformer models and made targeted weight changes.

How can you explore this in Grafy?

Grafy does not offer direct weight editing, and this work is best read as research rather than a ready-to-use Grafy feature. For readers who want to study the claim, Grafy Reader is the natural starting point: you can read, search and question PDFs and papers with AI, which makes it suitable for working through the original blog post and related papers on weight editing. Grafy Chat lets you compare how several models explain the difference between fine-tuning, retraining and direct weight edits side by side. If you want to reproduce the inspection side of the experiment at a smaller scale, Grafy Code provides an AI coding studio where you can run Python to analyse model outputs, repetition behaviour and evaluation results.

Frequently asked questions

Did Astra add new mathematical knowledge to Qwen?

No. According to VentureBeat, the authors attributed the gain to removing a numerical direction associated with repetition from layers 15 and 19. Before the edit, Qwen often repeated text until it reached the output limit. The patch reduced that obstruction, allowing the model to complete more answers using abilities it had already acquired, rather than inserting six times as much mathematical knowledge.

Did the researchers show direct editing is cheaper than fine-tuning?

No. VentureBeat reports that the approach still required computation, practice problems and testing, and that the researchers have not yet established savings in time or computing resources. The strongest Qwen attempt involved 158 conversational turns, 80 Python runs and 20.3 hours. The experiments therefore did not establish that direct editing would consistently outperform conventional training in speed, cost or effectiveness.

Did Astra improve Gemma 3 270M's mathematical performance?

No. According to the authors as reported by VentureBeat, Astra's edits produced no established mathematical improvement for Gemma 3 270M. Unlike Qwen in the researchers' setup, Gemma rarely became trapped in prolonged repetition. The result helped distinguish repairing a clear flaw from adding new capabilities, and showed that weight access alone did not consistently produce a successful intervention.

---

Source: VentureBeat — AI — 'Like gene editing for neural networks': MIT researchers used GPT-6 Astra to edit Qwen's open weights directly, boosting math performance