Open-Weight AI Models Nearly Match Frontier Models at Hacking

Open-weight models from China performed close to OpenAI's most capable cyber model in a simulated hacking experiment, according to WIRED. The result matters because it suggests closed labs may no longer hold a clear advantage in offensive cyber skills.

The research, reported by WIRED, was carried out by Vals, a company that measures AI capabilities. Vals asked different models to carry out complex, multi-step instructions on mock corporate networks. The findings point to a change that security specialists have been anticipating: capabilities once confined to frontier labs and trusted access programmes are now being approached by models that anyone can download.

For creators, developers and researchers who follow model releases, the study is less about a single benchmark score than about access. If a downloadable model can do much of what a restricted model can do, then decisions about release, pricing and defence have to be made on a different basis.

What did the experiment actually test?

According to WIRED, Vals set up two simulated scenarios with specific objectives for the models to achieve. In the first, a model had access to a mock corporate network through a public-facing website. In the second, it had to compromise one company network in order to reach a second, separate network.

That design matters because it tests sequence and adaptation, not just single answers. A model has to follow instructions across steps, respond to what it finds in the environment, and keep working toward a defined goal. WIRED describes the tasks as complex and multi-step, which is closer to how real intrusion work proceeds than a one-shot question-and-answer test.

The experiment was not a test of live corporate systems. The networks were simulated, and the tasks were set by the researchers. That limits what can be concluded about real-world attacks, but it does allow a controlled comparison between models given the same objectives.

How did the open-weight models perform?

OpenAI's Daybreak, described by WIRED as its most advanced cyber model and available only to trusted partners, was the most capable in the tests. It completed 81 percent of the tasks it was given, averaged across two tests.

DeepSeek, a Chinese open-weight model, completed 73 percent of objectives on average. That is eight percentage points behind Daybreak, and WIRED reports that Vals found the two models scored equally on some individual tasks.

Nikil Ravi, a researcher at Vals, told WIRED that frontier models kept inside labs and offered through trusted access are now starting to be matched by open models available for download. WIRED's summary of the work also placed GLM alongside DeepSeek as not far behind OpenAI's advanced cyber model, although the detailed figures reported in the article centre on DeepSeek.

The gap, then, is real but narrow. Daybreak led overall, DeepSeek was close behind, and on particular tasks the distinction disappeared. For a reader comparing model classes, that pattern is more significant than the headline percentages alone: parity on some tasks suggests the remaining advantage may depend on task type rather than a blanket superiority.

What does this mean for gated access to frontier models?

WIRED reports that the findings suggest closed models no longer have a definitive edge in cyber skills. That has direct implications for recent efforts by frontier labs to limit access in order to slow the spread of such capabilities.

The article notes that US frontier labs, particularly Anthropic, had based limited releases on the idea that gated access would buy time. Anthropic began that approach with Claude Mythos Preview in April, according to WIRED. Logan Graham, Anthropic's frontier red team lead, told WIRED at the time that its Project Glasswing, which provides trusted access to full model capabilities, was intended to put tools in defenders' hands early.

Graham also told WIRED in April that the industry needed to prepare for a world in which these capabilities would be broadly available within months or a couple of years, and that security would need to change accordingly. WIRED observes that, seven months later, that transition appears to have arrived on schedule.

The practical question is not whether gating has any effect at all, but whether it can preserve a meaningful lead for long. If open models are already close on these simulated tasks, a lab's window to use exclusive access to find and fix vulnerabilities before others do may be shorter than planned.

How much cheaper was the open-weight option?

Cost was another finding reported by WIRED. Vals said carrying out an attack cost between $17 and $46 in tokens for GPT-6-Sol, compared with between $2 and $3 for DeepSeek.

WIRED explains that open models are generally cheaper for this kind of work because they can be run on a user's own hardware rather than paid for through a proprietary API. Rayan Krishnan, CEO of Vals, told WIRED that this also gives defenders a strong incentive to use open models instead of proprietary ones.

For teams that run repeated tests, red-team exercises or large-scale vulnerability searches, that difference in token cost could shape which model they choose, even if the more expensive option scores slightly higher. A model that is a few points ahead on task completion may not be the best choice if running it at scale costs many times more.

The cost figures in the article relate to the specific Vals setup and token pricing at the time. They should not be read as a universal price for any hacking-related task, but they do show why open-weight models can be attractive for high-volume use.

What are experts still debating about the threat?

WIRED places the experiment in a wider debate about what AI has changed in cybersecurity. Much of the year, according to the article, has seen AI automate core hacking tasks for both attackers and defenders, helping find large numbers of new vulnerabilities and probe networks more quickly. Speed and scale have increased, and fixing uncovered flaws has intensified.

What remains contested is whether AI has fundamentally altered the conceptual basis of defence, or whether it mainly amplifies problems that already existed. In other words, experts are still discussing whether AI presents a new kind of threat or makes long-standing security challenges faster and larger.

WIRED also notes a gap between laboratory capability and observed behaviour. Real-world reporting on criminal and state-backed activity has repeatedly shown attackers conserving resources and using older, less capable models for most activity. Even with strict controls, defenders are told they must assume well-funded attackers may already have had some access to closed models.

Alex Zenla, cofounder and chief technology officer of Edera, told WIRED he generally agrees that open-weight models can produce advanced attacks and vulnerabilities today, especially when safety training is disabled by modifying weights. His caveat, as reported by WIRED, is that closed labs may have trained additional unreleased models beyond these capabilities, making the situation a continuing race. He described the period ahead as potentially dangerous for security.

How can you explore this in Grafy?

The most direct way to explore the topic is to compare how different models handle the same defensive or research prompt, side by side, in Grafy Chat. That lets a reader test wording, reasoning and refusal behaviour across models without assuming one model will always lead, and it makes the cost and capability trade-offs reported by WIRED easier to discuss in concrete terms.

For the research side, Grafy Reader can be used to read, search and question papers and PDFs, which is useful for following the underlying evaluation methods behind claims like those from Vals. Readers who want to understand how automated systems are built and tested may also look at Grafy Code, an AI coding studio, to explore defensive tooling and secure development workflows in a controlled environment.

None of these tools should be used to attack systems without authorisation. The value for most readers is in understanding evaluation, defence and model comparison, which is where the Vals results have their clearest lessons.

Frequently asked questions

Did DeepSeek complete as many tasks as Daybreak in the Vals tests?

No. According to WIRED, OpenAI's Daybreak completed 81 percent of the tasks on average across two tests, while DeepSeek completed 73 percent. The Vals researchers noted the two models scored equally on some individual tasks, but Daybreak remained the most capable overall in the experiment reported by WIRED.

Was OpenAI's Daybreak model available to the general public?

No. According to WIRED, Daybreak was available only to trusted partners. It was described as OpenAI's most advanced cyber model, and its restricted availability was part of the comparison with open-weight models that anyone can download.

Did Vals find that using DeepSeek cost less than using GPT-6-Sol for these attacks?

Yes. According to WIRED, Vals said an attack cost $17 to $46 in tokens for GPT-6-Sol, compared with $2 to $3 for DeepSeek. WIRED reported that open models can be cheaper because they can be run on a user's own hardware.

Source: WIRED — AI

---

Source: WIRED — AI — Free Chinese AI Models Are Getting Scary Good at Hacking