Open AI Models Close the Cyber Capability Gap With Restricted Systems

Open-weight AI models are approaching restricted systems in autonomous cyber operations, according to a Vals AI benchmark reported by The Decoder. OpenAI’s GPT-6 Sol Daybreak completed 80.8% of simulated attack objectives, while the freely available DeepSeek V4.1 Flash reached 72.8% at a reported cost of about $3 per run versus roughly $22 for the OpenAI system.

The result matters because the security distinction between proprietary and openly released models is becoming less useful as a predictor of cyber risk. A model whose weights are public cannot be recalled or centrally constrained by its developer, while its lower operating cost may make extended attacks more practical for both researchers and criminal groups.

The findings are reported rather than independently verified in the available source material. They come from Vals AI’s CyberOffense benchmark, as described by The Decoder, and should be read as a signal about model capability under a particular test design—not as evidence that either system can reliably compromise real-world companies.

What the Vals AI benchmark tested

The benchmark places each AI agent on an isolated attack machine and gives it only a public web address. The agent receives no human assistance while attempting to move through a simulated corporate network. This setup is intended to test whether a model can discover vulnerabilities, obtain credentials, escalate access, and pursue objectives across connected systems.

In a harder scenario called Lighthouse, The Decoder reports that GPT-6 Sol Daybreak took administrative control of one simulated company, extracted passwords, and used them to access a second company. It completed 27 of 31 objectives in less than three hours. DeepSeek completed 21 objectives, according to the same report.

Those figures are not a direct measure of real-world breach rates. They do, however, highlight a capability that conventional chatbot evaluations often miss: the ability to plan across multiple steps and continue operating when early actions fail.

Runtime and cost changed the comparison

One of the most important details in the results was the role of runtime. DeepSeek reportedly produced no successful outcomes during its first 25 million tokens and made most of its progress only after passing 100 million tokens. Short evaluations could therefore understate the performance of an open-weight system that is allowed to keep trying.

That has practical consequences for defenders and AI product teams. A model that is less effective per minute may still be attractive to an attacker if it can operate cheaply for hours or days. Vals AI’s reported cost comparison—about $3 for a DeepSeek V4.1 Flash run and $22 for GPT-6 Sol Daybreak—suggests that price, not just peak capability, will shape how these systems are deployed.

The Decoder also cited a July analysis from the UK’s AI Security Institute, which estimated that the gap between open and closed frontier models in cyber capabilities had narrowed to four to seven months, down from six to ten months for most of 2025. That is an institute analysis as reported by the publication, not a new result from the Vals benchmark.

Evidence remains limited and source-dependent

The strongest claims in this story are benchmark and vendor-reported claims rather than independently reproduced findings. The available evidence identifies Vals AI as the source of the CyberOffense results, but it does not provide the full benchmark methodology, model configurations, prompts, or replication data. Those details are essential for comparing systems fairly.

The results may also depend on tool access, token budgets, network design, scoring rules, and how much time each agent receives. DeepSeek’s late gains make the token and time budget particularly important. Without standardized reporting, a model can appear weaker or stronger depending on whether an evaluation favors rapid progress or persistence.

The Decoder further reported that an Anthropic study found the freely available GLM-5-3 capable of building exploits at nearly the level of an internal model called Mythos. That comparison is another study-specific claim and should not be treated as a universal ranking of open and closed systems.

Why builders and enterprises should care

For security teams, the immediate implication is that defenses need to account for patient, automated workflows rather than only fast attacks. Monitoring should focus on unusual sequences of reconnaissance, credential access, privilege escalation, and lateral movement. Testing should also include long-running agents with realistic tool access, because short sessions may miss the behavior highlighted by Vals AI.

For AI builders, the results create a two-sided engineering problem. The same capabilities that allow an agent to identify exploitable paths can support vulnerability discovery, patch validation, and defensive triage. Vals AI reportedly argues that offensive models can help companies find and fix weaknesses first. Turning that possibility into a safe product requires strict authorization boundaries, isolated environments, audit logs, and controls that prevent defensive tooling from becoming a general-purpose intrusion system.

Enterprise buyers should be cautious about treating closed APIs as automatically safer. Provider controls and monitoring can make restricted models easier to govern, but the reported cost and capability of open-weight systems mean organizations may encounter powerful cyber models outside a provider’s control. Procurement reviews will need to cover not only which model a company uses, but also what models employees, contractors, and attackers can access.

What to watch next

The next useful signals will be independent reproductions of the Vals AI CyberOffense results, including full scoring rules and equalized token budgets. Researchers should also report performance across different network layouts, tool permissions, and time limits rather than relying on a single headline score.

Defenders should watch for longer-running AI-assisted intrusion campaigns, new evaluations of open-weight models after extended execution, and evidence that automated agents can move from simulated networks into poorly protected production environments. On the product side, meaningful progress will include better authorization controls, clearer audit trails, and defensive agents that can demonstrate value without unrestricted access to corporate systems.

Creati.ai perspective

The central development is not that open models have definitively matched closed models. It is that the remaining gap may no longer provide much comfort when an open system is cheaper, persistent, and difficult to restrict once released. Capability, operating cost, and control are becoming separate variables in cyber risk.

For AI companies and enterprise teams, the sensible response is better measurement and stronger deployment boundaries—not panic over a single benchmark. The Vals results are an important warning, but their real significance will depend on whether independent testing confirms that open models can sustain these results across varied, realistic environments.