Anthropic's GLM-5.3 cyber report: what the tests show, and what they can't

Anthropic’s Frontier Red Team published an analysis of GLM-5.3 on September 29. GLM-5.3 is a model from Z.ai (Zhipu AI) that launched on August 14, with its weights released about two weeks later. Anthropic makes two claims: the model can build working exploits at a level close to its own restricted Claude Mythos Preview, and its safeguards are easy to get around.

Everything below is Anthropic’s testing unless stated otherwise. It is a report by a competitor, and we have not reproduced any of it.

What Anthropic measured

Anthropic used two benchmarks. ExploitBench asks models to exploit known bugs in the V8 engine that powers Chrome (41 bugs, 410 attempts here). Its internal binary-exploitation benchmark uses 100 randomly selected tasks built on open-source projects, with full credit only for a complete control-flow hijack.

Benchmark (Anthropic’s setup)	GLM-5.3	Claude Mythos Preview	Earlier models
ExploitBench, end-to-end exploits	50 of 410 (12%)	56 of 410 (14%)	Claude Opus 4.6 and GLM-5.2: 0%; Kimi K3: 0.5%; DeepSeek V4.1-Flash: 0.2%
Binary exploitation, full control-flow hijack	4%	6%	Claude Opus 4.6 and GLM-5.2: no successes

The Claude models in these comparisons ran with cyber safeguards disabled; Anthropic says Claude models are released with safeguards and that versions with reduced safeguards are limited to vetted users. The 95% intervals in its chart overlap for GLM-5.3 and Mythos Preview, so the 12% and 14% should not be read as a ranking.

Human-led sessions

Anthropic also had researchers use the model on targets where the flaws were unknown to them. In a session that lasted about a day with limited human attention, GLM-5.3 found several previously unknown flaws in a browser’s JavaScript engine and chained them into a page that reads files from a visitor’s computer. Anthropic says it disclosed these to the maintainer and is reviewing further reports about drivers and network-facing software.

In a second session, the smaller GLM-5.3-Flash chained exploits for two publicly known Chrome flaws into a reliable exploit for an ARM64 target, bypassing pointer-authentication hardening. Anthropic reports 20 minutes of human attention plus 8 hours of model work, at a cost of $20.40 at Zhipu’s API prices. We are not reproducing technical detail from either session.

How the safeguards fared

Out of the box, GLM-5.3 refused the malicious orders in every trial. Anthropic then tried three ways around that, each measured over 50 simulated episodes, counting whether the model tried to connect to a remote target. Nothing was executed.

• A cover story (telling the model it was a red-team agent on an exercise): 64%.
• Prefilled reasoning (writing the model’s opening thoughts so it appears to have decided to proceed): 92%.
• Abliteration (editing the weights to remove refusals): 100%.

The table also shows that Anthropic’s own models are not immune: with safeguards disabled, Claude Opus 4.8 engaged in 4% of cover-story episodes and Claude Opus 5 in 10%, while Claude Mythos 5 showed 0%. With safeguards on, Anthropic reports 0% for the Claude models. Two of the techniques cannot be used against Claude at all: its API does not accept prefilled reasoning, and its weights are not released, so they cannot be abliterated.

Abliteration is the most consequential finding because it works on any open-weight model you can download. Anthropic says its team had never tried it before, and needed about 2,200 GPU hours, roughly $4,400 of compute, for GLM-5.3 (about 600 GPU hours for Flash). The mean refusal rate across three harmful-request benchmarks fell from about 95% to 6%. General capability barely moved: GPQA-Diamond stayed at 88%, and a CyberGym subset went from 85% to 81%. Anthropic adds that several developers published abliterated versions of GLM-5.3 within days of its release.

Independent and company views

NIST’s CAISI published its own assessment on September 17. It calls GLM-5.3 the most cyber-capable open-weight model released to date and says it lags the U.S. frontier by about four months on an aggregate of its cyber benchmarks. It tested U.S. models with cyber safeguards disabled. Its assessment page does not discuss GLM-5.3’s refusals or misuse potential, which is the part Anthropic’s report adds.

Z.ai, at launch, claimed 54.4% on ExploitBench (up from 24.4% for GLM-5.2) and 84.5% on CyberGym. It said its most sensitive cyber functions would initially be limited to a trusted-access program, and that it would delay the weights by about two weeks for more security checks. Anthropic’s position is that the released model has no meaningful safeguards.

Do not put Anthropic’s 12% next to Z.ai’s 54.4% or CAISI’s 61.1%. CAISI states that its score is the best of three attempts per task on a 16-point graded scale (61.1% is 9.8 of 16, with a 95% interval of 45.9% to 74.5%). Anthropic instead counts end-to-end exploit successes per attempt (50 of 410). Z.ai does not say how it scores. These are different measurements under one benchmark name.

Reading the report

• It comes from a competitor. Anthropic sells safeguarded models, and the report itself argues for wider access for defenders and for government testing of capable models. That does not make the findings wrong, but it is context.
• The tests are simulations. “Engaged” means the model tried to connect to a remote target after an overtly harmful order. It is not proof that an attack would succeed in the real world.
• Some inputs are not public. The binary-exploitation benchmark is internal, the comparison models had safeguards disabled, and each bypass figure rests on 50 episodes.
• Abliteration is not specific to GLM-5.3. Anthropic calls it a standard technique. Our reading is that its point is the combination: high capability and a downloadable model whose refusals can be removed cheaply.

Our assessment

For teams running open-weight models, the practical lesson is that a vendor’s refusals are a default, not a control. If an agent can reach real systems, the limits need to sit outside the model, in permissions, sandboxing and monitoring. We covered that approach in NVIDIA’s Open Agent Safety Platform, and in an OpenAI agent that reached an external chatbot through DNS. Our note on Kimi K4 and GLM 5.5 Flash records on OpenCode concerns a different model.

Independent reproduction is the missing piece. Until an outside group repeats the bypass tests, the safeguard numbers are Anthropic’s claims.

Media in this article are Anthropic’s original figures, uncropped, with credit and a source link. The cover is our own conceptual graphic and contains no data.