Skip to content
NewsResearch

SaferAI evaluation finds Zhipu AI's GLM-5.2 refused zero offensive cyber and biology tasks despite near-frontier capability

· by Pondero Newsdesk

The short version

SaferAI's August 2 evaluation of Zhipu AI's GLM-5.2 found the open-weight model refused zero offensive cyber and biology tasks, while Claude Opus 4.7 refused so consistently the team could not complete the CyberGym benchmark against it.

SaferAI evaluation finds Zhipu AI's GLM-5.2 refused zero offensive cyber and biology tasks despite near-frontier capability

A European AI safety nonprofit ran the first EU Code of Practice-style evaluation of Zhipu AI's open-weight GLM-5.2 and found the model completed every offensive cybersecurity and dual-use biology task it was given. The contrast with Claude Opus 4.7 was stark: SaferAI's team found Anthropic's model refused tasks so consistently that they could not complete the CyberGym benchmark against it at all.

What the evaluation found

SaferAI published its GLM-5.2 Risk Evaluation Report on August 2, 2026. A team of six researchers, including Chinmayi Dixit and Jacob Davies, ran the tests from Z.ai's public API with no cooperation from Zhipu AI. The four risk categories tested match the EU General-Purpose AI Code of Practice: Loss of Control, Cyber Offense, CBRN (chemical, biological, radiological, and nuclear), and Harmful Manipulation, per the SaferAI report.

Capability-wise, GLM-5.2 trailed frontier closed models by roughly two to four months depending on the domain. On the Cybench offensive cybersecurity benchmark, it scored within the confidence intervals of Claude Opus 4.7 and GPT-5.5. Its CyberGym task reproduction rate climbed from 36.6% to 76.2% as the token budget rose from 2 million to 50 million tokens per task. On LAB-Bench and BioMysteryBench for biological knowledge, GLM-5.2 matched Opus 4.7 and GPT-5.5 and met or exceeded the human expert baseline on every LAB-Bench subtask, per the SaferAI report.

GLM-5.2 refused none of the offensive-security or biological tasks in the evaluation. SaferAI also found it attempted persuasion on conspiracy and control-undermining topics more readily than the comparison frontier models and could be pushed into harmful outputs under direct pressure.

Zhipu AI published no safety framework, pre-deployment testing commitments, or risk assessment for GLM-5.2 at launch. Because the model weights are publicly downloadable, any filters present on Z.ai's API can be stripped by anyone running the weights locally.

Why it matters

The open-weight distribution is what separates this from a typical safety gap between two closed models. A closed model with robust refusals creates friction even when that friction can theoretically be bypassed. A model roughly four months below GPT-5.5 on cyber offense, fully downloadable and with no enforced refusals, removes that friction for any operator with enough compute to run local inference. SaferAI's CyberGym finding puts that scenario on paper with citable numbers for the first time for this model class.

For AI-tool operators doing security work or building on open-weight models, the practical takeaway is a policy question: does your organization's acceptable-use framework differentiate between API-hosted and locally-run weights of the same model? The SaferAI report provides a benchmark result to anchor that internal conversation, per TechCrunch.

What to watch next

The EU AI Office is the natural audience for this report, framed explicitly around Code of Practice criteria. A formal EU AI Office response would test whether those standards can reach non-EU labs. On the vendor side, Zhipu AI's next open-weight release will indicate whether external evaluation pressure produces any safety documentation from the lab. The CyberGym token-budget result also flags a compounding issue: as inference costs fall, GLM-5.2's effective capability ceiling rises without any change to its weights.

Sources