Anthropic, Google, Microsoft Unveil First AI Jailbreak Scoring Scale

In a significant move for AI safety, leading tech giants including Anthropic, Google, and Microsoft have introduced a draft of the first-ever AI jailbreak scoring scale. This new framework aims to standardize how the industry assesses and responds to model vulnerabilities. The initiative represents a rapid shift from concept to draft policy.

Drawing on data from bug bounty programs, the proposed scale is designed to help major AI labs triage model vulnerabilities with the same rigor and clarity as the Common Vulnerabilities and Exposures (CVEs) system used in software security.

Draft AI Jailbreak Severity Scale

The proposed framework is a standardized system for measuring the severity of AI model jailbreaks. Developed by a coalition of major tech labs, it uses a five-tier logarithmic scale to rank vulnerabilities based on capability gain, breadth, and weaponization ease.

Led by a coalition comprising Anthropic, Amazon, Google, Microsoft, and reportedly OpenAI, the proposal introduces a five-tier, logarithmic scale that mirrors the logic of the established CVSS. The draft severity labels are structured as follows:

• Level 0: informational tactics that add no capability gain
• Level 1: low impact, limited breadth
• Level 2: medium impact with broader reuse
• Level 3: high impact and high weaponisation ease
• Level 4: critical impact, easy to discover and repurpose

Scores are calculated based on key axes including capability gain, breadth, ease of weaponization, and discoverability. The framework's development is influenced by regulatory considerations, with government agencies seeking oversight mechanisms for frontier AI models.

Early Lessons from Anthropic's HackerOne Bounties

Anthropic has been gathering data for its severity metrics through public red-teaming contests. A notable challenge on HackerOne involved security researchers testing the company's Constitutional Classifiers. While several teams successfully bypassed multiple jailbreak levels and earned significant rewards, a recap on the HackerOne blog confirmed that no participant discovered a "universal jailbreak" against the updated system.

Subsequent bug bounty rounds have offered substantial prizes for any universal technique. Over extensive red-team testing hours, only a limited number of high-risk vulnerabilities have been confirmed. Based on these results, Anthropic's internal metrics indicate its classifiers significantly reduced jailbreak success rates, though these figures are self-reported by the company.

The insights from this paid testing have informed the layered defenses in Claude's newer models. For instance, requests identified as relating to cyber, bio, or model-extraction topics are automatically rerouted to older model versions. This hand-off mechanism reportedly allows the vast majority of user sessions to remain on newer models without intervention. However, independent research shows that adaptive "tree-of-attacks" prompts can still achieve some bypass success, underscoring the ongoing need for external controls.

For security teams evaluating the proposed framework, these findings highlight key operational takeaways. Effective strategies include implementing input-output filtering before the LLM, using behavioral logging to flag users who repeatedly rephrase prompts after a refusal, and applying modest timeouts that increase with each failed attempt to deter automated attacks. These practices align with emerging security recommendations and suggest how severity scores may eventually map to specific, actionable mitigations.

---

What is the proposed AI jailbreak severity framework and how does it work?

The proposed AI jailbreak severity framework is an industry standard designed to bring consistent, quantifiable measurement to AI jailbreak risks. Modeled on the Common Vulnerability Scoring System (CVSS) used in traditional cybersecurity, it introduces a five-tier severity scale ranging from Level 0 (Informational) to Level 4 (Critical).

Each jailbreak is scored across multiple distinct axes:

• Capability gain - how far the technique extends beyond existing attacker tools
• Breadth - the range of offensive tasks it enables
• Ease of weaponization - how readily it converts to working attacks
• Discoverability - how easily threat actors could find or derive it independently

The scale is logarithmic, meaning each band represents a qualitatively more serious risk rather than incremental escalation. This structure allows security teams to prioritize responses using familiar triage logic borrowed from software vulnerability management.

Which companies are collaborating on this framework?

The framework represents a cross-industry coalition led by Anthropic in partnership with Amazon, Microsoft, Google, and OpenAI. This collaboration operates as a voluntary industry initiative with consideration for regulatory oversight.

The agreement includes provisions for government agency review of covered frontier models before public deployment.

How effective has Anthropic's HackerOne program been at discovering jailbreaks?

Anthropic's bug bounty program on HackerOne has yielded significant insights into model vulnerabilities while validating defensive improvements. In their Constitutional Classifiers challenge:

• Many security researchers contributed extensive collective hours
• A large number of chat interactions were generated
• Several teams successfully passed multiple jailbreaking levels
• Substantial rewards were distributed

Most importantly, no universal jailbreak (a single strategy bypassing all safeguards) was discovered for the updated classifier system. Follow-up programs have similarly found very few universal jailbreaks, with only a small number of high-risk vulnerabilities identified.

The program demonstrated that Constitutional Classifiers significantly reduced jailbreak success rates - blocking the vast majority of attempted attacks.

What specific mitigation techniques is Anthropic deploying?

Anthropic has implemented a multi-layered defense architecture centered on intelligent model routing and adaptive monitoring:

Safety classifiers with automatic handoff: When newer models detect requests related to cybersecurity, biology/chemistry, or distillation, responses are automatically routed to older model versions instead. Users are notified of this fallback, which occurs in a small percentage of sessions.

Invisible safeguards: For other categories, newer models employ prompt modification, steering vectors, and parameter-efficient fine-tuning to subtly degrade harmful output capability without visible refusals that attackers could exploit.

Automated red-teaming: Internal frameworks continuously mutate and reframe prompts to identify universal jailbreaks before external discovery.

However, research reveals remaining vulnerabilities: adaptive "tree-of-attacks" methods still bypass safeguards in some cases for harmful intents.

When will the framework become officially available?

The first draft has been published, with an announcement formalizing the full standards expected soon. This timeline reflects accelerated development driven by regulatory interest rather than purely voluntary industry coordination - government oversight signals that adoption will likely outpace typical standards-body processes.