Google: AI Models Found Zero-Day Flaw, Hackers Tried to Exploit It

Google reports that hackers used AI models to find a zero-day flaw and prepare an exploit for a mass attack. A new Google Cloud Threat Intelligence report details how an AI-assisted threat actor identified and weaponized a previously unknown software vulnerability before being stopped by the company's security teams. The incident confirms growing fears that AI could significantly accelerate the discovery of security bugs, making sophisticated attacks more accessible.

How Google Uncovered the AI-Generated Exploit

Google's threat intelligence team discovered a threat actor using a large language model (LLM) to find and weaponize a zero-day vulnerability. The AI-generated Python script, targeting a web admin tool's two-factor authentication, was stopped by Google's teams before a planned mass-exploitation campaign could be launched.

Google analysts detected the Python exploit script targeting a popular open-source tool's two-factor authentication. According to the Google Cloud Threat Intelligence Group report, the script exhibited clear signs of LLM assistance, and threat hunters successfully blocked the wide-scale campaign before it could be deployed.

Evidence Pointing to AI Involvement

Google's team identified several key indicators that the exploit was created with AI assistance:

• Formal, Repetitive Code Comments: The script's documentation used phrasing typical of security textbooks, a style common in LLM outputs.
• Inaccurate CVSS Score: The code included a hallucinated (non-existent) CVSS score.
• Familiar Code Structure: Key patterns in the code matched examples found in public datasets used for training large language models.

Google expressed "high confidence" that its Gemini model was not used in the attack. While the company has previously connected AI-powered reconnaissance to state-sponsored groups from China and North Korea, it has not attributed this specific zero-day incident to any nation-state.

A Milestone in Offensive AI Attacks

This incident is believed to be the first confirmed case of criminals using AI to generate a functional zero-day exploit in the wild, according to a SecurityWeek analysis. While security researchers have used AI defensively for years, this marks a significant escalation, suggesting offensive AI capabilities are rapidly advancing to match, and potentially surpass, defensive measures.

The Growing AI Arms Race

The zero-day discovery is part of a broader trend of malicious actors using AI for cyberattacks. Google has also observed other tactics from state-linked groups and criminals:

• China-linked groups have used role-play jailbreak prompts to audit embedded firmware.
• North Korean actors have sent numerous iterative prompts to analyze published CVEs.
• Criminal developers have attempted to fine-tune open-source models for exploit generation.

Why This Case Matters

This incident highlights a critical shift in the cybersecurity landscape, as AI drastically lowers the barrier to entry for creating powerful exploits. What once demanded weeks of expert-level reverse engineering can now potentially be accomplished in seconds by less-skilled actors. This trend puts immense pressure on software developers to implement more robust automated testing and faster patching cycles. The effectiveness of defensive countermeasures, such as prompt filtering and AI safety training, remains a crucial question as this new era of AI-driven threats unfolds.

---

What exactly did Google detect?

Google's Threat Intelligence Group (GTIG) saw a criminal group feed unknown software to an external AI model and receive back a working zero-day exploit - a Python script that bypasses two-factor authentication in a popular open-source web-admin tool. The crew had already registered the domain for a "mass exploitation event" when GTIG stepped in and neutralised the campaign.

How sure is Google that AI wrote the exploit?

The code carried hallucinated CVSS scores, textbook-style doc-strings and a "perfect" Pythonic layout - all hallmarks of large-language-model output. Analysts say this gives them "high confidence" the vulnerability was both found and weaponised with AI assistance, even though the exact model is still unknown.

Is this the first time attackers have used AI for zero-days?

Yes - at least in public incident reporting. GTIG labels the case the first confirmed criminal use of AI to discover and build a zero-day that was minutes away from live deployment. Earlier sightings were limited to reconnaissance or copy-paste of existing CVEs.

Which other actors are experimenting with AI-driven vuln research?

• China-linked groups - ran "senior security auditor" jailbreaks to probe firmware
• North Korean threat actors - submitted numerous recursive prompts to re-analyse old CVEs
• Cyber-crime affiliates - seen packaging AI-generated Android droppers such as PROMPTSPY

What are AI labs doing to slow the trend?

• Major AI companies have trimmed model release scopes, added constitutional refusal layers and rate-limit high-risk prompts
• Google's Project Zero counters with Big Sleep, an AI agent that has found memory-safety bugs in software before attackers could exploit them
• Across the industry, a growing number of organisations now run pre-deployment model scans, though many still skip security checks, showing the defence gap remains significant.