Palo Alto Networks tests Claude Mythos for vulnerability detection, finds 26 CVEs

Recent Palo Alto Networks tests of Claude Mythos for vulnerability detection highlight a critical question for security leaders: does the value of accelerated AI-powered code scanning justify the significant financial costs and governance overhead? While the pilot within the Project Glasswing consortium quickly surfaced numerous bugs, every finding carried both a literal invoice for compute tokens and a procedural burden for triage and privacy.

What the Mythos pilot actually found

The test of Claude Mythos by Palo Alto Networks identified multiple new vulnerabilities, none of which were critical or actively exploited. While the AI successfully surfaced legitimate vulnerabilities, the findings required significant human validation, highlighting that AI scanning supplements - rather than replaces - expert security team oversight and analysis.

According to industry reports, the AI-assisted scan generated numerous vulnerabilities from various code issues, none of which were actively exploited. Analysis of the findings revealed that most were of medium severity. This outcome demonstrates that while AI can surface meaningful security flaws, every discovery requires manual validation to confirm its real-world impact.

Furthermore, the project's economics point to significant long-term costs. Anthropic supplied Project Glasswing members with substantial usage credits, suggesting that commercial adoption will involve significant compute bills. Each finding also incurred operational overhead for human confirmation, patch development, and disclosure management - costs often overlooked in ROI calculations.

Economics of LLM security scans

Dimension	Possible benefit	Direct or indirect cost
Detection speed	Faster discovery of cross-file and semantic flaws	High token consumption during deep repository queries
Security outcome	Shorter exploit window if findings are confirmed	Extra logging and access controls to secure model APIs
Productivity	Fewer staff hours on manual review when signal is accurate	Triage queues for false positives and duplicate alerts
Financial impact	Incident avoidance that may dwarf tooling spend	Integration, staffing, and ongoing model usage credits

While a broader Forrester study on Palo Alto's platform cites significant ROI benefits, that figure does not isolate the impact of AI-powered scanning. Organizations must therefore build their own risk-adjusted financial models to evaluate this specific technology.

A practical evaluation framework

Security teams considering an LLM-powered scanner should implement a structured evaluation using four key checkpoints:

• Baseline cost-per-finding by running a pilot on a limited repository slice while tracking token consumption.
• Measure the false-positive ratio against authoritative sources like the CVE list or OWASP Top 10.
• Confirm the vendor's privacy posture, ensuring they disable model training on prompts and enforce strict data retention limits.
• Stage the rollout in concentric pilots, expanding only after human triage time drops by a predefined percentage.

Adoption trends suggest a cautious approach is common. Industry reports indicate that application security programs have been slower to adopt AI compared to threat detection programs. This gap may indicate that buyers are waiting for more mature quality and performance metrics.

Governance guardrails to prevent code leakage

To prevent a security tool from becoming a liability, strong governance is essential. Experts at Wiz recommend enforcing strict role-based access controls (RBAC), encrypting all data in transit and at rest, and continuously monitoring for prompt injection attacks. Similarly, legal and security advisors at Jones Walker stress that proprietary code and secrets should never be sent to third-party models, even if the vendor claims to have disabled training on customer prompts. These guardrails are critical for ensuring that code analysis does not inadvertently create new data exposure risks.

Deciding when value outweighs cost

The decision to adopt AI scanners involves balancing competing pressures. As Palo Alto Networks' own Unit 42 threat intelligence team warns, AI also accelerates attacker workflows, increasing the urgency for earlier detection. However, this same pressure magnifies the risk of misconfiguring a scanner or leaking proprietary code. Security leaders must weigh the potential for high-impact discovery against the consequences of a tool-induced breach.

Ultimately, the ROI depends on context. Organizations managing sprawling, legacy-heavy codebases may find that preventing a single major breach justifies the investment. In contrast, smaller teams with modern codebases and tight privacy constraints may determine that traditional SAST and targeted manual reviews offer a more cost-effective and lower-risk solution.

---

What did Palo Alto Networks actually find when it tested Claude Mythos for vulnerability detection?

Palo Alto Networks ran an AI-assisted scan on its own codebase inside the Project Glasswing consortium. The exercise delivered multiple vulnerabilities across numerous individual issues. None of these vulnerabilities were seen being exploited in the wild at time of disclosure, and the majority were of medium severity according to the company's Defender's Guide 2026. The test shows that modern LLMs can surface previously hidden flaws quickly, but each finding still needs manual triage before it becomes a patch.

How do the security benefits compare to the real-world costs of LLM-driven scans?

The upside is rapid vulnerability discovery, which can shorten the time-to-patch and reduce incident-response expense. Palo Alto's own platform-wide Forrester TEI study reports significant ROI and substantial NPV for its broader toolset, illustrating the kind of financial upside when flaws are caught early.
The downside is three-fold:

• Direct spend: Anthropic granted the consortium substantial usage credits, implying that at-scale token costs are material once the pilot ends.
• Triage overhead: every finding must be deduplicated, validated, and scheduled for remediation; the vulnerability findings still consumed analyst hours before publication.
• Governance overhead: new controls for prompt injection, token leakage and access logging all add security engineering tasks beyond the original scan.

A practical rule of thumb is that LLM scanning pays off for large, complex codebases where a single missed vulnerability could cost more than the entire pilot budget.

What hidden risks should enterprises consider before uploading code to a third-party LLM?

1. Data leakage: industry studies show that a significant portion of employee prompts accidentally contains secrets or proprietary code.
2. Prompt injection attacks: considered a major vulnerability in large language model applications.
3. Training-data retention: unless the vendor disables training on customer input, uploaded source could become part of a future model.
4. Licensing contamination: retrieved snippets may inadvertently embed GPL or other restrictive code into a proprietary project.
5. False-positive inflation: traditional SAST tools already suffer high false positive rates; an LLM can amplify the noise unless tightly tuned.

Which governance guardrails reduce the chance of sensitive code exposure?

• Classification first: inventory every repository and tag data by sensitivity before any model call.
• Least-privilege API keys: enforce role-based access, MFA and short-lived tokens for every service that can invoke the LLM.
• Prompt sanitization: strip secrets, PII and credentials at the CI/CD layer; block the job if sensitive patterns are detected.
• Ephemeral logging: store only redacted prompts/outputs and set retention to the minimum needed for audit.
• Continuous red-team: schedule regular tests for prompt injection, retrieval leakage and model output anomalies.

NIST SP 800-207 is the 2020 Zero Trust Architecture publication that describes zero trust principles such as treating all data sources and computing services as resources, securing communications regardless of network location, dynamically authenticating and authorizing access, monitoring asset integrity and posture, and collecting telemetry.

Where should an organization start if it wants to pilot LLM vulnerability scanning without betting the farm?

1. Scope: pick one high-value micro-service or library with clear ownership and a small team.
2. Goals: define limited KPIs such as cost-per-valid-finding and mean-time-to-triage.
3. Budget: allocate a capped usage-credit envelope and track token spend per sprint.
4. Controls: apply the governance guardrails above from day one; a secrets-scanning pre-step prevents embarrassing leaks.
5. Exit criteria: after 30 - 45 days, compare valid bugs found vs. engineering hours spent to decide whether to expand, tune or sunset the pilot.

Following this staged approach lets security teams harvest the upside of AI discovery while keeping operational and financial surprises under tight control.