AI Code is 86% Flawed in Benchmarks, CSET Study Finds
Serge Bulaev
A recent study suggests that AI-generated code has many flaws, with failure rates as high as 86% for some security issues. Researchers found that nearly half of AI code samples show problems like SQL injection and improper authentication. The main challenge may be checking and fixing AI code, not just creating it, so teams are adding more human reviews and strict checks. Productivity gains are mixed; AI might speed up some tasks, but could also slow down experienced programmers and increase maintenance work. Experts recommend careful oversight and collecting data to ensure code quality and safety.
Recent studies revealing significant AI-generated code flaws highlight a new bottleneck in software development: critical verification. While large language models (LLMs) accelerate code creation, teams now spend more time proving that the generated code is secure, maintainable, and production-ready than they do writing it.
The core issue is that AI models generate buggy code at an unsustainable rate. A landmark CSET study from Georgetown University found that nearly half of AI-generated code snippets from five popular models had security weaknesses, including SQL injection. Further benchmarks show failure rates as high as 86% for cross-site scripting and 47% for SQL injection, according to a CodeQA analysis.
As a result, code verification - not generation - is the new factor limiting delivery speed. Organizations are now re-architecting their development pipelines to incorporate mandatory human-in-the-loop (HITL) checkpoints to manage this risk.
Why security flaws persist
AI-generated code contains flaws because models are trained on vast amounts of public code, which includes outdated and insecure examples. They also lack the full context of a specific application's architecture, leading them to produce code with vulnerabilities that may not be immediately obvious to developers.
- Training data mirrors internet code, including outdated patterns.
- Models lack full context of a live stack, so they guess at dependencies.
- Prompted fixes are accepted at face value, reinforcing silent failures.
- Developers report overconfidence in AI output, skipping deeper audits.
A layered HITL workflow emerging in 2025-2026
| Stage | Primary gate | Typical escalation trigger |
|---|---|---|
| Automated lint + static analysis | Style errors, obvious bugs | Security-sensitive paths |
| AI bot review | Trivial suggestions, docstrings | Low model confidence |
| Selective human review | Business logic, auth, payments | Novel patterns or unresolved scans |
| CI test suite | Unit and scenario tests | Any failing check |
| Release monitoring | Telemetry, regression alerts | Quality drift or user incident |
Expert guidance recommends three core policies for managing AI code risk. First, define critical code paths - such as authentication, infrastructure, and public APIs - that require mandatory human review. Second, demand proof of correctness by gating merges on passing tests. Third, track key metrics like defect density and review turnaround to ensure the process remains effective.
Productivity trade-offs appear real
While marketing emphasizes speed, rigorous studies present a mixed view on productivity. A 2024 Copilot field study showed 26% more task completions with no quality degradation. Conversely, a separate trial found AI assistance slowed experienced open-source contributors by 19%. Furthermore, analysis from GitClear suggests a rise in future maintenance costs, indicated by increased code duplication and less refactoring.
Tooling is catching up
To manage these risks, modern development tools are evolving. Compliance and security teams now mandate comprehensive audit trails that log AI prompts, generated code, human reviews, and test results. This detailed record is essential for incident response and for proving regulatory compliance.
The Pragmatic Path Forward
The takeaway is pragmatic: AI tools can accelerate initial code drafting, but deploying them safely and effectively depends on a new discipline. This requires structured verification processes, targeted human oversight for critical code, and continuous measurement of quality and velocity.
What does the CSET study mean by "86% flawed" code?
CSET's benchmark evaluated five widely-used AI coding models and found that 86% of snippets failed at least one security test, with 47% vulnerable to SQL injection, 88% exposing log-injection flaws, and 86% exhibiting XSS issues. In practical terms, nearly half of the generated samples could be exploited in real deployments, echoing the broader finding that about 45% of AI-generated code contains security flaws.
Why are developers becoming more confident despite lower security quality?
A Stanford study cited in the research shows that AI-assisted developers believed their code was more secure than human-only code, even when it contained significantly more vulnerabilities. The theory: AI suggestions are delivered with high linguistic confidence, which users misinterpret as technical correctness. This "automation bias" can reduce manual review rigor and ultimately lower the team's security posture.
How does AI assistance affect experienced developers in familiar codebases?
A 2024 METR/Anthropic randomized trial found that experienced contributors were 19% slower when using AI assistants on codebases they already knew well. The extra time came from debugging AI-introduced regressions and re-validating correct but unfamiliar patterns. GitClear's 2025 report reinforces this: code churn increased and refactoring dropped, suggesting that even veterans spend more time cleaning up after the bot.
If generation is fast, what becomes the new bottleneck?
As AI writes code at machine speed, verification becomes the rate-limiting step. Security reviews, comprehensive test suites, dependency scans, and manual architecture checks now consume the majority of delivery time. Teams adopting AI coding tools in 2025 - 2026 report that shipping velocity is now gated by review capacity, not writing capacity.
What practical steps reduce risk without killing productivity?
The evidence supports a risk-tiered human-in-the-loop model:
1. Classify code paths - automatically label auth, payments, and infra modules as "critical," triggering mandatory human review.
2. Gate merges with tests - require unit, integration, and security scans to pass before any AI-generated diff can land.
3. Escalate on low confidence - route any snippet flagged by static analysis or with fuzzy AI confidence scores to a senior reviewer.
4. Track drift - monitor defect density and regression rates weekly; adjust review thresholds accordingly.
5. Maintain an audit trail - log prompts, model version, reviewer comments, and test results for every AI-assisted commit.
By focusing human effort on high-impact code and security edge cases, teams can preserve most of AI's speed while containing its risks.