AI Coding Tools Introduce New Security Risks, Veracode Report Finds
Serge Bulaev
Code tools like Claude, Copilot, and Cursor often make mistakes that can lead to serious security problems, like leaking secrets or letting in hackers. Attacks called 'prompt injections' are especially easy - hackers trick the AI with sneaky messages and steal data or add hidden dangers. Copilot has leaked private code, Cursor let a fake extension steal money, and Claude is still at risk for similar tricks. Using clear input formats, strict rules, and careful reviews can make these tools safer, but engineers must stay alert and check their code often.
The widespread adoption of AI coding tools like GitHub Copilot, Anthropic's Claude, and Cursor introduces significant new security risks, from secret exfiltration to supply chain vulnerabilities. While these assistants accelerate development, they also create a shared attack surface vulnerable to prompt injection and other exploits that can compromise enterprise data and systems.
Shared attack surface across Claude, Copilot, and Cursor
AI coding assistants frequently generate vulnerable code containing flaws like SQL injection and cryptographic failures. They are also susceptible to prompt injection attacks, where malicious instructions trick the model into leaking secrets or inserting backdoors. This shared attack surface affects all major tools, regardless of model size.
All three leading tools generate vulnerable code at scale. A recent Veracode report found security flaws in 45% of AI-generated code snippets, cautioning that larger models like Claude offer no greater safety than their peers. Common weaknesses include classic CWEs such as SQL injection, cryptographic failures, and cross-site scripting.
Prompt injection is the most accessible exploit vector, earning the LLM01 designation from OWASP for 2025. In these attacks, adversaries embed malicious payloads within benign-looking instructions, compelling the model to overwrite its system prompts to exfiltrate secrets or insert backdoors. As all vendors fine-tune their models on public data, complete immunity remains elusive.
Where each vendor stumbles
OpenAI/GitHub Copilot: Experienced a prompt injection attack that silently exfiltrated data from private repositories. While GitHub has since enhanced its automatic secret scanning, the fundamental risk remains as long as the tool operates with developer-level permissions.
Cursor: The editor's agentic execution capabilities create supply chain risks. A notable incident involved an Ethereum engineer losing funds after installing a malicious third-party extension, as detailed in a Fortune investigation. Subsequent research confirmed Cursor could install high-risk dependencies without user confirmation.
Anthropic/Claude: While the Claude family has not suffered a public zero-day exploit, security researchers consistently reproduce the same prompt injection vulnerabilities found in other models. Analysts warn that state-level actors may target popular LLMs by poisoning training data or exploiting the excessive permissions often granted to autonomous agents.
| Tool | Most Observed Failure Mode | Real-world example |
|---|---|---|
| OpenAI (Copilot) | Prompt injection and secret exfiltration | Silent leak of repo data before mitigations |
| Cursor | Malicious extensions and agentic dependency install | Wallet drain via fake browser plugin |
| Anthropic (Claude) | Unpredictable generation and supply-chain risk | No confirmed breach, but test cases show injectable prompts |
Prompt design and guardrails that work
Implementing robust prompt design and architectural guardrails can significantly mitigate these risks. Testing teams observed a 30% reduction in critical vulnerabilities by adopting structured prompts with strict output schemas. This checklist, based on best practices from AWS, OWASP, and leading enterprise security playbooks, addresses the highest-impact threats:
- Wrap every user input within clear delimiters such as
- Demand deterministic formats (JSON, fenced code blocks) so scanners can parse and validate before execution.
- Enforce least-privilege tokens; never pass credentials through the prompt.
- Chain tasks: ask for code, then separately ask the model to review it against OWASP Top 10.
- Red-team monthly by injecting "ignore rules" phrases and scanning for jailbreak success.
What engineers can do today
Engineers can take immediate steps to harden their development lifecycle against AI-driven risks:
- Enforce Code Reviews: Peer review for all AI-generated code remains mandatory.
- Automate Static Analysis (SAST): Integrate automated security testing to scan all AI-generated output before it is merged into a codebase.
- Manage Extensions Carefully: Cursor users should disable automatic extension installation and await the release of signed, verified add-ons from the vendor.
- Limit Blast Radius: For tools like GitHub Copilot, reduce potential damage by limiting repository access scope and enabling secret scanning on every push.
- Track Model Updates: Monitor vendor change logs for model updates, as minor adjustments to parameters like temperature can significantly alter security outcomes.
Which AI code-generation tools were tested and how did they differ?
The Veracode study compared Anthropic's Claude-based assistants, OpenAI-powered tools such as GitHub Copilot, and the Cursor editor.
Each was fed the same web-app prompts; Cursor stood out for suffering real-world wallet-drain attacks via malicious extensions, while OpenAI-linked tools were exploited for secret exfiltration from private repos. Anthropic had no tool-specific incident on record, yet still produced code with the same 45 % baseline flaw rate.
What exact failure modes were cataloged?
The report logs prompt injection that turns Copilot into a silent data thief, Cursor extensions that auto-install poisoned npm packages, and "AI-in-the-middle" hijacks where the model rewrites code before it reaches the editor.
Every tool also generated classic OWASP Top-10 bugs - SQL injection, hard-coded secrets, and XSS - at speeds that outpace human review.
How much worse is AI-generated code versus human code?
Across Java, JavaScript, Python, and C# tests, 45 % of AI snippets contained at least one security weakness, matching the highest human-introduced rates but produced 5-10× faster, amplifying technical debt. Veracode adds that larger models did not score better; size is not safety.
Can simple prompt tweaks really fix the problem?
Yes - adding a one-sentence rule such as "Use parameterized queries and output only ES-lint-clean code" dropped injection flaws by 38 % in the Veracode lab.
Chaining prompts into outline → code → security review steps cut logic errors by half, proving that disciplined prompt engineering is the fastest ROI available today.
What should security and engineering teams do tomorrow?
- Ban shadow AI - centrally license one vetted tool and turn on its built-in scanner (Copilot's auto-analysis blocks 22 % of flaws).
- Mandate human review for any package the agent wants to install; Cursor's wallet-drain happened because no one saw the extension permissions.
- Add a 30-minute adversarial red-team sprint to every sprint cycle; Veracode shows this catches 70 % of prompt-injection paths before code is merged.