AI risks in 2026: attacks, incidents, protection

In short

AI systems can be steered with text, and not only by you. Every web page, email and tool response an assistant reads can carry hidden instructions. It gets dangerous when an agent can see private data, read outside content and send data out at the same time. Simon Willison calls this the "lethal trifecta". OpenAI itself writes that prompt injection is unlikely ever to be fully solved.

The risks at a glance

The security community OWASP maintains the Top 10 for LLM Applications in its 2025 edition. It was published on 18 November 2024 and is still current in October 2026. At the top are prompt injection, sensitive information disclosure and supply chain weaknesses, meaning tampered models, datasets or libraries. Then come excessive agency, system prompt leakage, misinformation and unbounded consumption that makes costs explode.

For agents that operate tools themselves, a separate list appeared in December 2025. It warns of goal hijacking, tool misuse, malicious MCP servers and rogue agents that leave their mission even though each single action looks harmless. In 2026 the attack catalogue MITRE ATLAS mainly added agent techniques such as tool poisoning and the "supply chain rug pull": a tool that first builds trust and then turns malicious (changelog).

What happened in 2025 and 2026

• EchoLeak: One crafted email was enough to pull data out of Microsoft 365 Copilot without a single click (case study).
• Perplexity Comet: Hidden text on a web page made the AI browser send the user's email address and one-time code to the attacker. According to Brave, this was not fully fixed even after the patch.
• postmark-mcp: The first known malicious MCP server secretly blind-copied every email to the attacker, with 1,643 downloads (The Hacker News).
• ClawHavoc: Researchers found 341 malicious skills on the skill marketplace ClawHub. 335 of them installed the Atomic Stealer infostealer on macOS through fake "prerequisites" (The Hacker News). Of 3,984 skills examined, 36.82 percent had at least one security flaw (Snyk).
• Moltbook: A largely AI-built platform left its database open without access rules, exposing about 1.5 million API tokens and roughly 35,000 email addresses (Wiz).
• AI as an attack tool: According to Anthropic, a state-sponsored Chinese group used Claude Code against about 30 targets. The AI did 80 to 90 percent of the work and succeeded in a few cases. In 2025 the FBI warned of AI voices impersonating senior US officials.

The market for protection tools is moving too. Lakera went to Check Point, Protect AI to Palo Alto Networks, Promptfoo to OpenAI and Guardrails AI to Harvey in September 2026. The LLM Guard library was archived in July 2026. Tools can disappear.

What the EU AI Act requires

Since 2 August 2026 the AI Act largely applies and the transparency rules are in force. The "AI Omnibus" postponed the high-risk obligations: for use cases such as education or employment to 2 December 2027, and for AI in regulated products to 2 August 2028 (European Commission). Existing generative AI systems must meet the labelling duty from 2 December 2026 (overview). Most small creator works are not affected by the high-risk rules, but topics such as candidate screening or education can trigger them.

How to protect yourself

As a user:

• Give agents only the permissions the task needs, and read every confirmation before purchases, emails or transfers yourself.
• Do not copy commands from skill descriptions.
• Do not let AI browsers work unattended in email, banking or company accounts.
• Hang up on calls about money and call back on a number you know.

As a creator:

• Never combine private data, outside content and a channel to the outside without human approval.
• Scan skills, MCP servers and models before you publish. "No findings" does not mean "safe" (Cisco Skill Scanner).
• Keep API keys on the server and turn on your database's access rules.
• Label AI interaction and AI-generated content.

No tool makes an AI app completely secure, and guard filters usually catch only part of the risks (Microsoft). The rules above lower the risk, but they do not replace your own judgement.