Prompt injection and AI application security: attack types, layered defense and how to test

AI applications increasingly read things the business did not write itself: customer emails, web pages, uploaded documents, tool results. Any of these can carry text meant to steer the model, and this attack is called prompt injection.

This article brings together the definitions from OWASP and NIST, published incidents, guidance from Anthropic, OpenAI, Google, Microsoft and the UK National Cyber Security Centre (NCSC), and ways to test, checked as of October 21, 2026.

It answers 3 questions: why the problem is hard to close, what layered defense is, and what to verify before launch. Click Star already has articles on AI chatbots for websites and on protecting customer data in AI analysis; this one covers attacks and testing.

Key points

• OWASP ranks prompt injection as risk LLM01 (2025 edition), in 2 types: direct (the user types the instruction) and indirect (it sits in a web page, file or email the model reads).
• Root cause: NIST (March 2025) notes data and instructions share a single channel; the NCSC (December 8, 2025) argues it may never be fully mitigated like SQL injection.
• Vendors say it is not solved: Anthropic treats even a 1% attack success rate as meaningful risk.
• Published incidents (EchoLeak, GitHub MCP, Comet) are mostly research demonstrations; no real-world damage statistics have been published.
• No single layer is enough: a line in the prompt is a soft layer; add least privilege, data separation, human approval, tool limits, monitoring and isolation.
• Risk rises when the AI reads outside content, touches private data and can send things out; re-run the attack test set whenever the model, prompt or tools change.

What prompt injection is: direct and indirect

According to OWASP's LLM01:2025 page (read October 21, 2026), prompt injection happens when a prompt changes the model's behavior in unintended ways. The injected text can be invisible to a human reader, for example hidden text on a web page that a summarizer reads.

Direct means the user's input acts on the model itself. Indirect means the model takes data from an outside source such as a web page or file, and that content changes its behavior. A jailbreak is a form where the attacker wants the model to drop all safety rules.

NIST, in report AI 100-2 E2025 (final version March 24, 2025, read October 21, 2026), draws a similar split. In the indirect case the attacker is a third party, while the person harmed is often the application's own user, so a chatbot can be turned on its owner's customers.

Why it is hard to close

NIST writes that beneath many vulnerabilities in LLM applications is the fact that data and instructions do not reach the model in separate channels, a flaw similar to the one behind SQL injection. SQL injection could be closed with parameterized queries, which keep data from being read as commands.

The NCSC (post of December 8, 2025 by Dave Chismon) says current models enforce no security boundary between instructions and data, calling them an "inherently confusable deputy". The goal is to cut likelihood and impact; if a system cannot tolerate the residual risk, an LLM may be the wrong choice.

NIST also notes current models remain highly vulnerable to direct injection and recommends designing on the assumption the model can be steered. Anthropic (November 24, 2025) writes that prompt injection is far from solved; Google (April 2, 2026) treats it as ongoing mitigation.

Measurements show progress, not zero. In Anthropic's model report (read October 21, 2026), the chance of an attacker succeeding within 15 attempts on a Gray Swan, UK AISI and CAISI (US) benchmark fell from 83.2% (Haiku 4.5) to 7.1% (Haiku 5.5); the report lists Opus 5.5 at 1.0%.

Limits: models were measured without product protections, Haiku 5.5 still shows 24.4% on computer-use tasks, and this is a single benchmark, not a promise for a business's application.

A public competition (arXiv paper, March 16, 2026) logged 464 participants sending 272,000 attempts at 13 frontier models, with 8,648 successful attacks across 41 scenarios. Every model was breached, from 0.5% (Claude Opus 4.5) to 8.5% (Gemini 2.5 Pro).

Incidents that have been published

The cases below show how attacks work; they are not damage statistics, and Click Star found no public count of real incidents.

EchoLeak, CVE-2025-32711. An academic analysis (arXiv, September 6, 2025) describes a zero-click flaw in Microsoft 365 Copilot: a single crafted email could pull data out, through a chain bypassing the XPIA classifier, link redaction and auto-fetched images.

GitHub MCP, May 26, 2025. Invariant Labs showed a malicious Issue in a public repository could make an agent read private-repository data and post it publicly. They call it an architectural problem at the agent level, not a server bug, and note "Always Allow" makes odd actions harder to spot.

Comet, August 20, 2025. Brave showed hidden text in a Reddit comment making the Comet browser assistant fetch the user's email and a one-time login code, and noted the attack class was not fully fixed after its first report.

Web scan, April 23, 2026. Google scanned Common Crawl: most injected text was pranks, SEO manipulation or agent deterrents, a few aimed at theft or deletion, with low sophistication. The malicious group grew 32% (relative) from November 2025 to February 2026. The archive is mostly static English-language web.

Defense in layers

Microsoft (MSRC, July 29, 2025) combines probabilistic and deterministic controls (prevent, detect, limit impact) and does not rely on blocking every injection.

Layer 1, least privilege. OWASP advises giving the application its own API keys and limiting privileges to what is needed. The NCSC: when a model processes input from a party, its privileges drop to that party's level, e.g. a model reading outside email should hold no privileged tools.

Layer 2, separate untrusted data. Claude Platform docs advise placing third-party content only in tool-result blocks, labeling its source, JSON-encoding it and keeping your own instructions out. OpenAI advises passing untrusted input as user messages, not developer messages. Microsoft uses Spotlighting.

Layer 3, limit tools and outputs. OpenAI advises fixed formats between steps and extracting only validated fields from outside input. Google does not render external image URLs in Markdown; Microsoft blocks known leak paths by rule.

Layer 4, human approval. OWASP calls for human approval of privileged actions; OpenAI advises keeping approvals on for MCP tools. The weakness is click fatigue: Anthropic reports sandboxing cut permission prompts 84% (October 20, 2025, internal).

Layer 5, filtering and monitoring. Anthropic advises screening tool results with a small model. The NCSC advises logging and treating failed tool calls as an early sign. NIST warns detectors can be attacked too.

Layer 6, isolate the runtime. Anthropic (October 20, 2025) says file and network isolation are both needed: without the network a hijacked agent cannot send SSH keys out; without file isolation it could escape.

What is not enough: telling the model in the prompt

Telling the model "ignore strange instructions" is worth doing; OWASP lists constraining behavior via the system prompt as the first of 7 mitigations. But it is a soft layer, since the attacker prompts the same model.

The NCSC advises against deny-lists of phrases such as "ignore previous instructions", as there are endless rephrasings, and urges caution toward products claiming to "stop" prompt injection. OpenAI calls built-in guardrails a first wave that is not foolproof. Sturdier question: if the model is fooled, what is the most it can do?

Rising risk when AI has tools: agents, MCP, browsers

NIST notes that agents act through tools, so injection can lead to arbitrary code execution or data exfiltration. Simon Willison (June 16, 2025, a secondary source, a personal blog) calls the dangerous combination the "lethal trifecta": private data, untrusted content and the ability to communicate outward.

Meta (October 31, 2025) proposes the Agents Rule of Two: in a single session an agent should have at most 2 of 3 properties: processing untrusted input, accessing sensitive data, changing state or communicating externally. If all 3 are needed, a human must approve.

On MCP (a protocol that connects tools to AI), Claude Code's security documentation advises writing your own servers or using trusted providers, and says Anthropic does not security-audit each server. The table above is Click Star's frame: higher privilege, larger maximum damage.

What this means for Vietnamese businesses

Click Star found no public measurements of prompt injection in Vietnam, so the advice rests on international guidance and reasoning.

Example: a website chatbot connected to a CRM to look up orders and send confirmation emails. It reads customers' text, looks up order data and sends email, so all 3 sides of the trifecta are present. For a small business without a security team, the cheapest fix is cutting privileges at design time, above all automatic sending.

How to test and what to measure

Anthropic's documentation advises red-teaming your own agent before deployment with documents, emails and tool results that deliberately contain injections. Group the test set along OWASP's scenarios: instructions in the chat box; hidden in files, emails and web pages; reading the system prompt; out-of-scope tool calls; leaks through images and links; other languages and encodings.

NIST points to open tools: AgentDojo (97 tasks, 629 security test cases, per its 2024 arXiv paper), garak (NVIDIA) and PyRIT (Microsoft).

Metrics to track (Click Star's frame): attack success rate by number of attempts k; false-block rate on legitimate requests; number of tool calls outside the allow-list.

NIST reminds that evaluations capture a single moment and can underestimate vulnerabilities, so keep measuring after deployment. Some attacks leave no trace in the final answer (March 2026 paper), so log tool calls.

What to do before launch

• List the content sources the AI reads and the tools and data it touches; remove what is not needed.
• Check the trifecta: if the AI has all 3, drop 1 or add a human approver.
• Use dedicated, narrow-scope keys that can be revoked fast.
• Put an approval step on send, write, delete and payment actions.
• Log prompts, results and tool calls, with access control on the logs.
• Run the attack set, save the results as a baseline, and re-run on any change of model or tools.

Click Star's view. Technically, treat the model as an untrusted user inside the system: useful, but with privileges no higher than the business accepts if it is fooled. The key decision is which privileges to grant, not the wording of the prompt.

Disclosure: Click Star sells AI integration and AI automation services, so it has an interest in recommending safe design. A business can apply the list above itself.

Common mistakes

1. Believing a line in the system prompt is enough.
2. Giving a chatbot CRM write access, email sending or every customer's data from day one.
3. Testing only a few hand-typed "ignore previous instructions" lines, not instructions hidden in files, emails and web pages.
4. Mixing tools from several MCP servers without reviewing each source's privileges.
5. Auto-approving every action to avoid annoying users.
6. Measuring once at launch and never again after the model changes.
7. Reading only the final answer, without checking tool calls.

What is not known

• No public measurements of prompt injection prevalence in Vietnam, or statistics of real-world damage worldwide, were found.
• Vendors' attack success rates depend on benchmark, attempts and protections, so they cannot be compared directly.
• OpenAI's post on agents resisting prompt injection (March 11, 2026) could not be read (the page blocks the tool); only OpenAI documents that opened are used.
• Advice for small businesses is inference from international guidance; no study measures it.