LLM Security Risks: Where Each Threat Enters the Request Path and How to Reduce It

An internal retrieval-augmented generation (RAG) assistant receives a question from a credit analyst about next quarter’s headcount plan. The retriever pulls the closest-matching chunks from the vector index, one of them from an HR planning document restricted to the leadership team, and the model summarizes it in three clean paragraphs. Every component did what it was built to do, and the restricted HR content leaked to a user who was not cleared to see it.

The document permissions stayed on the file share, the ingestion job never copied them onto the vector chunks, and the retrieval query never checked who was asking. The authors of the OWASP RAG Security Cheat Sheet call access control that fails to carry through to vector chunks “the most common compliance failure in enterprise RAG deployments” [1].

An LLM security risk is any path by which a large language model (LLM) application can be made to disclose data, take an action, or produce output that its operator did not authorize, whether an attacker drives it or an ordinary user triggers it by accident. In chat applications and RAG or copilot applications, those paths run along the request: the user prompt, the system instructions, the documents and tool results pulled into context, the call to the model, and the response that reaches a browser or a downstream system, such as a database. Where a risk enters that path determines which control can stop it, which is why the sections below sort LLM security risks by entry point.

Why LLM Applications Need Their Own Security Model

LLM applications fail in ways string-matching controls cannot see, because the model does not separate instructions from data. The UK National Cyber Security Centre (NCSC) states that current large language models “simply do not enforce a security boundary between instructions and data inside a prompt” and warns that a deny-list faces “infinite ways to rephrase an attack that would avoid such a filter” [2]. Following the NCSC’s reasoning, any control that matches fixed strings inherits the deny-list problem. A natural-language attack has no fixed string. The same instruction can arrive as encoded or translated text, such as a Base64 blob or a second language, or in the pixels of an image.

Two more properties widen the attack surface:

• Non-deterministic outputs: The same prompt can produce a safe answer in one run and a disclosure in the next, and a passed test describes only the runs sampled.
• Data moving in both directions: Prompts, retrieved chunks, tool returns, and file reads enter the context. Anything the model writes can leave through the response or another output channel, such as a log line or a rendered link.

The OWASP Top 10 for LLM Applications 2026 states that for prompt injection “no reliable prevention mechanism exists today” [3].

The controls in the sections below are the runtime-enforcement slice of a wider AI governance program, which also holds the inventory of LLM applications, their owners, lifecycle status, policies, and the evidence retained when a control fires.

Input Risks: Direct Prompt Injection and Jailbreaks

Direct prompt injection enters at the user prompt, where an attacker types instructions that override the system prompt or the application’s task. OWASP files it under LLM01:2026 Prompt Injection and treats jailbreaking as the subset of direct injection aimed at violating the model’s safety protocols [3].

Persona-play is a common form. The attacker asks the model to role-play an unrestricted character and routes the harmful request through it. Volume works as well as framing. Anthropic’s many-shot jailbreaking study, published at NeurIPS 2024, found that on the models tested roughly 8 fabricated dialogue turns pushed the jailbreak success rate past 50% and 256 turns pushed it past 90% [4].

Static defenses do not hold against attackers who adapt. OWASP’s 2026 guidance cites Nasr et al., who found that most of 12 defenses showed near-zero attack success against static test sets and above 90% under adaptive attack [3].

The controls at this layer sit on the request path:

• Request check: Run it before model invocation, on the gateway the application already uses, with a trained classifier, because a keyword list inherits the deny-list problem.
• Role constraints: Add role constraints in the system prompt, which OWASP lists as a partial control only, and validate output against a schema in application code.
• Testing: Run adaptive, multi-turn attack suites alongside benign traffic to measure over-refusal, as the guide on red teaming techniques describes.

Context and Data Risks: Retrieved Content and Sensitive Data

Indirect prompt injection enters through content the application retrieves on the user’s behalf. A poisoned document, email, web page, or tool result carries instructions the model treats as authoritative, and the user never sees them. EchoLeak, CVE-2025-32711, let a single crafted email make Microsoft 365 Copilot exfiltrate internal data with no user interaction; Microsoft scored it CVSS 9.3 and reports it fully mitigated with no known exploitation [5]. The PoisonedRAG study at USENIX Security 2025 reached a 90% attack success rate by inserting five malicious texts per target question into a knowledge base of millions of documents [6].

Sensitive data disclosure, LLM02:2026 Sensitive Information Disclosure, enters at two points on this path. The first is the retrieval step, as in the opening scenario. OWASP’s 2026 mitigation for every deployment is authorization before retrieval, enforced at document and chunk level inside the index query [3]. The second is the prompt itself, where an employee pastes material the application was never cleared to hold. In 2023, Samsung employees reportedly entered confidential source code and meeting content into ChatGPT at least three times [7]. The same exposure applies inside a sanctioned application, because every prompt sent to a hosted model API leaves the network, along with any retrieved chunks and tool returns included in it. OWASP’s guidance is to minimize context sent to external providers and scrub logs and traces.

The controls at this layer sit on the retrieval and prompt path:

• Untrusted context: Treat every retrieved chunk and tool return as untrusted input and run the same injection check on it that runs on the user prompt. The guide on indirect prompt injection covers the runtime pattern.
• Permission-aware retrieval: Carry document permissions onto the vector chunks and filter inside the retrieval query, so the model never receives a chunk the user could not open directly [1].
• Inline redaction: Detect and redact PII/PHI before the prompt reaches the model and again before the response reaches the user. This reduces exposure regardless of a provider’s retention terms.

Output Risks: Insecure Handling, Harmful Content and Hidden Context Exposure

Output risks enter where the model’s response leaves the application and reaches a user or a downstream system, such as a browser or backend.

• Insecure output handling: The response reaches something that executes it. OWASP files this under LLM10:2026 Improper Output Handling and lists cross-site scripting in browsers and server-side request forgery on backends, along with privilege escalation and remote code execution, as the impacts. LangChain’s LLMMathChain passed model output to Python’s exec, so an attacker could use prompt injection to run arbitrary code (CVE-2023-29374; the National Vulnerability Database scored it 9.8) [8]. In browsers, a vector is a rendered Markdown image or link whose URL carries exfiltrated data. The mitigations include context-aware output encoding and a strict content security policy (CSP). Disable auto-rendering by default for Markdown images and for embedded content such as link previews and iframes.
• Hallucinated or harmful content: This harms the user directly. In a RAG application the question is whether the response is faithful to the retrieved context; in a customer-facing chat it is whether the response is toxic or out of scope, and OWASP tracks unfaithful output under LLM07:2026 Misinformation.
• Hidden context exposure: “What used to be System Prompt Leakage is now Hidden Context Exposure, a broader framework for the same failure to trust information that should have stayed out of reach,” the authors of the 2026 edition state. LLM08:2026 Hidden Context Exposure covers developer instructions, retrieved policy text, configuration, and tool and function schemas, and the guidance assumes all of it is discoverable [3]. The control is to keep credentials, including tokens, and connection strings out of hidden context and to enforce authorization in deterministic code.

Across all three, the response check runs before output reaches the user or a downstream system, validating it against a schema in application code, scoring it for faithfulness, toxicity, PII, and system-prompt strings, and blocking or redacting according to policy.

Model and Supply-Chain Risks: Poisoned Weights, Adapters and Training Data

Supply-chain risk enters before the first request, in the model artifacts, such as weights and adapters, and the fine-tuning data the application is built on. OWASP’s LLM04:2026 Supply Chain names hosted fine-tuning adapters and conversion and merge workflows as attack surfaces. LLM05:2026 Data and Model Poisoning warns that shared repositories can carry malicious pickle payloads and tampered chat templates and tokenizer configs. Souly et al. found that 250 malicious documents reliably backdoored the models tested, while 100 did not [9].

These are procurement and integrity controls applied before deployment. OWASP recommends a signed software bill of materials (SBOM) extended to model artifacts, including models and adapters, and to datasets, with CycloneDX ML-BOM and OWASP AIBOM as candidate formats.

OWASP also recommends signing and hashing every artifact. Treat templates and tokenizer configs, along with adapters, as code: sign them and review them through diffs and static analysis [3]. OWASP’s authors add that “Signing proves integrity and origin, not safety,” so a third-party model still needs red teaming before it is promoted.

Tool Risks: Excessive Permissions When an LLM Calls Tools

Excessive permissions enter where the model’s output becomes a tool call or an API request, including a database write. OWASP’s authors moved this risk up to LLM03:2026 Excessive Agency and describe the change as “the most consequential move on the list” [3]. The root causes are too much functionality, too many permissions and too much autonomy, and a hallucination can trigger the damage as easily as an injection.

The controls at this layer sit on the tool-call path:

• Least privilege: Scope each tool to the minimum permissions it needs.
• User context: Run tools in the user’s own context, so a call can do no more than the user could do directly.
• Human approval: Require human approval before a high-impact action runs.
• Code authorization: Enforce authorization in application code, where the model cannot override it.

For the full agent threat model, including multi-step and multi-agent paths, see the guide on agent security risks.

A Practical Control Set for Production LLM Applications

The table pairs each risk with where it enters the request path, the primary control, and the test that proves it works:

The OWASP Top 10 for LLM Applications 2026 from the OWASP GenAI Security Project is the standard reference list for these threats, and the table maps onto it. Input risks map to LLM01:2026 Prompt Injection [3]. Data risks map to LLM02:2026 Sensitive Information Disclosure and LLM09:2026 Vector and Embedding Weaknesses [3]. Insecure and unfaithful output risks map to LLM10:2026 Improper Output Handling and LLM07:2026 Misinformation [3]. Hidden context exposure maps to LLM08:2026 Hidden Context Exposure [3]. Supply-chain risks map to LLM04:2026 Supply Chain and LLM05:2026 Data and Model Poisoning [3]. Tool permissions map to LLM03:2026 Excessive Agency [3]. LLM06:2026 Unbounded Consumption falls outside this guide.

How Fiddler Reduces LLM Security Risks

Each risk above enters at a point a runtime check can inspect: the prompt, the retrieved context, the response or the tool call. Fiddler Guardrails, part of the Fiddler AI Observability and Security platform, applies those checks inline and can be deployed directly through Fiddler or through the LLM or MCP gateway a team already runs, such as LiteLLM, AgentGateway or Kong Gateway, so deployment needs no new infrastructure, agent-side integration or SDK to embed. Each check allows, blocks or redacts content, whichever the policy and use case call for, with under 80ms response time:

• Input: Checks the prompt for direct prompt injection and jailbreaks, and runs PII/PHI detection and redaction and secret detection, before the model is invoked.
• Retrieved context and data: Scans RAG chunks, tool returns and file reads for indirect prompt injection and secrets before the model reads them.
• Output: Checks the response for toxicity, PII/PHI and secrets, and scores hallucination and faithfulness for RAG applications, before it reaches the user or a downstream system.
• Tool calls: On the MCP gateway, enforces tool allowlists and validates tool-call arguments before a call runs.

The batteries-included, out-of-the-box Fiddler Centor Models score these checks in-environment, so prompts and responses never leave the environment for scoring (zero data egress). Deployment options include VPC and on-premise deployments, as well as AWS GovCloud.

Request a demo to review how inline enforcement would fit the existing LLM gateway and deployment environment.

References

[1] OWASP Cheat Sheet Series, “RAG Security Cheat Sheet,” OWASP Foundation. Accessed: Oct. 14, 2026. [Online]. Available: https://cheatsheetseries.owasp.org/cheatsheets/RAG_Security_Cheat_Sheet.html

[2] D. Chismon, “Prompt injection is not SQL injection (it may be worse),” National Cyber Security Centre, Dec. 8, 2025. Accessed: Oct. 14, 2026. [Online]. Available: https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection

[3] OWASP GenAI Security Project, “OWASP Top 10 for LLM Applications 2026,” OWASP, 2026. Accessed: Oct. 14, 2026. [Online]. Available: https://genai.owasp.org/download/56857/

[4] C. Anil et al., “Many-shot Jailbreaking,” in Advances in Neural Information Processing Systems (NeurIPS), 2024. Accessed: Oct. 14, 2026. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2024/file/ea456e232efb72d261715e33ce25f208-Paper-Conference.pdf

[5] Microsoft Security Response Center, “CVE-2025-32711,” Security Update Guide, Jun. 11, 2025. Accessed: Oct. 14, 2026. [Online]. Available: https://msrc.microsoft.com/update-guide/vulnerability/CVE-2025-32711

[6] W. Zou et al., “PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models,” in Proc. 34th USENIX Security Symposium, 2025. Accessed: Oct. 14, 2026. [Online]. Available: https://www.usenix.org/conference/usenixsecurity25/presentation/zou-poisonedrag

[7] The Register, “Samsung reportedly leaked its own secrets through ChatGPT,” Apr. 6, 2023. Accessed: Oct. 14, 2026. [Online]. Available: https://www.theregister.com/software/2023/04/06/samsung-reportedly-leaked-its-own-secrets-through-chatgpt/895536

[8] National Institute of Standards and Technology, “CVE-2023-29374 Detail,” National Vulnerability Database, Apr. 4, 2023. Accessed: Oct. 14, 2026. [Online]. Available: https://nvd.nist.gov/vuln/detail/CVE-2023-29374

[9] A. Souly et al., “Poisoning attacks on LLMs require a near-constant number of poison samples,” arXiv:2510.07192, 2025. Accessed: Oct. 14, 2026. [Online]. Available: https://arxiv.org/html/2510.07192

[10] National Cyber Security Centre, “ChatGPT and large language models: what’s the risk?,” Mar. 14, 2023. Accessed: Oct. 14, 2026. [Online]. Available: https://www.ncsc.gov.uk/blog-post/chatgpt-and-large-language-models-whats-the-risk