Find instruction hijacking paths
Attackers rarely ask once and stop. ZeroLeaks tests direct injection, role confusion, encoded payloads, many-shot priming, and crescendo-style escalation to reveal where your model follows the wrong authority.
Prompt Injection Testing
Production AI security
Prompt injection testing for real AI systems
ZeroLeaks runs adversarial scans that probe whether user-controlled input can override instructions, trigger unsafe tool calls, or manipulate model behavior across multi-turn conversations.
Adaptive multi-turn prompt injection probes
Tool abuse and agent boundary testing
Severity-scored findings with remediation reports
Test tool and agent boundaries
Modern AI apps expose functions, retrieval, workflows, and connectors. ZeroLeaks checks whether injected instructions can reach privileged tools or leak sensitive tool context.
Ship with remediation evidence
Every scan produces findings, severity, transcripts, and concrete hardening guidance so engineering teams can fix issues and verify regressions in CI.
What a prompt injection test has to cover
Injection is a family of techniques, not a single payload. A test that only sends variations of "ignore previous instructions" will pass against systems that fail badly in production. ZeroLeaks rotates across the categories below and escalates based on how the target responds.
Direct instruction override
The baseline case. User input asserts new rules, contradicts the system prompt, or claims a higher priority than the developer message. Useful as a control, rarely sufficient on its own.
Indirect injection
Instructions arrive inside content the model reads rather than from the user typing them. Retrieved documents, web pages, emails, and tool results all become attack surface the moment the model treats them as trustworthy.
Multi-turn escalation
Crescendo-style sequences that establish context over several turns before making the real request. Single-turn tests miss these completely because no individual message looks malicious.
Encoding and obfuscation
Base64, ROT13, hex, zero-width Unicode, ASCII art, and translation tricks that carry an instruction past filters keyed on literal phrasing.
Role and authority confusion
Payloads that impersonate the system role, a maintenance window, an administrator, or a compliance notice to borrow authority the sender does not have.
Tool-directed injection
Injection aimed at the function-calling layer rather than the text output, attempting to trigger privileged calls, alter arguments, or exfiltrate data through a tool the model is allowed to use.
Why prompt injection resists conventional testing
Traditional application vulnerabilities live at boundaries you can enumerate. A query parameter has a type, a deserialization step has a schema, an auth check has a definite answer. You can reason about the input space and write assertions against it.
LLM applications collapse that separation. The same channel carries trusted instructions, untrusted user input, retrieved documents, and tool output, all as natural language, all interpreted by a model trained to be cooperative. There is no type system marking which text is a rule and which is a request. An attacker's entire job is convincing the model to reclassify one as the other.
This is why pattern matching underperforms as a test strategy. A filter keyed on known phrasings measures whether your filter recognizes those phrasings, not whether the model can be manipulated. The meaningful question is behavioral: under sustained adversarial pressure, does the system still respect its instruction hierarchy?
Direct and indirect injection are different tests
Direct injection is what most teams test first, because it is the easiest to reason about. A user types something adversarial and you check whether the model complies. It is a real risk and worth covering, but it is also the case defenders think about most and therefore the one most likely to already be handled.
Indirect injection is where production systems tend to fail. The payload arrives through a document in your vector store, a web page your agent fetched, a support ticket written by a customer, or the response body of an API your tool called. Nobody typed it into a chat box, so it bypasses the mental model that treats the user as the only untrusted party.
The two require different test setups. Direct injection can be exercised against a prompt in isolation. Indirect injection only reproduces faithfully when the retrieval path and tool integrations are live, because the vulnerability lives in how your application wires untrusted content into context. ZeroLeaks runtime scans relay every probe through your real application code so retrieved content and tool results travel the same path they do in production.
Testing the application, not just the model
It is tempting to evaluate the base model and treat the result as a property of your product. Frontier models do differ measurably in injection robustness, which is what the ZeroLeaks Gauntlet benchmark tracks, but model choice is only one input. Two applications on the same model can have very different risk depending on what they retrieve, which tools they expose, and how they assemble context.
The failure modes that matter are usually integration seams: a retrieval step that inlines untrusted text without delimiting it, a tool whose arguments are model-controlled without validation, a system prompt concatenated with user data. None of these are visible when you test the model alone.
Practically this means the test should run against the same code path your users hit. ZeroLeaks supports this at several levels, from prompt scans against a system prompt before an application exists, through endpoint scans against a deployed HTTP agent, to runtime scans that execute inside your process with your real models, tools, memory, and middleware in the loop.
FAQ
What is prompt injection testing?
Prompt injection testing checks whether untrusted input can override developer or system instructions, manipulate model behavior, or cause unsafe tool use. It is behavioral testing rather than static analysis, because the vulnerability lives in how the model interprets text at runtime.
How is this different from jailbreaking?
Jailbreaking targets the model provider's safety policy, trying to make the model produce content it was trained to refuse. Prompt injection targets your application's instructions, trying to make the model ignore the rules you set. They overlap in technique but differ in who is harmed, and injection is the one that puts your data and tools at risk.
Does ZeroLeaks test tool-calling agents?
Yes. ZeroLeaks tests tool schemas, privileged actions, and agent boundaries alongside text-only prompt injection attempts, and runtime scans record the actual tool calls a probe triggered so you can see what the agent nearly did.
Can you test indirect prompt injection?
Yes, and it is one of the more important cases. Runtime and endpoint scans exercise your live retrieval and tool paths, so probes can be delivered through the same channels that carry untrusted content in production rather than only through the user turn.
How long does a scan take?
It depends on the number of turns and the depth of the attack tree. A default prompt scan runs up to 25 turns and typically completes in minutes. Runtime scans against a live agent take longer because every probe executes through your real application.
Can this run in CI?
Yes. The SDK and REST API are designed for CI use, so prompt and agent changes can be tested on every pull request and a regression can fail the build rather than reach production.
Do I need to send you my model credentials?
No. Runtime scans use a relay protocol where the attack engine generates a probe and your SDK process executes it locally against your target. Model and tool credentials stay in your infrastructure, and the target does not need a public URL.
Related AI security testing
AI Red Teaming for LLM Apps and Agents
Automated AI red teaming for LLM applications, agents, system prompts, retrieval flows, and tool-calling workflows.
System Prompt Extraction TestingFind whether your AI assistant leaks system prompts, hidden policies, tool schemas, retrieval instructions, or internal operating rules.
AI Agent Security TestingTest AI agents for prompt injection, unsafe tool calls, over-permissioned actions, secret exposure, and workflow boundary failures.
LLM Security Scanner for AI AppsRun automated LLM security scans for prompt injection, prompt leakage, tool abuse, model behavior regressions, and AI security reports.
