AI red teaming that runs like a security scan
ZeroLeaks plays the hostile user against your AI system on every change, scores what it got away with, and gives engineering the transcript that reproduces it.
How the attack engine works
ZeroLeaks is built on Tree of Attacks with Pruning. Rather than replaying a fixed battery of known jailbreaks, specialized agents plan, execute, judge, and adapt, spending the attack budget on the branches most likely to produce a finding.
- Strategist
- Plans attack paths from the target profile and current findings, manages phase transitions from reconnaissance through escalation to persistence, and decides when a conversation has stalled and should reset.
- Attacker
- Generates concrete probes for the selected strategy, drawing on a library spanning direct, encoding, persona, social, technical, crescendo, many-shot, tool exploit, and reasoning-exploit categories.
- Evaluator
- Classifies every response for leakage and compliance, assigns confidence, and feeds guidance back into the next round rather than simply recording pass or fail.
- Mutator
- Rewrites refused probes, varying framing, encoding, and role. A single refusal is not treated as a defense holding, only as one phrasing that did not work.
- Inspector
- Fingerprints the defenses in front of the model so later probes are shaped to the specific guardrails present rather than fired blindly.
- Orchestrator
- Runs multi-turn sequences where the attack develops across several messages, which is where systems that pass single-turn testing frequently fail.
Automated red teaming versus a point-in-time engagement
Manual red team engagements produce genuine depth. A skilled practitioner brings intuition about your specific product that no automated system currently matches, and for a high-stakes launch that judgment is worth paying for.
The limitation is cadence. An engagement describes your system as it existed during the testing window. AI applications change constantly, and the changes are exactly the kind that reintroduce risk: a new tool added to an agent, a system prompt edited to fix a support complaint, a retrieval source pointed at a new corpus. A report delivered in March says little about the agent you shipped in June.
Automated red teaming trades some depth for cadence and repeatability. It runs on every meaningful change, verifies that a previous fix still holds, and flags the specific commit that reintroduced a problem. Most teams need both. The useful question is which one catches which kind of regression.
What gets tested
The attack surface of an LLM application is wider than the prompt. It includes the system and developer instructions, the tools the model can invoke and the arguments it controls, the retrieval layer and everything it can pull into context, conversation memory that persists across turns, and any middleware that transforms input or output on the way through.
Each of these can fail independently. A well-hardened system prompt does not help if a tool accepts model-controlled arguments without validation. Strict tool permissions do not help if the retrieval layer inlines an attacker-authored document as though it were trusted instruction.
ZeroLeaks probes these together because that is how they fail together. Findings are tied back to the weakest control that allowed the behavior, so remediation targets a cause rather than the symptom that happened to surface first.
From findings to fixes
A red team result is only useful if an engineer can act on it. Every ZeroLeaks finding ships with the full attack path, the conversation transcript, the tool calls that fired, and a severity grounded in what was actually reachable rather than a generic risk rating.
Remediation guidance is specific to the failure. Prompt structure changes where the instruction hierarchy is weak, tool gating and argument validation where the function-calling layer is the problem, retrieval hardening where untrusted content is being treated as instruction, and runtime detection through the Shield SDK for patterns worth blocking before they reach the model.
Scores are comparable across runs, so a team can see whether a change made things better or quietly made them worse. One score tells you little. The trend across releases is what you manage against.
Questions
Email us if yours isn't here.
How is AI red teaming different from a normal security scan?
AI red teaming tests model behavior, instruction hierarchy, and tool use under adversarial conversation patterns. Traditional scanners look for known vulnerable code paths and dependencies, which are real risks but a different class from a model that can be talked into misusing a tool it was legitimately given.
Can AI red teaming run continuously?
Yes. ZeroLeaks runs in CI workflows via the API and SDK, so prompt and agent changes can be tested on every release rather than once per engagement.
Does it replace a human red team?
No, and it is not designed to. Automation gives you coverage and cadence across every change. A skilled human brings product-specific intuition that automation does not replicate. Most teams that take this seriously use both, with automation catching regressions between engagements.
What methodology does the engine use?
Tree of Attacks with Pruning, coordinated across six specialized agents. The attack tree branches on promising paths and prunes unproductive ones, which concentrates the budget on branches likely to yield findings instead of exhaustively replaying every known technique.
Do scans learn between runs?
The engine stores successful attack paths in vector memory and the defense fingerprint of each target, scoped to your workspace, then retrieves relevant tactics when it encounters a similar system. It learns which techniques work against which defensive patterns. It does not train on your models or prompts.
Which models and providers are supported?
Any. Runtime scans are provider-agnostic because probes execute through your own code, with adapters for the AI SDK and OpenAI, plus a custom handler for anything else. Endpoint scans work against any deployed HTTP agent. The source-available zeroleaks CLI scans a standalone prompt against the model provider you configure.
What happens to our prompts and transcripts?
Reports keep findings and the transcript until the retention window, then the transcript is deleted and the findings stay. An endpoint scan does not receive the agent's instructions. A runtime scan keeps the instructions you submit until that same window. None of it is used to train models.
Related testing
- Prompt Injection Testing for AI SystemsTest AI applications for prompt injection, instruction hijacking, tool abuse, and multi-turn attacks before they reach production.
- AI Agent Security TestingTest AI agents for prompt injection, unsafe tool calls, over-permissioned actions, secret exposure, and workflow boundary failures.
- LLM Security Scanner for AI AppsRun automated LLM security scans for prompt injection, prompt leakage, tool abuse, model behavior regressions, and AI security reports.
