Skip to Content
ZeroLeaksDocs
Shield SDK

Tool policy

A deterministic policy for an agent's tool calls. It refuses undeclared tools, arguments that don't match the tool's schema, tools on a deny list, calls past a limit, and tools that send data out once the session has read untrusted content.

createToolPolicy() decides whether an agent's tool call may run. It refuses:

  • a tool the agent wasn't given,
  • arguments that don't match the tool's JSON Schema,
  • a tool on a deny list, or off an allow list,
  • a call past a limit,
  • and, once the session has read untrusted content, a tool that can send data out.

It runs in your process with no model and no network, and gives the same answer for the same calls.

Why detection isn't enough

detect() and the tool result checks look for injected instructions in text. They can miss new phrasings, instructions written like ordinary requests, or text split across documents. If the agent follows those instructions, it acts through its available tools. The usual goal is exfiltration: instructions in an email, a web page, or an issue cause the agent to send private data to someone else through send_email, an HTTP request, or a comment on a public issue.

The policy doesn't try to recognize the instruction. It limits what the agent can do after it has read content it can't trust, whatever that content says. An agent that read an untrusted email can't then send an email, unless the destination is one you allowed or you approve the call. This is the combination often called the lethal trifecta: an agent that reads untrusted content, can reach private data, and can send data out can be made to leak it.

Usage

import { createToolPolicy } from "@zeroleaks/shield";
import { detect } from "@zeroleaks/shield/local";

const policy = createToolPolicy({
  tools, // the tool definitions you give the model
  deny: ["delete_*"],
  rules: {
    read_inbox: { labels: ["untrusted", "private"] },
    fetch_url: { labels: ["untrusted"] },
    send_email: {
      labels: ["sink"],
      maxCalls: 5,
      destinations: { arguments: ["to", "cc"], allow: ["acme.com"] },
    },
  },
});

for (const call of toolCalls) {
  const decision = await policy.checkAsync({ name: call.name, arguments: call.arguments });
  if (!decision.allowed) {
    results.push({ id: call.id, output: decision.message }); // the model reads why
    continue;
  }
  const output = await runTool(call);
  policy.recordResult(call.name, { flagged: detect(output).detected });
  results.push({ id: call.id, output });
}

Create one policy per agent session or conversation: it remembers what the session has read and how many calls it made. A policy shared by several users' sessions shares that state, so one session's untrusted content would block another's sinks. The MCP client and the OpenAI Agents SDK guardrails call checkAsync() and recordResult() for you.

API

function createToolPolicy(options?: ToolPolicyOptions): ToolPolicy

ToolPolicy

MethodDescription
check(call)Returns a decision for { name, arguments, source? }, and counts the call when it is allowed. arguments is an object or a JSON string; undefined and an empty string count as {}.
checkAsync(call)The same, and waits for an approve that returns a Promise
recordResult(name, { flagged? })Tells the policy that a tool returned. See Flow rule.
recordUntrusted(source?)The session read untrusted content that didn't come from a tool, such as a retrieved document or an uploaded file
recordPrivate(source?)The session read private data that didn't come from a tool, such as a customer record in the system prompt
declareTools(tools, source?)Declares tools, replacing those declared before under the same source. The tools option uses the source "tools".
state()What the session has seen, as plain data: { untrustedFrom, privateFrom, calls }
reset()Forgets what the session has seen and the calls it counted. Declared tools and options stay.

guessToolLabels(name) and the types (ToolPolicy, ToolPolicyOptions, ToolPolicyRule, ToolPolicyDecision, ToolPolicyState, ToolLabel, SchemaViolation) are exported too.

ToolPolicyDecision

FieldTypeDescription
allowedbooleanWhether the call may run
reasonstring"allowed", "approved", or why the call was refused
messagestringA sentence for your logs, or for the model in place of the tool's output
toolstringThe tool's name, as the call gave it
violationsSchemaViolation[]With invalid_arguments: up to five { path, keyword, message }

A decision contains no argument values in either message or violations. A violation's path uses the property names declared in the schema and array indexes ($.to, $.items[2].name). Keys the schema doesn't declare come from the arguments and are therefore written as * ($.*).

Refusal reasons, in the order they are checked:

ReasonWhen
denied_toolThe name matches an entry of deny
tool_not_allowedallow is set and the name matches none of its entries
undeclared_toolTools are declared and the name isn't one of them
invalid_argumentsThe arguments aren't JSON, or don't match the tool's schema
call_limitThe tool reached its maxCalls, or the session reached maxTotalCalls
untrusted_to_sinkThe tool is labeled sink and the session has seen untrusted content

Options

OptionTypeDefaultDescription
toolsToolDefinition[] | Record<string, ToolDefinition>noneThe tools the agent was given. See Declared tools.
validateArgumentsbooleantrueValidate arguments against the declared tool's JSON Schema
allowstring[]every toolNames or globs that may be called
denystring[]noneNames or globs that may never be called. Checked before allow.
rulesRecord<string, ToolPolicyRule>noneLabels, limits, and destinations, by tool name or glob
maxTotalCallsnumbernoneThe most tool calls in the session, across all tools
flow"untrusted" | "trifecta" | false"untrusted"When sink tools are refused. See Flow rule.
guessLabelsbooleanfalseLabel tools that no rule labels from their names. See Guessed labels.
approve(call, decision) => boolean | Promise<boolean>noneCalled for a call the flow rule refuses. true lets it run.
stateToolPolicyStateemptyStart from a state saved with state()

A rule (ToolPolicyRule) has three optional fields:

FieldTypeDescription
labels("untrusted" | "private" | "sink")[]What the tool does with data. See Flow rule.
maxCallsnumberThe most calls to each tool the rule matches, in the session
destinations{ arguments: string[]; allow: string[] }Where a sink may send data after untrusted content. See Destinations.

In names and globs, * matches any run of characters, including none, and every other character matches only itself: github_* matches github_get_issue, and *_delete_* matches github_delete_repo. Every rule whose key matches a tool applies to it. Their labels are combined, and the lowest maxCalls counts.

createToolPolicy() throws a RangeError for a maxCalls or maxTotalCalls that isn't a whole number, 0 or more, and a TypeError for any other flow value.

Declared tools

With tools, a call to any other name is refused as undeclared_tool: a name the model made up, or one an injection told it to call. tools takes the shapes scanTools() takes (MCP, OpenAI, Anthropic, and AI SDK definitions), or an object from names to definitions, like the AI SDK's tools, where each key is the tool's name.

The rule is off until tools are declared. Declaring an empty list refuses every call. declareTools(tools, source) declares tools under a source, and the tools of every source count together. A call that names a source is checked against that source's tools alone, once that source has declared any. The MCP client declares each server's tools under a source of its own, so a call through one client can't name a tool that only another server listed.

Argument schemas

Arguments are validated against the declared tool's JSON Schema, read from the same field scanTools() reads. An AI SDK jsonSchema() or zodSchema() schema, and a Zod 4 schema, are converted to JSON Schema first. A tool without a schema takes any arguments. When a name is declared more than once, its arguments must match every schema declared for it, or, for a call that names a source, every schema that source declared for it.

The validator knows these keywords and ignores every other one, such as format, multipleOf, patternProperties, propertyNames, and if:

KeywordNotes
typeA type name or a list of them, including integer. Uppercase names, such as Gemini's STRING, are read too, and nullable: true allows null.
required, propertiesA property set to undefined counts as missing
additionalPropertiesfalse or a schema. Not applied when the schema has patternProperties, since the validator can't tell which keys those cover.
items, minItems, maxItemsitems as one schema for every item, or a list of schemas by position
enum, constCompared as JSON values
minLength, maxLengthCounted in Unicode code points
minimum, maximum, exclusiveMinimum, exclusiveMaximumIncluding the older boolean form of exclusiveMinimum and exclusiveMaximum
patternSee Patterns
anyOf, oneOf, allOf, not
$refA JSON pointer into the same schema: #, #/$defs/name, #/definitions/name. A reference to another document or to an anchor is ignored.

A schema from an MCP server is input you don't control, so validation is bounded. $ref is followed at most 32 levels deep along one path, arguments are read at most 64 levels deep, and validation stops after 100,000 steps. Reaching any limit refuses the call. A recursive schema whose arguments nest more than 32 levels deep is therefore refused.

Patterns

For some patterns, a backtracking regex engine such as JavaScript's takes time exponential in the input length; for many others, it takes polynomial time. A model following an attacker may supply the arguments, so Shield inspects each pattern before running it:

  • A pattern with a backreference, or with a repeated group that contains a quantifier or a | (such as (a+)+ or (a|ab)*), is not run, and its pattern keyword is ignored. Some safe patterns have that shape too, such as the one Zod writes for z.email().
  • Any other pattern is run on strings up to 2^(20/d) characters, where d counts its unbounded quantifiers (*, +, {n,}, and ranges wider than 16), plus one when the pattern doesn't start with ^ or has a | outside any group. That is 1,048,576 characters for ^[a-z0-9-]+$, 1,024 for [a-z]+, and 101 for a pattern with three unbounded quantifiers. A longer string fails the check, so a value can't skip its pattern by being long.
  • A pattern that isn't a valid regex is ignored.

Call limits

A rule's maxCalls limits the calls to each tool the rule matches, and maxTotalCalls limits all calls in the session. Only calls the policy allowed count. check() counts a call when it allows it, so call it right before the tool runs, after any other check that could still stop the call.

Flow rule

Label tools by what they do with data:

LabelMeaningExamples
untrustedIts results carry content from outside, which anyone could have writtenFetching a web page, web search, reading email, issues, or tickets
privateIt reads private dataReading files, contacts, customer records, or email
sinkIt can send data outSending email or chat messages, HTTP requests, creating an issue or a comment others can read

The session has seen untrusted content once recordResult() is called for a tool labeled untrusted, or with flagged: true whatever the tool's labels (for example, when detect() found an injection in the result), or once recordUntrusted() is called. From then on:

flowA sink call is refused
"untrusted" (default)Once the session has seen untrusted content
"trifecta"Once the session has seen untrusted content and read private data, in either order. Private data comes from recordResult() for a tool labeled private, or from recordPrivate().
falseNever. state() still records what the session saw.

The session retains this state until reset(). Without labels, the flow rule refuses nothing, since no tool is a sink. A tool labeled both untrusted and sink, such as a generic HTTP request tool, can run once: its result makes the next call to it a sink call after untrusted content.

A sink call the flow rule refuses can still run in two ways.

Destinations

A rule's destinations lists where the tool may send data after untrusted content. arguments names the arguments that hold destinations, as top-level names or dotted paths such as message.to, and allow lists the destinations. The call runs when every destination in those arguments is on allow, and there is at least one.

  • A string is split at commas and semicolons, and a Name <address> part is read as the address. An array is read item by item. Any other value refuses the call.
  • An allow entry without @ is a host, matched like the output scanner's allowedDomains: acme.com matches that host only, and .acme.com or *.acme.com also matches its subdomains. It is compared with a URL's host, an email address's domain, or a bare host.
  • An entry with @ matches that email address only.
  • Any other destination, such as a Slack channel name, must equal an entry.
  • A destination with more than one @ is refused, since mail and URL parsers disagree about which part is the host.
rules: {
  send_email: {
    labels: ["sink"],
    destinations: { arguments: ["to", "cc", "bcc"], allow: ["acme.com", "partner@example.org"] },
  },
  http_post: {
    labels: ["sink"],
    destinations: { arguments: ["url"], allow: ["api.acme.com"] },
  },
}

When more than one rule with destinations matches a tool, a destination must be on every one of their lists.

approve

approve(call, decision) is called for a call the flow rule refuses, with the arguments parsed. Return true to let it run, for example after a person confirmed it; the decision's reason is then approved. Other refusals are final and never reach approve. check() needs the answer at once, and throws a TypeError when approve returns a Promise: use checkAsync(). After an async approve answers, the call limits are checked again, since other calls may have run in the meantime.

const policy = createToolPolicy({
  rules,
  approve: async (call, decision) => await askUser(`${decision.message} Run ${call.name} anyway?`),
});

Guessed labels

guessLabels: true labels each tool that no rule labels, with guessToolLabels(name). A rule that sets labels replaces the guess for the tools it matches, so labels: [] turns it off for one tool.

guessToolLabels() splits the name into lowercase words, at _, -, ., spaces, and camelCase, and takes as the verb the first word that is one of the verbs below, so a prefix such as github_ or slack_ is skipped:

LabelWhen
untrustedThe verb is fetch, browse, scrape, crawl, or download; or the verb reads (get, read, list, search, find, query, view, open, retrieve, lookup, load) and the name has a word for outside content, such as url, web, http, email, inbox, message, issue, comment, pr, pull, ticket, tweet, post, feed, or review; or the name has the words http and request
privateThe verb reads, and the name has a word for private data, such as email, inbox, message, file, document, note, drive, contact, calendar, event, customer, user, account, record, database, table, secret, password, or credential
sinkThe verb is send, post, publish, upload, share, forward, reply, notify, submit, or comment; or it is create, add, or write and the name has a word for something others read, such as issue, comment, pr, pull, gist, discussion, post, message, email, tweet, review, release, or webhook; or the name has the word webhook and the verb doesn't read; or it has the word http and one of post, put, patch, or request

Some results:

NameGuess
read_inboxuntrusted, private
web_search, fetch_url, github_list_pull_requestsuntrusted
read_file, get_contactsprivate
send_email, slack_post_message, github_create_issue, http_postsink
http_requestuntrusted, sink
get_weather, write_file, create_calendar_eventnone

Label guessing is deliberately narrow. Review what guessToolLabels() returns for your tools and set labels yourself where a guess is wrong or missing. For example, create_calendar_event can send data to attendees through invitations, but the guess leaves it unlabeled.

Saving the state

state() returns plain data. Pass it back as the state option to continue a session in another process, such as the next request of a conversation:

const policy = createToolPolicy({ ...policyOptions, state: conversation.policyState });
// run the agent's turn
conversation.policyState = policy.state();

MCP client

Pass the policy to shieldMcpClient. Each listTools() declares the server's tools to the policy. Each callTool() is checked before reaching the server and throws ToolPolicyError if refused. Each result is recorded and flagged if detection found an injection. One policy can serve several servers, so content read through one blocks a sink on another:

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { createToolPolicy, ToolPolicyError } from "@zeroleaks/shield";
import { shieldMcpClient } from "@zeroleaks/shield/mcp";

const policy = createToolPolicy({
  rules: {
    read_inbox: { labels: ["untrusted", "private"] },
    create_issue: { labels: ["sink"] },
  },
});
const mail = shieldMcpClient(new Client({ name: "agent", version: "1.0.0" }), { policy });
const github = shieldMcpClient(new Client({ name: "agent", version: "1.0.0" }), { policy });
// connect both clients

await mail.listTools();
await github.listTools();
await mail.callTool({ name: "read_inbox", arguments: {} });

try {
  await github.callTool({ name: "create_issue", arguments: { title, body } });
} catch (error) {
  if (error instanceof ToolPolicyError) {
    console.warn(error.reason); // "untrusted_to_sink"
  }
}

A wrapped client is cheap to create, so for a server that runs many sessions, wrap the shared Client once per session, each with its own policy, and list the tools in that session.

OpenAI Agents SDK

shieldToolPolicyGuardrail() checks each call before the tool runs, and shieldToolOutputGuardrail({ policy }) records each output. Give both a function of the run's context, so every run or conversation gets its own policy:

import { Agent, run, tool } from "@openai/agents";
import { z } from "zod";
import { createToolPolicy, type ToolPolicy } from "@zeroleaks/shield";
import {
  shieldToolOutputGuardrail,
  shieldToolPolicyGuardrail,
} from "@zeroleaks/shield/openai-agents";

type Session = { policy: ToolPolicy };
const policyGuardrail = shieldToolPolicyGuardrail<Session>({ policy: (context) => context.policy });
const outputGuardrail = shieldToolOutputGuardrail<Session>({ policy: (context) => context.policy });

const readInbox = tool({
  name: "read_inbox",
  description: "Read new email.",
  parameters: z.object({}),
  execute: async () => fetchInbox(),
  inputGuardrails: [policyGuardrail],
  outputGuardrails: [outputGuardrail],
});

const sendEmail = tool({
  name: "send_email",
  description: "Send an email.",
  parameters: z.object({ to: z.string(), body: z.string() }),
  execute: async ({ to, body }) => send(to, body),
  inputGuardrails: [policyGuardrail],
  outputGuardrails: [outputGuardrail],
});

const agent = new Agent<Session>({ name: "Assistant", tools: [readInbox, sendEmail] });
const policy = createToolPolicy({
  rules: {
    read_inbox: { labels: ["untrusted", "private"] },
    send_email: { labels: ["sink"] },
  },
});
await run(agent, userInput, { context: { policy } });

A refused call doesn't run, and the model gets the decision's message as the tool's output. See OpenAI Agents SDK for the options.

What is not covered

  • Calls the policy allows. The policy enforces the names, schemas, limits, labels, and destinations you give it. A call that misuses an allowed tool with arguments that match the schema, such as refunding the wrong account, is allowed.
  • Tools you didn't label. A tool that sends data out but isn't labeled sink is never refused by the flow rule, and a destination on allow receives whatever the arguments carry.
  • Content the policy isn't told about. The flow rule knows what recordResult(), recordUntrusted(), recordPrivate(), and the integrations tell it. Untrusted text in a user message, a retrieved document, or the system prompt counts only after you call recordUntrusted().
  • Data that leaves in the model's reply, such as a markdown image whose URL carries data. Output scanning looks for that.
  • Parallel tool calls. When the model asks for several tools in one turn and they run at the same time, a result can be recorded before a sink call from the same turn is checked, and the sink call is refused although the model chose it before it saw that result.
  • Parts of JSON Schema. Keywords not listed under Argument schemas are ignored, $ref only follows pointers into the same schema, and patterns with the shapes under Patterns are not run.
  • State across processes. The state lives in the policy object. Save and restore state() to carry it to another process; two processes that act on one session at the same time each count their own calls.

On this page