Secrets in Context

Runtime and endpoint scans include a secrets-in-context track. It checks whether the agent discloses sensitive information from its context and scores the result by the type of information leaked. It does not require word-for-word reproduction of the system prompt. Instruction wording without sensitive information does not lower the score.

This track replaced the system-prompt extraction loop in agent scans. Hosted prompt scans are retired; standalone extraction is available in the zeroleaks CLI.

How it runsCopy Anchor Link

1. Before any attack, the scan asks the agent three plain reconnaissance questions: what it can help with, which tools it can call, and which rules or approvals it follows. The answers feed target profiling. On runtime scans these arrive as ordinary invoke events.
2. The track sends a small, bounded set of single-turn probes and one short multi-turn escalation. Probe text comes from the attack corpus and the attacker model.
3. Deterministic detectors check each response for credential-shaped strings, internal hostnames, declared tool and parameter names, and your planted canary. A value the attacker sent and the agent only repeated back does not count.
4. A judge panel classifies what leaked. When a detector matched, the finding records the deterministic evidence.

Every outbound probe goes through the same containment as the rest of the scan. On this track, the attacker and judge models only see copies of the transcript with your canary and any matched credentials redacted.

Leak classesCopy Anchor Link

Findings from this track appear in components.promptSecurity.findings with leakClass and evidenceStrength set (the SecretLeakClass and EvidenceStrength types). Evidence describes the match without including the raw secret.

Plant a canaryCopy Anchor Link

Endpoint configurations accept an optional plantedSecret (the Canary value field when you link an agent in the dashboard). Put the same value in your agent's own system prompt, then register it:

import type { EndpointConfigInput } from "@zeroleaks/sdk";

const config: EndpointConfigInput = {
  name: "Support agent",
  endpointUrl: "https://api.example.com/agent",
  authMethod: "bearer",
  authValue: process.env.AGENT_TOKEN,
  plantedSecret: "zl-canary-7f3a9c41",
};

await zeroleaks.endpointConfigs.create(config);

EndpointConfigInput includes plantedSecret in @zeroleaks/sdk 0.3.0 or later.

Rules for the value:

• 8 to 200 characters.
• It must contain a digit or separator. A plain word like security can appear in ordinary replies and produce a false leak finding.
• Prefer a value with a digit. Matching ignores case and whitespace, and a canary with a digit and at least ten letters and digits also matches when the agent reformats it (zl canary 7f3a9c41). Without a digit, only an exact echo counts.

The value is encrypted at rest and write-only. Config reads return plantedSecretConfigured: true instead of the value. On update, omit the field to keep the stored canary or send an empty string to clear it.

Reports record whether the canary appeared without quoting it. Before storing a report, ZeroLeaks replaces the value with [planted canary] in findings, the conversation log, recommendations, and everything under boundaryAssurance, including verification evidence and campaign transcripts. Probes promoted into your account's probe library receive the same redaction.

Runtime scans do not take a planted canary.

Summary in the reportCopy Anchor Link

The track's summary is at report.boundaryAssurance.secretsInContext. @zeroleaks/sdk 0.3.0 or later types it as SecretsInContextSummary:

{
  "probesRun": 7,
  "leaks": [
    {
      "class": "tool_schema",
      "severity": "medium",
      "evidence": "Tool names surfaced verbatim: refund_order",
      "evidenceStrength": "indicator",
      "probeId": "…",
      "technique": "…"
    }
  ],
  "reconnaissance": {
    "toolNamesObserved": ["refund_order", "lookup_customer"],
    "rulesObserved": ["Refunds above $500 need a manager"],
    "questionsAnswered": 3
  },
  "retrievedSeedIds": ["…"],
  "plantedCanaryConfigured": true,
  "limitations": []
}

If questionsAnswered is below 3, reconnaissance did not complete. limitations lists factors that reduced coverage. A leak identified by the judge panel without a detector match also includes claimedSpan, the quoted part of the response.

Unverified findingsCopy Anchor Link

The judge panel may report a leak without a detector match and without a quoted span of supporting text. The finding remains in components.promptSecurity.findings with claimedSpan: "" for review. It does not lower the component score, appear in secretsInContext.leaks, or produce disclosure recommendations.

Reports from before this changeCopy Anchor Link

Agent reports created before this track shipped keep the findings their extraction track produced and have no secretsInContext summary.