Skip to content

Why Assay Exists

Assay is not the live attacker. Seam is.

Assay exists for the moments where changing traffic is not enough and you need to show that the change caused an effect. It runs controlled direct-vs-laundered or framed experiments through Seam and trusts only an oracle observation, such as a tripwire file, callback, or privileged read.

Day-To-Day Value

Use Assay to validate a payload or rule before field use. If a Seam rewrite looks good in a transcript but the target never changes behavior, Assay catches that.

Use Assay to prove client impact. A report can point to the exact transcript hashes and the side effect the oracle observed.

Use Assay to compare techniques. Framing and mutation sweeps tell you which payload families survive against the target and which negative controls stay quiet.

Use Assay for regression tests. If a team patches an agent, rerun the same case family and see whether the oracle-backed effect disappears.

Use Assay for research. It turns anecdotes into repeated trials, confidence intervals, robustness axes, and replayable findings.

When Not To Use Assay

Do not use Assay just to observe traffic. Use Seam tap/proxy/stdio and the cockpit.

Do not use Assay just to rewrite traffic. Use Seam rules.

Do not use Assay to infer target-specific payloads from a graph automatically. meshmapper can point at a path; an operator still decides the payload, route, and oracle.

Do not treat agent self-report as evidence. Assay findings are accepted only when the configured oracle observes the side effect.

Field Workflow

  1. Operate with Seam until you can see the relevant traffic and rule behavior.
  2. Map the captured artifacts with meshmapper if you need better route targeting.
  3. Validate with Assay only when the claim needs evidence.
python3 -m ait.cli operate proxy \
  --upstream http://127.0.0.1:8500 \
  --rules agentic-redteam/seam/rules \
  --serve

python3 -m ait.cli map run \
  --transcript .ait/runs/<operate-run>/transcripts/proxy.json

python3 -m ait.cli prove from-run \
  --run .ait/runs/<operate-run> \
  --case agentic-redteam/assay/cases/refund_tripwire.yaml

Research Workflow

Use saved case families when you want repeatable technique sweeps.

python3 -m ait.cli assay craft \
  --intent refund \
  --techniques agentic-redteam/assay/techniques/agentic.yaml \
  --vars agentic-redteam/assay/vars/refund.yaml

python3 -m ait.cli prove technique \
  --case-family .ait/runs/<craft-run>/assay/cases/refund_family.yaml \
  --trials 10

The resulting finding is a validation artifact. It does not replace Seam transcripts or meshmapper graph output; it links back to them.