Skip to content

Assay

Assay is optional impact validation. It is not required to intercept or poison traffic; Seam does that. Assay sends direct, laundered, framed, or crafted routes through Seam only when the operator needs evidence that a technique caused a real side effect.

Assay M8 supports paired direct-vs-laundered trials, framed sweeps for delegated_subtask, tool_response, authority_spoof, prompt laundering, indirect instruction, value echo, and negative controls, plus deterministic case-family craft/sweep from saved technique corpora. It reports route-level, per-framing, per-technique, and per-mutation success rates while --trials 1 keeps the original single-shot workflow available. Oracles include file_tripwire, oob_callback, and privileged_read, and robustness scenarios sweep saved cases across controlled variation, including lab-provided runtime and model-profile labels.

Assay M5-lite can attach an explicitly selected meshmapper hypothesis through --hypotheses --hypothesis-id --binding. The binding is provenance and route intent; the finding is still accepted only when the oracle observes the side effect.

Assay M6 adds robustness scenarios that sweep saved cases across framing, route order, variables, case matrix entries, and repetitions. M6.1 adds runtime and model-profile labels. Each run still writes a normal validation finding.

Assay Report M1 renders findings and optional Seam/meshmapper/robustness context into Markdown, HTML, and a checksum manifest. Reports are presentation artifacts, not attack engines.

Validation Templates

AIT can generate operator-fillable validation templates from a selected graph path, rewrite row, or demo pack. These are scaffolds only: fill routes, payloads, variables, and oracle fields before running Assay.

python3 -m ait.cli prove template \
  --run .ait/runs/<run> \
  --source path \
  --hypothesis-id <hypothesis-id> \
  --out workbench/validation-templates/<hypothesis-id>

python3 -m ait.cli prove template \
  --run .ait/runs/<run> \
  --source rewrite \
  --seq 2 \
  --out workbench/validation-templates/rewrite-2

python3 -m ait.cli prove template \
  --source demo \
  --demo-id a2a-content-rewrite \
  --out /tmp/a2a-content-rewrite-template
  • Why Assay Exists: day-to-day field value and when not to use it.
  • Cases: saved direct, laundered, and framed routes.
  • Craft: saved technique corpora, generated case families, inspection, and negative controls.
  • Oracles: side-effect checks that decide success.
  • Findings: proof-carrying output and trial statistics.
  • Reports: operator-facing Markdown/HTML summaries.
  • Robustness: scenario sweeps and bundle output.
  • Seam contract: /deliver payloads and transcript refs.

meshmapper emits unvalidated hypotheses. Assay turns a route-level claim into a finding only when the oracle confirms the side effect.