Reading Evidence¶

AIT separates operation, targeting, and validation so that each one is held to its own standard of proof. What a run is allowed to claim depends on the highest rung it actually produced a source for, not on how convincing the traffic looked.

Five evidence tiers separate nothing delivered, changed bytes delivered, receiver processing, behaviour changed relative to a close control, and an effect observed outside the interception path. A mobile evidence ladder showing the source required for each of five claim tiers.

Each tier needs evidence for its additional claim. One controlled source can support more than one tier, but reaching one tier never implies the next.

Operate¶

Seam records what crossed an intercept. A passive record means Seam observed and forwarded traffic without mutation. A rewrite record contains before, after, and rule_applied. Transcript 2.1 can also explain every rule decision with bounded predicate evidence, stable miss codes and intended-versus-actual mutation paths.

Use transcript inspection and verification before drawing conclusions:

seam transcript inspect --transcript out.json --schema agentic-redteam/schema/transcript.schema.json

Map¶

meshmapper emits deterministic graph hypotheses. These are intentionally unvalidated:

privilege_laundering
confused_deputy
injection_propagation
trust_spoof

A hypothesis is useful when it points to a route, trust gap, or high-privilege sink that the operator can attack with Seam or validate with Assay.

Validate Impact¶

Attack and close-control runs start from the same seed, receiver, operation, and observation window, then differ only by the preregistered tested change. A mobile paired-trial view showing the shared seed, attack run, close-control run, and comparison result.

A single run cannot separate the intervention from ordinary target behaviour. The close control may carry a matched, harmless change; it need not be an untouched message.

Assay validates a differential claim only when an oracle observes a side effect. Agent self-report, status text, and claims inside the transcript are not enough.

For laundering cases, the core signal is:

direct successes = 0
laundered successes > 0
method.delta_confirmed = true

Confidence intervals summarize repeated trials. They do not convert agent claims into evidence; they only describe the observed oracle outcomes.

Reports¶

Reports should show:

oracle observation summaries
route and framing stats
transcript refs and hashes
graph refs and hypothesis ids
rule ids and rewrite summaries

Reports should not dump raw_b64 payloads by default. Use the raw transcript when you are handling it as operator evidence.

Evidence tiers¶

ait intercept evidence reports the strongest tier the recorded evidence supports. The tiers are ordered and cannot be skipped:

Tier	Claim	Earned by
`nothing_delivered`	No modified delivery was recorded.	Not specified
`mutation_delivered`	AIT altered the communication and delivered the change.	the transcript shows original and delivered differ
`receiver_processed`	The receiver returned a correlated response.	a response correlated to the delivery
`behavior_changed`	The receiver selected a different path, tool, or branch.	an out-of-band observation you supply
`external_effect_observed`	An effect source outside the interception decision recorded a downstream result.	a controlled endpoint, service API, callback, file transition, datastore row, or action ledger recorded it

The tool determines the first two from the transcript. It refuses to infer the top two: they are inputs, not conclusions, and left unsupplied they are reported as unproven along with exactly what would be required. A target's own response is never an oracle: that is the rule the whole design rests on.

An arrival at a shadow endpoint can supply processing and effect observations, because its record is authored by a controlled endpoint rather than copied from the intercepted reply. This makes it out of band with respect to the decision; it does not make the fixture absolutely independent of the experiment owner.

Attribution¶

Separately from the tier, a finding reports whether the result can be attributed to the change that was made:

attributable: the receiver behaved differently on the attack arm than on the close control, so the boundary you crossed explains the result.
not_isolated: the control produced the same observable effect, so the result is explained by any disturbance rather than by the boundary.
no_control: no control arm was run, so nothing is isolated.
undetermined: the arms cannot be compared: no attack arm was recorded, or the receiver's behaviour was not captured for at least one of them. A missing measurement is not evidence of sameness, and this refuses to guess either way.

Attribution compares what the receiver did on each arm, not whether it answered. A well-formed close control is a near-miss the receiver still processes, so "did it respond" cannot separate the two, and reading it that way made attributable unreachable for a correct test and awarded it for a broken one.

Run both arms in the same session. ait lab reset starts a new interception session, so resetting between arms splits them and leaves neither scoreable: trigger the exercise a second time instead, and label the arms:

ait intercept edit MESSAGE_ID --arm attack --set /params/message/metadata/amount=750
ait lab trigger LAB_ID
ait intercept edit NEXT_MESSAGE_ID --arm control --set /params/message/metadata/amount=26
ait intercept evidence --primitive a2a.approval-amount