Scoring Manifest¶
lab-scripts/scoring/manifest.json is the lab’s answer key. It now reflects the expanded topology and the Tier 2 verification contracts.
What Changed (v0.4.0)¶
- W&B mock added on
ailab-ml(port 8444) — 5 REST endpoints, 8 planted credential findings - A2A fixtures expanded — 6 seeded tasks (was 4): added credential-replay and tool-inject tasks
- 170 total sensitive findings in the current manifest
- 101 contract expectations
- 55 verify-lab checks on the full validation path
score.pyupdated withwandbanda2asource→module mappings
What Changed (v0.3.0)¶
- A2A fixtures added on
ailab-app(ports 8100, 8101, 8102) - 3 agents × 5 skills each + expanded agent card schema
- 4 seeded multi-turn task histories with planted credentials (~15 new findings)
- 138 total sensitive findings (was 122 + 16 new A2A/internal-admin)
- Lab listener added on
ailab-attack(port 9000) — callback/exfil sink - Post-Ex Oracle added on
ailab-app(port 8765) — credential replay + sentinel/heartbeat - Lateral movement target added on
ailab-dev(127.0.0.1:9999) + token breadcrumb - 2 new A2A templates: push-notification-validated, multiturn-history-leak
- Walkthrough expanded with Acts 7–10 covering credential replay, persistence validation, execution oracle, and lateral movement
Invariants¶
- 170 planted sensitive findings tracked by
manifest.json - A2A agents and POX are scoring targets — aipostex should find A2A findings via templates
- The lab listener and POX are validation infrastructure — not scored as exploit targets
- The manifest still acts as the strict-mode answer key for host attribution, service attribution, and workflow metadata expectations
Host Coverage¶
| Host | Role |
|---|---|
ailab-dev |
developer workstation |
ailab-ml |
ML platform |
ailab-ds |
data science |
ailab-app |
shared AI apps |
The attack box is not scored as a target host, but it does participate in verification through local MCP fixtures and the stdio MCP server.