crucible-triage¶
Standalone tool for processing fuzzer crash outputs into deduplicated, classified, CVE-ready reports.
Usage¶
Flags¶
| Flag | Default | Description |
|---|---|---|
--crashes | ./crashes | Path to crash directory |
--output | ./reports | Path to report output directory |
--minimize | false | Minimize crash reproducers (recurses into subdirectories, preserves structure) |
--harness | (none) | Path to harness binary (required for binary crash replay and --minimize) |
--sarif | (none) | Write SARIF 2.1.0 output to this file path |
--replay-timeout | 30s | Timeout per harness replay execution |
--replay-env | (none) | Extra environment variables for replay (KEY=VALUE, repeatable) |
--target | (auto) | Target surface for reports (gguf, rpc, grammar, jinja, json-schema, tokenizer, whisper, sd); auto-detected from --harness name if empty |
Binary crash files require --harness
libFuzzer and AFL++ produce binary crash reproducers (not sanitizer text logs). Without --harness, binary files are skipped with a per-file warning printed to stderr, and the process exits non-zero with a summary count. Always pass --harness when triaging binary crashes:
How It Works¶
- Reads all files from the crash directory tree (recursing into subdirectories)
- Parses ASAN/UBSAN stack traces from crash output
- Deduplicates by hashing the top 5 stack frames (normalized for ASLR)
- Classifies each unique crash by bug type
- Maps each crash type to its CWE identifier
- Scores with automatic CVSS estimation
- Generates one markdown report per unique crash
- Exports SARIF 2.1.0 output (when
--sarifis set)
Output¶
╔══════════════════════════════════════════╗
║ CRUCIBLE CRASH TRIAGE ║
╠══════════════════════════════════════════╣
║ Files processed: 47 ║
║ Unique crashes: 5 ║
╚══════════════════════════════════════════╝
Crash Statistics:
heap-buffer-overflow: 2
integer-overflow: 1
use-after-free: 1
null-dereference: 1
[a3f8b2c1] heap-buffer-overflow CVSS 9.8 (Critical)
Location: gguf.c:342 in gguf_init_from_file
Report: reports/a3f8b2c1d4e5f678.md
[b7c9d3e2] integer-overflow CVSS 8.8 (High)
Location: ggml.c:1205 in ggml_nbytes
CWE: CWE-190 — Integer Overflow or Wraparound
Report: reports/b7c9d3e2a1b4c567.md
Report Format¶
Each unique crash produces a markdown file named <stack-hash>.md:
# Vulnerability Report: CRASH-0001
**Type:** Heap Buffer Overflow
**CWE:** CWE-122 — Heap-based Buffer Overflow
**CVSS Score:** 9.8 (Critical)
**Function:** gguf_init_from_file
**Location:** gguf.c:342
**Target:** gguf
**Harness:** ./harness/libfuzzer/crucible-libfuzzer
**Minimized:** reports/minimized/crash-a3f8b2c1 *(when `--minimize` is used)*
## Description
...
## Reproducer
Path: crashes/crash-a3f8b2c1
## CVE Submission Template
...
Bug Classification¶
| ASAN Keyword | Crash Type | CVSS | Severity |
|---|---|---|---|
heap-buffer-overflow | HeapOverflow | 9.8 | Critical |
global-buffer-overflow | GlobalOverflow | 9.8 | Critical |
heap-use-after-free | UseAfterFree | 9.8 | Critical |
double-free | DoubleFree | 9.8 | Critical |
integer overflow | IntegerOverflow | 8.8 | High |
stack-buffer-overflow | StackOverflow | 7.5 | High |
FPE (divide by zero) | DivByZero | 7.5 | High |
allocator is returning null | AllocTooBig | 7.5 | High |
SEGV on unknown address | NullDeref | 5.3 | Medium |
assertion failed | AssertionFailure | 5.3 | Medium |
use-after-poison | UseAfterPoison | 5.3 | Medium |
| Timeout | Timeout | 5.3 | Medium |
out-of-memory | OOM | 5.3 | Medium |
| UBSAN enum load | EnumLoad | 3.3 | Low |
misaligned-access | MisalignedAccess | 3.3 | Low |
| Unrecognized signal | Unknown | 0.0 | Unknown |
Examples¶
# Triage crashes from default directory (sanitizer text logs only)
crucible-triage
# Replay binary crash reproducers against a harness
crucible-triage --crashes ./crashes --harness ./harness/libfuzzer/crucible-libfuzzer
# Custom directories
crucible-triage --crashes ./findings/libfuzzer --output ./reports/libfuzzer
# Triage AFL++ output
crucible-triage --crashes ./crashes/afl/default/crashes --output ./reports/afl
# Generate SARIF output for CI integration
crucible-triage --crashes ./findings --output ./reports --sarif ./results.sarif
# Triage tokenizer crashes (needs CRUCIBLE_VOCAB for replay)
crucible-triage --crashes ./crashes/tokenizer \
--harness ./harness/libfuzzer/crucible-libfuzzer-tokenizer \
--replay-env CRUCIBLE_VOCAB=/path/to/ggml-vocab-llama-bpe.gguf
# Triage whisper crashes (needs CRUCIBLE_WHISPER_MODEL for replay)
crucible-triage --crashes ./crashes/whisper \
--harness ./harness/libfuzzer/crucible-libfuzzer-whisper \
--replay-env CRUCIBLE_WHISPER_MODEL=/path/to/ggml-tiny.bin
# Override target surface for report language
crucible-triage --crashes ./crashes/rpc --target rpc
# Longer timeout for slow harnesses
crucible-triage --crashes ./crashes --harness ./my-harness --replay-timeout 60s
SARIF Output¶
When --sarif is specified, crucible-triage writes a SARIF 2.1.0 JSON file alongside the markdown reports. SARIF is consumed by GitHub Code Scanning, VS Code SARIF Viewer, and other security tooling.
The SARIF output includes:
- Tool metadata: Crucible version and description
- Rules: One rule per crash type, with CWE references, severity levels, and
target/<surface>tags - Results: One result per unique crash, with source location, fingerprints (
crucible/stackHash/v1andcrucible/target), and crash type
# View SARIF summary
jq '.runs[0].results | length' results.sarif
# 5
jq '.runs[0].results[] | {ruleId, message: .message.text}' results.sarif
CWE Classification¶
Every crash type maps to a CWE (Common Weakness Enumeration) identifier. This mapping is included in both markdown reports and SARIF output.
| Crash Type | CWE ID | CWE Name |
|---|---|---|
| HeapOverflow | CWE-122 | Heap-based Buffer Overflow |
| GlobalOverflow | CWE-120 | Buffer Copy without Checking Size of Input |
| StackOverflow | CWE-121 | Stack-based Buffer Overflow |
| UseAfterFree | CWE-416 | Use After Free |
| DoubleFree | CWE-415 | Double Free |
| NullDeref | CWE-476 | NULL Pointer Dereference |
| IntegerOverflow | CWE-190 | Integer Overflow or Wraparound |
| DivByZero | CWE-369 | Divide By Zero |
| AssertionFailure | CWE-617 | Reachable Assertion |
| OOM | CWE-400 | Uncontrolled Resource Consumption |
| AllocTooBig | CWE-789 | Memory Allocation with Excessive Size |
| Timeout | CWE-400 | Uncontrolled Resource Consumption |
| UseAfterPoison | CWE-416 | Use After Free |
| EnumLoad | CWE-843 | Access of Resource Using Incompatible Type |
| MisalignedAccess | CWE-188 | Reliance on Data/Memory Layout |
watch Subcommand¶
Continuously monitors crash directories from active fuzzing campaigns and automatically deduplicates new findings in real time.
Usage¶
Flags¶
| Flag | Default | Description |
|---|---|---|
--harness | (required) | Path to harness binary for crash replay |
--crashes | ./crashes | Root crash directory to monitor |
--output | ./crashes/dedup | Directory for deduplicated crash files |
--interval | 30s | Poll interval for new crash files |
--replay-env | (none) | Extra env vars for replay (KEY=VALUE, repeatable) |
--replay-timeout | 30s | Timeout per harness replay execution |
How It Works¶
- Loads checkpoint — on startup, reads
.crucible-watch-state.jsonlfrom the output directory to restore previously seen files and dedup hashes - Scans crash directories on each polling interval
- Replays each new crash file against the harness binary to obtain ASAN/UBSAN output
- Deduplicates by stack trace hash (top 5 frames, ASLR-normalized)
- Copies unique crashes to the dedup directory with naming pattern
{stackhash[:8]}-{original_filename} - Persists state — writes the dedup artifact first, then appends the checkpoint entry only after the file is safely on disk (O_APPEND for crash safety). If the dedup write fails, the crash is retried on the next poll.
- Prints a live summary of unique findings while watching
The watcher runs until interrupted with Ctrl+C, then prints a final summary. On restart, previously seen crashes are skipped automatically via the checkpoint file.
Examples¶
# Watch crashes from a deep harness campaign
crucible-triage watch \
--harness ./harness/libfuzzer/crucible-libfuzzer-deep \
--crashes ./crashes \
--interval 60s
# Watch with fast polling during active fuzzing
crucible-triage watch \
--harness ./harness/libfuzzer/crucible-libfuzzer \
--crashes ./crashes \
--interval 10s \
--output ./crashes/unique
# Watch tokenizer crashes (requires CRUCIBLE_VOCAB for replay)
crucible-triage watch \
--harness ./harness/libfuzzer/crucible-libfuzzer-tokenizer \
--crashes ./crashes/tokenizer \
--replay-env CRUCIBLE_VOCAB=/path/to/ggml-vocab-llama-bpe.gguf
# Watch whisper crashes with relaxed timeout
crucible-triage watch \
--harness ./harness/libfuzzer/crucible-libfuzzer-whisper \
--crashes ./crashes/whisper \
--replay-env CRUCIBLE_WHISPER_MODEL=/path/to/ggml-tiny.bin \
--replay-timeout 60s