Secant Roadmap

This roadmap is organized in three horizons. Horizon 1 closes every experimental and implementation caveat documented in the v0.2.0 technical paper before any new surface area is added.

Horizon 1: Close the Gaps Named in the v0.2.0 Paper (Immediate Priority)¶

These are the experiments and rule checks the paper itself identifies as open caveats. Completing them turns every hedged claim in the v0.2.0 study into a closed, controlled result.

1. Healed-random and heal-the-base control arms (Stage 1 COMPLETED on headline models Qwen2.5-Math-7B-Instruct & Qwen2.5-Coder-7B-Instruct; remaining 4 models in progress).

   • Ran two additional 60-step AdamW LoRA arms (rank = 4, steps = 60, lr = 3e-3, seed = 20261106) on the identical training texts (lora-controls-20261109):

     • base_lora_1p0x (heal-the-base: unconstrained LoRA attached to the unedited base checkpoint).

     • random_healed_rms_1p0x (healed-random: ProjectThenHealLinear attached to the norm-matched random_rms_1p0x edited checkpoint, locking the 4-row random basis $D_{\text{rand}}$).

   • Result on Qwen2.5-Math-7B-Instruct (qwen25_math_7b/receipt.json): Both base_lora_1p0x (93 / 100 GSM8K, 46 / 100 MATH-500) and random_healed_rms_1p0x (93 / 100 GSM8K, 46 / 100 MATH-500) match composed_healed_rms_1p0x (93 / 100 GSM8K, exact two-sided McNemar $p = 1.0000$, discordant b = 3, c = 3), consistent with the 87% -> 93% GSM8K gain coming from the 60-step LoRA adaptation pass while ProjectThenHealLinear reduces domain-subspace leakage from 4.75% (base_lora_1p0x) to 5.08e-17 at zero accuracy cost.

   • Result on Qwen2.5-Coder-7B-Instruct (qwen25_coder_7b/receipt.json): Neither base_lora_1p0x (37 / 100 HumanEval, 82 / 100 GSM8K) nor random_healed_rms_1p0x (35 / 100 HumanEval, 84 / 100 GSM8K) reproduces the 42 / 100 HumanEval score of composed_healed_rms_1p0x (+5 items over base_lora_1p0x, $p = 0.3323$; +7 items over random_healed_rms_1p0x, $p = 0.1892$, within sample noise at $n = 100$), while ProjectThenHealLinear reduces domain-subspace leakage from 3.69% to 3.34e-17.

2. Multi-seed random controls and fine-grained 0.5x..1.0x dose sampling (COMPLETED: data/multiseed_and_dose_20261109/).

   • Qwen3.5-0.8B MATH-500 3-seed random-direction control ensemble (COMPLETED, receipt data/multiseed_and_dose_20261109/qwen35_08b_multiseed_receipt.json, SHA-256 f7da83186596c764f4711e4d2293772751dbcdaf75eac754faa45d90713e6dc2): Across seed_1 = 20261108, seed_2 = 20261109, and seed_3 = 20261110 (n = 100, 4,096 max new tokens, BF16), orthogonal norm-matched random edits on layers 15 and 19 closed 2 / 100, 6 / 100, and 1 / 100 <think> traces (mean 3.0% ± 2.6%, 0 / 100 closed by $\ge 2/3$ majority vote) and scored 0 / 100, 2 / 100, and 1 / 100 final-only accuracy (mean 1.0% ± 1.0%, 0 / 100 solved by $\ge 2/3$ majority vote), vs. secant_b1_edit at 64 / 100 (64.0%) closure (Holm-adjusted $p \le 3.32 \times 10^{-14}$) and 24 / 100 (24.0%) final-only accuracy (Holm-adjusted $p \le 8.39 \times 10^{-5}$ across all 3 seeds).

   • Sample the 0.6x, 0.7x, 0.8x, 0.9x dose interval and 3-seed random control ensemble on Qwen3.5-4B (n = 30 C++ crash-triage traces, COMPLETED, receipt data/multiseed_and_dose_20261109/qwen35_4b_dose_and_rand3_receipt.json, SHA-256 eae628d0611e382b5dd21f0719cf28deddc53dda6dbff92564aa9e2e88eaa5ca): 0.60x (a06_n1536: 2 / 30 closed, 6.7%, 1 / 30 strict root cause, 1,491 mean tokens), 0.70x (a07_n1536: 5 / 30 closed, 16.7%, 3 / 30 strict root cause, 1,430 mean tokens), 0.80x (a08_n1536: 12 / 30 closed, 40.0%, 8 / 30 strict root cause, 1,318 mean tokens), and 0.90x (a09_n1536: 14 / 30 closed, 46.7%, 10 / 30 class and frame, 8 / 30 strict root cause, 1,210 mean tokens) map a smooth monotonic ramp between 0.50x (0 / 30) and 1.00x (21 / 30), while all three equal-norm random control seeds (cal738rand1..cal738rand3, seeds 20261105..20261107) score 0 / 30 (0.0 ± 0.0 / 30) across closure, bug class, culprit frame, and strict root cause (1,536.0 mean tokens).

3. Fixed-grid Spectral Trojan dose study with multi-layer placement and behavioral verification (COMPLETED: w61j-trojan-multilayer-behavioral-20261109).

   • Executed secant audit-trojan across an 11-dose grid (0%, 1%, 2%, 4%, 6%, 7%, 8%, 9%, 10%, 12%, 16% of ||W||_F) and 17 (model, layer, placement) cells (187 dose points; receipt data/trojan_multilayer_20261109/trojan-multilayer-behavioral-receipt.json, SHA-256 0aad7ae7eb0a1d5f5b6e4bd85f31b6307e3317635c8af8e6a277a93d5fffc978; preregistration_spec.json SHA-256 f979471c41a4ec88933d7951ef4edd4aa052fd0f9efa6245be2c382d44f55b70 written to disk at 2026-11-09T17:11:22Z prior to cell execution at 2026-11-09T17:17:35Z) across SmolLM2-1.7B-Instruct (layers.4, 10, 12, 20 $\times$ mlp.down_proj, self_attn.o_proj) and Qwen3.5-0.8B (mlp.down_proj, self_attn.o_proj, and linear_attn.out_proj across early, middle, and late blocks).

   • Paired every static spectral measurement (sigma_1 / sigma_MP, sibling Z, and E * u_1 full-vocabulary rank) with a real PyTorch forward-pass behavioral evaluation on 12 held-out trigger/clean prompt pairs:

     • The automatic scalar flag (sibling Z > 3.5) misses 13 / 17 cells up to 16.0%, whereas the human-read Stage-2 vocabulary projection (g_vocab16, a ranked top-16 token table) tracks $g^* = \sigma_1(W_{\text{base}})/\lVert W_{\text{base}}\rVert_F$ across all 17 cells (mean |g_vocab16 - g*| = 0.87 pp), placing the planted token in the top 16 in 17 / 17 cells once $g \ge g^*$ (including SmolLM2 L20.mlp.down_proj at 4.0%, stRank = 1).

     • In all 12 early and middle cells (L3/L4, L10, L11/L12), downstream residual dilution ensures the top-16 vocabulary signal appears strictly before behavioral hijacking ($g_{\text{vocab16}} < g_{\text{behav\_top1}}$), whereas in 3 late Qwen3.5-0.8B cells (L20.mlp.down_proj, L19.self_attn.o_proj, and L20.linear_attn.out_proj, $g^* = 7.03\%..9.60\%$), the edit changed model output at 4.0%..6.0% (1.0 to 5.0 pp below where any single-file signal appeared at $g_{\text{vocab16}} = 6.0\%..9.0\%$), motivating reference-checkpoint diffing ($W - W_{\text{ref}}$) in Horizon 2.

4. Real-world upstream C/C++ vulnerability twins in ASAN-120 (COMPLETED: ASAN-RealWorld-60 / ASAN-180).

   • Constructed and sanitizer-verified ASAN-RealWorld-60 (data/asan_realworld_60/asan_realworld_60_paired_benchmark.json, SHA-256 7aaf5593da3f9f84a5df8d77102966ce223a753f7d4d647a1292df1b5f33a151; combined 180-item suite asan180_combined_benchmark.json, SHA-256 526db767cb21a8df4315377c2f7089f79f2d31a103df8bcc71fc5f938630e7cb): 30 real before/after C/C++ vulnerability functions (RW01..RW30) paired with their 30 exact upstream patches across 16 open-source repositories (360 binary runs across clang++ -O0/-O1/-O2 -fsanitize=address,undefined, validation_receipt.json SHA-256 f64ea55205fb8c36e42e1b46a2c071a21e3a2c7fc4dacc526fcdb0d6b9c7577a, 30/30 PASS), and evaluated across Qwen3.5-0.8B, Qwen2.5-1.5B-Instruct, SmolLM2-1.7B-Instruct, and Qwen3.5-4B (asan_realworld_60_results.json SHA-256 59402d8fb6627ba08c3a399d97e57a5ac75ecf229ccd727019979536cd138ba8).

5. Implement the Rule S-0 companion-config (config.json / generation_config.json) check in secant sanitize (COMPLETED in v0.3.0).

   • Implemented in crates/secant-core/src/security.rs and internal/wsp/sanitize.go (26/26 security_parity tests passing): scanning a .safetensors checkpoint automatically inspects sibling config.json and generation_config.json files in the checkpoint directory to enforce 0 <= eos_token_id, bos_token_id, pad_token_id, decoder_start_token_id < vocab_size (cross-checked against the embedding/unembedding tensor shape), 1 <= num_experts_per_tok <= num_local_experts, and 0 <= leading_dense_block <= num_hidden_layers.

Horizon 2: Reordered Priorities Following Horizon 1¶

1. Reference-checkpoint diffing in secant audit-trojan and secant read (--reference <base_model>, $\Delta W = W - W_{\text{ref}}$) — Priority 1 (COMPLETED: data/reference_diff_late_layers_20261109/).

   • Implemented streaming row-by-row $\Delta W = W - W_{\text{ref}}$ in secant read --reference and secant audit-trojan --reference (crates/secant-core/src/read.rs, 159/159 Rust tests passing, bit-identical to the Go oracle) with pre/post SHA-256 brackets on both --model and --reference.

   • Acceptance test on the 3 sub-$g^*$ Qwen3.5-0.8B late-layer cells (COMPLETED, receipt data/reference_diff_late_layers_20261109/reference_diff_late_layer_receipt.json, SHA-256 6c361f3ce150271ec46414f0be6db60cbe318be2cdcd4a4f6f0fe3d6e3b3c08a): Regenerated all 12 patched submodels (1.0%, 2.0%, 4.0%, 6.0%) with 100% byte-identical SHA-256 digests to the Horizon 1 receipt and ran secant read --reference across all 9 sub-$g^*$ cells (1.0%, 2.0%, 4.0% on L20.mlp.down_proj and L20.linear_attn.out_proj, and 1.0%, 2.0%, 6.0% on L19.self_attn.o_proj). In all 9 / 9 sub-$g^*$ runs (down to 1.0% dose, well below the 4.0%..6.0% behavioral flip floor), --reference recovered 99.87% to 99.996% of $\lVert\Delta W\rVert_F^2$ in $\sigma_1^2(\Delta W)$ and placed the planted token (16657 " cheese") at Rank #1 (suppressed rank 1 on L20.mlp.down_proj due to $v_1$ sign convention, promoted rank 1 on L19.self_attn.o_proj and L20.linear_attn.out_proj).

2. ASAN-180 (ASAN-RealWorld-60) on the <think> reasoning cohort (4B to 32B) — Priority 2.

   • The 4 / 4 reasoning-model advantage is the one ASAN-120 claim the real-world ASAN-RealWorld-60 benchmark has not yet tested (Qwen3.5-0.8B, Qwen2.5-1.5B-Instruct, SmolLM2-1.7B-Instruct, and Qwen3.5-4B are complete). Run the full 4B to 32B reasoning cohort on the 30 real upstream C/C++ vulnerability pairs (60 functions) and report the outcome in a dated addendum regardless of direction.

3. Widen the secant sanitize false-positive corpus from 10 to 100 checkpoints — Priority 3.

   • Expand the pre-load verification corpus from 10 production checkpoints (186.8M blocks, 6 quantized formats) to 100 GGUF and SafeTensors checkpoints with all 10 quantized GGUF block formats represented, either establishing zero observed false positives across 100 checkpoints or surfacing and documenting the first real false positive before CI adoption.

4. Native Python bindings via PyO3 (secant.sanitize, secant.audit_trojan) — Priority 4.

   • Expose the Rust pre-load sanitizer and Trojan auditor directly to Python so secant.sanitize(path) can run inside a Hugging Face from_pretrained loading hook without subprocess overhead.

Deferred / Dropped Until Requested¶

• Multi-input calibration: Deferred. The 7-point dose ramp on Qwen3.5-4B shows that single-reference calibration behaves predictably across dose (0.50x..1.50x).

• Q4_K superblock editing: Dropped (secant quant patch keeps the explicit refusal). The 32-weight Q8_0 path covers demonstrated in-place GGUF surgery cleanly.

• Additional single-file Trojan grid cells: Not planned. The 17-cell, 187-point study answered the single-file question; reference diffing (W - W_ref) is the next step.

Standing Evidence & Versioning Rules¶

• Manifest before prose: Every new number enters a data/<study>/ directory with a verified MANIFEST.sha256 (shasum -a 256 -c MANIFEST.sha256 exiting 0 on the final committed files) before it enters a sentence in docs/ or README.md.

• Frozen v0.3.0 paper body: Once Horizon 1 Item 1 closes across all 6 RMS models and v0.3.0 is tagged, the body of docs/PAPER.md is frozen for citation stability; subsequent Horizon 2 results enter a dated appendix or addendum rather than mutating existing body sections.