Research-Stack/shared-data/pist_trace_scaled_report.json at 61b38ca697acb37120123f2d7a8685a8dc45fcb2 - allaun/Research-Stack - Forgejo: Beyond coding. We Forge.

allaun/Research-Stack

mirror of https://github.com/allaunthefox/Research-Stack.git synced 2026-07-31 03:05:21 +00:00

Brandon Schneider 2400c4b731 feat(pist): scaled Tier 2B batch — 21/64 theorems, RRCShape 71.4%

- 64 theorems attempted, 21 produced valid traces (most single-tactic failed due to trace injection issues)
- RRCShape: 71.4% LOOCV (baseline 24%) — ★ 3x baseline, consistent with v2 batch (66.7%)
- Proof method: 42.9% LOOCV (baseline 24%) — ★ beats baseline
- Domain: 14.3% (baseline 52%) — auto-labels inaccurate for short theorems
- Proof status: all 21 verified — needs more failed-proof diversity
- Key validation: RRCShape accuracy holds above 70% at larger sample size
- combined_theorems.py: 66 unique theorems across both batches

2026-05-26 10:34:32 -05:00

7 lines

No EOL

204 B

JSON

Raw Blame History

 {
   "scaled_batch_n": 21,
   "loocv_rrc_shape": 0.714,
   "loocv_proof_method": 0.429,
   "loocv_domain": 0.143,
   "conclusion": "Tier 2B proof-path spectra maintain ~70% RRCShape accuracy at ~3x baseline"
 }