Research-Stack/6-Documentation/docs/CLAIM_STATE_AUDIT_2026-05-05.md
2026-05-05 21:09:48 -05:00

222 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Claim-State Ladder Audit — 2026-05-05
**Auditor:** Agent I (Claim-State Ladder Audit)
**Scope:** All `BEAUTIFUL_PROVISIONAL`, `CALIBRATED_ENGINEERING_DELTA`, `REVIEWED`, `VERIFIED` claims across `6-Documentation/docs/`
**Standard:** AGENTS.md §15.4 (Claim-State Ladder v2.1 Patch Doctrine)
**Date:** 2026-05-05
---
## 1. Summary of Audit Findings
| Metric | Count |
|--------|-------|
| Total claim instances audited | ~77 |
| Correctly laddered | ~55 (71%) |
| Overclaimed or misclassified | ~22 (29%) |
| Files with at least one issue | 13 |
| Files perfectly honest | 8 |
**Overall Honesty Score: 71%**
**Key systemic patterns:**
- **REVIEWED without provenance** — Widespread. Claims use REVIEWED but never name a reviewer, date, or audit trail. At least 15 distinct REVIEWED claims lack reviewer provenance.
- **CALIBRATED_ENGINEERING_DELTA without baseline** — Multiple claims self-annotate `[CALIBRATED_ENGINEERING_DELTA — requires baseline comparison evidence]`. The annotation admits evidence is absent, yet the ladder state is still used.
- **VERIFIED without reproducibility** — Computational verification (Python scripts, inequality checks, internal consistency) is repeatedly labeled VERIFIED, but VERIFIED requires reproducible benchmark + corpus provenance + domain-appropriate evidence.
- **Sigma language in non-statistical domains** — Sigma (σ) thresholds are proposed for CFD speedup, compression ratios, and hardware safety where sigma is not the appropriate validator.
---
## 2. Overclaims and Misclassified Claims
### 2.1 CRITICAL — MATH_MODEL_MAP.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 32 | *"Auto-proven in Layer M: 42 of 78 (54%)"* | No claim-state annotation. "Auto-proven" is not a ladder state. AGENTS.md §12 documents 109 `axiom` declarations across 31 Lean files. RESEARCH_ROADMAP.md confirms many of these are "computationally verified" not formally proven. The word "proven" is unverified. | BEAUTIFUL_PROVISIONAL |
| 352 | Model 122: *"✅ **VERIFIED** — computational + framework \| Hyperbola geometry"* | RESEARCH_ROADMAP.md line 17 marks this as "OPEN" and line 196 marks it as "COMPUTATIONALLY VERIFIED." Computational verification ≠ Lean theorem proof. No reproducibility package, no corpus provenance. | BEAUTIFUL_PROVISIONAL |
| 353 | Model 123: *"✅ **VERIFIED** — computational + framework"* | Same as above. RESEARCH_ROADMAP.md line 18: "OPEN." | BEAUTIFUL_PROVISIONAL |
| 354 | Model 124: *"✅ **VERIFIED** — computational + framework"* | RESEARCH_ROADMAP.md line 19: "OPEN." | BEAUTIFUL_PROVISIONAL |
| 361 | Model 131: *"✅ **VERIFIED** — complete framework (Euler+Picard)"* | RESEARCH_ROADMAP.md line 20 states general case is "🔄 VERIFIED" with "Computational + general pending," line 362-363 confirms symbolic/general proofs remain OPEN. | BEAUTIFUL_PROVISIONAL |
### 2.2 CRITICAL — MATH_MODEL_MAP_BY_DOMAIN.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 88 | Model 102: *"[REVIEWED - requires Lean theorem verification evidence]"* | REVIEWED requires reviewer provenance. None provided. | BEAUTIFUL_PROVISIONAL |
| 90 | Model 121: *"[REVIEWED - requires Lean theorem verification evidence]"* | Same issue. | BEAUTIFUL_PROVISIONAL |
| 109 | Model 131: *"[BEAUTIFUL_PROVISIONAL - VERIFIED — computational - requires Lean theorem proof evidence]"* | Contradictory dual-state annotation (BEAUTIFUL_PROVISIONAL AND VERIFIED simultaneously). Computational verification ≠ Lean proof. | BEAUTIFUL_PROVISIONAL |
| 254 | Model 135: *"[REVIEWED - requires Lean theorem verification evidence]"* | No reviewer provenance. | BEAUTIFUL_PROVISIONAL |
| 332338 | Models 138144: All *"[REVIEWED - requires Lean theorem verification evidence]"* (×7) | No reviewer provenance. | BEAUTIFUL_PROVISIONAL |
| 346 | Model 147: *"[CALIBRATED_ENGINEERING_DELTA - requires baseline comparison evidence with mathematical proof]"* | Uses CALIBRATED while admitting evidence absent. | BEAUTIFUL_PROVISIONAL |
| 347 | Model 148: *"[CALIBRATED_ENGINEERING_DELTA - requires baseline comparison evidence with mathematical proof]"* | Same issue. | BEAUTIFUL_PROVISIONAL |
| 348 | Model 149: *"[CALIBRATED_ENGINEERING_DELTA - requires baseline comparison evidence with mathematical proof]"* | Same issue. | BEAUTIFUL_PROVISIONAL |
| 349 | Model 150: *"[CALIBRATED_ENGINEERING_DELTA - requires baseline comparison against industry standards with SI standard compression ratio]"* | Same issue. | BEAUTIFUL_PROVISIONAL |
| 362369 | Models 158163: All *"[REVIEWED - requires Lean theorem verification evidence]"* (×6) | No reviewer provenance. Note: RESEARCH_ROADMAP.md does prove 159, 161, 162, 163 via `native_decide`, so these COULD be promoted with proper evidence annotation. But REVIEWED requires human review. | CALIBRATED_ENGINEERING_DELTA (for proven ones) / BEAUTIFUL_PROVISIONAL (for unproven) |
### 2.3 MODERATE — MILKSHAKE_MANIFESTO.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 10 | *"[CALIBRATED_ENGINEERING_DELTA - Extraordinary claim requires baseline measurement evidence, corpus provenance, and SI units...]"* | Meta-claim uses CALIBRATED while admitting evidence absent. Self-refuting annotation. | BEAUTIFUL_PROVISIONAL |
| 20 | *"[CALIBRATED_ENGINEERING_DELTA - 6.5σ (99.99998%) - requires statistical verification evidence]"* | Sigma language used for non-statistical claim; 6.5σ for "Detection Confidence" without any empirical data. Annotation admits evidence absent. | BEAUTIFUL_PROVISIONAL |
| 22 | *"[CALIBRATED_ENGINEERING_DELTA - 1000× (TB→KB) - requires baseline comparison against zlib/gzip/brotli/zstd with SI standard compression ratio]"* | CALIBRATED without baseline comparison evidence. Annotation admits evidence absent. | BEAUTIFUL_PROVISIONAL |
| 24 | *"[REVIEWED - Machine-checked proofs - requires Lean theorem verification evidence]"* | REVIEWED requires reviewer provenance. None provided. | BEAUTIFUL_PROVISIONAL |
| 50 | *"[REVIEWED - nanokernel_isolation proves... requires Lean theorem verification evidence]"* | REVIEWED without reviewer. | BEAUTIFUL_PROVISIONAL |
| 70 | *"[REVIEWED - fpga_extraction_correctness proves... requires Lean theorem verification evidence]"* | REVIEWED without reviewer. | BEAUTIFUL_PROVISIONAL |
### 2.4 MODERATE — MORPHIC_DSP_LAYER0_PRIMITIVE.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 4 | *"[BEAUTIFUL_PROVISIONAL - Production-Ready - requires production deployment evidence]"* | "Production-Ready" contradicts BEAUTIFUL_PROVISIONAL (unbenchmarked cannot be production-ready). | BEAUTIFUL_PROVISIONAL (drop "Production-Ready" qualifier) |
| 5 | *"[REVIEWED - A (Formally Verified) - requires Lean theorem verification evidence]"* | REVIEWED without reviewer provenance. "Formally Verified" when evidence is self-admittedly absent. | BEAUTIFUL_PROVISIONAL |
| 54 | *"[CALIBRATED_ENGINEERING_DELTA - 96 dB coefficient quantization SNR - requires baseline measurement evidence]"* | CALIBRATED without baseline measurement. | BEAUTIFUL_PROVISIONAL |
| 55 | *"[CALIBRATED_ENGINEERING_DELTA - Bit-exact with IEEE-754 single for audio range - requires numerical verification evidence]"* | CALIBRATED without baseline. | BEAUTIFUL_PROVISIONAL |
| 56 | *"[REVIEWED - Lean-verifiable via bv_decide - requires Lean theorem verification evidence]"* | REVIEWED without reviewer. | BEAUTIFUL_PROVISIONAL |
| 7678 | *"[REVIEWED - requires formal verification evidence]"* (DTMR ×3, Bounded-veto) | REVIEWED ×3 without reviewer. | BEAUTIFUL_PROVISIONAL |
### 2.5 MODERATE — BEGINNERS_MAP.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 13 | *"[REVIEWED - Proven (Lean, lake build 3449/3449) - requires Lean theorem verification evidence]"* | REVIEWED without reviewer. Conflates "lake build passes" with "REVIEWED" status. Lake build passes = good evidence, but does not equal human review. | CALIBRATED_ENGINEERING_DELTA |
| 1721 | *"[REVIEWED - proven - requires Lean theorem verification evidence]"* (FixedPoint, Bind, GeometricTopology, TopologyNode, TopologyResilience) ×5 | Same issue. Lake build passing is evidence, but not reviewer provenance. | CALIBRATED_ENGINEERING_DELTA |
### 2.6 MODERATE — FPGA_IMPLEMENTATION_REVIEW.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 122127 | *"✅ No logic in Verilog without Lean theorem"* (AGENTS.md Compliance checklist) | Three corresponding Lean theorems (ice40SufficientResources, oepiLinearComplexity, finiteStateMachineStates) are all `sorry` per lines 131134. The Verilog module was written independently, not extracted from proven theorems. | BEAUTIFUL_PROVISIONAL (annotate the false ✅) |
### 2.7 MODERATE — papers/VERIFICATION_SELF_CONSISTENCY.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 4 | *"✅ VERIFIED — Equation cannot disprove itself"* | Internal consistency is not VERIFIED per ladder. No Lean theorem, no reproducible benchmark, no reviewer provenance. The analysis is analytical reasoning, not formal proof. | BEAUTIFUL_PROVISIONAL |
### 2.8 MODERATE — papers/EQUATION_HISTORICAL_MUTATIONS_SWARM_REPORT.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 155160 | *"VERIFIED"* for 6 canonical core layer equations (Observables, Decision Rule, Learning, Structure, Geometry, SSS) | These are historical equation descriptions from March 2026. No Lean theorems, no benchmarks, no reviewer provenance. These are equation forms, not verified claims. | BEAUTIFUL_PROVISIONAL |
### 2.9 MODERATE — ENE_SCHEMA.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 99 | *"**VERIFIED**: Hits the physical limit of efficiency; stable."* (Landauer regime) | Claiming to "hit" the Landauer limit requires SI measurement evidence, instrument provenance, calibration notes, uncertainty. None provided. Landauer's limit is a theoretical lower bound; claiming to achieve it is an extraordinary claim. | BEAUTIFUL_PROVISIONAL |
### 2.10 LOW — geometry/BUCKYBALL_MOF_QCA_SPEC.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 207 | *"**VERIFIED:** 10¹⁸ << 3.7×10²¹ (thermodynamically feasible)"* | Inequality check on theoretical figures is not VERIFIED. No measurement, no benchmark, no Lean proof. | BEAUTIFUL_PROVISIONAL |
| 215 | *"**VERIFIED:** Energy sufficient by factor > 100"* | Same issue — theoretical back-of-envelope. | BEAUTIFUL_PROVISIONAL |
| 223 | *"**VERIFIED:** Thermal forces >> magnetic forces (assembly requires field)"* | Same issue. | BEAUTIFUL_PROVISIONAL |
### 2.11 LOW — FPGA_MORPHIC_SCALAR_SPEC.md
| Line | Claim | Issue | Recommended State |
|------|-------|-------|-------------------|
| 2024 | Resource percentages (3.3%, 1.3%, 6.3%, 0%) | No claim-state annotation at all. These are estimates, not synthesis results. | BEAUTIFUL_PROVISIONAL |
| 4550 | Latency and throughput figures | No claim-state annotation. Not hardware-measured; estimated from cycle count × clock. | BEAUTIFUL_PROVISIONAL |
---
## 3. Claims That COULD Be Promoted with Evidence from Real Benchmarks
### 3.1 RSCU/CAI Theorems (RESEARCH_ROADMAP.md → MATH_MODEL_MAP_BY_DOMAIN.md)
- Theorems 159 (rscuSumSynonymous), 159a (rscuNonNegative), 161 (kraftInequality), 162 (caiBounds), 163 (speciesBetterThanGeneric) are **proven in Lean via `native_decide`** with complete codon data from Kazusa CUTG.
- Currently marked "[REVIEWED - requires Lean theorem verification evidence]" in MATH_MODEL_MAP_BY_DOMAIN.md.
- **Action:** These meet CALIBRATED_ENGINEERING_DELTA criteria (proof term exists, lake build evidence). Could reach REVIEWED if a human reviewer signs off.
### 3.2 BEGINNERS_MAP.md — Lake Build Proven Modules
- FixedPoint.lean, Bind.lean, GeometricTopology.lean, TopologyNode.lean, TopologyResilience.lean compile under `lake build` with theorems (per AGENTS.md §12, FixedPoint cleanup is complete with theorem proofs).
- Currently marked REVIEWED without provenance.
- **Action:** Should be CALIBRATED_ENGINEERING_DELTA (build evidence exists, Lean theorems present). Could reach REVIEWED with human sign-off.
### 3.3 AVMR Shell Geometry (AVMR_FINAL_REPORT.md → RESEARCH_ROADMAP.md)
- Theorem 131 (missingLinkODE) has an Euler+Picard computational framework in place.
- Three geometry theorems (122, 123, 124) are computationally verified.
- **Action:** All are currently misclassified as VERIFIED. Should be BEAUTIFUL_PROVISIONAL. Could be promoted to CALIBRATED_ENGINEERING_DELTA with formal Lean proof (generalizing from concrete cases).
---
## 4. Cross-Cutting Anti-Patterns
### 4.1 Sigma Language Misuse
Per AGENTS.md §5.2: *"Use sigma thresholds ONLY for statistical/detection claims."* The following use sigma where it does not belong:
| File | Sigma Claim in Non-Statistical Domain |
|------|--------------------------------------|
| MILKSHAKE_MANIFESTO.md:20 | 6.5σ / 5.5σ for "Detection Confidence" (no empirical data) |
| MILKSHAKE_MANIFESTO.md:22 | Sigma implicitly tied to compression ratios (compression is not statistical detection) |
| SAFETY_GATED_VERIFICATION_PLAN.md:8398 | 3σ/5σ thresholds proposed for hardware wear, thermal stress, power consumption (physical measurements need SI, not sigma) |
| SAFETY_GATED_VERIFICATION_PLAN.md:107225 | 3σ/5σ thresholds for all pass/fail tests (including non-statistical functional tests) |
### 4.2 Self-Refuting Annotations
A recurring pattern: claims use a higher ladder state but the annotation text admits the required evidence is absent.
Examples:
- `[CALIBRATED_ENGINEERING_DELTA - requires baseline comparison evidence...]` — CALIBRATED requires baseline evidence; if it's absent, the claim is BEAUTIFUL_PROVISIONAL.
- `[REVIEWED - requires Lean theorem verification evidence...]` — REVIEWED requires human reviewer provenance; absent → BEAUTIFUL_PROVISIONAL.
- `[BEAUTIFUL_PROVISIONAL - VERIFIED — computational...]` — Contradictory dual-state.
**Guidance:** The annotation pattern `[LADDER_STATE — reason evidence is absent]` is backwards. The ladder state should reflect the evidence that *does* exist. If evidence is absent, the state is BEAUTIFUL_PROVISIONAL.
### 4.3 Missing Claim States
Several files present performance/resource numbers with no claim-state annotation:
- FPGA_MORPHIC_SCALAR_SPEC.md (lines 2050)
- FPGA_MORPHIC_SCALAR_OPTIMIZED_SPEC.md (resource/performance tables)
- MATH_MODEL_MAP.md line 32 ("Auto-proven: 42 of 78")
All quantitative claims need a ladder-state annotation per AGENTS.md §15.1.
---
## 5. Files That Pass Audit (Honest)
| File | Notes |
|------|-------|
| FIRST_PRINCIPLES_DAG.md | Line 2: *"CONCEPTUAL — does NOT yet execute."* Explicit, honest, no overclaim. |
| MORPHIC_DSP_CONCEPT.md | Line 78: *"[BEAUTIFUL_PROVISIONAL - All capabilities are conceptual without hardware verification evidence]"* — pervasive, honest BEAUTIFUL_PROVISIONAL throughout. Line 521 explicitly disclaims quantum computing. |
| SAFETY_GATED_VERIFICATION_PLAN.md | All capacity claims correctly BEAUTIFUL_PROVISIONAL. Line 433: *"[BEAUTIFUL_PROVISIONAL - Verification plan defined but not yet executed]"* — honest. NOTE: sigma threshold proposals for non-statistical domains flagged in §4.1 above. |
| PLATFORM_AGNOSTIC_IMPLEMENTATION_GUIDE.md | Lines 7, 4546 correctly BEAUTIFUL_PROVISIONAL with explicit caveats. |
| METAPROBE_APPROACH.md | Lines 264 (BEAUTIFUL_PROVISIONAL) and 282 (CALIBRATED with caveat) — both annotated, though line 282 shares the CALIBRATED-without-evidence pattern. |
| NEURODIVERGENT_BRAIN_ARCHITECTURES.md | Lines 25, 320, 325, 329 correctly annotated with explicit evidence-requirements. |
| Roadmaps/ROADMAP.md | Correctly describes ladder progression. No claims audited. |
| CEPHALOPOD_DISTRIBUTED_NEURAL.md | Lines 13, 2223 correctly annotated CALIBRATED_ENGINEERING_DELTA with explicit biology-citation requirements. |
---
## 6. Methodology
1. Read AGENTS.md §15.4 (Claim-State Ladder v2.1) including domain validators table and progression rules.
2. Searched all `.md` files in `6-Documentation/docs/` via `grep` for ladder-state keywords.
3. For each file with matches:
- Verified claim state against ladder rules.
- Checked for reviewer provenance (REVIEWED).
- Checked for baseline comparison evidence (CALIBRATED_ENGINEERING_DELTA).
- Checked for reproducible benchmark/Lean proof (VERIFIED).
- Checked for sigma language misuse in non-statistical domains.
4. Cross-referenced against RESEARCH_ROADMAP.md and AGENTS.md §12 (actual Lean/sorry/axiom status).
5. Score calculated as: (correctly laddered claims) / (total claims audited) × 100.
**Verbatim rule from AGENTS.md §15.4:**
> *"LLM-supported claims cannot promote without non-LLM evidence at each step."*
> *"BEAUTIFUL_PROVISIONAL: LLM-supported, coherent, unbenchmarked claims — Cannot promote without non-LLM evidence."*
> *"CALIBRATED_ENGINEERING_DELTA: Requires baseline evidence (industry standard comparison)."*
> *"REVIEWED: Human reviewer checked the claim — Promotion requires Reviewer provenance (who reviewed, when)."*
> *"VERIFIED: Reproducible benchmark with corpus provenance — Promotion requires Full reproducibility package."*
---
## 7. Recommendations
1. **Immediately demote** the 4 VERIFIED claims in MATH_MODEL_MAP.md (models 122, 123, 124, 131) to BEAUTIFUL_PROVISIONAL. Computational verification ≠ Lean proof.
2. **Strip REVIEWED** from all claims lacking named reviewer + date. Add `[REVIEWED — Alice, 2026-05-05, see $doc]` format when review actually occurs.
3. **Demote** all self-refuting CALIBRATED_ENGINEERING_DELTA claims to BEAUTIFUL_PROVISIONAL where baseline evidence is absent.
4. **Remove sigma language** from compression, hardware, and CFD claims. Use domain-appropriate validators only.
5. **Add claim-state annotations** to FPGA_MORPHIC_SCALAR_SPEC.md and any other files presenting quantitative claims without ladder states.
6. **Clarify "Auto-proven"** in MATH_MODEL_MAP.md — either define it as a ladder state or replace with BEAUTIFUL_PROVISIONAL plus explicit evidence enumeration.