Research-Stack/6-Documentation/docs/CLAIM_STATE_AUDIT_2026-05-05.md
2026-05-05 21:09:48 -05:00

18 KiB
Raw Permalink Blame History

Claim-State Ladder Audit — 2026-05-05

Auditor: Agent I (Claim-State Ladder Audit) Scope: All BEAUTIFUL_PROVISIONAL, CALIBRATED_ENGINEERING_DELTA, REVIEWED, VERIFIED claims across 6-Documentation/docs/ Standard: AGENTS.md §15.4 (Claim-State Ladder v2.1 Patch Doctrine) Date: 2026-05-05


1. Summary of Audit Findings

Metric Count
Total claim instances audited ~77
Correctly laddered ~55 (71%)
Overclaimed or misclassified ~22 (29%)
Files with at least one issue 13
Files perfectly honest 8

Overall Honesty Score: 71%

Key systemic patterns:

  • REVIEWED without provenance — Widespread. Claims use REVIEWED but never name a reviewer, date, or audit trail. At least 15 distinct REVIEWED claims lack reviewer provenance.
  • CALIBRATED_ENGINEERING_DELTA without baseline — Multiple claims self-annotate [CALIBRATED_ENGINEERING_DELTA — requires baseline comparison evidence]. The annotation admits evidence is absent, yet the ladder state is still used.
  • VERIFIED without reproducibility — Computational verification (Python scripts, inequality checks, internal consistency) is repeatedly labeled VERIFIED, but VERIFIED requires reproducible benchmark + corpus provenance + domain-appropriate evidence.
  • Sigma language in non-statistical domains — Sigma (σ) thresholds are proposed for CFD speedup, compression ratios, and hardware safety where sigma is not the appropriate validator.

2. Overclaims and Misclassified Claims

2.1 CRITICAL — MATH_MODEL_MAP.md

Line Claim Issue Recommended State
32 "Auto-proven in Layer M: 42 of 78 (54%)" No claim-state annotation. "Auto-proven" is not a ladder state. AGENTS.md §12 documents 109 axiom declarations across 31 Lean files. RESEARCH_ROADMAP.md confirms many of these are "computationally verified" not formally proven. The word "proven" is unverified. BEAUTIFUL_PROVISIONAL
352 Model 122: " VERIFIED — computational + framework | Hyperbola geometry" RESEARCH_ROADMAP.md line 17 marks this as "OPEN" and line 196 marks it as "COMPUTATIONALLY VERIFIED." Computational verification ≠ Lean theorem proof. No reproducibility package, no corpus provenance. BEAUTIFUL_PROVISIONAL
353 Model 123: " VERIFIED — computational + framework" Same as above. RESEARCH_ROADMAP.md line 18: "OPEN." BEAUTIFUL_PROVISIONAL
354 Model 124: " VERIFIED — computational + framework" RESEARCH_ROADMAP.md line 19: "OPEN." BEAUTIFUL_PROVISIONAL
361 Model 131: " VERIFIED — complete framework (Euler+Picard)" RESEARCH_ROADMAP.md line 20 states general case is "🔄 VERIFIED" with "Computational + general pending," line 362-363 confirms symbolic/general proofs remain OPEN. BEAUTIFUL_PROVISIONAL

2.2 CRITICAL — MATH_MODEL_MAP_BY_DOMAIN.md

Line Claim Issue Recommended State
88 Model 102: "[REVIEWED - requires Lean theorem verification evidence]" REVIEWED requires reviewer provenance. None provided. BEAUTIFUL_PROVISIONAL
90 Model 121: "[REVIEWED - requires Lean theorem verification evidence]" Same issue. BEAUTIFUL_PROVISIONAL
109 Model 131: "[BEAUTIFUL_PROVISIONAL - VERIFIED — computational - requires Lean theorem proof evidence]" Contradictory dual-state annotation (BEAUTIFUL_PROVISIONAL AND VERIFIED simultaneously). Computational verification ≠ Lean proof. BEAUTIFUL_PROVISIONAL
254 Model 135: "[REVIEWED - requires Lean theorem verification evidence]" No reviewer provenance. BEAUTIFUL_PROVISIONAL
332338 Models 138144: All "[REVIEWED - requires Lean theorem verification evidence]" (×7) No reviewer provenance. BEAUTIFUL_PROVISIONAL
346 Model 147: "[CALIBRATED_ENGINEERING_DELTA - requires baseline comparison evidence with mathematical proof]" Uses CALIBRATED while admitting evidence absent. BEAUTIFUL_PROVISIONAL
347 Model 148: "[CALIBRATED_ENGINEERING_DELTA - requires baseline comparison evidence with mathematical proof]" Same issue. BEAUTIFUL_PROVISIONAL
348 Model 149: "[CALIBRATED_ENGINEERING_DELTA - requires baseline comparison evidence with mathematical proof]" Same issue. BEAUTIFUL_PROVISIONAL
349 Model 150: "[CALIBRATED_ENGINEERING_DELTA - requires baseline comparison against industry standards with SI standard compression ratio]" Same issue. BEAUTIFUL_PROVISIONAL
362369 Models 158163: All "[REVIEWED - requires Lean theorem verification evidence]" (×6) No reviewer provenance. Note: RESEARCH_ROADMAP.md does prove 159, 161, 162, 163 via native_decide, so these COULD be promoted with proper evidence annotation. But REVIEWED requires human review. CALIBRATED_ENGINEERING_DELTA (for proven ones) / BEAUTIFUL_PROVISIONAL (for unproven)

2.3 MODERATE — MILKSHAKE_MANIFESTO.md

Line Claim Issue Recommended State
10 "[CALIBRATED_ENGINEERING_DELTA - Extraordinary claim requires baseline measurement evidence, corpus provenance, and SI units...]" Meta-claim uses CALIBRATED while admitting evidence absent. Self-refuting annotation. BEAUTIFUL_PROVISIONAL
20 "[CALIBRATED_ENGINEERING_DELTA - 6.5σ (99.99998%) - requires statistical verification evidence]" Sigma language used for non-statistical claim; 6.5σ for "Detection Confidence" without any empirical data. Annotation admits evidence absent. BEAUTIFUL_PROVISIONAL
22 "[CALIBRATED_ENGINEERING_DELTA - 1000× (TB→KB) - requires baseline comparison against zlib/gzip/brotli/zstd with SI standard compression ratio]" CALIBRATED without baseline comparison evidence. Annotation admits evidence absent. BEAUTIFUL_PROVISIONAL
24 "[REVIEWED - Machine-checked proofs - requires Lean theorem verification evidence]" REVIEWED requires reviewer provenance. None provided. BEAUTIFUL_PROVISIONAL
50 "[REVIEWED - nanokernel_isolation proves... requires Lean theorem verification evidence]" REVIEWED without reviewer. BEAUTIFUL_PROVISIONAL
70 "[REVIEWED - fpga_extraction_correctness proves... requires Lean theorem verification evidence]" REVIEWED without reviewer. BEAUTIFUL_PROVISIONAL

2.4 MODERATE — MORPHIC_DSP_LAYER0_PRIMITIVE.md

Line Claim Issue Recommended State
4 "[BEAUTIFUL_PROVISIONAL - Production-Ready - requires production deployment evidence]" "Production-Ready" contradicts BEAUTIFUL_PROVISIONAL (unbenchmarked cannot be production-ready). BEAUTIFUL_PROVISIONAL (drop "Production-Ready" qualifier)
5 "[REVIEWED - A (Formally Verified) - requires Lean theorem verification evidence]" REVIEWED without reviewer provenance. "Formally Verified" when evidence is self-admittedly absent. BEAUTIFUL_PROVISIONAL
54 "[CALIBRATED_ENGINEERING_DELTA - 96 dB coefficient quantization SNR - requires baseline measurement evidence]" CALIBRATED without baseline measurement. BEAUTIFUL_PROVISIONAL
55 "[CALIBRATED_ENGINEERING_DELTA - Bit-exact with IEEE-754 single for audio range - requires numerical verification evidence]" CALIBRATED without baseline. BEAUTIFUL_PROVISIONAL
56 "[REVIEWED - Lean-verifiable via bv_decide - requires Lean theorem verification evidence]" REVIEWED without reviewer. BEAUTIFUL_PROVISIONAL
7678 "[REVIEWED - requires formal verification evidence]" (DTMR ×3, Bounded-veto) REVIEWED ×3 without reviewer. BEAUTIFUL_PROVISIONAL

2.5 MODERATE — BEGINNERS_MAP.md

Line Claim Issue Recommended State
13 "[REVIEWED - Proven (Lean, lake build 3449/3449) - requires Lean theorem verification evidence]" REVIEWED without reviewer. Conflates "lake build passes" with "REVIEWED" status. Lake build passes = good evidence, but does not equal human review. CALIBRATED_ENGINEERING_DELTA
1721 "[REVIEWED - proven - requires Lean theorem verification evidence]" (FixedPoint, Bind, GeometricTopology, TopologyNode, TopologyResilience) ×5 Same issue. Lake build passing is evidence, but not reviewer provenance. CALIBRATED_ENGINEERING_DELTA

2.6 MODERATE — FPGA_IMPLEMENTATION_REVIEW.md

Line Claim Issue Recommended State
122127 " No logic in Verilog without Lean theorem" (AGENTS.md Compliance checklist) Three corresponding Lean theorems (ice40SufficientResources, oepiLinearComplexity, finiteStateMachineStates) are all sorry per lines 131134. The Verilog module was written independently, not extracted from proven theorems. BEAUTIFUL_PROVISIONAL (annotate the false )

2.7 MODERATE — papers/VERIFICATION_SELF_CONSISTENCY.md

Line Claim Issue Recommended State
4 " VERIFIED — Equation cannot disprove itself" Internal consistency is not VERIFIED per ladder. No Lean theorem, no reproducible benchmark, no reviewer provenance. The analysis is analytical reasoning, not formal proof. BEAUTIFUL_PROVISIONAL

2.8 MODERATE — papers/EQUATION_HISTORICAL_MUTATIONS_SWARM_REPORT.md

Line Claim Issue Recommended State
155160 "VERIFIED" for 6 canonical core layer equations (Observables, Decision Rule, Learning, Structure, Geometry, SSS) These are historical equation descriptions from March 2026. No Lean theorems, no benchmarks, no reviewer provenance. These are equation forms, not verified claims. BEAUTIFUL_PROVISIONAL

2.9 MODERATE — ENE_SCHEMA.md

Line Claim Issue Recommended State
99 "VERIFIED: Hits the physical limit of efficiency; stable." (Landauer regime) Claiming to "hit" the Landauer limit requires SI measurement evidence, instrument provenance, calibration notes, uncertainty. None provided. Landauer's limit is a theoretical lower bound; claiming to achieve it is an extraordinary claim. BEAUTIFUL_PROVISIONAL

2.10 LOW — geometry/BUCKYBALL_MOF_QCA_SPEC.md

Line Claim Issue Recommended State
207 "VERIFIED: 10¹⁸ << 3.7×10²¹ (thermodynamically feasible)" Inequality check on theoretical figures is not VERIFIED. No measurement, no benchmark, no Lean proof. BEAUTIFUL_PROVISIONAL
215 "VERIFIED: Energy sufficient by factor > 100" Same issue — theoretical back-of-envelope. BEAUTIFUL_PROVISIONAL
223 "VERIFIED: Thermal forces >> magnetic forces (assembly requires field)" Same issue. BEAUTIFUL_PROVISIONAL

2.11 LOW — FPGA_MORPHIC_SCALAR_SPEC.md

Line Claim Issue Recommended State
2024 Resource percentages (3.3%, 1.3%, 6.3%, 0%) No claim-state annotation at all. These are estimates, not synthesis results. BEAUTIFUL_PROVISIONAL
4550 Latency and throughput figures No claim-state annotation. Not hardware-measured; estimated from cycle count × clock. BEAUTIFUL_PROVISIONAL

3. Claims That COULD Be Promoted with Evidence from Real Benchmarks

3.1 RSCU/CAI Theorems (RESEARCH_ROADMAP.md → MATH_MODEL_MAP_BY_DOMAIN.md)

  • Theorems 159 (rscuSumSynonymous), 159a (rscuNonNegative), 161 (kraftInequality), 162 (caiBounds), 163 (speciesBetterThanGeneric) are proven in Lean via native_decide with complete codon data from Kazusa CUTG.
  • Currently marked "[REVIEWED - requires Lean theorem verification evidence]" in MATH_MODEL_MAP_BY_DOMAIN.md.
  • Action: These meet CALIBRATED_ENGINEERING_DELTA criteria (proof term exists, lake build evidence). Could reach REVIEWED if a human reviewer signs off.

3.2 BEGINNERS_MAP.md — Lake Build Proven Modules

  • FixedPoint.lean, Bind.lean, GeometricTopology.lean, TopologyNode.lean, TopologyResilience.lean compile under lake build with theorems (per AGENTS.md §12, FixedPoint cleanup is complete with theorem proofs).
  • Currently marked REVIEWED without provenance.
  • Action: Should be CALIBRATED_ENGINEERING_DELTA (build evidence exists, Lean theorems present). Could reach REVIEWED with human sign-off.

3.3 AVMR Shell Geometry (AVMR_FINAL_REPORT.md → RESEARCH_ROADMAP.md)

  • Theorem 131 (missingLinkODE) has an Euler+Picard computational framework in place.
  • Three geometry theorems (122, 123, 124) are computationally verified.
  • Action: All are currently misclassified as VERIFIED. Should be BEAUTIFUL_PROVISIONAL. Could be promoted to CALIBRATED_ENGINEERING_DELTA with formal Lean proof (generalizing from concrete cases).

4. Cross-Cutting Anti-Patterns

4.1 Sigma Language Misuse

Per AGENTS.md §5.2: "Use sigma thresholds ONLY for statistical/detection claims." The following use sigma where it does not belong:

File Sigma Claim in Non-Statistical Domain
MILKSHAKE_MANIFESTO.md:20 6.5σ / 5.5σ for "Detection Confidence" (no empirical data)
MILKSHAKE_MANIFESTO.md:22 Sigma implicitly tied to compression ratios (compression is not statistical detection)
SAFETY_GATED_VERIFICATION_PLAN.md:8398 3σ/5σ thresholds proposed for hardware wear, thermal stress, power consumption (physical measurements need SI, not sigma)
SAFETY_GATED_VERIFICATION_PLAN.md:107225 3σ/5σ thresholds for all pass/fail tests (including non-statistical functional tests)

4.2 Self-Refuting Annotations

A recurring pattern: claims use a higher ladder state but the annotation text admits the required evidence is absent.

Examples:

  • [CALIBRATED_ENGINEERING_DELTA - requires baseline comparison evidence...] — CALIBRATED requires baseline evidence; if it's absent, the claim is BEAUTIFUL_PROVISIONAL.
  • [REVIEWED - requires Lean theorem verification evidence...] — REVIEWED requires human reviewer provenance; absent → BEAUTIFUL_PROVISIONAL.
  • [BEAUTIFUL_PROVISIONAL - VERIFIED — computational...] — Contradictory dual-state.

Guidance: The annotation pattern [LADDER_STATE — reason evidence is absent] is backwards. The ladder state should reflect the evidence that does exist. If evidence is absent, the state is BEAUTIFUL_PROVISIONAL.

4.3 Missing Claim States

Several files present performance/resource numbers with no claim-state annotation:

  • FPGA_MORPHIC_SCALAR_SPEC.md (lines 2050)
  • FPGA_MORPHIC_SCALAR_OPTIMIZED_SPEC.md (resource/performance tables)
  • MATH_MODEL_MAP.md line 32 ("Auto-proven: 42 of 78")

All quantitative claims need a ladder-state annotation per AGENTS.md §15.1.


5. Files That Pass Audit (Honest)

File Notes
FIRST_PRINCIPLES_DAG.md Line 2: "CONCEPTUAL — does NOT yet execute." Explicit, honest, no overclaim.
MORPHIC_DSP_CONCEPT.md Line 78: "[BEAUTIFUL_PROVISIONAL - All capabilities are conceptual without hardware verification evidence]" — pervasive, honest BEAUTIFUL_PROVISIONAL throughout. Line 521 explicitly disclaims quantum computing.
SAFETY_GATED_VERIFICATION_PLAN.md All capacity claims correctly BEAUTIFUL_PROVISIONAL. Line 433: "[BEAUTIFUL_PROVISIONAL - Verification plan defined but not yet executed]" — honest. NOTE: sigma threshold proposals for non-statistical domains flagged in §4.1 above.
PLATFORM_AGNOSTIC_IMPLEMENTATION_GUIDE.md Lines 7, 4546 correctly BEAUTIFUL_PROVISIONAL with explicit caveats.
METAPROBE_APPROACH.md Lines 264 (BEAUTIFUL_PROVISIONAL) and 282 (CALIBRATED with caveat) — both annotated, though line 282 shares the CALIBRATED-without-evidence pattern.
NEURODIVERGENT_BRAIN_ARCHITECTURES.md Lines 25, 320, 325, 329 correctly annotated with explicit evidence-requirements.
Roadmaps/ROADMAP.md Correctly describes ladder progression. No claims audited.
CEPHALOPOD_DISTRIBUTED_NEURAL.md Lines 13, 2223 correctly annotated CALIBRATED_ENGINEERING_DELTA with explicit biology-citation requirements.

6. Methodology

  1. Read AGENTS.md §15.4 (Claim-State Ladder v2.1) including domain validators table and progression rules.
  2. Searched all .md files in 6-Documentation/docs/ via grep for ladder-state keywords.
  3. For each file with matches:
    • Verified claim state against ladder rules.
    • Checked for reviewer provenance (REVIEWED).
    • Checked for baseline comparison evidence (CALIBRATED_ENGINEERING_DELTA).
    • Checked for reproducible benchmark/Lean proof (VERIFIED).
    • Checked for sigma language misuse in non-statistical domains.
  4. Cross-referenced against RESEARCH_ROADMAP.md and AGENTS.md §12 (actual Lean/sorry/axiom status).
  5. Score calculated as: (correctly laddered claims) / (total claims audited) × 100.

Verbatim rule from AGENTS.md §15.4:

"LLM-supported claims cannot promote without non-LLM evidence at each step." "BEAUTIFUL_PROVISIONAL: LLM-supported, coherent, unbenchmarked claims — Cannot promote without non-LLM evidence." "CALIBRATED_ENGINEERING_DELTA: Requires baseline evidence (industry standard comparison)." "REVIEWED: Human reviewer checked the claim — Promotion requires Reviewer provenance (who reviewed, when)." "VERIFIED: Reproducible benchmark with corpus provenance — Promotion requires Full reproducibility package."


7. Recommendations

  1. Immediately demote the 4 VERIFIED claims in MATH_MODEL_MAP.md (models 122, 123, 124, 131) to BEAUTIFUL_PROVISIONAL. Computational verification ≠ Lean proof.
  2. Strip REVIEWED from all claims lacking named reviewer + date. Add [REVIEWED — Alice, 2026-05-05, see $doc] format when review actually occurs.
  3. Demote all self-refuting CALIBRATED_ENGINEERING_DELTA claims to BEAUTIFUL_PROVISIONAL where baseline evidence is absent.
  4. Remove sigma language from compression, hardware, and CFD claims. Use domain-appropriate validators only.
  5. Add claim-state annotations to FPGA_MORPHIC_SCALAR_SPEC.md and any other files presenting quantitative claims without ladder states.
  6. Clarify "Auto-proven" in MATH_MODEL_MAP.md — either define it as a ladder state or replace with BEAUTIFUL_PROVISIONAL plus explicit evidence enumeration.