Research-Stack/audit/documentation_audit.md

Documentation Audit — Research Stack (OTOM)

Audit Date: 2026-06-22 Scope: All .md files (1,956), .yaml/.yml files (172), .lean files (1,295), and documentation directories Methodology: Cross-referencing documentation claims against actual repository contents via GitHub raw URLs and file tree analysis

---

Summary

Metric	Count
Total .md files audited	1,956
Total .lean files	1,295
Stale claims	8
Broken cross-references	16
Old TODOs (unaddressed >2 months)	8
Duplicate documentation pairs	45 (16 docs + 29 obsidian-vault)
Empty/minimal READMEs	15
Undocumented major modules	6
AGENTS.md files (potential conflict)	7

Overall Assessment: The documentation set is extensive (1,956 markdown files, ~9.3MB in 6-Documentation/ alone) but suffers from significant staleness, broken cross-references, inflated proof claims, duplicate content across multiple directory trees, and receipts that describe files and test results that do not exist in the repository.

---

Critical Issues

C1: E2E and library Receipts Describe Non-Existent Files

Files: e2e/E2E_MASTER_RECEIPT.md, library/CONCEPTUAL_UPGRADE_RECEIPT.md

Severity: CRITICAL

Both receipt files claim detailed test results and reference source files that do not exist:

• e2e/ directory is completely empty (0 files), yet E2E_MASTER_RECEIPT.md describes:

  • e2e/E2EMasterTrace.lean (~540 lines)
  • e2e/run_e2e_trace.py (~650 lines)
  • Step-by-step execution traces with theorem names and line numbers
  • Receipt SHA-256: c8ad995a0fdd9bd0160ae5e20ca27b89a5ca759ef0465b7d0472d0901b3efcfa

• library/ directory is completely empty (0 files), yet CONCEPTUAL_UPGRADE_RECEIPT.md describes:

  • library/hachimoji_codec_v2.py — "17/17 PASSED"
  • library/HachimojiCodecV2.lean — "4 theorems proven"
  • library/counterexample_detector.py — "10/10 PASSED"
  • All test claims are unverifiable

Impact: These receipts are fabrication — they document code and test results that were never committed to the repository. Any downstream consumer trusting these receipts is being misled.

---

C2: "3,500+ Proofs" Claim is Inflated

Files: README.md, EXPLANATION_FOR_HUMANS.md, GETTING_STARTED.md

Severity: CRITICAL

Multiple documents claim:

• EXPLANATION_FOR_HUMANS.md: "3,500+ strict mathematical proofs" and "checked against 3,500+ strict mathematical proofs"
• README.md: "746 verified Lean modules"
• GETTING_STARTED.md: "3,500+ verified proofs" and "all 3,500+ proofs are verified"

Reality:

• There are 1,295 .lean files, not 746 "verified modules"
• The "3,500" number matches the build job count at various points in time (3,314 / 3,529 / 3,592 / 8,332 jobs), not individual proof theorems
• Build jobs are compilation units (module imports, definitions, structures), not proof theorems
• Many .lean files contain only def, structure, #eval statements with no theorem or lemma
• Foundations files like CarnotEfficiency.lean (377 bytes) are just a single definition with #eval — no theorem
• LandauerBound.lean (457 bytes) uses a placeholder kB value (ofRawInt 6) with comment "Simplified kB" — not physically meaningful

Impact: This claim misleads readers about the maturity and depth of the formalization. Conflating build jobs with proof theorems is a significant misrepresentation.

---

C3: eigensolid_convergence Theorem is a Tautology

File: 0-Core-Formalism/lean/Semantics/Semantics/BraidEigensolid.lean

Severity: CRITICAL

The flagship theorem, referenced throughout documentation (AGENTS.md, ARCHITECTURE.md, README.md) as proof of compressor correctness:

theorem eigensolid_convergence
    (s : BraidState)
    (h_eig : IsEigensolid (crossStep s)) :
    ∀ i : Fin 8, (crossStep (crossStep s)).strands i = (crossStep s).strands i :=
  h_eig

This theorem literally restates its hypothesis. IsEigensolid (crossStep s) means exactly ∀ i, (crossStep (crossStep s)).strands i = (crossStep s).strands i. The "proof" is just returning the hypothesis.

Impact: The core compressor convergence guarantee is vacuous. The theorem proves nothing beyond what was assumed. ARCHITECTURE.md section 13 and TODO_MAP.md B11 both cite this as a completed proof.

---

C4: CONCEPTS.md Referenced but Missing

Files: GETTING_STARTED.md (section 6), ARCHITECTURE.md (section 10)

Severity: HIGH

CONCEPTS.md is listed as a "Key File to Read First" in GETTING_STARTED.md and linked from ARCHITECTURE.md's "Further Reading" section. The file does not exist in the repository.

---

C5: Circular Reference in docs/README.md

File: docs/README.md (also 6-Documentation/docs/README.md — duplicate)

Severity: HIGH

The README states:

"The authoritative documentation tree is 6-Documentation/docs/."

But this file IS in 6-Documentation/docs/. It points to itself via a relative path. This is a broken self-reference that would confuse any reader.

---

Warnings

W1: Duplicate Documentation (45+ files)

Severity: HIGH

Duplicate Set	Count	Locations
docs/ ↔ 6-Documentation/docs/	16 files	README.md, WIKI.md, rrc_equation_classification.md, roadmaps/RESEARCH_STACK_FOREST_MAP_WATERFALL.md, etc.
obsidian-vault/ ↔ 6-Documentation/wiki/obsidian-vault/	29 files	Complete layer READMEs (L0-L6), main vault README

The root-level docs/ and obsidian-vault/ directories appear to be stale copies of the authoritative 6-Documentation/ tree. The docs/README.md even admits: "This directory is a legacy alias." Yet 16 files and 29 obsidian-vault files remain duplicated, creating confusion about which is authoritative.

Risk: Readers may find stale copies first, and edits to one may not propagate to the other.

---

W2: Inconsistent Build Job Counts Across Documents

Severity: MEDIUM

Document	Claimed Jobs	Context
README.md	3,314	"lake build Compiler"
ARCHITECTURE.md	8,332	"full workspace"
ARCHITECTURE.md	3,314	"Compiler surface"
AGENTS.md	3,314	"Compiler surface" (commit e61bb627)
TODO_MAP.md	3,529	"deterministic build receipt"
TODO_MAP.md	3,592	"build jobs green"
CHANGELOG.md	3,529	"deterministic build receipt"

Different documents cite different commit hashes for these numbers. There is no single source of truth for the current build status.

---

W3: 7 AGENTS.md Files with Potential Overlap/Conflict

Severity: MEDIUM

File	Size
AGENTS.md (root)	28,853 bytes
6-Documentation/docs/AGENTS.md	48,594 bytes
0-Core-Formalism/lean/Semantics/AGENTS.md	38,223 bytes
4-Infrastructure/AGENTS.md	27,403 bytes
5-Applications/text-to-cad/AGENTS.md	7,740 bytes
scripts/qc-flag/AGENTS.md	2,414 bytes
shared-data/artifacts/lean_expert_agent/AGENTS.md	2,893 bytes

Total: ~156KB of agent instructions. Root AGENTS.md directs to nested ones, but there's no mechanism ensuring consistency. The root file and 6-Documentation/docs/AGENTS.md are the largest and may contain conflicting guidance.

---

W4: 15 Empty or Minimal READMEs

Severity: MEDIUM

The following READMEs are under 500 bytes and provide no useful guidance:

File	Size	Issue
0-Core-Formalism/lean/Semantics/README.md	11 bytes	Just "# Semantics"
2-Search-Space/GhostPivot/README.md	31 bytes	Nearly empty
2-Search-Space/simulations/chunked-audio-DSP/README.md	70 bytes	Nearly empty
5-Applications/audio-dsp/README.md	70 bytes	Nearly empty
6-Documentation/docs/README.md	311 bytes	Circular reference only
docs/README.md	311 bytes	Circular reference only
pending/lean_unification/README.md	446 bytes	Minimal
3-Mathematical-Models/genetics/parametric-learn/README.md	493 bytes	Minimal

These create navigation dead-ends for explorers.

---

W5: TODO_MAP.md Staleness

Severity: MEDIUM

TODO_MAP.md is dated "2026-04-24" (nearly 2 months old). Items marked:

• ✅ "DONE" that reference missing files (equations_forest.jsonl, supernodes.json)
• 🔄 "IN_PROGRESS" for months with no evidence of progress
• 📋 "TODO" items (B5-B9, C1-C4, F1-F5, G1-G6) untouched since creation
• ⏳ "BLOCKED" items with no unblock pathway documented

The "Metrics" table at the bottom shows many targets unmet (e.g., "Equations in registry: 200+ | 0", "Supernodes: 30-45 | 0", "Foundation kernel theorems: 12 | ~3 partial").

---

W6: AGENTS.md Contains Local Deployment Details

Severity: LOW

Root AGENTS.md contains:

• Personal IP addresses (100.88.57.96, 100.92.88.64)
• Tailscale hostnames (qfox-1, neon-64gb)
• Local file paths (/home/allaun/, ~/.hermes/)
• Hardware details specific to the author's setup (RTX 4070, CachyOS)

This information is only relevant to the original author and serves no purpose for external contributors.

---

W7: Wrong Path in TODO_MAP.md for Foundations/

Severity: LOW

TODO_MAP.md section B1 references:

0-Core-Formalism/lean/Semantics/Foundations/ShannonEntropy.lean

Actual path in repository:

0-Core-Formalism/lean/Semantics/Semantics/Foundations/ShannonEntropy.lean

The Semantics/ namespace is duplicated. This indicates a namespace refactoring occurred without updating documentation.

---

Per-Doc Breakdown

README.md (root)

• Stale claim: "746 verified Lean modules" — count is not verifiable
• Stale claim: "3,314-job deterministic lake build Compiler" — job count varies across docs
• Broken ref: "Gremlin mathblob database" with "14,449 vertices and 29,379 edges" — unverifiable
• Grade: B- (Good structure but inflated metrics)

EXPLANATION_FOR_HUMANS.md

• Stale claim: "3,500+ strict mathematical proofs" — conflates build jobs with theorems
• Stale claim: "There are zero errors" — refers to build, not proofs
• Grade: C+ (Good plain-English explanation but propagates inflated proof count)

ARCHITECTURE.md

• Stale claim: "8332 jobs full workspace, 3314 jobs Compiler surface, 0 errors" — job count inconsistent
• Broken ref: CONCEPTS.md in "Further Reading" section 10
• Broken ref: .devcontainer/flake.nix in Storage & Persistence section (only Dockerfile exists)
• Broken ref: 4-Infrastructure/infra/credential_server.py in Infrastructure table
• Inconsistent: Section 13 says receipt_invertible is pending, but TODO_MAP.md B11 says eigensolid convergence is ✅ DONE
• Grade: B (Good architecture overview but stale references and inconsistent claim tracking)

AGENTS.md (root)

• Stale claim: "3314 jobs, 0 errors" with commit e61bb627 — other docs cite different commits
• Personal info: Contains local IP addresses, hostnames, personal paths
• Grade: B (Good operating rules but contains personal deployment data and stale metrics)

TODO_MAP.md

• Old TODOs: 8 items pending since 2026-04-24
• False completions: Items marked ✅ DONE that reference missing files
• Stale metrics: "Equations in registry: 0 (ChatGPT sandbox only)" — months old
• Stale blocker: "Tang Nano 9K not physically connected" still listed as blocker
• Grade: C (Comprehensive but significantly stale; many false completions)

GETTING_STARTED.md

• Broken ref: CONCEPTS.md in "Key Files to Read First" table
• Broken ref: 5-Applications/hutter_prize/ARCHITECTURE.md in same table (hutter_prize has its own ARCHITECTURE.md but it's not the system overview)
• Stale claim: "3,500+ verified proofs" in build instructions
• Stale paths: References /home/allaun/CascadeProjects/Research-Stack — author's local path
• Grade: B- (Useful setup guide but stale claims and broken refs)

E2E_MASTER_RECEIPT.md

• CRITICAL: References files in e2e/ directory that is completely empty
• CRITICAL: Claims 3 PROVEN + 3 COMPUTED + 2 STATED steps with detailed line numbers — all unverifiable
• CRITICAL: Claims SHA-256 receipt hash — cannot be verified
• Fabrication: Entire document describes a trace through code that does not exist
• Grade: F (Receipt for non-existent code)

CONCEPTUAL_UPGRADE_RECEIPT.md

• CRITICAL: References files in library/ directory that is completely empty
• CRITICAL: Claims test suite results (17/17, 14/14, 10/10) that cannot be verified
• Fabrication: Describes V2 codec with theorems and tests that don't exist in repo
• Grade: F (Receipt for non-existent code)

docs/README.md + 6-Documentation/docs/README.md

• Circular ref: Points to 6-Documentation/docs/ which is the SAME directory
• Grade: D (Self-referential brokenness)

CITATION.cff

• Note: Contains ~100+ references, many from 2026-06-19 session. Well-structured.
• Minor issue: Some references have placeholder/generic notes
• Grade: B+ (Good citation management, some entries are recent)

CLAUDE.md / GEMINI.md

• Duplicate content: GEMINI.md appears to be a subset/superset of CLAUDE.md guidance
• Broken ref: Points to AGENTS.md files but doesn't distinguish between 7 copies
• Grade: B- (Useful but could be consolidated)

---

Recommendations

Immediate (P0)

1. Remove or quarantine E2E_MASTER_RECEIPT.md and CONCEPTUAL_UPGRADE_RECEIPT.md — they document non-existent code
2. Fix or remove the "3,500+ proofs" claim — replace with actual theorem count or clarify "build jobs"
3. Create CONCEPTS.md or remove references to it
4. Fix eigensolid_convergence — replace tautology with actual proof or downgrade claim status

Short-term (P1)

5. Deduplicate docs/ → merge into 6-Documentation/docs/, then remove root docs/
6. Deduplicate obsidian-vault/ → merge into 6-Documentation/wiki/obsidian-vault/, remove root copy
7. Consolidate AGENTS.md — keep one root file, have nested ones reference it explicitly
8. Remove personal info from AGENTS.md (IPs, hostnames, local paths)
9. Update TODO_MAP.md — mark items referencing missing files as not-done, refresh metrics
10. Write READMEs for empty directories (GhostPivot, audio-dsp, etc.)

Medium-term (P2)

11. Single source of truth for build metrics — one file that documents current job count
12. Automated broken link checker for cross-references
13. Documentation freshness dating — add "last verified" timestamps to key docs
14. Consolidate CLAUDE.md/GEMINI.md into single LLM_INSTRUCTIONS.md

---

Audit completed. 1,956 markdown files, 1,295 Lean files, 172 YAML files analyzed. All findings mapped to specific file paths and verifiable against the repository at https://github.com/allaunthefox/Research-Stack branch main.