mirror of
https://github.com/allaunthefox/Research-Stack.git
synced 2026-07-30 18:56:16 +00:00
8e51acad08
CRITICAL (7): - C1: Hardcoded Wolfram Alpha API key (revoke immediately) - C2: Q16_16 rounding diverges Lean↔Python↔C (data corruption) - C3: Receipts describe non-existent files (fabrication) - C4: '3,500+ proofs' = compilation units, not theorems - C5: NaN sentinel collision Lean↔C (silent corruption) - C6: eigensolid_convergence theorem is a tautology - C7: 74% CI failure rate HIGH (12): duplicate definitions, SQL injection, command injection, 1,029 duplicate files, 66 large files in git, AGENTS.md drift MEDIUM (23): stale docs, broken cross-refs, unhandled errors LOW (31): naming violations, misplaced files, cleanup Master synthesis: audit/MASTER_AUDIT_SYNTHESIS.md Per-dimension reports: audit/*_audit.md
199 lines
9.2 KiB
Markdown
199 lines
9.2 KiB
Markdown
# Master Audit Synthesis
|
|
## Research-Stack Full Codebase Audit — June 21, 2026
|
|
### 6 parallel auditors, 2,068 lines of findings, 9,606 objects inspected
|
|
|
|
---
|
|
|
|
## The Honest Picture After 10+ Iterations
|
|
|
|
This repository has been through more than 10 development iterations. Each iteration added value. Each iteration also left debris. The debris has accumulated to the point where **the documentation describes a project that doesn't exist**, the **CI fails 74% of the time**, and **441 files are duplicates** of other files in the same repo.
|
|
|
|
This is not unusual for a solo researcher using AI agents. It's the expected state. What matters is seeing it clearly.
|
|
|
|
---
|
|
|
|
## Summary by Severity
|
|
|
|
| Severity | Count | Meaning |
|
|
|----------|-------|---------|
|
|
| **CRITICAL** | 7 | Security risk, data corruption, or fundamental dishonesty |
|
|
| **HIGH** | 12 | Broken functionality that affects correctness |
|
|
| **MEDIUM** | 23 | Quality issues that impede development |
|
|
| **LOW** | 31 | Cleanup, consistency, polish |
|
|
|
|
---
|
|
|
|
## The 7 Critical Issues
|
|
|
|
### C1: Hardcoded API Key (Security — revoke today)
|
|
**File:** `wolfram_verify.py`
|
|
**Finding:** Hardcoded Wolfram Alpha API key `APPID = "HYJE3R3R63"`
|
|
**Impact:** Anyone with git access can use this key. If repo is public (it is: 0 stars but public), key is exposed.
|
|
**Fix:** Revoke key immediately. Move to env var. Scrub from git history.
|
|
|
|
### C2: Q16_16 Conversion Divergence (Data Corruption)
|
|
**Files:** `FixedPoint.lean`, `lib/q16.py`, `manifold_binary.h`
|
|
**Finding:** Lean uses `floor()`, Python uses `round()`, C uses truncation. Values at half-LSB boundaries diverge silently across the entire pipeline.
|
|
**Impact:** The same Q16_16 value means different things in different languages. This corrupts every cross-language data flow.
|
|
**Fix:** Define canonical rounding mode in one file. All languages must implement identical conversion.
|
|
|
|
### C3: Receipts Describe Non-Existent Files (Fundamental Dishonesty)
|
|
**Files:** `e2e/E2E_MASTER_RECEIPT.md`, `library/CONCEPTUAL_UPGRADE_RECEIPT.md`
|
|
**Finding:** Both receipts claim detailed test results from files (`E2EMasterTrace.lean`, `hachimoji_codec_v2.py`, `ClosedTrace.lean`, etc.) that **do not exist** in the `e2e/` or `library/` directories.
|
|
**Impact:** The receipts are fabrications. This is the most dangerous kind of technical debt — the kind that makes you believe things are proven when they aren't.
|
|
**Fix:** Generate the actual files, or remove the receipts until the files exist.
|
|
|
|
### C4: "3,500+ Proofs" Claim Is Inflated (Marketing vs. Reality)
|
|
**Files:** `README.md`, `GETTING_STARTED.md`, `EXPLANATION_FOR_HUMANS.md`
|
|
**Finding:** The "3,500+ proofs" number matches the **compilation unit count**, not actual proved theorems. Many `.lean` files contain only definitions (e.g., `CarnotEfficiency.lean` = 377 bytes, one definition + `#eval`). The 26 `sorry` markers in core theorems are not counted.
|
|
**Impact:** Visitors and potential collaborators are misled about the project's maturity.
|
|
**Fix:** Replace with accurate count (e.g., "883 lines of Chentsov proof, 0 sorry; 26 sorry markers in core theorems under active development").
|
|
|
|
### C5: NaN Sentinel Collision (Silent Data Corruption)
|
|
**Files:** `BindServer.lean` (Lean side), `manifold_binary.h` (C side)
|
|
**Finding:** Lean uses `0xFFFFFFFF` as NaN sentinel. C interprets `0xFFFFFFFF` as a valid large positive integer. On overflow, Lean sends "NaN", C receives a valid number.
|
|
**Impact:** Overflow errors become silently wrong data instead of errors.
|
|
**Fix:** Use IEEE 754 NaN encoding on both sides, or explicit error flags.
|
|
|
|
### C6: Eigensolid Convergence Theorem Is a Tautology
|
|
**File:** `BraidEigensolid.lean`
|
|
**Finding:** The flagship compressor correctness theorem `eigensolid_convergence` literally restates its hypothesis (`h_eig`). It proves nothing. Cited as complete across multiple documents.
|
|
**Impact:** A core correctness claim is vacuous.
|
|
**Fix:** Either prove it properly or downgrade the claim to a conjecture.
|
|
|
|
### C7: 74% CI Failure Rate (Systemic Unreliability)
|
|
**Files:** `.github/workflows/`
|
|
**Finding:** Math-First Checks, Pages Build, and Batch Compute workflows fail consistently. Only Wolfram Alpha verification passes. `.pre-commit-config.yaml` is referenced by CI but **does not exist**.
|
|
**Impact:** Every push to main has a 74% chance of failing CI. This trains developers to ignore CI.
|
|
**Fix:** Fix the 3 broken workflows. Create `.pre-commit-config.yaml`.
|
|
|
|
---
|
|
|
|
## The 12 High Issues
|
|
|
|
### H1: IsSidonSet Defined Twice (Lean)
|
|
Two different definitions in `SidonSets.lean` and `ErdosRenyiPipeline.lean` with slightly different signatures. Any theorem using one won't work with the other.
|
|
|
|
### H2: SQL Injection Pattern (Python)
|
|
`finance_manager.py` lines 46, 62-64: f-string SQL with user input. Unfixed despite PR #76 claiming to patch it.
|
|
|
|
### H3: Command Injection (Python)
|
|
`verify_all_shims.py` uses `subprocess` with `shell=True` and user-controlled paths.
|
|
|
|
### H4: Duplicate q16.py (Python)
|
|
Two copies (`lib/q16.py` and `auto/lib/q16.py`) with different subsets of operations. Import ambiguity.
|
|
|
|
### H5: Stats_show Destructive Read (C)
|
|
`pist_neuromorphic.c`: `stats_show()` mutates frequency data during read. Subsequent reads return wrong values.
|
|
|
|
### H6: AGENTS.md References Non-Existent Components
|
|
The 27KB AGENTS.md describes extracted/verified components that don't exist in the codebase.
|
|
|
|
### H7: No Q16_16 Roundtrip Test
|
|
The full Lean→Python→C→Lean data path is completely untested in CI.
|
|
|
|
### H8: 66 Large Files in Git (Not LFS)
|
|
44.88 MB PDF, 25 MB GIF (duplicated), 187 MB WAV files, 24 MB compressed file. Should be in Git LFS.
|
|
|
|
### H9: 192 Build Artifacts Committed to Main
|
|
`.pist` and `.pist.meta` files are generated by Lean build but committed to git. Should be `.gitignore`d.
|
|
|
|
### H10: Dependencies Unpinned
|
|
`requirements.txt` uses `>=` for 20+ critical deps (qiskit, numpy, scipy) with no lock files.
|
|
|
|
### H11: 111 Misplaced Files
|
|
Documentation scattered in core formalism (24 files) and applications (75 files) instead of `6-Documentation/`.
|
|
|
|
### H12: NoDupeLabs Duplicated (1,029 Files)
|
|
Full project copied in both `4-Infrastructure/` and `5-Applications/`. 100+ confirmed identical via git blob hashes.
|
|
|
|
---
|
|
|
|
## Per-Dimension Breakdown
|
|
|
|
### Lean Formalism (1,295 files)
|
|
- 26+ `sorry` markers across core theorems
|
|
- T1_Coherence.lean: 12 sorry (the hardest — Chentsov-level)
|
|
- ErdosRenyiPipeline.lean: 6 sorry
|
|
- E8Sidon.lean: 5 sorry
|
|
- 4 duplicate definitions detected
|
|
- 220 naming convention violations (mostly lowercase_snake in Lean where PascalCase expected)
|
|
- ~119 files under 500 bytes (likely stubs)
|
|
|
|
### Infrastructure (Python/Rust/C/Shell)
|
|
- 3 secrets found (1 critical API key)
|
|
- 6 security issues (SQL injection, command injection, unsafe cast)
|
|
- 3 dead code functions
|
|
- 2 broken imports
|
|
- 3 path mismatches
|
|
- 2 build problems
|
|
|
|
### Documentation (1,956 Markdown files)
|
|
- 8 stale claims (including "3,500+ proofs")
|
|
- 16 broken cross-references
|
|
- 8 old TODOs (>2 months)
|
|
- 45 duplicate documentation pairs
|
|
- 15 empty/minimal READMEs
|
|
- 6 undocumented major modules
|
|
|
|
### Integration (Lean ↔ Python ↔ C)
|
|
- 18 integration points identified
|
|
- 3 critical type mismatches (Q16_16 rounding)
|
|
- 7 error handling gaps
|
|
- 2 broken CLI contracts
|
|
- 5 workflow gaps
|
|
- 2 race conditions
|
|
|
|
### Quality (CI/Tests/Cleanup)
|
|
- CI failure rate: 74%
|
|
- Only 2 actual pytest files out of 1,282 "test" named files
|
|
- 627 duplicate basenames
|
|
- 57 experimental files on main
|
|
- 66 large files (>1MB)
|
|
- 101 binary files in git
|
|
- ~297 files needing cleanup
|
|
|
|
### Consistency (Organization/Naming)
|
|
- 220 naming violations
|
|
- 111 misplaced files
|
|
- 5 version skew issues
|
|
- 2 dead symlinks
|
|
- 441 duplicate files
|
|
- 4 removed feature references
|
|
- 179 gitignore gaps
|
|
|
|
---
|
|
|
|
## The Brutal Truth
|
|
|
|
This is not a codebase that's ready for external use. It's a **research notebook** that has grown to 9,606 objects across 6 architecture layers. The core mathematical insight — chaos game + Sidon + Chentsov + Hachimoji — is sound and genuinely novel. But the implementation around it has accumulated 10 iterations of debris.
|
|
|
|
**What works:**
|
|
- ChentsovFinite.lean (883 lines, 0 sorry) — the keystone theorem
|
|
- HachimojiSubstitution.lean — the 8-state encoding with bijection
|
|
- qaoa_adapter.py — the QUBO ↔ QAOA bridge (solid engineering)
|
|
- The 6-layer architecture concept (even if the files are misplaced)
|
|
|
|
**What needs immediate attention:**
|
|
1. Revoke the API key (C1) — 5 minutes
|
|
2. Fix Q16_16 rounding to be canonical (C2) — 1 day
|
|
3. Delete or generate the missing files that receipts claim exist (C3) — 1 day
|
|
4. Fix the 3 broken CI workflows (C7) — 1 day
|
|
5. Remove 1,029 duplicate NoDuPeLabs files (H12) — 30 minutes
|
|
|
|
**What needs a focused cleanup sprint:**
|
|
- All 26 sorry markers in core theorems
|
|
- The SQL injection and command injection
|
|
- The 66 large files → Git LFS
|
|
- The 192 build artifacts → .gitignore
|
|
- The 111 misplaced files → correct directories
|
|
|
|
The ship in the bottle is still impressive. But the bottle has accumulated 10 layers of dust, a hardcoded API key, and 441 duplicate planks. Time to clean the glass.
|
|
|
|
---
|
|
|
|
*Audit conducted: June 21, 2026*
|
|
*Auditors: 6 parallel specialist agents*
|
|
*Objects inspected: 9,606*
|
|
*Findings: 2,068 lines across 6 reports*
|
|
*This synthesis: 1 report bringing all findings into focus*
|