From abf8329921f8791df46784dc76524aea0e72de2c Mon Sep 17 00:00:00 2001
From: openresearch <bot@openresearch.sh>
Date: Fri, 3 Jul 2026 20:48:28 +0000
Subject: [PATCH] docs: record LLM recoverable drop analysis (same conservation
 law)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

LLMs 'drop data recoverably' via four mechanisms:
1. Residual stream (accumulate, never drop — workspace not compressor)
2. Superposition (pack N features into d<N, exact only when k-sparse)
3. Attention (soft retrieval, KV eviction = explicitly lossy)
4. Quantization (drop bits, recover approximately)

Law: recoverable ⟺ sparse/redundant. Same wall as every compression
branch. Dense/random data → recovery fails → entropy floor.

Pipeline connection:
- QR decomposition (O-AMMR) IS compressed sensing
- GW ringdown at 30dB: k=5 sparse, d=9 < RIP bound → lossy → 1.5x
- Order-2 PPM: 256 contexts packing 65K transitions → interference
  on dense data → 3.088 b/B residual
- Mass number = honesty tag for what was kept vs lost (receipt)

Doctrine consistency: 'MassNumber = recoverability RECEIPT' confirmed.
LLM superposition = same mechanism, same RIP bound, same lossy floor.
---
 docs/weird_machine_conservation_law.md | 56 ++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/docs/weird_machine_conservation_law.md b/docs/weird_machine_conservation_law.md
index c1f42c82..244eeb67 100644
--- a/docs/weird_machine_conservation_law.md
+++ b/docs/weird_machine_conservation_law.md
@@ -114,3 +114,59 @@ One rule: you can move bits between columns, never beat K(data).
 Everything that "compresses" is either base conversion (bijection,
 no gain) or arithmetic coding (needs model, ship cost). The clever
 geometry buys nothing over boring xz/LPC.
+
+## LLM Recoverable Drop: Same Law, Different Substrate
+
+### How LLMs Actually "Drop Data Recoverably"
+
+1. **Residual stream** = accumulate, don't overwrite. No bytes saved —
+   the data was never dropped. The stream is a workspace, not a compressor.
+
+2. **Superposition** = pack N features into d < N dimensions via
+   near-orthogonal directions (Anthropic's Toy Models of Superposition).
+   Recovery is exact only when k-sparse: d ≥ k·log(N/k) (RIP bound).
+   Past that → interference → lossy.
+
+3. **Attention** = soft retrieval, keeps everything. KV-cache eviction
+   (StreamingLLM/beacons) is explicitly lossy — evicted tokens gone,
+   reconstructed only approximately.
+
+4. **Quantization** = drop precision bits, recover approximately.
+   Lossy, bounded by weight tolerance.
+
+### The Law Under All of It
+
+Recoverable ⟺ sparse/redundant. Every mechanism recovers the
+structured part and loses the noise part. Feed it dense/random data
+and recovery fails — same wall as every compression branch measured.
+
+### Connection to Pipeline Measurements
+
+- GW ringdown at 30dB: signal is k=5 sparse in N=10,000 samples.
+  RIP bound: d ≥ 5·log(2000) ≈ 37.5. Our d=9 polynomial is BELOW
+  the bound → lossy recovery, residual = noise → 1.5x (ties LPC).
+
+- Order-2 PPM on enwik8: 256×256 co-occurrence packs N=65,536
+  transitions into d=256 contexts. Sparse text = good recovery.
+  Dense Wikipedia = interference = 3.088 b/B residual.
+
+- QR decomposition (O-AMMR) IS compressed sensing. Eigenvalue
+  spectrum IS the sparse feature set. Golden spiral IS RIP-compliant
+  recovery. But it's LOSSY — the residual is lost.
+
+### The Honest Split
+
+- **Lossless**: dead. K(data) is conserved across every substrate.
+- **Lossy recoverability**: real, useful, must be judged vs
+  JPEG/Opus at equal distortion. The QR projection recovers the
+  dominant structure (sparse part), loses the residual (noise part).
+  The mass number IS the honesty tag for what was kept vs lost.
+
+### Final Consistency Check
+
+The doctrine says: "MassNumber is the admissibility / recoverability
+RECEIPT projected from SemanticMass." The measurement confirms:
+- As lossless compressor: dead (base conversion, 1.00x)
+- As lossy projection: real (recovers sparse structure, loses noise)
+- As receipt: exactly what the spec says (records what was kept)
+- LLM superposition: same mechanism, same RIP bound, same lossy floor