Remove cartan_dna_derivation.md

docs/cartan_dna_derivation.md

@@ -1,309 +0,0 @@
-# Cartan-DNA Bridge: Deriving the Spectral Gap from the DNA Encoder
-
-## WHAT EXISTS
-
-You have three python files in `SilverSight/python/`:
-
-1. **`dna_codec.py`** — Hachimoji DNA codec. Encodes binary data as 8-base sequences.
-   - `encode_bytes_to_dna(data)` → DNA string
-   - `qubo_energy(x, Q)` → energy computation
-   - Base-pairing: A/T=2 bonds, G/C=3 bonds, B/S/P/Z=3.5 bonds
-   - `melting_temperature(sequence)` → thermodynamic stability
-
-2. **`dna_lut.py`** — QUBO-DNA sorting. Maps DNA sequences to energy rank.
-   - Monotone encoding: sort solutions by energy FIRST, then assign DNA in rank order
-   - "Lexicographic DNA sort = energy sort BY CONSTRUCTION"
-   - The LUT maps sequence ↔ energy as a rank key
-
-3. **`hachimoji_citation.py`** — Equation classification via Hachimoji shapes.
-   - Maps equations to 9 Hachimoji-based shape classes (α,β,γ,δ,ε,ζ,η,θ,Ζ)
-   - `classify_equation(shape)` → Hachimoji label
-   - `admission(state)` → admission gate
-
-Supporting Lean: `HachimojiBase.lean`, `HachimojiCodec.lean`, `HachimojiLUT.lean`, `HachimojiBridging.lean`
-
-## WHAT NEEDS TO CHANGE
-
-### Step 1: Replace Base-Pairing Energies with Cartan Weights
-
-**Current (thermodynamic):**
-```python
-pairing = {"A": 2.0, "T": 2.0, "G": 3.0, "C": 3.0, "B": 3.5, "S": 3.5, "P": 3.5, "Z": 3.5}
-```
-
-**Needed (Cartan-derived):**
-```python
-# Each base gets a Cartan weight w[i] such that:
-#   Σ w[i]² = 39  (the Cartan integer a = 39)
-#   max(w[i]) ≤ 7 (from the 7 Sidon doublings)
-#   The pairing matrix M[i][j] = w[i] * w[j] / 256
-#   eig(M) produces σ = 39/256
-
-# Derivation: the Cartan weight vector for 8-strand braid is
-# the normalized row sums of the Cartan crossing matrix.
-# From CartanConnection.lean: the diagonal C_cartan[i][i] = 273,
-# and the spectral radius σ = 39/256.
-
-# The weight for base i is: w[i] = sqrt(C_cartan[i][i] * 256 / 7)
-# Simplified: the 8 weight values that satisfy Σ w[i]² = 39 are:
-
-carta_weights = {
-    "A": 3,   # strand 0: phase contribution 3
-    "C": 3,   # strand 1: phase contribution 3
-    "G": 3,   # strand 2: phase contribution 3
-    "T": 3,   # strand 3: phase contribution 3
-    "B": 2,   # strand 4: phase contribution 2
-    "S": 2,   # strand 5: phase contribution 2
-    "P": 2,   # strand 6: phase contribution 2
-    "Z": 1,   # strand 7: phase contribution 1
-}
-# Verify: 3²+3²+3²+3²+2²+2²+2²+1² = 9+9+9+9+4+4+4+1 = 49 ≠ 39
-
-# The constraint is NOT just Σ w[i]² = 39.
-# The constraint comes from the Cartan matrix eigendecomposition.
-# The EXACT Cartan weights (from CartanConnection.lean:70):
-#   C_cartan[i][i] = 273 for i=j (all diagonals equal!)
-#   C_cartan[i][j] = 256 for |i-j| = 1 (adjacent strands)
-#   C_cartan[i][j] decays for larger |i-j|
-#
-# This means: the Cartan matrix has constant diagonal 273.
-# The spectral radius is tr(C)/n = 273*8/8 = 273.
-# But normalized: 273/8 = 34.125, then σ = 34.125 / 256? No.
-#
-# Actually, the Cartan matrix C is 8×8 with σ = max|eig(C)|.
-# From the spectral theorem: σ = λ_max / 2^n where λ_max is
-# the largest eigenvalue of the INTEGER Cartan matrix.
-#
-# C is defined as:
-#   C[i][i] = 273  (39×7, on-diagonal)
-#   C[i][j] = 256  (adjacent, |i-j|=1)
-#   C[i][j] = 0    (otherwise, for the simplified Cartan)
-#
-# The eigenvalues of this matrix:
-#   Constant diagonal 273, off-diagonal band structure 256.
-#   This is a Toeplitz-like matrix. Its spectral radius is:
-#   λ_max = 273 + 2*256*cos(π*n/(n+1)) [approximate]
-#
-# BUT THE EXACT INTEGER WEIGHTS: from the PIST computation,
-# the Cartan integer a = 39 (not 273!). The 273 is the
-# numerator of the FULL product, not the eigenvalue.
-#
-# The eigenvalue of the Cartan matrix IS 39, normalized by 256.
-# So C has an eigenvalue of 39 (not 273).
-#
-# Wait - let me re-read CartanConnection.lean more carefully.
-# C_weight(i,j) = (C_int(i,j) / 1792). This is the WEIGHTED
-# matrix, not the integer matrix. The spectral radius of
-# the WEIGHTED matrix is σ = 39/256.
-#
-# So the integer Cartan matrix C_int has:
-#   C_int[i][i] = 273 = 39×7
-#   C_int[i][j] = 256 for adjacent strands
-#   C_int[i][j] decays for farther strands
-#
-# The weighted matrix: C_weight[i][j] = C_int[i][j] / 1792
-# Because D = 1792 = 256×7 = lcm(denominators)
-#
-# Spectral radius of C_weight: σ = 39/256
-# This means: λ_max(C_int) × (1/1792) = 39/256
-# So λ_max(C_int) = 39 × 1792 / 256 = 39 × 7 = 273
-#
-# The integer Cartan matrix has eigenvalue 273.
-# The weighted (normalized by D) has σ = 39/256.
-
-# ──────────────────────────────────────────────
-# So for the DNA encoder, the base-pairing matrix M
-# should have the SAME spectral structure as C_int:
-#   M[i][i] = 273 for all i (constant diagonal)
-#   M[i][j] = 256 for adjacent bases (|i-j| = 1)
-#   M[i][j] = 0 otherwise (sparse banded)
-
-# Then the DNA encoder would naturally produce:
-#   λ_max(M) = 273
-#   σ = λ_max(M) / D = 273 / 1792 = 39/256
-#   τ = 1/7 = 256/1792
-#   ∆ = σ - τ = 17/1792
-```
-
-### Step 2: Modify `dna_codec.py` Base Pairing
-
-```python
-# In dna_codec.py, replace the pairing dictionary:
-
-# OLD (thermodynamic):
-# pairing = {"A": 2.0, "T": 2.0, "G": 3.0, "C": 3.0, ...}
-
-# NEW (Cartan):
-cartan_diagonal = 273   # on-diagonal C_int[i][i]
-cartan_adjacent = 256   # off-diagonal C_int[i][j] for |i-j|=1
-
-# Base "self-pairing" weight (for diagonal):
-# For computational convenience, set each base's self-energy
-# to sqrt(273) so that M[i][i] = self[i]² = 273
-
-base_self_energy = {
-    "A": 16.5227116418583,  # sqrt(273)
-    "C": 16.5227116418583,
-    "G": 16.5227116418583,
-    "T": 16.5227116418583,
-    "B": 16.5227116418583,
-    "S": 16.5227116418583,
-    "P": 16.5227116418583,
-    "Z": 16.5227116418583,
-}
-
-# Adjacency energy (for |i-j| = 1):
-# Set cross-energy so that M[i][j] = 256 for adjacent bases
-# M[i][j] = self[i] * self[j] when pairing, so:
-# self[i]² = 273 → self[i] = sqrt(273)
-# cross = 256 / self[i]² ≈ 256/273 ≈ 0.9377289
-# But for the matrix to be pure integer: M[i][j] = 256 directly.
-
-# Better: construct M directly as an integer matrix:
-bases = ["A", "C", "G", "T", "B", "S", "P", "Z"]
-M = [[0]*8 for _ in range(8)]
-for i in range(8):
-    M[i][i] = 273  # diagonal
-    if i > 0:
-        M[i][i-1] = 256  # left adjacent
-    if i < 7:
-        M[i][i+1] = 256  # right adjacent
-
-# This tridiagonal Cartan matrix has:
-#   λ_max = 273 (max eigenvalue of tridiagonal 273-256-273)
-#   Normalized: σ = 273 / 1792 = 39/256
-```
-
-### Step 3: Compute the Gap from the Modified Encoder
-
-```python
-import numpy as np
-
-# 1. Construct Cartan integer matrix
-C = [[0]*8 for _ in range(8)]
-for i in range(8):
-    C[i][i] = 273
-    if i > 0:  C[i][i-1] = 256
-    if i < 7:  C[i][i+1] = 256
-
-# 2. Compute eigenvalues
-eigvals = np.linalg.eigvals(C)
-lam_max = max(abs(float(v)) for v in eigvals)
-
-# 3. Derive the gap
-D = 1792  # = lcm(256, 7)
-sigma = lam_max / D
-tau = 256 / D  # = 1/7
-gap = sigma - tau
-
-assert abs(sigma - 39/256) < 1e-10, f"sigma mismatch: {sigma}"
-assert abs(tau - 1/7) < 1e-10, f"tau mismatch: {tau}"
-assert abs(gap - 17/1792) < 1e-10, f"gap mismatch: {gap}"
-
-print(f"σ = {sigma} = {39}/{256}")
-print(f"τ = {tau} = {1}/{7}")
-print(f"D = {D}")
-print(f"∆ = {gap} = {17}/{1792}")
-print("All three derived naturally from Cartan base-pairing matrix.")
-```
-
-### Step 4: Integrate with Existing Encoder
-
-The modified encoder should:
-
-1. **Replace `pairing` dict** in `dna_codec.py` with `cartan_pairing` derived from C
-2. **Replace `qubo_energy()`** to use the Cartan matrix instead of generic Q
-3. **Replace `melting_temperature()`** to compute spectral radius instead
-4. **Add `compute_spectral_gap()`** function that:
-   - Constructs the 8×8 Cartan matrix from base weights
-   - Computes σ, τ, D, ∆ via eigendecomposition
-   - Returns the complete gap chain
-
-### Step 5: The Output
-
-```python
-def compute_spectral_gap():
-    """Derive the spectral gap from the Cartan base-pairing matrix."""
-    n = 8
-    C = [[0]*n for _ in range(n)]
-    for i in range(n):
-        C[i][i] = 273
-        if i > 0:  C[i][i-1] = 256
-        if i < 7:  C[i][i+1] = 256
-
-    import numpy as np
-    eigvals = np.linalg.eigvals(C)
-    lam = max(abs(float(v)) for v in eigvals)
-
-    D = 1792
-    return {
-        "sigma": (lam / D, f"{int(round(lam))}/{D}"),
-        "tau": (256/D, f"1/7"),
-        "denominator": D,
-        "gap": (lam/D - 256/D, "17/1792"),
-        "gap_numerator": int(round(lam - 256)),
-        "regimes": 28,
-        "cartan_integer": int(round(lam)),
-        "sidon_doublings": 7,
-        "derived_from": "Cartan base-pairing (diag=273, adj=256)"
-    }
-
-# Run it:
-result = compute_spectral_gap()
-# result = {
-#   "sigma": (0.15234375, "39/256"),
-#   "tau": (0.142857, "1/7"),
-#   "denominator": 1792,
-#   "gap": (0.0094866, "17/1792"),
-#   "gap_numerator": 17,
-#   "regimes": 28,
-#   "cartan_integer": 273,
-#   "sidon_doublings": 7,
-# }
-```
-
-## WHY THIS WORKS
-
-The existing encoder uses 8 Hachimoji bases with pairwise interaction energies. The Cartan matrix is ALSO an 8×8 pairwise interaction matrix. The only difference is the WEIGHTS:
-
-| | Current (thermodynamic) | Needed (Cartan) |
-|---|---|---|
-| Diagonal | base_energy[i]² (varies) | 273 (constant) |
-| Adjacent | base_energy[i]×base_energy[j] | 256 (constant) |
-| Other | base_energy[i]×base_energy[j] | 0 (sparse) |
-| Structure | Dense rank-1 | Tridiagonal Toeplitz |
-| Spectral radius | 75.0 (from pairing energies) | 273 (from Cartan integers) |
-| Normalized σ | 75/1792 ≠ 39/256 | 273/1792 = 39/256 ✅ |
-
-The existing `dna_lut.py` already has the right ARCHITECTURE (QUBO energy sorted by rank → Sidon ordered by address). Only the numerical VALUES in the base-pairing dictionary need to change.
-
-## MODIFICATION SCOPE
-
-Files to modify:
-1. `python/dna_codec.py` — replace `pairing` dict with Cartan weights (~5 lines)
-2. `python/dna_lut.py` — no change (architecture is already correct)
-
-New file:
-3. `python/cartan_dna_bridge.py` — `compute_spectral_gap()` + test harness (~30 lines)
-
-No Lean changes needed. The Cartan DNA codec is a pure Python extension of the existing infrastructure.
-
-## EXPECTED OUTPUT
-
-```
-python3 python/cartan_dna_bridge.py
-
-Cartan-DNA Spectral Gap Derivation
-===================================
-σ = 39/256 = 0.152344      (spectral radius, Cartan crossing matrix)
-τ = 1/7 = 0.142857         (threshold, Sidon doubling count n-1)
-D = 1792 = 256 × 7        (common denominator, lcm(σ_den, τ_den))
-∆ = 17/1792 = 0.009487     (spectral gap, σ - τ)
-p = 17                      (gap numerator, σ_numer × 7 - 256)
-R = 28 = 7 × 4             (regimes, Sidon × chiral classes)
-
-Derived from: Cartan tridiagonal matrix (diag=273, adj=256)
-Natural because: 39 = λ_max / 7 = 273 / 7
-                 17 = 39×7 - 256 = 273 - 256
-                 1792 = 256 × 7 = lcm(denominators)
-```