SilverSight/docs/SPECTRAL_CODEBOOK_ANALYSIS.md

Spectral Codebook Analysis

Date: 2026-07-01 (revised after independent review) Source: 250 equations × 8×8 braid adjacency matrices from rrc_pist_predictions_250_v1.json Method: Exact eigenvalues via numpy (characteristic polynomial), NOT power iteration

⚠️ Corrections (2026-07-01 review)

The original analysis used power iteration (300 iterations) on raw integer matrices. This introduced systematic errors:

1. The "mid-band" (9 matrices, λ ∈ [0.67, 0.95]) does not exist. All 20 peripheral matrices have exact spectral radius ρ = 1.0. Power iteration fails on matrices with eigenvalues on the unit circle at 60° angles (peripheral spectrum non-convergence).
2. The 7-cluster table was incorrect. Only 4 data points exist above λ=11, not ~25.
3. 7.57 bits was a count, not Shannon entropy. True band entropy is 1.257 bits.
4. 12 matrices are exact duplicates. The 250-row corpus contains 238 distinct matrices.
5. Characteristic polynomials (192 unique) outperform spectral radius (180 unique) as a codebook key.
6. Power iteration in MatrixN.lean inherits the same bug — 22 of 250 matrices disagree with exact eigenvalues by >1e-3.

Executive Summary

The 250-equation corpus contains 180 distinguishable spectral radii (at 10dp) and 192 unique characteristic polynomials. The eigensolid is a near-perfect topological fingerprint: density–ρ correlation = 0.9806.

The real structure has 3 natural bands (not 5): null (ρ=0), peripheral-unit (ρ=1), and bulk (ρ>1). The Cartan gap Δ = 17/1792 ≈ 0.00949 is the principled distinguishability floor.

Corpus Statistics

Metric	Value
Total matrices	250
Distinct matrices	238
Unique spectral radii (10dp)	180
Unique characteristic polynomials	192
ρ range	[0.0, 16.99]
ρ mean	2.75
Density–ρ correlation	0.9806

Band Distribution (exact eigenvalues)

Band	ρ range	Count	Interpretation
null	0.0	35	Nilpotent — no operator interaction
peripheral	1.0	20	Unit spectral radius — Cartan boundary
bulk	(1.0, 4.0]	165	Moderate operator density
high-bulk	(4.0, ∞)	50	Dense operator graphs

Band entropy: 1.257 bits (max 2.322 for 5 bands, but only 3 active).

Interpretation

• null band (35 matrices): Empty or near-empty operator graphs. ρ=0 means the adjacency matrix is nilpotent. These are the "unclassified" equations.
• peripheral band (20 matrices): All have exact ρ=1.0. These are unitary-like — the operator graph has spectral content on the unit circle. The Cartan gap Δ = 17/1792 separates these from the null band.
• bulk band (165 matrices): The main body. ρ > 1 means the operator graph has amplifying modes. Each matrix is distinguishable by its spectral radius.
• high-bulk band (50 matrices): ρ > 4. Dense operator graphs with strong amplification.

Correlations (exact eigenvalues)

Pair	Pearson r
ρ ↔ density	0.9806

The spectral radius and edge density are near-interchangeable. ρ alone captures 96% of the variance (r² = 0.9616). The codebook can be 1-dimensional.

Key difference from power-iteration results: The exact correlation is nearly identical (0.9806 vs 0.9807), confirming that the correlation is real even though the individual λ values were wrong for 22 matrices.

Spectral Gaps (exact eigenvalues)

The top 5 gaps in the sorted spectral radius distribution:

Rank	Gap size	Lower ρ	Upper ρ	Interpretation
1	4.8807	11.68	16.56	Major complexity wall
2	1.5862	10.10	11.68	Secondary wall
3	1.0000	0.00	1.00	Null → peripheral transition
4	0.8895	9.21	10.10	Tertiary wall
5	0.7051	8.50	9.21	Quaternary wall

Note: The null→peripheral gap (exactly 1.0) is the third largest gap, not the fifth as in the power-iteration analysis.

The Cartan Gap

The Cartan gap Δ = 17/1792 ≈ 0.00949 (proven exact in CartanConnection.lean:70) is the minimum eigenvalue of the crossing blocks — the resolution floor of the braid operator on Δ₇.

This provides a principled, Lean-proven distinguishability floor:

• Two spectral radii separated by less than Δ are provably indistinguishable by the operator dynamics
• The current statistical heuristic (3× median gap) should be replaced by Δ as the minimum inter-codeword distance
• This transforms the codebook from "looks like a gap in this sample" to "provably resolvable by the operator"

Encoding Capacity

Spectral radius alone

• 180 unique values (10dp) → 7.49 bits/equation
• But 85 matrices are in collision groups (15 groups with identical ρ)

Characteristic polynomial

• 192 unique polynomials → 7.59 bits/equation
• 74 matrices in 16 cospectral groups (same polynomial, different matrix)
• Better discriminant than ρ alone: +12 unique keys

Band-level encoding

• 3 active bands → 1.257 bits at band level
• Within-band: additional bits from ρ or polynomial

Conditional on density

• ρ adds only ~2.2 bits beyond raw edge count
• Most of the "signal" is edge count, not spectral structure

λ Collisions (exact)

Metric	Count
Collision groups (same ρ at 10dp)	15
Matrices in collisions	85
Cospectral groups (same char poly)	16
Matrices in cospectral groups	74

Implication: The characteristic polynomial is a better codebook key than spectral radius alone. 192 vs 180 unique keys, and the polynomial is exact (integer coefficients) while ρ is a floating-point approximation.

The F Function (byte frequency) and Cartan Orthogonality

The F function (byte-frequency vector) is orthogonal to the Cartan spectral structure:

	F (byte frequency)	Cartan (spectral)
What it sees	Surface characters: 'a', '+', '=', '1'	Operator graph topology
After normalization	All vars → 'V', all digits → 'N'	Sidon addresses preserved
Sensitivity	6 unique vectors	180 unique spectral radii
Correlation with ρ	r ≈ 0.00	Deterministic

This orthogonality is by design: the normalization in F erases surface variation to expose structural invariants. The Cartan structure captures the algebraic content. They measure different things.

Where F matters: 85 matrices collide on ρ (15 groups). F distinguishes 10 of those 16 groups — it's a tiebreaker for Cartan degeneracies.

The f(n) corkscrew function is information-neutral — it's an injective spiral encoding from the spectral index. It contributes geometry (golden-angle low-discrepancy layout for nearest-neighbor decoding) but no new information. The contribution is at the torus transition: the winding pair (a,b) ∈ H₁(T²) is a genuine homological invariant.

Pipeline Architecture

Equation text
    → tokenize → 8×8 strand adjacency matrix (exact integers)
    → characteristic polynomial (exact, integer coefficients)
    → spectral radius ρ (from polynomial, not power iteration)
    → spectralRadiusToColor → shape classification
    → determineAlignment → alignment score
    → ncDerived → witness strength
    → JSON receipt

Recommended encoding hierarchy

1. Identity: base-B integer hash of matrix (injectivity provable for B ≥ 10)
2. Similarity: exact characteristic polynomial (192 unique, integer-only)
3. Layout: f(n) spiral with integer packing (no phinary floats)
4. Distinguishability floor: Cartan Δ = 17/1792 on Δ₇

Known Bugs (from review)

1. Power iteration non-convergence: 22 of 250 matrices have ρ values wrong by >1e-3. The Lean powerIteration in MatrixN.lean inherits this. Fix: use characteristic polynomial instead.
2. Phinary packing not injective: integration_sprint.py:269 uses float accumulation + truncation. Fix: replace with positional integer packing of charpoly coefficients.
3. Torus winding saturates at n ≥ 65536: pist_braid_bridge.py stores n//2 as Q16.16 which clamps at 32768. Fix: use integer winding counts.
4. 120 stale DB rows: ene.rrc_classifications on neon-64gb was written under the old buggy power-iteration regime. Fix: re-classify with exact eigenvalues.

Files

• formal/SilverSight/PIST/ClassifyN.lean — spectral thresholds
• formal/SilverSight/PIST/Matrices250.lean — 250 8×8 matrices
• formal/SilverSight/PIST/SpectralN.lean — spectral profile computation (uses power iteration — needs fix)
• formal/SilverSight/PIST/CartanConnection.lean — Cartan gap proof (Δ = 17/1792)
• formal/CoreFormalism/CharacterTransform.lean — Sidon → Z₂⁴ → Cartan chain
• formal/SilverSight/RRC/Q16_16Manifold.lean — 278-row fixture corpus
• python/silversight_engine.py — Python implementation of the pipeline
• data/spectral_codebook_raw.json — 250-entry raw spectral data (from power iteration — needs regeneration)