SilverSight/docs/cmix_epigenetic_analysis.md

cmix Weight Matrix Analysis: Compression Shape as Epigenetic Landscape

The cmix Architecture

cmix (fx2-cmix) is the current Hutter Prize record holder for enwik9 compression:

• Total size: 110,793,128 bytes (including 441,463-byte decompressor)
• Previous record: 112,578,322 bytes
• Improvement: 1.585%
• Decompression time: 65 hours (4 vCPU, 16GB RAM)

Model Inventory (461 models)

The 461 models are the "genes" of the compressor:

Model Type	Count	Role	Epigenetic analog
FXCM (context-mapped)	431	Primary predictors	Housekeeping genes (bulk of expression)
Match models	10	Sequence matching	Repeat-element silencing
Indirect (nonstationary)	15	Adaptive context	Histone modification
Indirect (run-map)	1	Run-length context	CpG methylation
Direct	1	Direct byte context	Promoter accessibility
Bracket	1	Nesting structure	Chromatin looping
PPMD	1	Context-tree model	Enhancer element
Byte mixer (LSTM)	1	Neural prediction	Transcription factor
Total	461

Weight Matrix Structure (the "Genome")

Layer 0: 23 mixers × 461 inputs

• 23 context-dependent mixers, each with 461 weight parameters
• Each mixer has a per-context hash map (up to 10,000 contexts)
• Total active parameters: ~106M floats (424MB)
• This is the primary genome — the compression shape

Layer 1: 1 mixer × 25 inputs

• 23 layer-0 outputs + 2 auxiliary (fxcm + byte_mixer)
• Single mixer with per-context weights
• This is the epigenetic controller — which layer-0 mixers to trust

The 23 Layer-0 Mixer Contexts

Each mixer operates on a different context (the "regulatory region"):

#	Context	Learning rate	Role
1	mx9	0.005	General context
2	mx10	0.0005	Slow adaptation
3	mx11	0.005	General context
4	mx12	0.0005	Slow adaptation
5	mx13	0.005	General context
6	mxx	0.001	Cross-model
7	recent_bytes[2]	0.002	Local history
8	line_break	0.0007	Structural
9	longest_match	0.0005	Match quality
10	mx19cxt	0.002	Extended context
11	auxiliary	0.0005	Auxiliary feedback
12	mx18	0.001	Special context
13	mx7	0.001	Special context
14	wordscxt	0.005	Word-level
15	b2streamcxt	0.001	Byte-2 stream
16	mx5	0.001	Special context
17	mx6	0.005	Special context
18	b3streamcxt	0.001	Byte-3 stream
19	mx8	0.001	Special context
20	mx17	0.005	Special context
21	mx16	0.005	Special context
22	mx14	0.005	Special context
23	mx15	0.005	Special context

The Compression Shape in Math Space

The 23 × 461 weight matrix W defines the compression shape. Since it's non-square, the SVD gives:

W = U Σ V^T

Where:

• U: 23 × 23 orthogonal matrix (which mixers cooperate)
• Σ: 23 singular values (the "active genes" — which directions matter)
• V: 461 × 23 matrix (which models contribute to each direction)

The 23 singular values ARE the eigenvalues of the compression shape. They tell us:

• High singular values: "overexpressed genes" — models that contribute most
• Near-zero singular values: "silenced genes" — redundant or useless models
• The spectrum shape: the "epigenetic landscape" — how the compressor distributes its prediction capacity across models

Epigenetic Mapping

Compression concept	Epigenetic concept
461 models	461 genes
23 mixers	23 regulatory regions
Layer-1 mixer	Master regulator
Learning rate	Methylation rate
Context hash map	Chromatin accessibility
Weight update	Histone modification
Weight decay	DNA methylation decay
Article reordering	Chromatin looping
NLP stemmer	Transcription factor
Dictionary	Reference genome
LSTM	Neural crest prediction

How to Extract the Weight Matrix

To compute the SVD, we need the actual trained weight matrix from a cmix run on enwik8. The process:

1. Modify cmix to dump the layer-0 weight matrix after compression
2. Run cmix on enwik8 (100MB, ~10 hours)
3. Extract the 23 × 461 matrix (one per context, average over contexts)
4. Compute SVD → 23 singular values
5. The singular value spectrum IS the compression shape equation

The "Equation" of the Compression Shape

The characteristic polynomial of W^T W (461 × 461) gives the equation. Its roots are the squared singular values — the eigenvalues of the compression shape.

For a 23 × 461 matrix, W^T W has at most 23 nonzero eigenvalues. These 23 values describe the entire compression capacity.

Storing these 23 values (instead of the full weight matrix) would be the "equation" — analogous to how the minimal polynomial of a matrix captures its eigenvalue structure from n+1 coefficients instead of n² entries.

Alternative Expression Patterns

If the SVD reveals that 5 singular values capture 95% of the energy, then 5 mixers suffice — the other 18 are redundant. This means:

• The compression shape lives in a 5-dimensional subspace
• Alternative configurations in this subspace may compress better
• The "locality" to search is the neighborhood of the current weight matrix in this 5D subspace

This is the "map its locality for other combinations" step:

• Current cmix: a point in 23D weight space
• SVD reduces to 5D principal subspace
• Search the 5D neighborhood for better compression

Connection to SilverSight

The SilverSight pipeline's universal pipeline already implements:

• QR decomposition (O-AMMR): the SVD is the symmetric version
• Chiral hachimoji LUT: encodes the eigenvalue spectrum
• GCCL Admit: validates that a weight configuration is lawful
• AngrySphinx: budgets the search through weight space
• MMR: tracks the compression history as Mountains

The cmix weight matrix IS the manifold. The SVD IS the eigenvalue decomposition. The chiral hachimoji LUT IS the receipt. The epigenetic framing tells us what to look for: which "genes" (models) are overexpressed (high singular value), which are silenced (near-zero), and what alternative "expression patterns" (nearby weight matrices) might compress better.

Next Steps

1. Extract weights: modify cmix to dump layer-0 weight matrix
2. Compute SVD: get the 23 singular values
3. Encode as hachimoji LUT: 23 coefficients × 30 bases = 690 bytes
4. Map in math space: the singular value spectrum is the "equation"
5. Search neighborhood: vary weights along top-5 singular directions
6. Measure: does the variation improve compression?

The epigenetic framing predicts: the compression shape is low-rank (most models are redundant), and the active subspace is small (5-10 dimensions out of 23). If confirmed, the search space for better compressors is much smaller than the full 23 × 461 space.