SilverSight/docs/cmix_epigenetic_analysis.md

# cmix Weight Matrix Analysis: Compression Shape as Epigenetic Landscape

## The cmix Architecture

cmix (fx2-cmix) is the current Hutter Prize record holder for enwik9 compression:
- **Total size**: 110,793,128 bytes (including 441,463-byte decompressor)
- **Previous record**: 112,578,322 bytes
- **Improvement**: 1.585%
- **Decompression time**: 65 hours (4 vCPU, 16GB RAM)

## Model Inventory (461 models)

The 461 models are the "genes" of the compressor:

| Model Type | Count | Role | Epigenetic analog |
|---|---|---|---|
| FXCM (context-mapped) | 431 | Primary predictors | Housekeeping genes (bulk of expression) |
| Match models | 10 | Sequence matching | Repeat-element silencing |
| Indirect (nonstationary) | 15 | Adaptive context | Histone modification |
| Indirect (run-map) | 1 | Run-length context | CpG methylation |
| Direct | 1 | Direct byte context | Promoter accessibility |
| Bracket | 1 | Nesting structure | Chromatin looping |
| PPMD | 1 | Context-tree model | Enhancer element |
| Byte mixer (LSTM) | 1 | Neural prediction | Transcription factor |
| **Total** | **461** | | |

## Weight Matrix Structure (the "Genome")

### Layer 0: 23 mixers × 461 inputs
- 23 context-dependent mixers, each with 461 weight parameters
- Each mixer has a per-context hash map (up to 10,000 contexts)
- Total active parameters: ~106M floats (424MB)
- This is the **primary genome** — the compression shape

### Layer 1: 1 mixer × 25 inputs
- 23 layer-0 outputs + 2 auxiliary (fxcm + byte_mixer)
- Single mixer with per-context weights
- This is the **epigenetic controller** — which layer-0 mixers to trust

### The 23 Layer-0 Mixer Contexts

Each mixer operates on a different context (the "regulatory region"):

| # | Context | Learning rate | Role |
|---|---|---|---|
| 1 | mx9 | 0.005 | General context |
| 2 | mx10 | 0.0005 | Slow adaptation |
| 3 | mx11 | 0.005 | General context |
| 4 | mx12 | 0.0005 | Slow adaptation |
| 5 | mx13 | 0.005 | General context |
| 6 | mxx | 0.001 | Cross-model |
| 7 | recent_bytes[2] | 0.002 | Local history |
| 8 | line_break | 0.0007 | Structural |
| 9 | longest_match | 0.0005 | Match quality |
| 10 | mx19cxt | 0.002 | Extended context |
| 11 | auxiliary | 0.0005 | Auxiliary feedback |
| 12 | mx18 | 0.001 | Special context |
| 13 | mx7 | 0.001 | Special context |
| 14 | wordscxt | 0.005 | Word-level |
| 15 | b2streamcxt | 0.001 | Byte-2 stream |
| 16 | mx5 | 0.001 | Special context |
| 17 | mx6 | 0.005 | Special context |
| 18 | b3streamcxt | 0.001 | Byte-3 stream |
| 19 | mx8 | 0.001 | Special context |
| 20 | mx17 | 0.005 | Special context |
| 21 | mx16 | 0.005 | Special context |
| 22 | mx14 | 0.005 | Special context |
| 23 | mx15 | 0.005 | Special context |

## The Compression Shape in Math Space

The 23 × 461 weight matrix W defines the compression shape.
Since it's non-square, the SVD gives:

```
W = U Σ V^T
```

Where:
- U: 23 × 23 orthogonal matrix (which mixers cooperate)
- Σ: 23 singular values (the "active genes" — which directions matter)
- V: 461 × 23 matrix (which models contribute to each direction)

The 23 singular values ARE the eigenvalues of the compression shape.
They tell us:
- **High singular values**: "overexpressed genes" — models that contribute most
- **Near-zero singular values**: "silenced genes" — redundant or useless models
- **The spectrum shape**: the "epigenetic landscape" — how the compressor
  distributes its prediction capacity across models

## Epigenetic Mapping

| Compression concept | Epigenetic concept |
|---|---|
| 461 models | 461 genes |
| 23 mixers | 23 regulatory regions |
| Layer-1 mixer | Master regulator |
| Learning rate | Methylation rate |
| Context hash map | Chromatin accessibility |
| Weight update | Histone modification |
| Weight decay | DNA methylation decay |
| Article reordering | Chromatin looping |
| NLP stemmer | Transcription factor |
| Dictionary | Reference genome |
| LSTM | Neural crest prediction |

## How to Extract the Weight Matrix

To compute the SVD, we need the actual trained weight matrix from a
cmix run on enwik8. The process:

1. Modify cmix to dump the layer-0 weight matrix after compression
2. Run cmix on enwik8 (100MB, ~10 hours)
3. Extract the 23 × 461 matrix (one per context, average over contexts)
4. Compute SVD → 23 singular values
5. The singular value spectrum IS the compression shape equation

## The "Equation" of the Compression Shape

The characteristic polynomial of W^T W (461 × 461) gives the equation.
Its roots are the squared singular values — the eigenvalues of the
compression shape.

For a 23 × 461 matrix, W^T W has at most 23 nonzero eigenvalues.
These 23 values describe the entire compression capacity.

Storing these 23 values (instead of the full weight matrix) would be
the "equation" — analogous to how the minimal polynomial of a matrix
captures its eigenvalue structure from n+1 coefficients instead of n² entries.

## Alternative Expression Patterns

If the SVD reveals that 5 singular values capture 95% of the energy,
then 5 mixers suffice — the other 18 are redundant. This means:
- The compression shape lives in a 5-dimensional subspace
- Alternative configurations in this subspace may compress better
- The "locality" to search is the neighborhood of the current
  weight matrix in this 5D subspace

This is the "map its locality for other combinations" step:
- Current cmix: a point in 23D weight space
- SVD reduces to 5D principal subspace
- Search the 5D neighborhood for better compression

## Connection to SilverSight

The SilverSight pipeline's universal pipeline already implements:
- QR decomposition (O-AMMR): the SVD is the symmetric version
- Chiral hachimoji LUT: encodes the eigenvalue spectrum
- GCCL Admit: validates that a weight configuration is lawful
- AngrySphinx: budgets the search through weight space
- MMR: tracks the compression history as Mountains

The cmix weight matrix IS the manifold. The SVD IS the eigenvalue
decomposition. The chiral hachimoji LUT IS the receipt. The epigenetic
framing tells us what to look for: which "genes" (models) are
overexpressed (high singular value), which are silenced (near-zero),
and what alternative "expression patterns" (nearby weight matrices)
might compress better.

## Next Steps

1. **Extract weights**: modify cmix to dump layer-0 weight matrix
2. **Compute SVD**: get the 23 singular values
3. **Encode as hachimoji LUT**: 23 coefficients × 30 bases = 690 bytes
4. **Map in math space**: the singular value spectrum is the "equation"
5. **Search neighborhood**: vary weights along top-5 singular directions
6. **Measure**: does the variation improve compression?

The epigenetic framing predicts: the compression shape is low-rank
(most models are redundant), and the active subspace is small
(5-10 dimensions out of 23). If confirmed, the search space for
better compressors is much smaller than the full 23 × 461 space.