openresearch
|
4910e3877c
|
Add cmix weight matrix analysis: compression shape as epigenetic landscape
Documents the cmix (fx2-cmix) architecture from the epigenetic
perspective:
- 461 models = 461 genes
- 23 layer-0 mixers = 23 regulatory regions
- Layer-1 mixer = master regulator
- Weight matrix (23 × 461) = the compression genome
- SVD of weight matrix = the compression shape equation
- Singular values = active genes (high) vs silenced (low)
- Learning rates = methylation rates
- Context hash map = chromatin accessibility
The key prediction: the compression shape is low-rank.
If 5 singular values capture 95% of energy, then:
- 5 mixers suffice (18 are redundant)
- The search space for better compressors is 5D, not 23 × 461
- Alternative expression patterns (nearby weight matrices) may compress better
Next steps: extract trained weights from cmix, compute SVD, encode
as hachimoji LUT, map in math space, search neighborhood.
|
2026-07-03 16:10:01 +00:00 |
|