Archive SYMBOLIC_REGRESSION_DESIGN.md

archive/2026-07-02/docs/SYMBOLIC_REGRESSION_DESIGN.md

@@ -0,0 +1,131 @@
+# SilverSight Symbolic Regression — Design Document
+
+## Principle
+
+**No external imports.** Use existing SilverSight infrastructure:
+- `EquationShape` + `classifyEquation` — structural classification
+- `spectral_profile` — 8D spectral features
+- `chaos_game` — IFS contraction search
+- `sidon_address` — deterministic addressing
+- `qubo/` — binary optimization
+
+## Architecture
+
+```
+Input: (X, y) data points
+  ↓
+1. Generate candidate expression trees
+  ↓
+2. Classify each with classifyEquation → Hachimoji state
+  ↓
+3. Compute spectral profile for each expression
+  ↓
+4. Apply linear scaling (solve for a, b)
+  ↓
+5. Compute fitness = BIC(scaled_expression, data)
+  ↓
+6. Use chaos game to navigate state space
+  ↓
+7. Use QUBO to select optimal expression
+  ↓
+Output: Best expression with R² score
+```
+
+## Missing Pieces (to implement)
+
+### 1. Expression Tree Data Structure
+
+```python
+@dataclass
+class ExprNode:
+    op: str           # '+', '-', '*', '/', 'sqrt', 'sin', 'cos', 'log', 'exp', 'x', 'const'
+    left: Optional['ExprNode']
+    right: Optional['ExprNode']
+    value: Optional[float]  # for const nodes
+    depth: int = 0
+    size: int = 0
+```
+
+### 2. Linear Scaling (Keijzer 2003)
+
+Given expression g(x) and target y, solve:
+```
+f(x) = a·g(x) + b
+where a, b = argmin Σ(f(x_i) - y_i)²
+```
+
+Closed-form solution:
+```
+a = cov(g, y) / var(g)
+b = mean(y) - a·mean(g)
+```
+
+### 3. BIC Fitness
+
+```
+fitness = n·ln(MSE) + k·ln(n)
+where:
+  n = data points
+  k = tree complexity (weighted: var=1, const=3)
+  MSE = mean squared error after linear scaling
+```
+
+### 4. Chaos Game Search
+
+Use existing `chaos_game.py` to navigate expression space:
+- Each expression maps to a spectral profile
+- Spectral profile maps to a Sidon address
+- Chaos game contracts toward good expressions
+
+### 5. ε-Lexicase Selection
+
+Select on individual data points, not aggregate fitness:
+- For each data point, keep only expressions within ε of best
+- This preserves behavioral diversity
+
+### 6. Operator Diversity via Hachimoji States
+
+- Φ (trivial): simple expressions (x, x², √x)
+- Σ (symmetric): balanced expressions (x·y, x+y)
+- Λ (quantified): expressions with constants
+- Π (complex): deep expressions (sin(1/x), exp(x²))
+- Ω (contradiction): degenerate expressions (constant, NaN)
+
+Force diversity by requiring expressions in multiple Hachimoji states.
+
+## Implementation Plan
+
+### Phase 1: Core Infrastructure
+1. `python/expr_tree.py` — expression tree data structure
+2. `python/linear_scaling.py` — Keijzer linear scaling
+3. `python/bic_fitness.py` — BIC fitness function
+
+### Phase 2: Search
+4. `python/chaos_game_search.py` — chaos game navigation
+5. `python/lexicase_selection.py` — ε-lexicase selection
+
+### Phase 3: Integration
+6. `python/symbolic_regression.py` — main entry point
+7. `tests/test_symbolic_regression.py` — verification
+
+### Phase 4: QUBO Enhancement
+8. Use QUBO to select optimal expression from candidates
+9. Use Hachimoji states to enforce diversity
+
+## Key Advantage Over GP-ELITE
+
+GP-ELITE uses genetic programming (random mutation + crossover).
+SilverSight uses **chaos game** (deterministic IFS contraction) + **Hachimoji states** (structural classification).
+
+This means:
+- Deterministic: same input → same output
+- Structured: expressions are classified by type, not just fitness
+- Efficient: chaos game contracts faster than random search
+
+## Test Cases
+
+1. **Kepler's Third Law**: T = a^1.5 from 8 planets
+2. **Simple polynomial**: y = 2x² + 3x + 1
+3. **Trigonometric**: y = sin(x) + 0.5·cos(2x)
+4. **Exponential**: y = exp(-x²)
+5. **BMCTE entropy**: H(p) = f(N, p) from experiment data