diff --git a/docs/SYMBOLIC_REGRESSION_DESIGN.md b/docs/SYMBOLIC_REGRESSION_DESIGN.md deleted file mode 100644 index d8e51c87..00000000 --- a/docs/SYMBOLIC_REGRESSION_DESIGN.md +++ /dev/null @@ -1,131 +0,0 @@ -# SilverSight Symbolic Regression — Design Document - -## Principle - -**No external imports.** Use existing SilverSight infrastructure: -- `EquationShape` + `classifyEquation` — structural classification -- `spectral_profile` — 8D spectral features -- `chaos_game` — IFS contraction search -- `sidon_address` — deterministic addressing -- `qubo/` — binary optimization - -## Architecture - -``` -Input: (X, y) data points - ↓ -1. Generate candidate expression trees - ↓ -2. Classify each with classifyEquation → Hachimoji state - ↓ -3. Compute spectral profile for each expression - ↓ -4. Apply linear scaling (solve for a, b) - ↓ -5. Compute fitness = BIC(scaled_expression, data) - ↓ -6. Use chaos game to navigate state space - ↓ -7. Use QUBO to select optimal expression - ↓ -Output: Best expression with R² score -``` - -## Missing Pieces (to implement) - -### 1. Expression Tree Data Structure - -```python -@dataclass -class ExprNode: - op: str # '+', '-', '*', '/', 'sqrt', 'sin', 'cos', 'log', 'exp', 'x', 'const' - left: Optional['ExprNode'] - right: Optional['ExprNode'] - value: Optional[float] # for const nodes - depth: int = 0 - size: int = 0 -``` - -### 2. Linear Scaling (Keijzer 2003) - -Given expression g(x) and target y, solve: -``` -f(x) = a·g(x) + b -where a, b = argmin Σ(f(x_i) - y_i)² -``` - -Closed-form solution: -``` -a = cov(g, y) / var(g) -b = mean(y) - a·mean(g) -``` - -### 3. BIC Fitness - -``` -fitness = n·ln(MSE) + k·ln(n) -where: - n = data points - k = tree complexity (weighted: var=1, const=3) - MSE = mean squared error after linear scaling -``` - -### 4. Chaos Game Search - -Use existing `chaos_game.py` to navigate expression space: -- Each expression maps to a spectral profile -- Spectral profile maps to a Sidon address -- Chaos game contracts toward good expressions - -### 5. ε-Lexicase Selection - -Select on individual data points, not aggregate fitness: -- For each data point, keep only expressions within ε of best -- This preserves behavioral diversity - -### 6. Operator Diversity via Hachimoji States - -- Φ (trivial): simple expressions (x, x², √x) -- Σ (symmetric): balanced expressions (x·y, x+y) -- Λ (quantified): expressions with constants -- Π (complex): deep expressions (sin(1/x), exp(x²)) -- Ω (contradiction): degenerate expressions (constant, NaN) - -Force diversity by requiring expressions in multiple Hachimoji states. - -## Implementation Plan - -### Phase 1: Core Infrastructure -1. `python/expr_tree.py` — expression tree data structure -2. `python/linear_scaling.py` — Keijzer linear scaling -3. `python/bic_fitness.py` — BIC fitness function - -### Phase 2: Search -4. `python/chaos_game_search.py` — chaos game navigation -5. `python/lexicase_selection.py` — ε-lexicase selection - -### Phase 3: Integration -6. `python/symbolic_regression.py` — main entry point -7. `tests/test_symbolic_regression.py` — verification - -### Phase 4: QUBO Enhancement -8. Use QUBO to select optimal expression from candidates -9. Use Hachimoji states to enforce diversity - -## Key Advantage Over GP-ELITE - -GP-ELITE uses genetic programming (random mutation + crossover). -SilverSight uses **chaos game** (deterministic IFS contraction) + **Hachimoji states** (structural classification). - -This means: -- Deterministic: same input → same output -- Structured: expressions are classified by type, not just fitness -- Efficient: chaos game contracts faster than random search - -## Test Cases - -1. **Kepler's Third Law**: T = a^1.5 from 8 planets -2. **Simple polynomial**: y = 2x² + 3x + 1 -3. **Trigonometric**: y = sin(x) + 0.5·cos(2x) -4. **Exponential**: y = exp(-x²) -5. **BMCTE entropy**: H(p) = f(N, p) from experiment data