SilverSight/docs/SYMBOLIC_REGRESSION_DESIGN.md

# SilverSight Symbolic Regression — Design Document

## Principle

**No external imports.** Use existing SilverSight infrastructure:
- `EquationShape` + `classifyEquation` — structural classification
- `spectral_profile` — 8D spectral features
- `chaos_game` — IFS contraction search
- `sidon_address` — deterministic addressing
- `qubo/` — binary optimization

## Architecture

```
Input: (X, y) data points
  ↓
1. Generate candidate expression trees
  ↓
2. Classify each with classifyEquation → Hachimoji state
  ↓
3. Compute spectral profile for each expression
  ↓
4. Apply linear scaling (solve for a, b)
  ↓
5. Compute fitness = BIC(scaled_expression, data)
  ↓
6. Use chaos game to navigate state space
  ↓
7. Use QUBO to select optimal expression
  ↓
Output: Best expression with R² score
```

## Missing Pieces (to implement)

### 1. Expression Tree Data Structure

```python
@dataclass
class ExprNode:
    op: str           # '+', '-', '*', '/', 'sqrt', 'sin', 'cos', 'log', 'exp', 'x', 'const'
    left: Optional['ExprNode']
    right: Optional['ExprNode']
    value: Optional[float]  # for const nodes
    depth: int = 0
    size: int = 0
```

### 2. Linear Scaling (Keijzer 2003)

Given expression g(x) and target y, solve:
```
f(x) = a·g(x) + b
where a, b = argmin Σ(f(x_i) - y_i)²
```

Closed-form solution:
```
a = cov(g, y) / var(g)
b = mean(y) - a·mean(g)
```

### 3. BIC Fitness

```
fitness = n·ln(MSE) + k·ln(n)
where:
  n = data points
  k = tree complexity (weighted: var=1, const=3)
  MSE = mean squared error after linear scaling
```

### 4. Chaos Game Search

Use existing `chaos_game.py` to navigate expression space:
- Each expression maps to a spectral profile
- Spectral profile maps to a Sidon address
- Chaos game contracts toward good expressions

### 5. ε-Lexicase Selection

Select on individual data points, not aggregate fitness:
- For each data point, keep only expressions within ε of best
- This preserves behavioral diversity

### 6. Operator Diversity via Hachimoji States

- Φ (trivial): simple expressions (x, x², √x)
- Σ (symmetric): balanced expressions (x·y, x+y)
- Λ (quantified): expressions with constants
- Π (complex): deep expressions (sin(1/x), exp(x²))
- Ω (contradiction): degenerate expressions (constant, NaN)

Force diversity by requiring expressions in multiple Hachimoji states.

## Implementation Plan

### Phase 1: Core Infrastructure
1. `python/expr_tree.py` — expression tree data structure
2. `python/linear_scaling.py` — Keijzer linear scaling
3. `python/bic_fitness.py` — BIC fitness function

### Phase 2: Search
4. `python/chaos_game_search.py` — chaos game navigation
5. `python/lexicase_selection.py` — ε-lexicase selection

### Phase 3: Integration
6. `python/symbolic_regression.py` — main entry point
7. `tests/test_symbolic_regression.py` — verification

### Phase 4: QUBO Enhancement
8. Use QUBO to select optimal expression from candidates
9. Use Hachimoji states to enforce diversity

## Key Advantage Over GP-ELITE

GP-ELITE uses genetic programming (random mutation + crossover).
SilverSight uses **chaos game** (deterministic IFS contraction) + **Hachimoji states** (structural classification).

This means:
- Deterministic: same input → same output
- Structured: expressions are classified by type, not just fitness
- Efficient: chaos game contracts faster than random search

## Test Cases

1. **Kepler's Third Law**: T = a^1.5 from 8 planets
2. **Simple polynomial**: y = 2x² + 3x + 1
3. **Trigonometric**: y = sin(x) + 0.5·cos(2x)
4. **Exponential**: y = exp(-x²)
5. **BMCTE entropy**: H(p) = f(N, p) from experiment data