SilverSight/docs/HACHIMOJI_DNA_SYNTAX.md

# Hachimoji DNA Encoding Syntax — Formal Specification

**Version:** 1.0
**Date:** 2026-06-23
**Status:** Active
**Purpose:** Computational substrate for manifold/QUBO/eigenvalue work.

---

## 1. Alphabet

### 1.1 Base Set

Eight bases, ordered by ASCII value for monotone lexicographic sorting:

| Index | Base | ASCII | Phase | Binary (3-bit) |
|-------|------|-------|-------|----------------|
| 0 | A | 0x41 | 0° | 000 |
| 1 | B | 0x42 | 45° | 001 |
| 2 | C | 0x43 | 90° | 010 |
| 3 | G | 0x47 | 135° | 011 |
| 4 | P | 0x50 | 180° | 100 |
| 5 | S | 0x53 | 225° | 101 |
| 6 | T | 0x54 | 270° | 110 |
| 7 | Z | 0x5A | 315° | 111 |

### 1.2 Ordering Axiom

```
A < B < C < G < P < S < T < Z
```

This ordering is **canonical** and **immutable**. It satisfies:

1. **ASCII order = index order.** `ord(A) < ord(B) < ... < ord(Z)`.
2. **Index order = lexicographic rank.** For any two sequences of equal length, `s₁ < s₂` (lexicographic) if and only if `dna_to_int(s₁) < dna_to_int(s₂)`.
3. **Monotone encoding.** Assigning sequences by increasing integer rank produces lexicographically sorted output.

**Proof:** The bases are chosen such that their ASCII codes are in ascending order: 0x41 < 0x42 < 0x43 < 0x47 < 0x50 < 0x53 < 0x54 < 0x5A. Since lexicographic comparison proceeds character-by-character using ASCII ordering, and our index ordering matches ASCII ordering, integer rank ordering implies lexicographic ordering. ∎

---

## 2. Integer ↔ DNA Conversion

### 2.1 Encoding (integer → DNA)

```
int_to_dna(value: int, length: int) → string
```

Converts a non-negative integer to a fixed-length DNA sequence using base-8 representation, most-significant digit first.

```
Algorithm:
  seq = []
  for i in 1..length:
      seq.append(BASES[value mod 8])
      value = value ÷ 8
  return reverse(seq)
```

**Constraints:**
- `value ≥ 0`
- `length ≥ 1`
- `value < 8^length` (otherwise the sequence cannot represent the value)

**Examples:**
```
int_to_dna(0, 3)   → "AAA"
int_to_dna(1, 3)   → "AAB"
int_to_dna(7, 3)   → "AAZ"
int_to_dna(8, 3)   → "ABA"
int_to_dna(511, 3) → "ZZZ"
```

### 2.2 Decoding (DNA → integer)

```
dna_to_int(sequence: string) → int
```

Converts a DNA sequence back to its integer value.

```
Algorithm:
  value = 0
  for each base b in sequence:
      value = value × 8 + BASE_TO_INDEX[b]
  return value
```

**Examples:**
```
dna_to_int("AAA") → 0
dna_to_int("AAB") → 1
dna_to_int("ABA") → 8
dna_to_int("ZZZ") → 511
```

### 2.3 Roundtrip Axiom

```
∀ value ∈ [0, 8^length):
    dna_to_int(int_to_dna(value, length)) = value
```

### 2.4 Lexicographic Ordering Axiom

```
∀ v₁, v₂ ∈ [0, 8^length):
    v₁ < v₂ ⟺ int_to_dna(v₁, length) < int_to_dna(v₂, length)
    (where < on strings is lexicographic comparison)
```

---

## 3. Symbol Encoding

### 3.1 Chunks

A **chunk** is a contiguous group of bytes treated as a single symbol.

| Chunk size | Range | Symbols | Bases needed |
|---|---|---|---|
| 1 byte | 0x00–0xFF | 256 | 3 (8³ = 512 ≥ 256) |
| 2 bytes | 0x0000–0xFFFF | 65,536 | 6 (8⁶ = 262,144 ≥ 65,536) |
| n unique | — | n | ⌈log₈(n)⌉ |

### 3.2 Bases Per Symbol

```
bases_needed(n_symbols: int) → int
    length = 1
    while 8^length < n_symbols:
        length += 1
    return length
```

---

## 4. Monotone LUT

### 4.1 Definition

A **monotone LUT** is a bijection:

```
L: {0, 1, ..., n-1} → DNA_sequences × Solutions × Energies
```

such that:

```
∀ i < j: L(i).energy ≤ L(j).energy
```

and:

```
∀ i < j: L(i).sequence < L(j).sequence  (lexicographic)
```

### 4.2 Construction

```
build_monotone_lut(solutions, energies) → LUT

Algorithm:
  1. Sort solutions by energy (ascending)
  2. Assign DNA sequences in order:
       rank 0 → int_to_dna(0, seq_len)
       rank 1 → int_to_dna(1, seq_len)
       ...
       rank n-1 → int_to_dna(n-1, seq_len)
  3. Return LUT: sequence → (solution, energy)
```

### 4.3 Properties

1. **Monotonicity.** Lexicographic sort of sequences = energy sort of solutions.
2. **Completeness.** Every solution has exactly one DNA sequence.
3. **Injectivity.** Every DNA sequence maps to at most one solution.
4. **Minimal encoding.** The optimal solution always maps to `AAA...A` (the lexicographically smallest sequence).

### 4.4 Verification

```
verify_monotone(lut) → (bool, float)

  is_monotone = (sort_by_sequence(lut) == sort_by_energy(lut))
  rank_correlation = spearman(sequence_indices, energy_ranks)
  return (is_monotone, rank_correlation)
```

A valid monotone LUT has `is_monotone = true` and `rank_correlation = 1.0`.

---

## 5. File Formats

### 5.1 DNA File (`.dna`)

Plain text file containing a single DNA sequence.

```
Format: [ACGTBPSZ]+
Encoding: ASCII
Line ending: LF (optional)
```

### 5.2 LUT File (`.lut`)

JSON file mapping DNA sequences to solutions and energies.

```json
{
  "format": "hachimoji_monotone_lut_v1",
  "bases": "ABCGPSTZ",
  "n_vars": 20,
  "n_solutions": 1048576,
  "seq_length": 7,
  "encoding": "monotone",
  "monotone": true,
  "rank_correlation": 1.0,
  "qubo_matrix": [[...]],
  "entries": {
    "AAAAAAA": {"x": [0,0,...,0], "energy": 0.0},
    "AAAAAAB": {"x": [1,0,...,0], "energy": 3.074},
    ...
  }
}
```

**Required fields:**
- `format` — always `"hachimoji_monotone_lut_v1"`
- `bases` — the base alphabet (must be `"ABCGPSTZ"`)
- `n_vars` — number of variables in the problem
- `n_solutions` — total number of entries
- `seq_length` — bases per sequence
- `encoding` — always `"monotone"`
- `monotone` — must be `true` for a valid LUT
- `rank_correlation` — must be `1.0` for a valid LUT
- `entries` — the mapping: sequence → {x, energy}

### 5.3 Metadata File (`.json`)

Problem-level metadata (optional).

```json
{
  "problem": "banded_qubo_20var",
  "n_vars": 20,
  "n_solutions": 1048576,
  "optimal": {"x": [...], "energy": 0.0, "seq": "AAAAAAA"},
  "worst": {"x": [...], "energy": 60.66, "seq": "GZZZZZZ"},
  "timing": {"generate": 0.185, "energy": 0.113, "sort": 0.096, "total": 0.394}
}
```

---

## 6. Operations

### 6.1 Encode

```
encode(data: bytes, chunk_size: int) → (dna: string, lut: dict)

  1. Split data into chunks of chunk_size bytes
  2. Rank chunks by frequency (most frequent → rank 0)
  3. Assign DNA sequences by rank
  4. Concatenate sequences
  5. Return (dna_string, decode_lut)
```

### 6.2 Decode

```
decode(dna: string, lut: dict, bases_per_symbol: int) → bytes

  1. Split dna into groups of bases_per_symbol
  2. Look up each group in lut
  3. Concatenate results
  4. Return bytes
```

### 6.3 Roundtrip

```
decode(encode(data)) = data
```

This must hold for all valid inputs. Verified at encode time.

---

## 7. QUBO Integration

### 7.1 Problem Encoding

A QUBO problem `minimize x^T Q x` over `x ∈ {0,1}^n` is encoded as:

1. **Matrix:** QUBO matrix Q encoded as bytes → DNA (via `encode`)
2. **Solutions:** All (or sampled) solutions encoded as DNA sequences (via monotone LUT)
3. **LUT:** The monotone LUT maps DNA sequences to (solution, energy) pairs

### 7.2 Solving

```
solve_qubo(Q) → (optimal_x, optimal_energy, optimal_seq)

  1. Enumerate all 2^n solutions (or sample)
  2. Compute energies: E_i = x_i^T Q x_i
  3. Build monotone LUT
  4. Return: optimal = LUT["AAA...A"]
```

### 7.3 Sorting as Computation

The act of sorting DNA sequences IS the act of solving the QUBO:

```
sorted(dna_sequences) → solutions in energy order
first(sorted) = optimal solution
last(sorted) = worst solution
```

This is the core insight: **sorting is solving**.

---

## 8. GPU Integration

### 8.1 Radix Sort

DNA sequences are base-8 digit arrays. Radix sort on these arrays is:

- **O(n · k)** where n = number of sequences, k = sequence length
- For constant k, this is **O(n)** — linear time
- Each digit is 3 bits, perfectly suited for GPU parallel processing

### 8.2 Zero Copy

CPU writes DNA sequences to GPU-accessible unified memory. GPU sorts in-place. CPU reads result. No memcpy.

```
CPU → [unified memory] → GPU (radix sort) → [unified memory] → CPU
```

### 8.3 Braid Sort Kernel

The GPU compute shader performs braid crossings:

```
braid_cross(a, b):
    if a > b: return (b, a)  // triangle rotation
    else:     return (a, b)  // eigensolid (converged)
```

Each workgroup processes a chunk of the array. After log₂(n) passes, the array is sorted.

---

## 9. Surface Rendering

### 9.1 8×8 Hachimoji Surface

A QUBO solution is rendered as an 8×8 pixel grid:

- Each pixel = one variable
- x[i] = 0 → dark (A-state, RGB: 13,13,13)
- x[i] = 1 → bright (G-state, RGB: 26,204,77)
- Variables laid out in row-major order

### 9.2 Color Map

| Base | Color | RGB | Meaning |
|---|---|---|---|
| A | Near black | (13, 13, 13) | x = 0 |
| B | Deep purple | (51, 26, 77) | synthetic |
| C | Ocean blue | (26, 77, 128) | synthetic |
| G | Hachimoji green | (26, 204, 77) | x = 1 |
| P | Plasma orange | (230, 102, 26) | synthetic |
| S | Spectral violet | (153, 51, 204) | synthetic |
| T | Teal | (26, 179, 179) | synthetic |
| Z | Near white | (242, 242, 242) | synthetic |

### 9.3 Eigenvalue Fingerprint

The 8×8 surface is the **eigenvalue fingerprint** of the QUBO solution. Different QUBOs produce different surfaces. The surface IS the answer.

---

## 10. Invariants

The following properties must hold for any valid Hachimoji DNA encoding:

1. **Alphabet consistency.** All sequences use only bases from `{A, B, C, G, P, S, T, Z}`.
2. **Ordering consistency.** ASCII order = index order = lexicographic rank.
3. **Monotonicity.** In a monotone LUT, `sort(sequence) = sort(energy)`.
4. **Roundtrip.** `decode(encode(data)) = data` for all valid inputs.
5. **Uniqueness.** Each solution maps to exactly one DNA sequence.
6. **Minimality.** The optimal (lowest-energy) solution maps to `AAA...A`.
7. **Completeness.** Every entry in the LUT has a valid solution and energy.
8. **Correlation.** Rank correlation between sequence index and energy = 1.0.

---

## 11. Anti-Patterns

The following are **forbidden**:

1. **Non-ASCII bases.** Sequences must use only the 8 canonical bases.
2. **Variable-length symbols within a LUT.** All sequences in a LUT must have the same length.
3. **Non-monotone assignment.** If `encoding = "monotone"`, the LUT must satisfy the monotonicity axiom.
4. **Lossy encoding.** Roundtrip must be exact. No approximation.
5. **Mutable base ordering.** The base ordering `A < B < C < G < P < S < T < Z` is fixed forever.

---

## 12. Extensions

Future extensions (not yet specified):

- **Multi-pass radix sort** for sequences longer than 8 bases
- **Hierarchical LUTs** for problems with structure (banded, sparse, block-diagonal)
- **Streaming encode/decode** for large files
- **WebGPU compute shader** for GPU-accelerated sorting
- **Finsler metric integration** for manifold-aware encoding
- **Eigenvalue surface** for visual comparison of solutions

---

## Appendix A: Reference Implementation

| Component | File | Language |
|---|---|---|
| LUT builder | `python/dna_lut.py` | Python |
| File encoder | `python/dna_encode_file.py` | Python |
| Radix sort | `python/dna_radix_gpu.py` | Python + NumPy |
| GPU kernel | `python/dna_braid.wgsl` | WGSL |
| WebGPU host | `python/dna_webgpu.js` | JavaScript |
| Surface render | `python/dna_surface.html` | HTML + Canvas |

## Appendix B: Proof of Monotonicity

**Theorem:** The monotone encoding satisfies the lexicographic ordering axiom.

**Proof:**

1. Let `S = {s₀, s₁, ..., s_{n-1}}` be solutions sorted by energy: `E(s₀) ≤ E(s₁) ≤ ... ≤ E(s_{n-1})`.
2. Assign `seq_i = int_to_dna(i, k)` where `k = ⌈log₈(n)⌉`.
3. By construction, `i < j ⟹ seq_i < seq_j` (lexicographic), because `int_to_dna` preserves ordering (§2.4).
4. Therefore, `seq_i < seq_j ⟹ E(s_i) ≤ E(s_j)`.
5. The LUT is monotone. ∎

## Appendix C: Worked Example

**Problem:** 3-variable diagonal QUBO, Q = diag(3, 2, 1).

| Rank | DNA | Solution | Energy |
|---|---|---|---|
| 0 | AAA | [0,0,0] | 0.0 |
| 1 | AAB | [0,0,1] | 1.0 |
| 2 | AAC | [0,1,0] | 2.0 |
| 3 | AAG | [0,1,1] | 3.0 |
| 4 | AAP | [1,0,0] | 3.0 |
| 5 | AAS | [1,0,1] | 4.0 |
| 6 | AAT | [1,1,0] | 5.0 |
| 7 | AAZ | [1,1,1] | 6.0 |

**Verification:**
- Lexicographic sort: AAA < AAB < AAC < AAG < AAP < AAS < AAT < AAZ
- Energy sort: 0.0 ≤ 1.0 ≤ 2.0 ≤ 3.0 ≤ 3.0 ≤ 4.0 ≤ 5.0 ≤ 6.0
- Monotone: ✓
- Optimal: AAA → E=0.0
- Worst: AAZ → E=6.0