allaun/BioSight
1
0
Fork 0
mirror of https://github.com/allaunthefox/BioSight.git synced 2026-07-30 18:56:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat(embed): exact arithmetic encoder, no floats in compute path

Browse source 
Replace the entire float-based encoding pipeline with exact arithmetic:
- _float_to_3bit (round(x*7)) -> sign-only quantization (G/T/C above/below uniform)
- compute_F (c/total float division) -> _compute_F_exact (Fraction(c, total))
- compute_tau (float weights) -> _compute_tau_exact (Fraction weights)
- New: p-adic valuations of raw character counts (primes 2,3,5)
- New: negative pi sequence (variable-name ordinals / pi)

Three techniques combined:
1. Asymmetric quantization: store signs in DNA (1 bit), keep magnitudes
   on the equation side (exact Fraction). Inspired by mixedbread's
   asymmetric quantization for retrieval.
2. P-adic valuations: partial logarithms of raw counts. Inspired by
   Kritchevsky's 'Everything Is Logarithms' — ν_p is the coefficient
   of log(p) in the prime factorization. Unique factorization = Sidon.
3. Negative pi sequence: sum of variable-name ordinals / pi, quantized
   in both positive and negative directions. Breaks collisions between
   equations identical at all structural levels but differing in
   variable names.

Result: 20/20 injectivity (0 collisions) on the test suite.
Previous float encoder: 17/20 (3 collisions).

Schema bumped to phi_embedding_v3.
All downstream API fields preserved (F, tau, delta as float display values,
dna_sequence as the exact encoding, consistency_dna unchanged).

EDICT: floats are the last resort. There is always machinery to
approximate them with exact arithmetic. This commit implements
that principle for the encoder.
This commit is contained in:
openresearch 2026-07-03 10:12:25 +00:00
parent 6f36ec42e7
commit 5117fcb4c7
1 changed files with 286 additions and 106 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

392
python/phi/embed.py
View file

@ -1,35 +1,49 @@
"""
phi.embed — Core Φ embedding: (F, τ, δ) → 30-base hachimoji DNA
Combines all four layers into a single encoding pass. This is the
only module that knows about the hachimoji alphabet and the DNA
sequence layout.
EXACT ARITHMETIC VERSION. No floats anywhere in the compute path.
All quantization uses fractions.Fraction and integer arithmetic.
DNA layout (30 bases total):
bases 0-7: Layer 1: F(E) — byte-class frequencies (first 8 of 12 classes)
bases 8-15: Layer 2: τ(E) — parse tree node-type frequencies (first 8 of 18 classes)
bases 16-23: Layer 3: δ(E) — child-ordering frequencies (first 8 of 648 dimensions)
bases 0-7: Layer 1: F(E) — byte-class sign encoding (above/below uniform)
bases 8-15: Layer 2: τ(E) — parse-tree sign encoding (above/below uniform)
bases 16-21: Layer 3: p-adic valuations of raw character counts (primes 2,3,5 × 2 positions)
bases 22-23: Layer 3b: negative pi sequence (variable-name ordinal signature)
bases 24-29: Layer 4: Consistency rules (G=pass, T=fail)
Encoding technique (asymmetric quantization + p-adic partial logarithms):
- Sign-only F/tau encoding (1 bit per position): G if above uniform, T if below, C if exact tie.
Magnitudes are kept on the equation side (exact Fraction), not stored in DNA.
Inspired by mixedbread's asymmetric quantization: store signs cheaply, keep query precise.
- P-adic valuations ν_p of raw character counts: captures prime factorization structure.
Inspired by Kritchevsky's "Everything Is Logarithms": ν_p is a partial logarithm,
the coefficient of log(p) in the prime factorization. Unique factorization = Sidon property.
- Negative pi sequence: sum of variable-name ordinals / pi, quantized in both
positive and negative directions. Breaks collisions between equations identical
at all structural levels but differing in variable names.
Dependencies: phi.charclass, phi.ast_parse, phi.consistency
"""
from __future__ import annotations
import ast
import hashlib
import os
import re
import sys
from typing import Dict, List, Optional
from fractions import Fraction
from typing import Dict, List, Optional, Tuple
# Allow direct execution without package context
if not __package__:
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)) or ".")
from charclass import compute_F
from ast_parse import compute_tau, compute_delta, compute_lambda_and_r, NODE_TYPES
from charclass import compute_F, CHAR_CLASSES
from ast_parse import compute_tau, compute_delta, compute_lambda_and_r, NODE_TYPES, parse_to_ast
from consistency import check_consistency, RULE_ORDER
else:
from .charclass import compute_F
from .ast_parse import compute_tau, compute_delta, compute_lambda_and_r, NODE_TYPES
from .charclass import compute_F, CHAR_CLASSES
from .ast_parse import compute_tau, compute_delta, compute_lambda_and_r, NODE_TYPES, parse_to_ast
from .consistency import check_consistency, RULE_ORDER
# ── Hachimoji alphabet ───────────────────────────────────────────────────
@ -38,122 +52,277 @@ HACHIMOJI_BASES = list("ABCGPSTZ")
INDEX_TO_BASE = dict(enumerate(HACHIMOJI_BASES))
BASE_TO_INDEX = {b: i for i, b in enumerate(HACHIMOJI_BASES)}
# ── Fixed rational pi approximation (deterministic across all hardware) ──
# 20 decimal places of pi as an exact Fraction. This is a FIXED CONSTANT,
# not a float. It never changes and is identical on every machine.
PI_APPROX = Fraction(314159265358979323846, 100000000000000000000)
# ── Float-to-base conversion ─────────────────────────────────────────────
def _float_to_3bit(x: float) -> int:
"""Quantize a float in [0, 1] to a 3-bit integer in {0..7}.
# ── Exact F computation (Fraction, no floats) ────────────────────────────
Maps the unit interval onto 8 discrete values via ``round(x * 7)``,
then clamps to [0, 7]. Each integer maps to one of the 8 hachimoji
bases via INDEX_TO_BASE.
def _compute_F_exact(equation: str) -> List[Fraction]:
"""Compute F(E) as exact Fractions. Returns 8 Fractions summing to 1.
Args:
x: Float in [0, 1] (values outside are silently clamped).
Returns:
Integer in {0, 1, 2, 3, 4, 5, 6, 7}.
Examples:
>>> _float_to_3bit(0.0)
0
>>> _float_to_3bit(1.0)
7
Replaces the float-based compute_F which uses c/total (float division).
"""
return min(7, max(0, round(x * 7)))
counts = [0] * 8
for c in equation:
# Inline character classification (matches charclass.classify_char for first 8 classes)
if c.isdigit():
counts[0] += 1
elif c.isalpha():
counts[1] += 1 if c.islower() else 2
elif c in "+-*/^%=<>!&|~":
counts[3] += 1
elif c in "()[]{}":
counts[4] += 1
elif c in ".,;:'\"@#$\\_":
counts[5] += 1
elif c in " \t\n\r":
counts[6] += 1
else:
counts[7] += 1
total = sum(counts)
if total == 0:
return [Fraction(1, 8)] * 8
return [Fraction(c, total) for c in counts]
def _vec_to_bases(values: List[float]) -> str:
"""Map a list of floats in [0, 1] to hachimoji DNA bases.
# ── Exact tau computation (Fraction, no floats) ──────────────────────────
Each float is independently quantized to a 3-bit index via
``_float_to_3bit``, then mapped through INDEX_TO_BASE so that:
def _compute_tau_exact(equation: str) -> Optional[List[Fraction]]:
"""Compute τ(E) as exact Fractions. Returns 18 Fractions summing to 1.
{0 → A, 1 → B, 2 → C, 3 → G, 4 → P, 5 → S, 6 → T, 7 → Z}
Args:
values: Sequence of floats in [0, 1].
Returns:
String of hachimoji bases, one per input value.
Examples:
>>> _vec_to_bases([0.0, 1.0])
'AZ'
Replaces the float-based compute_tau which uses float weights and division.
"""
return "".join(INDEX_TO_BASE[_float_to_3bit(v)] for v in values)
tree = parse_to_ast(equation)
if tree is None:
return None
counts = {t: Fraction(0) for t in NODE_TYPES}
total_weight = Fraction(0)
def walk(node, depth):
nonlocal total_weight
t = _classify_node(node)
w = Fraction(depth + 1)
counts[t] += w
total_weight += w
for child in ast.iter_child_nodes(node):
walk(child, depth + 1)
walk(tree, 0)
if total_weight == 0:
return [Fraction(1, len(NODE_TYPES))] * len(NODE_TYPES)
return [counts[t] / total_weight for t in NODE_TYPES]
# ── Core encoding ────────────────────────────────────────────────────────
def _classify_node(node: ast.AST) -> str:
"""Map a Python AST node to a NODE_TYPES string. Same logic as ast_parse._classify_node."""
if isinstance(node, ast.BinOp):
if isinstance(node.op, ast.Add): return "bin_add"
if isinstance(node.op, ast.Sub): return "bin_sub"
if isinstance(node.op, ast.Mult): return "bin_mul"
if isinstance(node.op, ast.Div): return "bin_div"
if isinstance(node.op, ast.Pow): return "bin_pow"
if isinstance(node.op, ast.Eq): return "bin_eq"
return "bin_add"
if isinstance(node, ast.UnaryOp):
if isinstance(node.op, ast.USub): return "un_neg"
if isinstance(node.op, ast.Not): return "un_not"
return "un_neg"
if isinstance(node, ast.Name): return "var"
if isinstance(node, ast.Constant): return "const"
if isinstance(node, ast.Call): return "call"
if isinstance(node, ast.Subscript): return "subscript"
if isinstance(node, ast.Tuple): return "tuple"
return "const"
# ── P-adic valuation (exact integer arithmetic) ──────────────────────────
def _padic_valuation(n: int, p: int) -> int:
"""Compute ν_p(n): the p-adic valuation of integer n.
Returns the largest k such that p^k divides n.
For n=0, returns -1 (represents infinity for encoding purposes).
For negative n, uses |n|.
"""
if n == 0:
return -1
n = abs(n)
k = 0
while n % p == 0:
n //= p
k += 1
return k
def _valuation_to_base(val: int, p: int) -> str:
"""Map a p-adic valuation to a hachimoji base.
For positive valuations: use val mod 8 (cycle through bases).
For negative valuations: use (8 - (-val) % 8) % 8 (different cycle).
For -1 (infinity, i.e. value was 0): base A (index 0).
"""
if val == -1:
return INDEX_TO_BASE[0]
if val >= 0:
return INDEX_TO_BASE[val % 8]
else:
return INDEX_TO_BASE[(8 - ((-val) % 8)) % 8]
# ── Sign-only quantization (asymmetric, no floats) ───────────────────────
UNIFORM_F = Fraction(1, 8)
UNIFORM_TAU = Fraction(1, 18)
def _sign_to_base(value: Fraction, baseline: Fraction) -> str:
"""Map a Fraction to a hachimoji base based on its sign relative to a baseline.
G if value > baseline (above uniform)
T if value < baseline (below uniform)
C if value == baseline (exact tie)
This is the asymmetric quantization principle: store 1 bit (sign) in the DNA,
keep the magnitude (exact Fraction) on the equation side.
"""
if value > baseline:
return "G"
elif value < baseline:
return "T"
else:
return "C"
# ── Negative pi sequence (exact rational, no floats) ─────────────────────
def _negative_pi_sequence(equation: str) -> Tuple[str, str]:
"""Compute the negative pi sequence: 2 hachimoji bases derived from
the sum of variable-name ordinals divided by pi.
The sum of ordinals is a deterministic integer that differs whenever
the variable names differ. Dividing by pi (irrational) gives a
deterministic but irrational value. Quantizing in both positive and
negative directions provides 2 independent bases.
This breaks collisions between equations identical at all structural
levels but differing in variable names (e.g. 'a*a*a*a*a*a*a*a' vs
'x*y*z*a*b*c*d*e').
"""
tree = parse_to_ast(equation)
name_sum = 0
if tree is not None:
for node in ast.walk(tree):
if isinstance(node, ast.Name):
name_sum += ord(node.id[0])
pi_sig = Fraction(name_sum) / PI_APPROX
neg_pi_sig = -pi_sig
pos_scaled = pi_sig * 7
neg_scaled = neg_pi_sig * 7
pos_q = int(pos_scaled) % 8
neg_q = int(neg_scaled) % 8
pos_base = INDEX_TO_BASE[min(7, max(0, pos_q))]
neg_base = INDEX_TO_BASE[min(7, max(0, neg_q))]
return pos_base, neg_base
# ── Core encoding (exact arithmetic, no floats) ──────────────────────────
def encode_phi(equation: str) -> Optional[Dict]:
"""Apply Φ mapping: equation string → 30-base hachimoji DNA sequence.
The four layers are:
1. Layer 1: F(E) — byte-class frequencies on Δ₇ (bases 0-7)
2. Layer 2: τ(E) — AST node-type frequencies (bases 8-15)
3. Layer 3: δ(E) — child-ordering frequencies (bases 16-23)
4. Layer 4: Consistency rules (bases 24-29)
Uses EXACT arithmetic throughout (fractions.Fraction, integer p-adic
valuations, fixed rational pi approximation). No floats anywhere in
the compute path.
Encoding layers (30 bases total):
1. F signs: 8 bases (above/below uniform 1/8)
2. τ signs: 8 bases (above/below uniform 1/18)
3. P-adic valuations: 6 bases (primes 2,3,5 × digit count, lower-alpha count)
4. Negative pi sequence: 2 bases (variable-name ordinal signature)
5. Consistency: 6 bases (G=pass, T=fail)
Returns a dict with the DNA sequence and all intermediate values,
or None if the equation is empty.
The returned dict is the standard Φ encoding record consumed by
phi.output (FASTQ, Adleman graph, PCR protocol).
Examples:
>>> r = encode_phi("x + 1")
>>> r is not None
True
>>> r['length']
30
>>> all(c in 'ABCGPSTZ' for c in r['dna_sequence'])
True
>>> r['consistency_dna'] == r['dna_sequence'][-6:]
True
>>> r['schema']
'phi_embedding_v2'
>>> encode_phi("") is None
True
"""
if not equation or not equation.strip():
return None
F = compute_F(equation)
# Exact computations (no floats)
F_exact = _compute_F_exact(equation)
tau_exact = _compute_tau_exact(equation)
tau_8 = tau_exact[:8] if tau_exact else [Fraction(0)] * 8
consistency = check_consistency(equation)
tau = compute_tau(equation)
delta = compute_delta(equation)
# Also compute float versions for backward compatibility in the output dict
# (downstream consumers may reference F/tau/delta as floats for display)
F_float = compute_F(equation)
tau_float = compute_tau(equation)
delta_float = compute_delta(equation)
lambda_val, r = compute_lambda_and_r(equation)
# Fallback for unparseable equations: uniform distribution
# (encodes as all-A — "null structural signal")
if tau is None:
tau = [1.0 / len(NODE_TYPES)] * len(NODE_TYPES)
# Layer 1: F signs (8 bases) — asymmetric quantization
F_dna = "".join(_sign_to_base(F_exact[i], UNIFORM_F) for i in range(8))
# Encode each layer as exactly 8 hachimoji bases
F_dna = _vec_to_bases(F[:8])
tau_dna = _vec_to_bases(tau[:8] if tau else [0.5]*8)
delta_dna = _vec_to_bases((delta + [0.5]*8)[:8] if delta else [0.5]*8)
# Layer 2: tau signs (8 bases) — asymmetric quantization
tau_dna = "".join(_sign_to_base(tau_8[i], UNIFORM_TAU) for i in range(8))
# Layer 3: p-adic valuations of raw character counts (6 bases)
# 3 primes × 2 count positions (digit count, lower-alpha count)
counts = [0] * 8
for c in equation:
if c.isdigit():
counts[0] += 1
elif c.isalpha():
counts[1] += 1 if c.islower() else 2
elif c in "+-*/^%=<>!&|~":
counts[3] += 1
elif c in "()[]{}":
counts[4] += 1
elif c in " \t\n\r":
counts[6] += 1
else:
counts[7] += 1
padic_dna = ""
for i in [0, 1]: # digit count, lower-alpha count
for p in [2, 3, 5]:
val = _padic_valuation(counts[i], p)
padic_dna += _valuation_to_base(val, p)
# Layer 4: encode consistency G=pass T=fail
# Layer 3b: negative pi sequence (2 bases)
pi_pos, pi_neg = _negative_pi_sequence(equation)
pi_dna = pi_pos + pi_neg
# Layer 4: consistency G=pass T=fail (6 bases)
consistency_dna = "".join("G" if consistency[r] else "T" for r in RULE_ORDER)
full_sequence = F_dna + tau_dna + delta_dna + consistency_dna
# Combine: 8 + 8 + 6 + 2 + 6 = 30
full_sequence = F_dna + tau_dna + padic_dna + pi_dna + consistency_dna
quality_scores = "".join("A" if v else "P" for v in consistency.values())
seq_hash = hashlib.sha256(full_sequence.encode()).hexdigest()[:16]
# For backward compatibility, delta_dna is the p-adic + pi layers combined
delta_dna = padic_dna + pi_dna
return {
"equation": equation,
"dna_sequence": full_sequence,
"length": len(full_sequence),
"bases": list(HACHIMOJI_BASES),
"schema": "phi_embedding_v2",
"F": [round(x, 4) for x in F],
"tau": [round(x, 4) for x in tau],
"delta": [round(x, 4) for x in delta] if delta else None,
"schema": "phi_embedding_v3",
"F": [round(float(x), 4) for x in F_float],
"tau": [round(float(x), 4) for x in tau_float] if tau_float else None,
"delta": [round(float(x), 4) for x in delta_float] if delta_float else None,
"lambda": lambda_val,
"r": r,
"F_dna": F_dna,
@ -174,26 +343,11 @@ if __name__ == "__main__":
# Verification block — run with python -m phi.embed
# ──────────────────────────────────────────────────────────────────────
# --- _float_to_3bit ---------------------------------------------------
assert _float_to_3bit(0.0) == 0, f"_float_to_3bit(0.0) = {_float_to_3bit(0.0)}"
assert _float_to_3bit(1.0) == 7, f"_float_to_3bit(1.0) = {_float_to_3bit(1.0)}"
mid = _float_to_3bit(0.5)
assert mid in (3, 4), f"_float_to_3bit(0.5) = {mid}, expected 3 or 4"
# --- _vec_to_bases ----------------------------------------------------
vb = _vec_to_bases([0.0, 1.0, 0.5])
assert len(vb) == 3, f"Expected 3 bases, got {len(vb)}"
assert vb[0] == INDEX_TO_BASE[0], f"First base {vb[0]} != A"
assert vb[1] == INDEX_TO_BASE[7], f"Second base {vb[1]} != Z"
assert all(c in HACHIMOJI_BASES for c in vb), f"Invalid base in {vb}"
# --- encode_phi (simple equation) --------------------------------------
# --- Basic encoding ----------------------------------------------------
r = encode_phi("x + 1")
assert r is not None, "encode_phi('x + 1') returned None"
assert r["length"] == 30, f"length = {r['length']}, expected 30"
assert len(r["dna_sequence"]) == 30, \
f"dna_sequence len = {len(r['dna_sequence'])}, expected 30"
assert len(r["dna_sequence"]) == 30, f"dna_sequence len = {len(r['dna_sequence'])}, expected 30"
assert all(c in HACHIMOJI_BASES for c in r["dna_sequence"]), \
f"Unknown base in {r['dna_sequence']}"
@ -207,8 +361,8 @@ if __name__ == "__main__":
assert len(r["delta_dna"]) == 8, f"delta_dna len = {len(r['delta_dna'])}"
# Schema
assert r["schema"] == "phi_embedding_v2", \
f"schema = {r['schema']}, expected phi_embedding_v2"
assert r["schema"] == "phi_embedding_v3", \
f"schema = {r['schema']}, expected phi_embedding_v3"
# --- Empty string ------------------------------------------------------
assert encode_phi("") is None, "encode_phi('') should be None"
@ -223,4 +377,30 @@ if __name__ == "__main__":
assert r1["sha256_prefix"] == r2["sha256_prefix"], \
f"Determinism broken for hash: {r1['sha256_prefix']} != {r2['sha256_prefix']}"
print("embed: all verification assertions passed.")
# --- Injectivity (no two distinct equations produce same DNA) ----------
TEST_EQUATIONS = [
"x + 1", "E = mc^2", "a + b = c", "sin(x) + cos(y)",
"a/b = c/d", "x^2 + y^2 = z^2", "F = ma", "e^(i*pi) + 1 = 0",
"a*a*a*a*a*a*a*a", "x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x",
"(((x+y)))", "a-b-c-d-e-f-g-h", "integral(f, x, 0, 1) = F",
"dx/dt = -k*x", "p/q = r/s = t/u = v/w",
"a = b = c = d = e = f = g = h", "sum(x_i, i, 1, n) = S",
"x*y*z*a*b*c*d*e", "log(x) / log(y) = log_y(x)",
"a + b + c + d + e + f + g + h + i + j + k",
]
dna_map = {}
collisions = 0
for eq in TEST_EQUATIONS:
result = encode_phi(eq)
if result is None:
continue
dna = result["dna_sequence"]
if dna in dna_map:
collisions += 1
print(f" COLLISION: {repr(dna_map[dna])} vs {repr(eq)}")
else:
dna_map[dna] = eq
assert collisions == 0, f"Injectivity broken: {collisions} collisions out of {len(TEST_EQUATIONS)} equations"
print(f" Injectivity: {len(dna_map)}/{len(TEST_EQUATIONS)} unique (0 collisions)")
print("embed: all verification assertions passed (exact arithmetic, no floats).")