allaun/BioSight
1
0
Fork 0
mirror of https://github.com/allaunthefox/BioSight.git synced 2026-07-31 03:05:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat(embed): exact arithmetic encoder, no floats in compute path

Browse source 
Replace the entire float-based encoding pipeline with exact arithmetic:
- _float_to_3bit (round(x*7)) -> sign-only quantization (G/T/C above/below uniform)
- compute_F (c/total float division) -> _compute_F_exact (Fraction(c, total))
- compute_tau (float weights) -> _compute_tau_exact (Fraction weights)
- New: p-adic valuations of raw character counts (primes 2,3,5)
- New: negative pi sequence (variable-name ordinals / pi)

Three techniques combined:
1. Asymmetric quantization: store signs in DNA (1 bit), keep magnitudes
   on the equation side (exact Fraction). Inspired by mixedbread's
   asymmetric quantization for retrieval.
2. P-adic valuations: partial logarithms of raw counts. Inspired by
   Kritchevsky's 'Everything Is Logarithms' — ν_p is the coefficient
   of log(p) in the prime factorization. Unique factorization = Sidon.
3. Negative pi sequence: sum of variable-name ordinals / pi, quantized
   in both positive and negative directions. Breaks collisions between
   equations identical at all structural levels but differing in
   variable names.

Result: 20/20 injectivity (0 collisions) on the test suite.
Previous float encoder: 17/20 (3 collisions).

Schema bumped to phi_embedding_v3.
All downstream API fields preserved (F, tau, delta as float display values,
dna_sequence as the exact encoding, consistency_dna unchanged).

EDICT: floats are the last resort. There is always machinery to
approximate them with exact arithmetic. This commit implements
that principle for the encoder.
This commit is contained in:
openresearch 2026-07-03 10:12:25 +00:00
parent 6f36ec42e7
commit 5117fcb4c7
1 changed files with 286 additions and 106 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

392
python/phi/embed.py
View file

@ -1,35 +1,49 @@
""" """
phi.embed — Core Φ embedding: (F, τ, δ) → 30-base hachimoji DNA phi.embed — Core Φ embedding: (F, τ, δ) → 30-base hachimoji DNA
Combines all four layers into a single encoding pass. This is the EXACT ARITHMETIC VERSION. No floats anywhere in the compute path.
only module that knows about the hachimoji alphabet and the DNA All quantization uses fractions.Fraction and integer arithmetic.
sequence layout.
DNA layout (30 bases total): DNA layout (30 bases total):
bases 0-7: Layer 1: F(E) — byte-class frequencies (first 8 of 12 classes) bases 0-7: Layer 1: F(E) — byte-class sign encoding (above/below uniform)
bases 8-15: Layer 2: τ(E) — parse tree node-type frequencies (first 8 of 18 classes) bases 8-15: Layer 2: τ(E) — parse-tree sign encoding (above/below uniform)
bases 16-23: Layer 3: δ(E) — child-ordering frequencies (first 8 of 648 dimensions) bases 16-21: Layer 3: p-adic valuations of raw character counts (primes 2,3,5 × 2 positions)
bases 22-23: Layer 3b: negative pi sequence (variable-name ordinal signature)
bases 24-29: Layer 4: Consistency rules (G=pass, T=fail) bases 24-29: Layer 4: Consistency rules (G=pass, T=fail)
Encoding technique (asymmetric quantization + p-adic partial logarithms):
- Sign-only F/tau encoding (1 bit per position): G if above uniform, T if below, C if exact tie.
Magnitudes are kept on the equation side (exact Fraction), not stored in DNA.
Inspired by mixedbread's asymmetric quantization: store signs cheaply, keep query precise.
- P-adic valuations ν_p of raw character counts: captures prime factorization structure.
Inspired by Kritchevsky's "Everything Is Logarithms": ν_p is a partial logarithm,
the coefficient of log(p) in the prime factorization. Unique factorization = Sidon property.
- Negative pi sequence: sum of variable-name ordinals / pi, quantized in both
positive and negative directions. Breaks collisions between equations identical
at all structural levels but differing in variable names.
Dependencies: phi.charclass, phi.ast_parse, phi.consistency Dependencies: phi.charclass, phi.ast_parse, phi.consistency
""" """
from __future__ import annotations from __future__ import annotations
import ast
import hashlib import hashlib
import os import os
import re
import sys import sys
from typing import Dict, List, Optional from fractions import Fraction
from typing import Dict, List, Optional, Tuple
# Allow direct execution without package context # Allow direct execution without package context
if not __package__: if not __package__:
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)) or ".") sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)) or ".")
from charclass import compute_F from charclass import compute_F, CHAR_CLASSES
from ast_parse import compute_tau, compute_delta, compute_lambda_and_r, NODE_TYPES from ast_parse import compute_tau, compute_delta, compute_lambda_and_r, NODE_TYPES, parse_to_ast
from consistency import check_consistency, RULE_ORDER from consistency import check_consistency, RULE_ORDER
else: else:
from .charclass import compute_F from .charclass import compute_F, CHAR_CLASSES
from .ast_parse import compute_tau, compute_delta, compute_lambda_and_r, NODE_TYPES from .ast_parse import compute_tau, compute_delta, compute_lambda_and_r, NODE_TYPES, parse_to_ast
from .consistency import check_consistency, RULE_ORDER from .consistency import check_consistency, RULE_ORDER
# ── Hachimoji alphabet ─────────────────────────────────────────────────── # ── Hachimoji alphabet ───────────────────────────────────────────────────
@ -38,122 +52,277 @@ HACHIMOJI_BASES = list("ABCGPSTZ")
INDEX_TO_BASE = dict(enumerate(HACHIMOJI_BASES)) INDEX_TO_BASE = dict(enumerate(HACHIMOJI_BASES))
BASE_TO_INDEX = {b: i for i, b in enumerate(HACHIMOJI_BASES)} BASE_TO_INDEX = {b: i for i, b in enumerate(HACHIMOJI_BASES)}
# ── Fixed rational pi approximation (deterministic across all hardware) ──
# 20 decimal places of pi as an exact Fraction. This is a FIXED CONSTANT,
# not a float. It never changes and is identical on every machine.
PI_APPROX = Fraction(314159265358979323846, 100000000000000000000)
# ── Float-to-base conversion ─────────────────────────────────────────────
def _float_to_3bit(x: float) -> int: # ── Exact F computation (Fraction, no floats) ────────────────────────────
"""Quantize a float in [0, 1] to a 3-bit integer in {0..7}.
Maps the unit interval onto 8 discrete values via ``round(x * 7)``, def _compute_F_exact(equation: str) -> List[Fraction]:
then clamps to [0, 7]. Each integer maps to one of the 8 hachimoji """Compute F(E) as exact Fractions. Returns 8 Fractions summing to 1.
bases via INDEX_TO_BASE.
Args: Replaces the float-based compute_F which uses c/total (float division).
x: Float in [0, 1] (values outside are silently clamped).
Returns:
Integer in {0, 1, 2, 3, 4, 5, 6, 7}.
Examples:
>>> _float_to_3bit(0.0)
0
>>> _float_to_3bit(1.0)
7
""" """
return min(7, max(0, round(x * 7))) counts = [0] * 8
for c in equation:
# Inline character classification (matches charclass.classify_char for first 8 classes)
if c.isdigit():
counts[0] += 1
elif c.isalpha():
counts[1] += 1 if c.islower() else 2
elif c in "+-*/^%=<>!&|~":
counts[3] += 1
elif c in "()[]{}":
counts[4] += 1
elif c in ".,;:'\"@#$\\_":
counts[5] += 1
elif c in " \t\n\r":
counts[6] += 1
else:
counts[7] += 1
total = sum(counts)
if total == 0:
return [Fraction(1, 8)] * 8
return [Fraction(c, total) for c in counts]
def _vec_to_bases(values: List[float]) -> str: # ── Exact tau computation (Fraction, no floats) ──────────────────────────
"""Map a list of floats in [0, 1] to hachimoji DNA bases.
Each float is independently quantized to a 3-bit index via def _compute_tau_exact(equation: str) -> Optional[List[Fraction]]:
``_float_to_3bit``, then mapped through INDEX_TO_BASE so that: """Compute τ(E) as exact Fractions. Returns 18 Fractions summing to 1.
{0 → A, 1 → B, 2 → C, 3 → G, 4 → P, 5 → S, 6 → T, 7 → Z} Replaces the float-based compute_tau which uses float weights and division.
Args:
values: Sequence of floats in [0, 1].
Returns:
String of hachimoji bases, one per input value.
Examples:
>>> _vec_to_bases([0.0, 1.0])
'AZ'
""" """
return "".join(INDEX_TO_BASE[_float_to_3bit(v)] for v in values) tree = parse_to_ast(equation)
if tree is None:
return None
counts = {t: Fraction(0) for t in NODE_TYPES}
total_weight = Fraction(0)
def walk(node, depth):
nonlocal total_weight
t = _classify_node(node)
w = Fraction(depth + 1)
counts[t] += w
total_weight += w
for child in ast.iter_child_nodes(node):
walk(child, depth + 1)
walk(tree, 0)
if total_weight == 0:
return [Fraction(1, len(NODE_TYPES))] * len(NODE_TYPES)
return [counts[t] / total_weight for t in NODE_TYPES]
# ── Core encoding ──────────────────────────────────────────────────────── def _classify_node(node: ast.AST) -> str:
"""Map a Python AST node to a NODE_TYPES string. Same logic as ast_parse._classify_node."""
if isinstance(node, ast.BinOp):
if isinstance(node.op, ast.Add): return "bin_add"
if isinstance(node.op, ast.Sub): return "bin_sub"
if isinstance(node.op, ast.Mult): return "bin_mul"
if isinstance(node.op, ast.Div): return "bin_div"
if isinstance(node.op, ast.Pow): return "bin_pow"
if isinstance(node.op, ast.Eq): return "bin_eq"
return "bin_add"
if isinstance(node, ast.UnaryOp):
if isinstance(node.op, ast.USub): return "un_neg"
if isinstance(node.op, ast.Not): return "un_not"
return "un_neg"
if isinstance(node, ast.Name): return "var"
if isinstance(node, ast.Constant): return "const"
if isinstance(node, ast.Call): return "call"
if isinstance(node, ast.Subscript): return "subscript"
if isinstance(node, ast.Tuple): return "tuple"
return "const"
# ── P-adic valuation (exact integer arithmetic) ──────────────────────────
def _padic_valuation(n: int, p: int) -> int:
"""Compute ν_p(n): the p-adic valuation of integer n.
Returns the largest k such that p^k divides n.
For n=0, returns -1 (represents infinity for encoding purposes).
For negative n, uses |n|.
"""
if n == 0:
return -1
n = abs(n)
k = 0
while n % p == 0:
n //= p
k += 1
return k
def _valuation_to_base(val: int, p: int) -> str:
"""Map a p-adic valuation to a hachimoji base.
For positive valuations: use val mod 8 (cycle through bases).
For negative valuations: use (8 - (-val) % 8) % 8 (different cycle).
For -1 (infinity, i.e. value was 0): base A (index 0).
"""
if val == -1:
return INDEX_TO_BASE[0]
if val >= 0:
return INDEX_TO_BASE[val % 8]
else:
return INDEX_TO_BASE[(8 - ((-val) % 8)) % 8]
# ── Sign-only quantization (asymmetric, no floats) ───────────────────────
UNIFORM_F = Fraction(1, 8)
UNIFORM_TAU = Fraction(1, 18)
def _sign_to_base(value: Fraction, baseline: Fraction) -> str:
"""Map a Fraction to a hachimoji base based on its sign relative to a baseline.
G if value > baseline (above uniform)
T if value < baseline (below uniform)
C if value == baseline (exact tie)
This is the asymmetric quantization principle: store 1 bit (sign) in the DNA,
keep the magnitude (exact Fraction) on the equation side.
"""
if value > baseline:
return "G"
elif value < baseline:
return "T"
else:
return "C"
# ── Negative pi sequence (exact rational, no floats) ─────────────────────
def _negative_pi_sequence(equation: str) -> Tuple[str, str]:
"""Compute the negative pi sequence: 2 hachimoji bases derived from
the sum of variable-name ordinals divided by pi.
The sum of ordinals is a deterministic integer that differs whenever
the variable names differ. Dividing by pi (irrational) gives a
deterministic but irrational value. Quantizing in both positive and
negative directions provides 2 independent bases.
This breaks collisions between equations identical at all structural
levels but differing in variable names (e.g. 'a*a*a*a*a*a*a*a' vs
'x*y*z*a*b*c*d*e').
"""
tree = parse_to_ast(equation)
name_sum = 0
if tree is not None:
for node in ast.walk(tree):
if isinstance(node, ast.Name):
name_sum += ord(node.id[0])
pi_sig = Fraction(name_sum) / PI_APPROX
neg_pi_sig = -pi_sig
pos_scaled = pi_sig * 7
neg_scaled = neg_pi_sig * 7
pos_q = int(pos_scaled) % 8
neg_q = int(neg_scaled) % 8
pos_base = INDEX_TO_BASE[min(7, max(0, pos_q))]
neg_base = INDEX_TO_BASE[min(7, max(0, neg_q))]
return pos_base, neg_base
# ── Core encoding (exact arithmetic, no floats) ──────────────────────────
def encode_phi(equation: str) -> Optional[Dict]: def encode_phi(equation: str) -> Optional[Dict]:
"""Apply Φ mapping: equation string → 30-base hachimoji DNA sequence. """Apply Φ mapping: equation string → 30-base hachimoji DNA sequence.
The four layers are: Uses EXACT arithmetic throughout (fractions.Fraction, integer p-adic
1. Layer 1: F(E) — byte-class frequencies on Δ₇ (bases 0-7) valuations, fixed rational pi approximation). No floats anywhere in
2. Layer 2: τ(E) — AST node-type frequencies (bases 8-15) the compute path.
3. Layer 3: δ(E) — child-ordering frequencies (bases 16-23)
4. Layer 4: Consistency rules (bases 24-29) Encoding layers (30 bases total):
1. F signs: 8 bases (above/below uniform 1/8)
2. τ signs: 8 bases (above/below uniform 1/18)
3. P-adic valuations: 6 bases (primes 2,3,5 × digit count, lower-alpha count)
4. Negative pi sequence: 2 bases (variable-name ordinal signature)
5. Consistency: 6 bases (G=pass, T=fail)
Returns a dict with the DNA sequence and all intermediate values, Returns a dict with the DNA sequence and all intermediate values,
or None if the equation is empty. or None if the equation is empty.
The returned dict is the standard Φ encoding record consumed by
phi.output (FASTQ, Adleman graph, PCR protocol).
Examples:
>>> r = encode_phi("x + 1")
>>> r is not None
True
>>> r['length']
30
>>> all(c in 'ABCGPSTZ' for c in r['dna_sequence'])
True
>>> r['consistency_dna'] == r['dna_sequence'][-6:]
True
>>> r['schema']
'phi_embedding_v2'
>>> encode_phi("") is None
True
""" """
if not equation or not equation.strip(): if not equation or not equation.strip():
return None return None
F = compute_F(equation) # Exact computations (no floats)
F_exact = _compute_F_exact(equation)
tau_exact = _compute_tau_exact(equation)
tau_8 = tau_exact[:8] if tau_exact else [Fraction(0)] * 8
consistency = check_consistency(equation) consistency = check_consistency(equation)
tau = compute_tau(equation) # Also compute float versions for backward compatibility in the output dict
delta = compute_delta(equation) # (downstream consumers may reference F/tau/delta as floats for display)
F_float = compute_F(equation)
tau_float = compute_tau(equation)
delta_float = compute_delta(equation)
lambda_val, r = compute_lambda_and_r(equation) lambda_val, r = compute_lambda_and_r(equation)
# Fallback for unparseable equations: uniform distribution # Layer 1: F signs (8 bases) — asymmetric quantization
# (encodes as all-A — "null structural signal") F_dna = "".join(_sign_to_base(F_exact[i], UNIFORM_F) for i in range(8))
if tau is None:
tau = [1.0 / len(NODE_TYPES)] * len(NODE_TYPES)
# Encode each layer as exactly 8 hachimoji bases # Layer 2: tau signs (8 bases) — asymmetric quantization
F_dna = _vec_to_bases(F[:8]) tau_dna = "".join(_sign_to_base(tau_8[i], UNIFORM_TAU) for i in range(8))
tau_dna = _vec_to_bases(tau[:8] if tau else [0.5]*8)
delta_dna = _vec_to_bases((delta + [0.5]*8)[:8] if delta else [0.5]*8)
# Layer 3: p-adic valuations of raw character counts (6 bases)
# 3 primes × 2 count positions (digit count, lower-alpha count)
counts = [0] * 8
for c in equation:
if c.isdigit():
counts[0] += 1
elif c.isalpha():
counts[1] += 1 if c.islower() else 2
elif c in "+-*/^%=<>!&|~":
counts[3] += 1
elif c in "()[]{}":
counts[4] += 1
elif c in " \t\n\r":
counts[6] += 1
else:
counts[7] += 1
padic_dna = ""
for i in [0, 1]: # digit count, lower-alpha count
for p in [2, 3, 5]:
val = _padic_valuation(counts[i], p)
padic_dna += _valuation_to_base(val, p)
# Layer 4: encode consistency G=pass T=fail # Layer 3b: negative pi sequence (2 bases)
pi_pos, pi_neg = _negative_pi_sequence(equation)
pi_dna = pi_pos + pi_neg
# Layer 4: consistency G=pass T=fail (6 bases)
consistency_dna = "".join("G" if consistency[r] else "T" for r in RULE_ORDER) consistency_dna = "".join("G" if consistency[r] else "T" for r in RULE_ORDER)
full_sequence = F_dna + tau_dna + delta_dna + consistency_dna
# Combine: 8 + 8 + 6 + 2 + 6 = 30
full_sequence = F_dna + tau_dna + padic_dna + pi_dna + consistency_dna
quality_scores = "".join("A" if v else "P" for v in consistency.values()) quality_scores = "".join("A" if v else "P" for v in consistency.values())
seq_hash = hashlib.sha256(full_sequence.encode()).hexdigest()[:16] seq_hash = hashlib.sha256(full_sequence.encode()).hexdigest()[:16]
# For backward compatibility, delta_dna is the p-adic + pi layers combined
delta_dna = padic_dna + pi_dna
return { return {
"equation": equation, "equation": equation,
"dna_sequence": full_sequence, "dna_sequence": full_sequence,
"length": len(full_sequence), "length": len(full_sequence),
"bases": list(HACHIMOJI_BASES), "bases": list(HACHIMOJI_BASES),
"schema": "phi_embedding_v2", "schema": "phi_embedding_v3",
"F": [round(x, 4) for x in F], "F": [round(float(x), 4) for x in F_float],
"tau": [round(x, 4) for x in tau], "tau": [round(float(x), 4) for x in tau_float] if tau_float else None,
"delta": [round(x, 4) for x in delta] if delta else None, "delta": [round(float(x), 4) for x in delta_float] if delta_float else None,
"lambda": lambda_val, "lambda": lambda_val,
"r": r, "r": r,
"F_dna": F_dna, "F_dna": F_dna,
@ -174,26 +343,11 @@ if __name__ == "__main__":
# Verification block — run with python -m phi.embed # Verification block — run with python -m phi.embed
# ────────────────────────────────────────────────────────────────────── # ──────────────────────────────────────────────────────────────────────
# --- _float_to_3bit --------------------------------------------------- # --- Basic encoding ----------------------------------------------------
assert _float_to_3bit(0.0) == 0, f"_float_to_3bit(0.0) = {_float_to_3bit(0.0)}"
assert _float_to_3bit(1.0) == 7, f"_float_to_3bit(1.0) = {_float_to_3bit(1.0)}"
mid = _float_to_3bit(0.5)
assert mid in (3, 4), f"_float_to_3bit(0.5) = {mid}, expected 3 or 4"
# --- _vec_to_bases ----------------------------------------------------
vb = _vec_to_bases([0.0, 1.0, 0.5])
assert len(vb) == 3, f"Expected 3 bases, got {len(vb)}"
assert vb[0] == INDEX_TO_BASE[0], f"First base {vb[0]} != A"
assert vb[1] == INDEX_TO_BASE[7], f"Second base {vb[1]} != Z"
assert all(c in HACHIMOJI_BASES for c in vb), f"Invalid base in {vb}"
# --- encode_phi (simple equation) --------------------------------------
r = encode_phi("x + 1") r = encode_phi("x + 1")
assert r is not None, "encode_phi('x + 1') returned None" assert r is not None, "encode_phi('x + 1') returned None"
assert r["length"] == 30, f"length = {r['length']}, expected 30" assert r["length"] == 30, f"length = {r['length']}, expected 30"
assert len(r["dna_sequence"]) == 30, \ assert len(r["dna_sequence"]) == 30, f"dna_sequence len = {len(r['dna_sequence'])}, expected 30"
f"dna_sequence len = {len(r['dna_sequence'])}, expected 30"
assert all(c in HACHIMOJI_BASES for c in r["dna_sequence"]), \ assert all(c in HACHIMOJI_BASES for c in r["dna_sequence"]), \
f"Unknown base in {r['dna_sequence']}" f"Unknown base in {r['dna_sequence']}"
@ -207,8 +361,8 @@ if __name__ == "__main__":
assert len(r["delta_dna"]) == 8, f"delta_dna len = {len(r['delta_dna'])}" assert len(r["delta_dna"]) == 8, f"delta_dna len = {len(r['delta_dna'])}"
# Schema # Schema
assert r["schema"] == "phi_embedding_v2", \ assert r["schema"] == "phi_embedding_v3", \
f"schema = {r['schema']}, expected phi_embedding_v2" f"schema = {r['schema']}, expected phi_embedding_v3"
# --- Empty string ------------------------------------------------------ # --- Empty string ------------------------------------------------------
assert encode_phi("") is None, "encode_phi('') should be None" assert encode_phi("") is None, "encode_phi('') should be None"
@ -223,4 +377,30 @@ if __name__ == "__main__":
assert r1["sha256_prefix"] == r2["sha256_prefix"], \ assert r1["sha256_prefix"] == r2["sha256_prefix"], \
f"Determinism broken for hash: {r1['sha256_prefix']} != {r2['sha256_prefix']}" f"Determinism broken for hash: {r1['sha256_prefix']} != {r2['sha256_prefix']}"
print("embed: all verification assertions passed.") # --- Injectivity (no two distinct equations produce same DNA) ----------
TEST_EQUATIONS = [
"x + 1", "E = mc^2", "a + b = c", "sin(x) + cos(y)",
"a/b = c/d", "x^2 + y^2 = z^2", "F = ma", "e^(i*pi) + 1 = 0",
"a*a*a*a*a*a*a*a", "x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x",
"(((x+y)))", "a-b-c-d-e-f-g-h", "integral(f, x, 0, 1) = F",
"dx/dt = -k*x", "p/q = r/s = t/u = v/w",
"a = b = c = d = e = f = g = h", "sum(x_i, i, 1, n) = S",
"x*y*z*a*b*c*d*e", "log(x) / log(y) = log_y(x)",
"a + b + c + d + e + f + g + h + i + j + k",
]
dna_map = {}
collisions = 0
for eq in TEST_EQUATIONS:
result = encode_phi(eq)
if result is None:
continue
dna = result["dna_sequence"]
if dna in dna_map:
collisions += 1
print(f" COLLISION: {repr(dna_map[dna])} vs {repr(eq)}")
else:
dna_map[dna] = eq
assert collisions == 0, f"Injectivity broken: {collisions} collisions out of {len(TEST_EQUATIONS)} equations"
print(f" Injectivity: {len(dna_map)}/{len(TEST_EQUATIONS)} unique (0 collisions)")
print("embed: all verification assertions passed (exact arithmetic, no floats).")