Stage stack solidification source slice

This commit is contained in:
Brandon Schneider 2026-05-11 22:08:10 -05:00
parent cabf709253
commit 29f9b78b6d
70 changed files with 14378 additions and 0 deletions

View file

@ -0,0 +1,38 @@
# AGENTS.md - Lean/Semantics
Scope: `0-Core-Formalism/lean/Semantics/`
The strict operating rules live in `../../../6-Documentation/docs/AGENTS.md`.
Follow those rules for all Lean, proof, fixed-point, hardware-extraction, and
shim-boundary work.
## Local Rules
- Keep module names aligned with file names and namespaces.
- Prefer small domain modules over utility files.
- Every new computational gate needs an executable witness: theorem, `#eval`,
or native-decision proof.
- Run the narrow build target first, for example:
```bash
lake build Semantics.BeaverMaskFreshness
```
- Run the broader build before claiming a stable Lean surface:
```bash
lake build
```
- Do not delete difficult theorems to make builds pass. Fix proofs or quarantine
with an explicit `TODO(lean-port): ...` boundary.
- Treat generated Python, Rust, Verilog, and JSON as shims or receipts, not as
the formal source of truth.
## Current Stack-Solidification Anchors
- `Semantics.BeaverMaskFreshness` is a finite admission gate for Beaver-mask
freshness negative controls.
- Stack status receipts live under `shared-data/data/stack_solidification/`.
- The current staged slice is documented in
`../../../6-Documentation/docs/stack_solidification_staging_manifest_2026-05-09.md`.

View file

@ -0,0 +1,94 @@
namespace Semantics.BeaverMaskFreshness
/-- Source class for a Beaver-style mask coefficient. Only `freshRandom`
is admissible for privacy-equivalent Beaver masking in this gate model. -/
inductive MaskSource where
| freshRandom
| reused
| topologyDerived
| adversarialChosen
deriving Repr, DecidableEq, BEq
/-- A finite audit event for one mask coefficient. -/
structure MaskEvent where
epoch : Nat
party : Nat
maskId : Nat
source : MaskSource
deriving Repr, DecidableEq, BEq
def sourceFreshIndependent (source : MaskSource) : Bool :=
source == MaskSource.freshRandom
def eventFreshIndependent (event : MaskEvent) : Bool :=
sourceFreshIndependent event.source
def maskIdUsedBefore (maskId : Nat) : List MaskEvent → Bool
| [] => false
| event :: rest => (event.maskId == maskId) || maskIdUsedBefore maskId rest
/-- Admission gate for treating a coefficient as a privacy-equivalent mask.
It must be fresh/independent and its mask id must not already occur in the
receipt history. -/
def admissibleMaskEvent (history : List MaskEvent) (event : MaskEvent) : Bool :=
eventFreshIndependent event && !maskIdUsedBefore event.maskId history
def admitMaskEvent (history : List MaskEvent) (event : MaskEvent) : Option (List MaskEvent) :=
if admissibleMaskEvent history event then
some (event :: history)
else
none
def freshA : MaskEvent :=
{ epoch := 0, party := 0, maskId := 1001, source := MaskSource.freshRandom }
def freshB : MaskEvent :=
{ epoch := 0, party := 1, maskId := 1002, source := MaskSource.freshRandom }
def reusedA : MaskEvent :=
{ epoch := 1, party := 0, maskId := 1001, source := MaskSource.reused }
def topologyA : MaskEvent :=
{ epoch := 1, party := 0, maskId := 2001, source := MaskSource.topologyDerived }
def adversarialA : MaskEvent :=
{ epoch := 1, party := 0, maskId := 3001, source := MaskSource.adversarialChosen }
/-- Positive control: a fresh random mask with an unused id admits. -/
theorem freshUnusedAdmits :
admissibleMaskEvent [] freshA = true := by
native_decide
/-- Positive control: distinct fresh random mask ids may both admit. -/
theorem distinctFreshSequenceAdmits :
admitMaskEvent [freshA] freshB = some [freshB, freshA] := by
native_decide
/-- Negative control: an explicit reused-source coefficient is rejected. -/
theorem reusedSourceRejected :
admissibleMaskEvent [freshA] reusedA = false := by
native_decide
/-- Negative control: even if source were mislabeled fresh, mask-id reuse is rejected. -/
theorem reusedMaskIdRejected :
admissibleMaskEvent [freshA] { reusedA with source := MaskSource.freshRandom } = false := by
native_decide
/-- Negative control: topology-derived adaptive coefficients are not treated as
privacy-equivalent random masks by this gate. -/
theorem topologyDerivedRejected :
admissibleMaskEvent [] topologyA = false := by
native_decide
/-- Negative control: adversarially chosen coefficients are rejected. -/
theorem adversarialChosenRejected :
admissibleMaskEvent [] adversarialA = false := by
native_decide
#eval admissibleMaskEvent [] freshA
#eval admissibleMaskEvent [freshA] reusedA
#eval admissibleMaskEvent [freshA] { reusedA with source := MaskSource.freshRandom }
#eval admissibleMaskEvent [] topologyA
#eval admissibleMaskEvent [] adversarialA
end Semantics.BeaverMaskFreshness

View file

@ -0,0 +1,78 @@
namespace Semantics.WhitespaceFreeGrammar
/-!
Whitespace-free grammar gate.
The intended compression rule is narrow:
* symbol payloads are stored;
* ordinary inter-symbol whitespace is derived from symbol count/order;
* no whitespace symbol is admitted into the payload stream;
* non-canonical spacing needs a residual and therefore remains outside this
zero-whitespace gate.
-/
/-- A finite grammar atom represented by its payload byte count. -/
structure GrammarAtom where
payloadBytes : Nat
deriving Repr, DecidableEq, BEq
/-- Payload cost counts symbols only. Whitespace is not a symbol. -/
def payloadBytes : List GrammarAtom -> Nat
| [] => 0
| atom :: rest => atom.payloadBytes + payloadBytes rest
/-- Number of derivable inter-symbol boundaries. -/
def derivedBoundaryCount : List GrammarAtom -> Nat
| [] => 0
| [_] => 0
| _ :: rest => 1 + derivedBoundaryCount rest
/-- Stored whitespace codes are always zero in this gate. -/
def storedWhitespaceCodes (_atoms : List GrammarAtom) : Nat := 0
/-- Stored cost under the zero-whitespace grammar. -/
def storedBytes (atoms : List GrammarAtom) : Nat :=
payloadBytes atoms + storedWhitespaceCodes atoms
/-- Decoded display bytes if every derived boundary replays as one ASCII space.
This is a replay cost, not a stored cost. -/
def canonicalDisplayBytes (atoms : List GrammarAtom) : Nat :=
payloadBytes atoms + derivedBoundaryCount atoms
def atomA : GrammarAtom := { payloadBytes := 5 }
def atomB : GrammarAtom := { payloadBytes := 4 }
def atomC : GrammarAtom := { payloadBytes := 7 }
def exampleAtoms : List GrammarAtom := [atomA, atomB, atomC]
theorem exampleStoredWhitespaceZero :
storedWhitespaceCodes exampleAtoms = 0 := by
native_decide
theorem exampleStoredCostDropsDerivedSpaces :
storedBytes exampleAtoms = payloadBytes exampleAtoms := by
native_decide
theorem exampleDerivedBoundaryCount :
derivedBoundaryCount exampleAtoms = 2 := by
native_decide
theorem exampleCanonicalDisplayCost :
canonicalDisplayBytes exampleAtoms = storedBytes exampleAtoms + 2 := by
native_decide
theorem emptyHasNoWhitespaceCodes :
storedWhitespaceCodes [] = 0 ∧ derivedBoundaryCount [] = 0 := by
native_decide
theorem singletonHasNoDerivedBoundary :
storedWhitespaceCodes [atomA] = 0 ∧ derivedBoundaryCount [atomA] = 0 := by
native_decide
#eval storedWhitespaceCodes exampleAtoms
#eval payloadBytes exampleAtoms
#eval storedBytes exampleAtoms
#eval derivedBoundaryCount exampleAtoms
#eval canonicalDisplayBytes exampleAtoms
end Semantics.WhitespaceFreeGrammar

View file

@ -0,0 +1,40 @@
# AGENTS.md - Infrastructure And Hardware
Scope: `4-Infrastructure/`
## Rules
- Keep infrastructure scripts receipt-bearing: every probe should have a
machine-readable output or update an existing receipt.
- Separate software witnesses from live hardware witnesses.
- Do not claim FPGA acceleration from bitstream generation alone.
- Do not claim UART/fabric success without observed bytes or a matching hardware
receipt.
- Treat `/usr/bin/sem` as GNU Parallel on this machine unless proven otherwise;
use the isolated `sem` binary documented in stack solidification receipts when
needed.
## Preferred Checks
```bash
python3 -m py_compile 4-Infrastructure/shim/<script>.py
python3 -m json.tool <receipt>.json >/dev/null
```
For Tang Nano 9K work, keep the boundaries explicit:
- bitstream present
- SRAM load
- flash persistence
- UART beacon
- Q16/software witness
- Q16/live hardware witness
## Current Stack-Solidification Anchors
- `4-Infrastructure/shim/stack_solidification_audit.py`
- `4-Infrastructure/shim/stack_fail_closure_register.py`
- `4-Infrastructure/shim/beaver_mask_freshness_negative_controls.py`
- `4-Infrastructure/shim/tang9k_uart_beacon_probe.py`
- `4-Infrastructure/shim/hutter_jxl_starfield_eigenprobe.py`
- `4-Infrastructure/shim/hutter_jxl_starfield_replay_verify.py`

View file

@ -0,0 +1,44 @@
#!/usr/bin/env bash
set -euo pipefail
TOP="tangnano9k_uart_beacon"
DEVICE="GW1NR-LV9QN88PC6/I5"
FAMILY="GW1N-9C"
FREQ_MHZ="27"
CST="${CST:-tangnano9k_uart_loopback.cst}"
JSON="${TOP}.json"
PNR="${TOP}_pnr.json"
FS="${TOP}.fs"
CHIPDB="${CHIPDB:-/usr/share/nextpnr/himbaechel/gowin/chipdb-${FAMILY}.bin}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_NEXTPNR="${SCRIPT_DIR}/../bin/nextpnr-himbaechel"
cd "${SCRIPT_DIR}"
echo "=== Tang Nano 9K UART Beacon Build ==="
yosys -p "read_verilog ${TOP}.v uart_tx.v; synth_gowin -top ${TOP} -json ${JSON}; stat"
if [ -x "${REPO_NEXTPNR}" ]; then
PNR_CMD="${REPO_NEXTPNR}"
else
PNR_CMD="nextpnr-himbaechel"
fi
PNR_ARGS=(
--device "${DEVICE}"
--json "${JSON}"
--write "${PNR}"
--freq "${FREQ_MHZ}"
--vopt "family=${FAMILY}"
--vopt "cst=${CST}"
)
if [ -f "${CHIPDB}" ]; then
PNR_ARGS+=(--chipdb "${CHIPDB}")
fi
"${PNR_CMD}" "${PNR_ARGS[@]}"
gowin_pack -d "${FAMILY}" -o "${FS}" "${PNR}"
sha256sum "${FS}"
echo "=== Build complete: ${SCRIPT_DIR}/${FS} ==="

View file

@ -0,0 +1,84 @@
// Tang Nano 9K UART TX beacon.
//
// This isolates the fabric TX pin from RX framing and host protocol logic. It
// emits a repeating receipt-like byte pattern at 115200 baud:
// A6 42 51 31 36 0A
// which reads as a board-level "Q16" beacon after the magic/version bytes.
`timescale 1ns / 1ps
module tangnano9k_uart_beacon (
input clk,
input rst_n,
input uart_rx_pin,
output uart_tx_pin,
output [5:0] led
);
localparam GAP_CYCLES = 24'd2700000; // about 100 ms at 27 MHz
wire tx_busy;
reg tx_start;
reg [7:0] tx_data;
reg [23:0] gap_counter;
reg [2:0] byte_index;
reg [5:0] sent_count;
reg [7:0] reset_counter = 8'd0;
wire internal_rst_n = &reset_counter;
uart_tx tx_inst (
.clk(clk),
.rst_n(internal_rst_n),
.tx_start(tx_start),
.tx_data(tx_data),
.uart_tx(uart_tx_pin),
.tx_busy(tx_busy)
);
function [7:0] beacon_byte;
input [2:0] idx;
begin
case (idx)
3'd0: beacon_byte = 8'hA6;
3'd1: beacon_byte = 8'h42;
3'd2: beacon_byte = 8'h51;
3'd3: beacon_byte = 8'h31;
3'd4: beacon_byte = 8'h36;
default: beacon_byte = 8'h0A;
endcase
end
endfunction
always @(posedge clk) begin
if (!internal_rst_n) begin
reset_counter <= reset_counter + 8'd1;
tx_start <= 1'b0;
tx_data <= 8'h00;
gap_counter <= 24'd0;
byte_index <= 3'd0;
sent_count <= 6'd0;
end else begin
tx_start <= 1'b0;
if (gap_counter < GAP_CYCLES) begin
gap_counter <= gap_counter + 24'd1;
end else if (!tx_busy) begin
tx_data <= beacon_byte(byte_index);
tx_start <= 1'b1;
sent_count <= sent_count + 6'd1;
if (byte_index == 3'd5) begin
byte_index <= 3'd0;
gap_counter <= 24'd0;
end else begin
byte_index <= byte_index + 3'd1;
end
end
end
end
assign led = ~sent_count;
// Keep RX referenced so the shared UART CST can be reused unchanged.
wire unused_rx = uart_rx_pin;
wire unused_rst = rst_n;
endmodule

View file

@ -0,0 +1,27 @@
// Tang Nano 9K UART diagnostic constraints with TX/RX swapped.
// Use only for transport isolation. Standard board examples commonly use
// uart_tx on 17 and uart_rx on 18; this file tests the opposite direction.
IO_LOC "clk" 52;
IO_PORT "clk" IO_TYPE=LVCMOS33 PULL_MODE=NONE;
IO_LOC "rst_n" 4;
IO_PORT "rst_n" IO_TYPE=LVCMOS33 PULL_MODE=UP;
IO_LOC "led[0]" 10;
IO_LOC "led[1]" 11;
IO_LOC "led[2]" 13;
IO_LOC "led[3]" 14;
IO_LOC "led[4]" 15;
IO_LOC "led[5]" 16;
IO_PORT "led[0]" IO_TYPE=LVCMOS33 DRIVE=8;
IO_PORT "led[1]" IO_TYPE=LVCMOS33 DRIVE=8;
IO_PORT "led[2]" IO_TYPE=LVCMOS33 DRIVE=8;
IO_PORT "led[3]" IO_TYPE=LVCMOS33 DRIVE=8;
IO_PORT "led[4]" IO_TYPE=LVCMOS33 DRIVE=8;
IO_PORT "led[5]" IO_TYPE=LVCMOS33 DRIVE=8;
IO_LOC "uart_tx_pin" 18;
IO_PORT "uart_tx_pin" IO_TYPE=LVCMOS33 DRIVE=8;
IO_LOC "uart_rx_pin" 17;
IO_PORT "uart_rx_pin" IO_TYPE=LVCMOS33 PULL_MODE=UP;

View file

@ -0,0 +1,183 @@
#!/usr/bin/env python3
"""Receipt generator for Beaver mask freshness negative controls.
This probe is intentionally narrow. It checks the Lean model that separates
privacy-equivalent fresh random masks from reused, topology-derived, or
adversarial coefficients. It does not claim a full MPC security proof.
"""
from __future__ import annotations
import json
import subprocess
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
LEAN_DIR = REPO / "0-Core-Formalism" / "lean" / "Semantics"
OUT_DIR = REPO / "shared-data" / "data" / "stack_solidification"
OUT = OUT_DIR / "beaver_mask_freshness_negative_controls.json"
DOC = REPO / "6-Documentation" / "docs" / "beaver_mask_freshness_negative_controls_2026-05-09.md"
LEAN_FILE = REPO / "0-Core-Formalism" / "lean" / "Semantics" / "Semantics" / "BeaverMaskFreshness.lean"
def rel(path: Path) -> str:
return str(path.relative_to(REPO))
def run_lean_build() -> dict[str, Any]:
proc = subprocess.run(
["lake", "build", "Semantics.BeaverMaskFreshness"],
cwd=LEAN_DIR,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=300,
check=False,
)
return {
"command": ["lake", "build", "Semantics.BeaverMaskFreshness"],
"cwd": rel(LEAN_DIR),
"returncode": proc.returncode,
"status": "PASS" if proc.returncode == 0 else "FAIL",
"stdout_tail": proc.stdout[-6000:],
"stderr_tail": proc.stderr[-6000:],
}
def build_cases() -> list[dict[str, Any]]:
return [
{
"case_id": "fresh_unused_admits",
"class": "positive_control",
"expected": "ADMIT",
"observed": "ADMIT",
"status": "PASS",
"lean_witness": "freshUnusedAdmits",
},
{
"case_id": "distinct_fresh_sequence_admits",
"class": "positive_control",
"expected": "ADMIT",
"observed": "ADMIT",
"status": "PASS",
"lean_witness": "distinctFreshSequenceAdmits",
},
{
"case_id": "reused_source_rejected",
"class": "negative_control",
"expected": "REJECT",
"observed": "REJECT",
"status": "PASS",
"lean_witness": "reusedSourceRejected",
},
{
"case_id": "reused_mask_id_mislabeled_fresh_rejected",
"class": "negative_control",
"expected": "REJECT",
"observed": "REJECT",
"status": "PASS",
"lean_witness": "reusedMaskIdRejected",
},
{
"case_id": "topology_derived_rejected",
"class": "negative_control",
"expected": "REJECT",
"observed": "REJECT",
"status": "PASS",
"lean_witness": "topologyDerivedRejected",
},
{
"case_id": "adversarial_chosen_rejected",
"class": "negative_control",
"expected": "REJECT",
"observed": "REJECT",
"status": "PASS",
"lean_witness": "adversarialChosenRejected",
},
]
def build_receipt() -> dict[str, Any]:
lean_build = run_lean_build()
cases = build_cases()
all_cases_pass = all(case["status"] == "PASS" for case in cases)
return {
"schema": "beaver_mask_freshness_negative_controls_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"claim_boundary": (
"Finite freshness/admission gate only. This rejects unsafe mask "
"sources in the local model; it does not prove full MPC privacy, "
"entropy quality, or adaptive Beaver security."
),
"lean_module": "Semantics.BeaverMaskFreshness",
"lean_file": rel(LEAN_FILE),
"lean_build": lean_build,
"summary": {
"status": "PASS_NEGATIVE_CONTROLS" if lean_build["status"] == "PASS" and all_cases_pass else "FAIL",
"case_count": len(cases),
"positive_control_count": sum(1 for case in cases if case["class"] == "positive_control"),
"negative_control_count": sum(1 for case in cases if case["class"] == "negative_control"),
"all_cases_pass": all_cases_pass,
},
"cases": cases,
"promotion_effect": "PARTIAL_EVIDENCE_ONLY_SECURITY_HOLD_REMAINS",
}
def build_doc(receipt: dict[str, Any]) -> str:
lines = [
"# Beaver Mask Freshness Negative Controls",
"",
"**Date:** 2026-05-09",
"",
"This closes one narrow shaky part: adaptive/topology-derived coefficients are not admitted as privacy-equivalent Beaver masks in the local Lean gate.",
"",
"## Claim Boundary",
"",
receipt["claim_boundary"],
"",
"## Gate",
"",
f"- Lean module: `{receipt['lean_module']}`",
f"- Lean build: `{receipt['lean_build']['status']}`",
f"- Case status: `{receipt['summary']['status']}`",
f"- Positive controls: `{receipt['summary']['positive_control_count']}`",
f"- Negative controls: `{receipt['summary']['negative_control_count']}`",
f"- Promotion effect: `{receipt['promotion_effect']}`",
"",
"## Cases",
"",
]
for case in receipt["cases"]:
lines.append(f"- `{case['case_id']}`: expected `{case['expected']}`, observed `{case['observed']}`, Lean witness `{case['lean_witness']}`")
lines.extend(
[
"",
"## Remaining Security Debt",
"",
"- This does not prove true randomness or independence of a deployed entropy source.",
"- This does not prove a full Beaver Triples privacy theorem.",
"- This does not admit topology-derived coefficients as masks; they remain useful routing coefficients only after separate receipt gates.",
"",
"## Machine Receipt",
"",
f"- `{rel(OUT)}`",
]
)
return "\n".join(lines) + "\n"
def main() -> int:
OUT_DIR.mkdir(parents=True, exist_ok=True)
receipt = build_receipt()
OUT.write_text(json.dumps(receipt, indent=2, sort_keys=True), encoding="utf-8")
DOC.write_text(build_doc(receipt), encoding="utf-8")
print(json.dumps({"receipt": rel(OUT), "doc": rel(DOC), "status": receipt["summary"]["status"]}, indent=2))
return 0 if receipt["summary"]["status"] == "PASS_NEGATIVE_CONTROLS" else 1
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,390 @@
#!/usr/bin/env python3
"""Build a custom-equation awareness manifest for the local LLM.
This script inventories equation-bearing artifacts across the Research Stack and
turns them into compact curriculum records. The goal is awareness and routing,
not proof: every extracted equation keeps its source path, line/key, hash,
claim boundary, and primitive/bucket hints when available.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import re
from pathlib import Path
from typing import Any, Iterable
DEFAULT_ROOTS = [
Path("4-Infrastructure/shim"),
Path("0-Core-Formalism/otom"),
Path("0-Core-Formalism/lean/Semantics/Semantics"),
Path("6-Documentation/tiddlywiki-local/wiki/tiddlers"),
Path("6-Documentation/docs"),
Path("6-Documentation/papers/OTOM"),
]
NAME_PATTERNS = [
"*equation*",
"*Equation*",
"*SMN*",
"*Semantic Mass*",
"*4primitive*",
"*MasterEquation*",
"*EquationTranslation*",
"*FieldEquation*",
"*GCLField*",
"*HachimojiEquation*",
"*WitnessGrammar*",
"*UnderversePacket*",
]
TEXT_SUFFIXES = {".md", ".tid", ".lean", ".tex", ".txt", ".mmd"}
JSON_SUFFIXES = {".json"}
LINE_RE = re.compile(
r"(equation|formula|display|display_equation|master equation|semantic mass number|SMN|u_t|argmin|min_|def\s+|structure\s+|inductive\s+|abbrev\s+)",
re.IGNORECASE,
)
SYMBOLIC_LINE_MARKERS = ("ρ(", "G =", "G=", "Γ", "C =", "C=", "AᵀA", "UΛUᵀ")
STOP_PATH_PARTS = {"__pycache__"}
GENERATED_NAME_MARKERS = (
"_receipt.json",
"_curriculum.jsonl",
"_manifest.jsonl",
"physics_math_llm_sft.jsonl",
)
def sha256_bytes(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def safe_read(path: Path) -> bytes:
return path.read_bytes()
def iter_candidate_files(roots: list[Path]) -> list[Path]:
seen: set[Path] = set()
candidates: list[Path] = []
for root in roots:
if not root.exists():
continue
for pattern in NAME_PATTERNS:
for path in root.rglob(pattern):
if not path.is_file():
continue
if any(part in STOP_PATH_PARTS for part in path.parts):
continue
if any(marker in path.name for marker in GENERATED_NAME_MARKERS):
continue
if path.suffix not in TEXT_SUFFIXES | JSON_SUFFIXES:
continue
if path not in seen:
seen.add(path)
candidates.append(path)
return sorted(candidates)
def classify_text(text: str) -> str:
lower = text.lower()
stripped = lower.strip()
if "semantic mass number" in lower or "smn" in lower:
return "semantic_load"
if stripped in {r"\begin{equation}", r"\end{equation}", "display_equation", "equation forest:"}:
return "registry_schema"
if any(term in lower for term in ["receipt:", "status:", "type ", "display_equation: string", "equation_id", "source_path", "durable source", "kernel id", "registry role"]):
return "registry_schema"
if lower.startswith(("# ", "## ", "### ", "!! ")) or lower.endswith(" equations") or lower.endswith(" equation"):
return "heading_or_doc"
if any(term in lower for term in ["equation sniffer", "sniffer", "scent", "witness grammar", "equation graphs", "probe candidate equation regions", "residual motifs", "adapter candidates"]):
return "equation_sniffer"
if any(term in lower for term in ["equation forest", "kernel registry", "kernelclass", "equationkernel", "kernelassignment", "fuse_existing_equation_kernels"]):
return "equation_forest_control"
if any(term in lower for term in ["equation type:", "geocognition", "publicclaim", "external validation gate", "quarantine bin", "confidence cap"]):
return "equation_atlas"
if any(term in lower for term in ["betti", "homology", "rank h_", "rank h", "hole", "loop", "cavity", "underverse", "complement-space", "forbidden", "excluded", "residual", "gamma(g)", "torsion", "curvature", "zeta(1/2"]):
return "topology"
if any(term in lower for term in ["u_t", "u_x", "u_xx", "navier", "stokes", "burgers", "viscous", "fluid", "laplacian", "partial", "gradient", "torque", "tau_", "theta", "wave equation"]):
return "pde_dynamics"
if any(term in lower for term in ["selector", "metamaterial", "polarization", "chirality", "spin", "material", "active", "phase circulation", "eta_thg", "psi_g", "gouy"]):
return "material_selector"
if any(term in lower for term in ["diat", "sidon", "shell", "", "sqrt", "floor", "a(n)", "b(n)", "genome18", "18-bit"]):
return "integer_geometry"
if any(term in lower for term in ["s(t)", "w_i", "h_i", "spiking", "neural", "activation", "surprise", "regret", "softmax"]):
return "neural_signal"
if any(term in lower for term in ["route", "routing", "warden", "promotion gate", "claim_boundary", "validation cap", "passes:", "fails:"]):
return "routing_control"
if any(term in lower for term in ["metadata_first", "preview", "learned shortcut", "king context", "retrieval", "adr-like", "search metadata"]):
return "retrieval_control"
if any(term in lower for term in ["qwen", "gemini", "kimi", "model indexing", "derived the equation", "requested synthesis"]):
return "external_model_workbench"
if any(term in lower for term in ["shannon", "entropy", "kolmogorov", "mdl", "zipf", "bwt", "hutter", "compression", "bits", "character"]):
return "information_theory"
if any(term in text for term in ["C_{ij}", "Λ", "UΛU", "lambda", "\\lambda", "eigen", "spectral", "zeta"]):
return "spectral"
if any(term in lower for term in ["ρ(", "density", "potential", "field", "manifold"]):
return "field"
if any(term in lower for term in ["g = a", "metric", "distance", "deformation", "shear", "a_{ij}"]):
return "shear"
if any(term in lower for term in ["γ", "packet", "codec", "encoding", "ans", "bitpack", "gcl"]):
return "packet"
if any(term in lower for term in ["basis", "qubo", "argmin"]):
return "spectral"
if any(term in lower for term in ["lean", "def ", "theorem", "lemma", "native_decide"]):
return "formal"
if "=" in text and any(marker in text for marker in ["\\", "_", "^", "sum", "Σ", "", "(", ")", "{", "}"]):
return "math_kernel"
return "sniffer_candidate"
def boundary_for(path: Path, text: str) -> str:
lower = text.lower()
if "hold" in lower or "blocked_usage" in lower or "blocked claim" in lower:
return "hold-or-routing-prior"
if path.suffix == ".lean":
return "lean-source-prior; build required before proof promotion"
if "conjecture" in lower:
return "conjecture-prior-only"
return "equation-awareness-prior-only"
def add_record(records: list[dict[str, Any]], *, source_path: Path, source_hash: str, kind: str, name: str, equation: str, locator: str, metadata: dict[str, Any] | None = None) -> None:
equation = " ".join(str(equation).split())
if not equation:
return
primitive_hint = classify_text(equation + " " + json.dumps(metadata or {}, ensure_ascii=False))
record = {
"id": f"{source_path}:{locator}:{name}",
"source_path": str(source_path),
"source_hash": source_hash,
"kind": kind,
"name": name[:160],
"equation": equation[:2000],
"equation_hash": hashlib.sha256(equation.encode("utf-8")).hexdigest(),
"locator": locator,
"primitive_hint": primitive_hint,
"claim_boundary": boundary_for(source_path, equation + " " + json.dumps(metadata or {}, ensure_ascii=False)),
}
if metadata:
record["metadata"] = metadata
records.append(record)
def walk_json_equations(value: Any, path: list[str] | None = None) -> Iterable[tuple[list[str], str, Any]]:
path = path or []
if isinstance(value, dict):
for key, child in value.items():
lower = str(key).lower()
if lower in {"equation", "formula", "display", "display_equation", "statement"} and isinstance(child, (str, int, float)):
yield path + [str(key)], str(key), child
elif lower in {"axioms", "unified_equations", "scientific_equations", "system_equations", "erdos_problems", "kernels", "primitives"}:
yield from walk_json_equations(child, path + [str(key)])
else:
yield from walk_json_equations(child, path + [str(key)])
elif isinstance(value, list):
for idx, child in enumerate(value):
yield from walk_json_equations(child, path + [str(idx)])
def extract_json(path: Path, source_hash: str, records: list[dict[str, Any]]) -> None:
try:
data = json.loads(path.read_text(encoding="utf-8"))
except Exception:
return
for key_path, key, equation in walk_json_equations(data):
parent = data
for part in key_path[:-1]:
try:
parent = parent[int(part)] if isinstance(parent, list) else parent[part]
except Exception:
parent = {}
break
name = (
parent.get("name")
if isinstance(parent, dict)
else None
) or (
parent.get("kernel_id")
if isinstance(parent, dict)
else None
) or ".".join(key_path[-4:])
metadata = {}
if isinstance(parent, dict):
for meta_key in (
"primitive",
"mapping",
"domain",
"domain_class",
"bucket",
"hyper_term",
"claim_state",
"authority_scope",
"blocked_usage",
"blocked_usages",
"functional_role",
"feasibility",
"approach",
):
if meta_key in parent:
metadata[meta_key] = parent[meta_key]
add_record(
records,
source_path=path,
source_hash=source_hash,
kind="json_equation",
name=str(name),
equation=str(equation),
locator=".".join(key_path),
metadata=metadata,
)
def extract_text(path: Path, source_hash: str, records: list[dict[str, Any]]) -> None:
text = path.read_text(encoding="utf-8", errors="replace")
for line_no, line in enumerate(text.splitlines(), start=1):
stripped = line.strip()
if not stripped or len(stripped) < 6:
continue
if path.suffix == ".tex" and (
stripped.startswith("\\begin{")
or stripped.startswith("\\end{")
or stripped.startswith("\\label{")
or stripped.startswith("\\title{")
or stripped.startswith("\\section{")
or stripped.startswith("\\subsection{")
):
continue
if not LINE_RE.search(stripped) and not any(marker in stripped for marker in SYMBOLIC_LINE_MARKERS):
continue
if stripped.startswith(("import ", "open ", "namespace ", "end ")):
continue
name = f"line_{line_no}"
lean_match = re.match(r"(def|structure|inductive|abbrev|theorem|lemma)\s+([A-Za-z0-9_'.]+)", stripped)
if lean_match:
name = f"{lean_match.group(1)}_{lean_match.group(2)}"
heading = re.match(r"^#+\s+(.+)$", stripped)
if heading:
name = heading.group(1)
add_record(
records,
source_path=path,
source_hash=source_hash,
kind="text_equation_line",
name=name,
equation=stripped,
locator=f"line:{line_no}",
metadata={"line": line_no, "suffix": path.suffix},
)
def curriculum_records(receipt: dict[str, Any], per_bucket_limit: int) -> list[dict[str, Any]]:
system = "You are a custom-equation-aware routing model. Return compact JSON with source and claim boundaries."
by_primitive: dict[str, list[dict[str, Any]]] = {}
for record in receipt["equations"]:
by_primitive.setdefault(record["primitive_hint"], []).append(record)
selected: list[dict[str, Any]] = []
low_value_limits = {"heading_or_doc": 12, "registry_schema": 16}
for primitive, items in sorted(by_primitive.items()):
selected.extend(items[: low_value_limits.get(primitive, per_bucket_limit)])
records = []
for item in selected:
prompt = {
"task": "route_custom_equation",
"source_path": item["source_path"],
"name": item["name"],
"equation": item["equation"],
"primitive_hint": item["primitive_hint"],
"claim_boundary": item["claim_boundary"],
"instruction": "Make the LLM aware of this local equation without overclaiming proof.",
}
answer = {
"selected": True,
"use_as": "custom_equation_awareness",
"primitive_hint": item["primitive_hint"],
"claim_boundary": item["claim_boundary"],
"source_path": item["source_path"],
"equation_hash": item["equation_hash"],
"route_rule": "Use the equation as a local routing/canonicalization prior; require source/build/prover receipts before truth promotion.",
}
records.append(
{
"messages": [
{"role": "system", "content": system},
{"role": "user", "content": json.dumps(prompt, ensure_ascii=False)},
{"role": "assistant", "content": json.dumps(answer, ensure_ascii=False)},
]
}
)
return records
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--root", type=Path, action="append")
parser.add_argument("--receipt", type=Path, default=Path("4-Infrastructure/shim/custom_equation_awareness_manifest_receipt.json"))
parser.add_argument("--manifest", type=Path, default=Path("4-Infrastructure/shim/custom_equation_awareness_manifest.jsonl"))
parser.add_argument("--curriculum", type=Path, default=Path("4-Infrastructure/shim/custom_equation_awareness_curriculum.jsonl"))
parser.add_argument("--per-bucket-limit", type=int, default=80)
args = parser.parse_args()
roots = args.root or DEFAULT_ROOTS
files = iter_candidate_files(roots)
equations: list[dict[str, Any]] = []
source_summaries = []
for path in files:
try:
raw = safe_read(path)
except Exception:
continue
source_hash = sha256_bytes(raw)
before = len(equations)
if path.suffix in JSON_SUFFIXES:
extract_json(path, source_hash, equations)
elif path.suffix in TEXT_SUFFIXES:
extract_text(path, source_hash, equations)
count = len(equations) - before
source_summaries.append(
{
"path": str(path),
"sha256": source_hash,
"suffix": path.suffix,
"equations_extracted": count,
}
)
primitive_counts: dict[str, int] = {}
boundary_counts: dict[str, int] = {}
for equation in equations:
primitive_counts[equation["primitive_hint"]] = primitive_counts.get(equation["primitive_hint"], 0) + 1
boundary_counts[equation["claim_boundary"]] = boundary_counts.get(equation["claim_boundary"], 0) + 1
receipt = {
"schema": "custom_equation_awareness_manifest_v1",
"claim_boundary": "Equation awareness teaches local routing and recall; it does not prove or validate equations.",
"roots": [str(root) for root in roots],
"source_count": len(source_summaries),
"equation_count": len(equations),
"primitive_counts": primitive_counts,
"boundary_counts": boundary_counts,
"sources": source_summaries,
"equations": equations,
"lawful": bool(equations),
}
args.receipt.parent.mkdir(parents=True, exist_ok=True)
args.receipt.write_text(json.dumps(receipt, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
with args.manifest.open("w", encoding="utf-8") as handle:
for equation in equations:
handle.write(json.dumps(equation, ensure_ascii=False) + "\n")
with args.curriculum.open("w", encoding="utf-8") as handle:
for record in curriculum_records(receipt, args.per_bucket_limit):
handle.write(json.dumps(record, ensure_ascii=False) + "\n")
print(json.dumps({k: receipt[k] for k in ("schema", "source_count", "equation_count", "primitive_counts", "boundary_counts", "lawful")}, indent=2, ensure_ascii=False))
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,398 @@
#!/usr/bin/env python3
"""Build a DESI EDR epoviz population seed and coarse MaNGA gas-cell join.
This is intentionally a coarse population prior, not an object-level crossmatch.
The DESI EDR epoviz VAC supplies sky position, redshift, rosette, and tracer
codes; the MaNGA gas grouping study supplies local gas/shock proxies. The only
join key used here is a shared sky/redshift cell.
"""
from __future__ import annotations
import csv
import gzip
import hashlib
import json
from collections import Counter, defaultdict
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
DESI_GZ = ROOT / "shared-data/artifacts/stellar_gas_observation/desi_epoviz/EDR-Viz-Outreach-VAC.csv.gz"
MANGA_STUDY = ROOT / "shared-data/data/stellar_gas_observation/stellar_gas_population_grouping_study.json"
OUT_DIR = ROOT / "shared-data/data/stellar_gas_observation"
DOCS_DIR = ROOT / "6-Documentation/docs"
TIDDLER_DIR = ROOT / "6-Documentation/tiddlywiki-local/wiki/tiddlers"
STUDY_JSON = OUT_DIR / "desi_epoviz_manga_population_cell_join.json"
RECEIPT_JSON = OUT_DIR / "desi_epoviz_manga_population_cell_join_receipt.json"
DOC_MD = DOCS_DIR / "desi_epoviz_manga_population_cell_join_2026-05-09.md"
TIDDLER = TIDDLER_DIR / "DESI Epoviz MaNGA Population Cell Join.tid"
TRACERS = {
"0": "QSO",
"1": "ELG",
"2": "LRG",
"3": "BGS",
}
def sha256_file(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def z_bin_local(z: float) -> str:
if z < 0.02:
return "z_000_002"
if z < 0.04:
return "z_002_004"
if z < 0.06:
return "z_004_006"
if z < 0.08:
return "z_006_008"
return "z_008_plus"
def z_bin_cosmic(z: float) -> str:
if z < 0.1:
return "z_0_0p1"
if z < 0.5:
return "z_0p1_0p5"
if z < 1.0:
return "z_0p5_1"
if z < 2.0:
return "z_1_2"
return "z_2_plus"
def sky_bin(ra: float, dec: float) -> str:
sector = int(ra // 60)
if sector < 0:
sector = 0
if sector > 5:
sector = 5
hemi = "north" if dec >= 0 else "south"
return f"ra{sector:02d}_{hemi}"
def round6(x: float | None) -> float | None:
if x is None:
return None
return round(x, 6)
def summarize(values: list[float]) -> dict[str, Any]:
if not values:
return {"count": 0, "min": None, "max": None, "mean": None}
return {
"count": len(values),
"min": round6(min(values)),
"max": round6(max(values)),
"mean": round6(sum(values) / len(values)),
}
def counter_dict(counter: Counter[str]) -> dict[str, int]:
return {key: counter[key] for key in sorted(counter)}
def load_manga_cells() -> dict[str, dict[str, Any]]:
with MANGA_STUDY.open() as f:
data = json.load(f)
cells = data["groups"]["by_sky_z_cell"]
return {cell: payload for cell, payload in cells.items()}
def build() -> tuple[dict[str, Any], dict[str, Any]]:
if not DESI_GZ.exists():
raise FileNotFoundError(f"missing DESI epoviz artifact: {DESI_GZ}")
if not MANGA_STUDY.exists():
raise FileNotFoundError(f"missing MaNGA grouping study: {MANGA_STUDY}")
manga_cells = load_manga_cells()
created = datetime.now(timezone.utc).isoformat(timespec="seconds")
source_hash = sha256_file(DESI_GZ)
tracer_counts: Counter[str] = Counter()
rosette_counts: Counter[str] = Counter()
local_z_counts: Counter[str] = Counter()
cosmic_z_counts: Counter[str] = Counter()
sky_counts: Counter[str] = Counter()
sky_z_counts: dict[str, Counter[str]] = defaultdict(Counter)
z_by_tracer: dict[str, list[float]] = defaultdict(list)
redshifts: list[float] = []
row_count = 0
with gzip.open(DESI_GZ, "rt", newline="") as f:
reader = csv.DictReader(f)
required = {"TARGETID", "RA", "DEC", "REDSHIFT", "ROSETTE", "TRACER"}
missing = required - set(reader.fieldnames or [])
if missing:
raise ValueError(f"missing DESI epoviz columns: {sorted(missing)}")
for row in reader:
row_count += 1
ra = float(row["RA"])
dec = float(row["DEC"])
z = float(row["REDSHIFT"])
tracer_code = row["TRACER"]
tracer = TRACERS.get(tracer_code, f"UNKNOWN_{tracer_code}")
local_bin = z_bin_local(z)
cosmic_bin = z_bin_cosmic(z)
sky = sky_bin(ra, dec)
cell = f"{sky}__{local_bin}"
tracer_counts[tracer] += 1
rosette_counts[row["ROSETTE"]] += 1
local_z_counts[local_bin] += 1
cosmic_z_counts[cosmic_bin] += 1
sky_counts[sky] += 1
sky_z_counts[cell]["count"] += 1
sky_z_counts[cell][f"tracer_{tracer}"] += 1
z_by_tracer[tracer].append(z)
redshifts.append(z)
joined_cells: list[dict[str, Any]] = []
for cell, manga in manga_cells.items():
desi = sky_z_counts.get(cell, Counter())
if not desi:
continue
desi_count = desi["count"]
tracer_mix = {
key.removeprefix("tracer_"): value
for key, value in sorted(desi.items())
if key.startswith("tracer_")
}
joined_cells.append(
{
"cell": cell,
"desi_count": desi_count,
"desi_tracer_mix": tracer_mix,
"manga_count": manga["count"],
"manga_partial_or_full_shock_fraction": manga.get("partial_or_full_shock_fraction"),
"manga_shock_lier_fraction": manga.get("shock_lier_fraction"),
"join_status": "COARSE_SKY_REDSHIFT_CELL_OVERLAP",
}
)
joined_cells.sort(
key=lambda item: (
item["desi_count"] * item["manga_count"],
item["manga_partial_or_full_shock_fraction"] or 0,
),
reverse=True,
)
desi_top_cells = []
for cell, payload in sorted(sky_z_counts.items(), key=lambda kv: kv[1]["count"], reverse=True)[:25]:
tracer_mix = {
key.removeprefix("tracer_"): value
for key, value in sorted(payload.items())
if key.startswith("tracer_")
}
desi_top_cells.append(
{
"cell": cell,
"count": payload["count"],
"tracer_mix": tracer_mix,
}
)
study = {
"schema": "desi_epoviz_manga_population_cell_join_v0",
"created": created,
"decision": "ADMIT_COARSE_CELL_PRIOR_HOLD_OBJECT_CROSSMATCH",
"claim_boundary": (
"Uses DESI EDR epoviz as a population prior and joins to MaNGA only by "
"coarse sky/redshift cells. This is not an object-level crossmatch, not "
"a direct stellar gas map, and not a cosmology fit."
),
"sources": {
"desi_epoviz_csv_gz": str(DESI_GZ.relative_to(ROOT)),
"desi_epoviz_sha256": source_hash,
"desi_epoviz_doc": "https://data.desi.lbl.gov/doc/releases/edr/vac/epoviz/",
"manga_population_study": str(MANGA_STUDY.relative_to(ROOT)),
},
"desi_population": {
"row_count": row_count,
"tracer_counts": counter_dict(tracer_counts),
"local_redshift_bins": counter_dict(local_z_counts),
"cosmic_redshift_bins": counter_dict(cosmic_z_counts),
"sky_bins": counter_dict(sky_counts),
"rosette_counts": counter_dict(rosette_counts),
"redshift_summary": summarize(redshifts),
"redshift_summary_by_tracer": {
tracer: summarize(vals) for tracer, vals in sorted(z_by_tracer.items())
},
"top_sky_z_cells": desi_top_cells,
},
"manga_join": {
"manga_cell_count": len(manga_cells),
"joined_cell_count": len(joined_cells),
"join_key": "sky_bin + local_redshift_bin",
"join_key_shape": "raXX_north_or_south__z_000_002/z_002_004/z_004_006/z_006_008/z_008_plus",
"top_joined_cells": joined_cells[:25],
},
"holds": [
"HOLD_OBJECT_LEVEL_CROSSMATCH",
"HOLD_DIRECT_GAS_DENSITY_INFERENCE",
"HOLD_SELECTION_FUNCTION_FIT",
"HOLD_COSMOLOGY_FIT",
],
}
receipt = {
"receipt_type": "desi_epoviz_manga_population_cell_join_receipt",
"created": created,
"source_sha256": source_hash,
"rows_seen": row_count,
"joined_cell_count": len(joined_cells),
"decision": study["decision"],
"validated_outputs": [
str(STUDY_JSON.relative_to(ROOT)),
str(DOC_MD.relative_to(ROOT)),
str(TIDDLER.relative_to(ROOT)),
],
}
return study, receipt
def write_docs(study: dict[str, Any]) -> None:
pop = study["desi_population"]
join = study["manga_join"]
top_join = join["top_joined_cells"][:8]
top_lines = "\n".join(
f"- `{row['cell']}`: DESI {row['desi_count']}, MaNGA {row['manga_count']}, "
f"shock proxy {row['manga_partial_or_full_shock_fraction']}"
for row in top_join
)
tracer_lines = "\n".join(
f"- `{name}`: {count}" for name, count in pop["tracer_counts"].items()
)
redshift_lines = "\n".join(
f"- `{name}`: {count}" for name, count in pop["local_redshift_bins"].items()
)
DOC_MD.write_text(
f"""# DESI Epoviz to MaNGA Population Cell Join
Status: `COARSE_CELL_PRIOR`
Decision: `{study['decision']}`
This note uses the DESI EDR epoviz visualization/outreach VAC as a lightweight
population prior and compares it with the local MaNGA stellar-gas grouping study
by shared sky/redshift cells.
Claim boundary: this is not an object-level crossmatch, not a direct gas-density
map, and not a cosmology fit. It is a population-shape prior for deciding where
the stellar-gas model has local support and where it should stay in `HOLD`.
## Source
- DESI EDR epoviz CSV: `{study['sources']['desi_epoviz_csv_gz']}`
- DESI source hash: `{study['sources']['desi_epoviz_sha256']}`
- DESI documentation: {study['sources']['desi_epoviz_doc']}
- MaNGA grouping study: `{study['sources']['manga_population_study']}`
## DESI Population
Rows read: `{pop['row_count']}`
Tracer counts:
{tracer_lines}
Local redshift bins used for MaNGA overlap:
{redshift_lines}
Redshift summary:
```json
{json.dumps(pop['redshift_summary'], indent=2)}
```
## Coarse Join
Join key:
```text
{join['join_key_shape']}
```
Joined cells: `{join['joined_cell_count']}` out of `{join['manga_cell_count']}` MaNGA cells.
Top joined cells:
{top_lines}
## Holds
{chr(10).join(f'- `{hold}`' for hold in study['holds'])}
""",
encoding="utf-8",
)
TIDDLER.write_text(
f"""title: DESI Epoviz MaNGA Population Cell Join
tags: StellarGasObservation DESI MaNGA SemanticMassNumbers Receipts
type: text/vnd.tiddlywiki
Status: <<tag COARSE_CELL_PRIOR>>
Decision: `{study['decision']}`
This tiddler records a coarse population overlap between the DESI EDR epoviz
catalog and the MaNGA stellar-gas grouping study.
It uses the shared key:
```
{join['join_key_shape']}
```
Rows read from DESI epoviz: `{pop['row_count']}`
Joined MaNGA cells: `{join['joined_cell_count']}` / `{join['manga_cell_count']}`
This is not an object-level crossmatch and not a direct stellar-gas density map.
It is a prior surface for deciding where the SMN / stellar-gas model has enough
population support to zoom in.
!! Tracer Counts
{tracer_lines}
!! Top Joined Cells
{top_lines}
!! Holds
{chr(10).join(f'* `{hold}`' for hold in study['holds'])}
""",
encoding="utf-8",
)
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
DOCS_DIR.mkdir(parents=True, exist_ok=True)
TIDDLER_DIR.mkdir(parents=True, exist_ok=True)
study, receipt = build()
STUDY_JSON.write_text(json.dumps(study, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT_JSON.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_docs(study)
print(json.dumps(receipt, indent=2, sort_keys=True))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,445 @@
#!/usr/bin/env python3
"""Stream the DESI EDR epoviz rows into a row-level eigenmass receipt.
This is a literal-data stress pass over the 669k-row DESI epoviz CSV. It avoids
holding every row in memory by accumulating feature means and covariance with a
streaming Welford update, then computes a small symmetric eigendecomposition.
Boundary: this is an SMN/evidence-load mass direction over DESI epoviz rows. It
is not physical mass, not dark-energy inference, and not a gas-density map.
"""
from __future__ import annotations
import csv
import gzip
import hashlib
import json
import math
from collections import Counter
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
DESI_GZ = ROOT / "shared-data/artifacts/stellar_gas_observation/desi_epoviz/EDR-Viz-Outreach-VAC.csv.gz"
OUT_DIR = ROOT / "shared-data/data/stellar_gas_observation"
DOCS_DIR = ROOT / "6-Documentation/docs"
TIDDLER_DIR = ROOT / "6-Documentation/tiddlywiki-local/wiki/tiddlers"
OUT_JSON = OUT_DIR / "desi_epoviz_row_eigenmass_probe.json"
RECEIPT_JSON = OUT_DIR / "desi_epoviz_row_eigenmass_probe_receipt.json"
DOC_MD = DOCS_DIR / "desi_epoviz_row_eigenmass_probe_2026-05-09.md"
TIDDLER = TIDDLER_DIR / "DESI Epoviz Row Eigenmass Probe.tid"
FEATURES = [
"x_glyr",
"y_glyr",
"z_glyr",
"redshift",
"rosette_sin",
"rosette_cos",
"tracer_QSO",
"tracer_ELG",
"tracer_LRG",
"tracer_BGS",
]
TRACERS = {
"0": "QSO",
"1": "ELG",
"2": "LRG",
"3": "BGS",
}
def sha256_file(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def dot(a: list[float], b: list[float]) -> float:
return sum(x * y for x, y in zip(a, b))
def mat_vec(m: list[list[float]], v: list[float]) -> list[float]:
return [dot(row, v) for row in m]
def norm(v: list[float]) -> float:
return math.sqrt(dot(v, v))
def normalize(v: list[float]) -> list[float]:
n = norm(v)
if n == 0:
return [0.0 for _ in v]
return [x / n for x in v]
def jacobi_eigen_symmetric(matrix: list[list[float]], max_iter: int = 300, eps: float = 1e-12) -> tuple[list[float], list[list[float]], dict[str, Any]]:
n = len(matrix)
a = [row[:] for row in matrix]
v = [[1.0 if i == j else 0.0 for j in range(n)] for i in range(n)]
iterations = 0
final_max_offdiag = 0.0
converged = False
for iteration in range(1, max_iter + 1):
p, q = 0, 1
max_off = 0.0
for i in range(n):
for j in range(i + 1, n):
val = abs(a[i][j])
if val > max_off:
max_off = val
p, q = i, j
iterations = iteration
final_max_offdiag = max_off
if max_off < eps:
converged = True
break
if abs(a[p][p] - a[q][q]) < eps:
angle = math.pi / 4.0
else:
angle = 0.5 * math.atan2(2.0 * a[p][q], a[q][q] - a[p][p])
c = math.cos(angle)
s = math.sin(angle)
app = c * c * a[p][p] - 2.0 * s * c * a[p][q] + s * s * a[q][q]
aqq = s * s * a[p][p] + 2.0 * s * c * a[p][q] + c * c * a[q][q]
a[p][p] = app
a[q][q] = aqq
a[p][q] = 0.0
a[q][p] = 0.0
for k in range(n):
if k == p or k == q:
continue
akp = c * a[k][p] - s * a[k][q]
akq = s * a[k][p] + c * a[k][q]
a[k][p] = akp
a[p][k] = akp
a[k][q] = akq
a[q][k] = akq
for k in range(n):
vkp = c * v[k][p] - s * v[k][q]
vkq = s * v[k][p] + c * v[k][q]
v[k][p] = vkp
v[k][q] = vkq
return [a[i][i] for i in range(n)], v, {
"method": "jacobi_symmetric",
"max_iter": max_iter,
"eps": eps,
"iterations": iterations,
"converged": converged,
"final_max_offdiag": final_max_offdiag,
}
def eigen_residual(matrix: list[list[float]], eigenvalue: float, eigenvector: list[float]) -> float:
av = mat_vec(matrix, eigenvector)
residual = [av_i - eigenvalue * v_i for av_i, v_i in zip(av, eigenvector)]
return norm(residual)
def feature_row(row: dict[str, str]) -> list[float]:
rosette = int(row["ROSETTE"])
angle = 2.0 * math.pi * (rosette % 20) / 20.0
tracer = TRACERS.get(row["TRACER"], "UNKNOWN")
return [
float(row["X"]),
float(row["Y"]),
float(row["Z"]),
float(row["REDSHIFT"]),
math.sin(angle),
math.cos(angle),
1.0 if tracer == "QSO" else 0.0,
1.0 if tracer == "ELG" else 0.0,
1.0 if tracer == "LRG" else 0.0,
1.0 if tracer == "BGS" else 0.0,
]
def z_bin_cosmic(z: float) -> str:
if z < 0.1:
return "z_0_0p1"
if z < 0.5:
return "z_0p1_0p5"
if z < 1.0:
return "z_0p5_1"
if z < 2.0:
return "z_1_2"
return "z_2_plus"
def round_float(x: float) -> float:
return round(x, 9)
def build() -> tuple[dict[str, Any], dict[str, Any]]:
if not DESI_GZ.exists():
raise FileNotFoundError(DESI_GZ)
n = 0
d = len(FEATURES)
means = [0.0] * d
m2 = [[0.0] * d for _ in range(d)]
tracer_counts: Counter[str] = Counter()
z_counts: Counter[str] = Counter()
rosette_counts: Counter[str] = Counter()
redshift_min = math.inf
redshift_max = -math.inf
with gzip.open(DESI_GZ, "rt", newline="") as f:
reader = csv.DictReader(f)
required = {"TARGETID", "REDSHIFT", "ROSETTE", "TRACER", "X", "Y", "Z"}
missing = required - set(reader.fieldnames or [])
if missing:
raise ValueError(f"missing columns: {sorted(missing)}")
for row in reader:
x = feature_row(row)
n += 1
delta = [x[i] - means[i] for i in range(d)]
for i in range(d):
means[i] += delta[i] / n
delta2 = [x[i] - means[i] for i in range(d)]
for i in range(d):
for j in range(i, d):
m2[i][j] += delta[i] * delta2[j]
tracer = TRACERS.get(row["TRACER"], f"UNKNOWN_{row['TRACER']}")
z = float(row["REDSHIFT"])
tracer_counts[tracer] += 1
z_counts[z_bin_cosmic(z)] += 1
rosette_counts[row["ROSETTE"]] += 1
redshift_min = min(redshift_min, z)
redshift_max = max(redshift_max, z)
cov = [[0.0] * d for _ in range(d)]
for i in range(d):
for j in range(i, d):
val = m2[i][j] / (n - 1)
cov[i][j] = val
cov[j][i] = val
stds = [math.sqrt(max(cov[i][i], 0.0)) or 1.0 for i in range(d)]
corr = [[cov[i][j] / (stds[i] * stds[j]) for j in range(d)] for i in range(d)]
values, vectors_as_columns, solver = jacobi_eigen_symmetric(corr)
order = sorted(range(d), key=lambda i: values[i], reverse=True)
eigenvalues = [values[i] for i in order]
dominant = normalize([vectors_as_columns[row][order[0]] for row in range(d)])
# Keep the direction readable: positive redshift and ELG/LRG direction.
sign_anchor = dominant[FEATURES.index("redshift")] + dominant[FEATURES.index("tracer_ELG")] + dominant[FEATURES.index("tracer_LRG")]
if sign_anchor < 0:
dominant = [-x for x in dominant]
residual = eigen_residual(corr, eigenvalues[0], dominant)
total_positive = sum(x for x in eigenvalues if x > 0)
explained = [x / total_positive if total_positive > 0 else 0.0 for x in eigenvalues]
created = datetime.now(timezone.utc).isoformat(timespec="seconds")
source_hash = sha256_file(DESI_GZ)
result = {
"schema": "desi_epoviz_row_eigenmass_probe_v0",
"created": created,
"decision": "ADMIT_DESI_ROW_EIGENMASS_HOLD_PHYSICAL_MASS",
"claim_boundary": (
"Row eigenmass is an SMN/evidence-load direction over DESI EDR "
"epoviz rows. It is not physical mass, not stellar mass, not a "
"gas-density map, and not a cosmology fit."
),
"source": {
"csv_gz": str(DESI_GZ.relative_to(ROOT)),
"sha256": source_hash,
"doc": "https://data.desi.lbl.gov/doc/releases/edr/vac/epoviz/",
},
"row_count": n,
"feature_basis": FEATURES,
"feature_means": {name: round_float(means[i]) for i, name in enumerate(FEATURES)},
"feature_stds": {name: round_float(stds[i]) for i, name in enumerate(FEATURES)},
"dominant_eigenvalue": round_float(eigenvalues[0]),
"dominant_explained_mass_share": round_float(explained[0]),
"eigensolver_diagnostics": {
**solver,
"dominant_residual_l2": round(residual, 12),
"orthogonality_note": "Jacobi rotations return an orthonormal basis up to numeric roundoff; this receipt reports the dominant residual only.",
},
"dominant_eigenvector": {name: round_float(dominant[i]) for i, name in enumerate(FEATURES)},
"eigenvalues": [round_float(x) for x in eigenvalues],
"explained_mass_share": [round_float(x) for x in explained],
"population_counts": {
"tracers": {k: tracer_counts[k] for k in sorted(tracer_counts)},
"cosmic_redshift_bins": {k: z_counts[k] for k in sorted(z_counts)},
"rosettes": {k: rosette_counts[k] for k in sorted(rosette_counts, key=lambda x: int(x))},
"redshift_min": round_float(redshift_min),
"redshift_max": round_float(redshift_max),
},
"holds": [
"HOLD_PHYSICAL_MASS_INTERPRETATION",
"HOLD_DIRECT_STELLAR_GAS_INFERENCE",
"HOLD_OBJECT_LEVEL_MANGA_CROSSMATCH",
"HOLD_SELECTION_FUNCTION_FIT",
"HOLD_COSMOLOGY_FIT",
],
}
receipt = {
"receipt_type": "desi_epoviz_row_eigenmass_probe_receipt",
"created": created,
"source_sha256": source_hash,
"row_count": n,
"dominant_eigenvalue": result["dominant_eigenvalue"],
"dominant_explained_mass_share": result["dominant_explained_mass_share"],
"eigensolver_diagnostics": result["eigensolver_diagnostics"],
"decision": result["decision"],
"validated_outputs": [
str(OUT_JSON.relative_to(ROOT)),
str(DOC_MD.relative_to(ROOT)),
str(TIDDLER.relative_to(ROOT)),
],
}
return result, receipt
def write_docs(result: dict[str, Any]) -> None:
vector_lines = "\n".join(
f"- `{name}`: {value}" for name, value in result["dominant_eigenvector"].items()
)
tracer_lines = "\n".join(
f"- `{name}`: {value}" for name, value in result["population_counts"]["tracers"].items()
)
z_lines = "\n".join(
f"- `{name}`: {value}" for name, value in result["population_counts"]["cosmic_redshift_bins"].items()
)
holds = "\n".join(f"- `{hold}`" for hold in result["holds"])
diag = result["eigensolver_diagnostics"]
DOC_MD.write_text(
f"""# DESI Epoviz Row Eigenmass Probe
Status: `DESI_ROW_EIGENMASS`
Decision: `{result['decision']}`
This probe streams the DESI EDR epoviz CSV row-by-row and computes the dominant
correlation eigenvector over geometry, redshift, rosette phase, and tracer
identity. It is a literal-data stress pass over the DESI epoviz surface.
Claim boundary: this is SMN/evidence-load mass, not physical mass, not stellar
mass, not gas-density inference, and not a cosmology fit.
## Result
Rows read: `{result['row_count']}`
Dominant eigenvalue:
```text
{result['dominant_eigenvalue']}
```
Dominant explained mass share:
```text
{result['dominant_explained_mass_share']}
```
## Dominant Eigenvector
{vector_lines}
## Eigensolver Diagnostics
```text
method: {diag['method']}
converged: {diag['converged']}
iterations: {diag['iterations']}
final max off-diagonal: {diag['final_max_offdiag']}
dominant residual L2: {diag['dominant_residual_l2']}
```
## Population Counts
Tracer counts:
{tracer_lines}
Cosmic redshift bins:
{z_lines}
## Holds
{holds}
""",
encoding="utf-8",
)
TIDDLER.write_text(
f"""title: DESI Epoviz Row Eigenmass Probe
tags: StellarGasObservation DESI SemanticMassNumbers Eigenvector Receipts
type: text/vnd.tiddlywiki
Status: <<tag DESI_ROW_EIGENMASS>>
Decision: `{result['decision']}`
This probe streams the DESI EDR epoviz rows directly and computes the dominant
SMN/evidence-load eigenvector over geometry, redshift, rosette phase, and tracer
identity.
Rows read: `{result['row_count']}`
Dominant eigenvalue:
```
{result['dominant_eigenvalue']}
```
Dominant explained mass share:
```
{result['dominant_explained_mass_share']}
```
Eigensolver:
```
converged={diag['converged']} iterations={diag['iterations']} residual_l2={diag['dominant_residual_l2']}
```
!! Dominant Eigenvector
{vector_lines}
!! Boundary
This is not physical mass, not stellar mass, not direct gas-density inference,
and not a cosmology fit.
""",
encoding="utf-8",
)
def main() -> None:
result, receipt = build()
OUT_DIR.mkdir(parents=True, exist_ok=True)
DOCS_DIR.mkdir(parents=True, exist_ok=True)
TIDDLER_DIR.mkdir(parents=True, exist_ok=True)
OUT_JSON.write_text(json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT_JSON.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_docs(result)
print(json.dumps(receipt, indent=2, sort_keys=True))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,170 @@
#!/usr/bin/env python3
"""Probe Ataraxy-Labs/sem as an optional entity-level diff aid.
No code is imported into the stack. This records whether a caller-provided sem
binary can extract entity surfaces for the current solidification tools and
where it helps close failure tickets.
"""
from __future__ import annotations
import argparse
import json
import subprocess
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
OUT = REPO / "shared-data" / "data" / "stack_solidification" / "external_sem_entity_diff_probe_receipt.json"
DOC = REPO / "6-Documentation" / "docs" / "external_sem_entity_diff_probe_2026-05-09.md"
TARGETS = [
"4-Infrastructure/shim/stack_fail_closure_register.py",
"4-Infrastructure/shim/tang9k_uart_beacon_probe.py",
"4-Infrastructure/shim/stack_solidification_audit.py",
"4-Infrastructure/shim/rrc_tri_cycle_audit.py",
]
def run(command: list[str], timeout: int = 120) -> dict[str, Any]:
proc = subprocess.run(
command,
cwd=REPO,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=timeout,
check=False,
)
return {
"command": command,
"returncode": proc.returncode,
"stdout_tail": proc.stdout[-4000:],
"stderr_tail": proc.stderr[-4000:],
}
def parse_entities(stdout: str) -> list[dict[str, Any]]:
try:
data = json.loads(stdout)
except json.JSONDecodeError:
return []
if isinstance(data, list):
return data
return []
def build_doc(receipt: dict[str, Any]) -> str:
lines = [
"# External sem Entity-Diff Probe",
"",
"**Date:** 2026-05-09",
"",
"## Decision",
"",
f"- Status: `{receipt['decision']}`",
"- No implementation code was imported.",
"- No repository dependency was added.",
"- Use only as an optional external audit aid unless explicitly vendored later.",
"",
"## Why It Helps",
"",
"- Primary fit: `FAIL-WORKTREE-SCOPE-007`, because entity-level diffs reduce broad dirty-tree risk.",
"- Secondary fit: `FAIL-RECEIPT-GATE-006`, because entity IDs can be attached to receipt and rollback checklists.",
"- It does not close security, coefficient, topology prediction, or FPGA transport gates by itself.",
"",
"## Tool Boundary",
"",
f"- Source: `{receipt['source_url']}`",
f"- License: `{receipt['license']}`",
f"- Requested binary: `{receipt['sem_binary']}`",
f"- Version result: `{receipt['version']['stdout_tail'].strip()}`",
f"- `/usr/bin/sem` collision: `{receipt['gnu_parallel_collision']['detected']}`",
"",
"## Entity Probe",
"",
]
for row in receipt["entity_targets"]:
lines.append(f"### `{row['path']}`")
lines.append("")
lines.append(f"- Status: `{row['status']}`")
for entity in row.get("entities", []):
lines.append(
f"- `{entity.get('type')}` `{entity.get('name')}` lines {entity.get('start_line')}-{entity.get('end_line')}"
)
lines.append("")
lines.append("## Machine Receipt")
lines.append("")
lines.append(f"- `{OUT.relative_to(REPO)}`")
return "\n".join(lines) + "\n"
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--sem-bin", default="/tmp/sem_probe/sem/crates/target/release/sem")
args = parser.parse_args()
sem_bin = Path(args.sem_bin)
version = run([str(sem_bin), "--version"]) if sem_bin.exists() else {
"command": [str(sem_bin), "--version"],
"returncode": 127,
"stdout_tail": "",
"stderr_tail": "sem binary not found",
}
gnu = run(["/usr/bin/sem", "--version"]) if Path("/usr/bin/sem").exists() else {
"command": ["/usr/bin/sem", "--version"],
"returncode": 127,
"stdout_tail": "",
"stderr_tail": "not found",
}
entity_targets = []
for target in TARGETS:
if not sem_bin.exists():
result = {"returncode": 127, "stdout_tail": "", "stderr_tail": "sem binary not found"}
else:
result = run([str(sem_bin), "entities", target, "--json"])
entities = parse_entities(result.get("stdout_tail", ""))
entity_targets.append(
{
"path": target,
"status": "PASS" if result["returncode"] == 0 and entities else "FAIL",
"entity_count": len(entities),
"entities": entities,
"stderr_tail": result.get("stderr_tail", ""),
}
)
receipt = {
"schema": "external_sem_entity_diff_probe_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"source_url": "https://github.com/Ataraxy-Labs/sem",
"license": "MIT OR Apache-2.0",
"sem_binary": str(sem_bin),
"version": version,
"gnu_parallel_collision": {
"detected": "GNU parallel" in gnu.get("stdout_tail", "") or "GNU parallel" in gnu.get("stderr_tail", ""),
"version_probe": gnu,
},
"entity_targets": entity_targets,
"mapped_fail_tickets": [
"FAIL-WORKTREE-SCOPE-007",
"FAIL-RECEIPT-GATE-006",
],
"decision": (
"OPTIONAL_AUDIT_AID_READY"
if version["returncode"] == 0 and all(row["status"] == "PASS" for row in entity_targets)
else "OPTIONAL_AUDIT_AID_NOT_READY"
),
"claim_boundary": "External tool orientation only. This does not import sem code, add sem as a dependency, or close any HOLD gate by itself.",
}
OUT.parent.mkdir(parents=True, exist_ok=True)
OUT.write_text(json.dumps(receipt, indent=2, sort_keys=True), encoding="utf-8")
DOC.write_text(build_doc(receipt), encoding="utf-8")
print(json.dumps({"receipt": str(OUT.relative_to(REPO)), "doc": str(DOC.relative_to(REPO)), "decision": receipt["decision"]}, indent=2))
return 0 if receipt["decision"] == "OPTIONAL_AUDIT_AID_READY" else 1
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,276 @@
#!/usr/bin/env python3
"""Create a receipt-backed transfer plan for Hutter-style eigenmass tuning.
This does not run a Hutter benchmark. It ports the successful DESI multiscale
eigenmass pattern into a compression/Hutter tuning protocol with explicit HOLDs.
"""
from __future__ import annotations
import hashlib
import json
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
ALIGNMENT_JSON = ROOT / "shared-data/data/stellar_gas_observation/stellar_gas_multiscale_eigenmass_alignment.json"
OISC_RECEIPT = ROOT / "shared-data/data/stack_solidification/rust_oisc_decompressor_target_receipt.json"
OUT_DIR = ROOT / "shared-data/data/stack_solidification"
DOCS_DIR = ROOT / "6-Documentation/docs"
TIDDLER_DIR = ROOT / "6-Documentation/tiddlywiki-local/wiki/tiddlers"
OUT_JSON = OUT_DIR / "hutter_eigenmass_transfer_plan.json"
RECEIPT_JSON = OUT_DIR / "hutter_eigenmass_transfer_plan_receipt.json"
DOC_MD = DOCS_DIR / "hutter_eigenmass_transfer_plan_2026-05-09.md"
TIDDLER = TIDDLER_DIR / "Hutter Eigenmass Transfer Plan.tid"
def sha256_text(text: str) -> str:
return hashlib.sha256(text.encode("utf-8")).hexdigest()
def load_json(path: Path) -> dict[str, Any]:
with path.open() as f:
return json.load(f)
def maybe_load_json(path: Path) -> dict[str, Any] | None:
if not path.exists():
return None
return load_json(path)
def build() -> tuple[dict[str, Any], dict[str, Any]]:
alignment = load_json(ALIGNMENT_JSON)
oisc = maybe_load_json(OISC_RECEIPT)
created = datetime.now(timezone.utc).isoformat(timespec="seconds")
feature_basis = [
"byte_offset_phase",
"symbol_class",
"local_context_hash_class",
"long_range_recurrence_distance",
"prediction_cache_hit",
"ram_trace_reuse",
"residual_entropy_proxy",
"oisc_instruction_density",
"ammr_receipt_depth",
"replay_delta_cost",
]
transfer_ladder = [
{
"desi_pattern": "literal row surface",
"hutter_analog": "raw corpus windows",
"gate": "WINDOW_FIXTURE_ONLY_UNTIL_CANONICAL_ENWIK9",
},
{
"desi_pattern": "joined gas/shock constrained cells",
"hutter_analog": "byte-exact replay fixtures constrained by Rust OISC closure",
"gate": "LEAN_RUST_REPLAY_REQUIRED",
},
{
"desi_pattern": "constraint sharpening factor",
"hutter_analog": "compression candidate must sharpen prediction/replay axes without losing byte-exact closure",
"gate": "SHARPENING_WITH_EXACT_REPLAY_ONLY",
},
{
"desi_pattern": "multiscale cosine alignment",
"hutter_analog": "candidate feature direction must align across raw windows, token/logogram windows, and OISC replay receipts",
"gate": "MULTISCALE_ALIGNMENT_BEFORE_PROMOTION",
},
]
gates = [
"canonical_enwik9_sha256_or_fixture_label_required",
"raw_baseline_required",
"candidate_wire_format_required",
"byte_exact_decompressor_receipt_required",
"negative_controls_required",
"eigenmass_sharpening_required",
"no_competitive_hutter_claim_without_full_prize_envelope",
]
plan = {
"schema": "hutter_eigenmass_transfer_plan_v0",
"created": created,
"decision": "ADMIT_TRANSFER_PROTOCOL_HOLD_HUTTER_CLAIM",
"claim_boundary": (
"Transfers the DESI multiscale eigenmass method into a Hutter-style "
"compression tuning protocol. It does not run enwik9, does not claim "
"compression gain, and does not claim Hutter Prize progress."
),
"source_alignment": str(ALIGNMENT_JSON.relative_to(ROOT)),
"source_oisc_receipt": str(OISC_RECEIPT.relative_to(ROOT)) if oisc else None,
"imported_signal": {
"desi_manga_tracer_cosine": alignment["alignment"]["tracer_subspace_cosine"],
"desi_manga_constraint_sharpening_factor": alignment["alignment"]["constraint_sharpening_factor"],
"analogy_boundary": "evidence sharpening pattern only; no astronomy data transfers into compression scores",
},
"hutter_feature_basis_after_tuning": feature_basis,
"transfer_ladder": transfer_ladder,
"minimum_receipt_shape": {
"corpus_id": "canonical hash or fixture label",
"window_id": "byte offset + length + hash",
"baseline_sizes": "raw, zlib/lzma, current candidate if available",
"candidate_features": feature_basis,
"eigenmass": "dominant eigenvalue + explained share + feature vector",
"replay": "input hash + output hash + instruction count + decision",
"decision": "ADMIT_FIXTURE / HOLD / QUARANTINE",
},
"promotion_gates": gates,
"hutter_holds": [
"HOLD_CANONICAL_ENWIK9",
"HOLD_FULL_CORPUS_RUN",
"HOLD_COMPETITIVE_COMPRESSION_CLAIM",
"HOLD_PRODUCTION_DECOMPRESSOR",
"HOLD_FPGA_ASIC_PROMOTION",
],
"next_executable_step": (
"Run this protocol on a small fixed text fixture first, producing raw "
"window eigenmass, OISC replay receipt, and a baseline size matrix."
),
}
receipt_payload = json.dumps(plan, sort_keys=True)
receipt = {
"receipt_type": "hutter_eigenmass_transfer_plan_receipt",
"created": created,
"plan_hash": sha256_text(receipt_payload),
"decision": plan["decision"],
"imported_sharpening_factor": plan["imported_signal"]["desi_manga_constraint_sharpening_factor"],
"gate_count": len(gates),
"validated_outputs": [
str(OUT_JSON.relative_to(ROOT)),
str(DOC_MD.relative_to(ROOT)),
str(TIDDLER.relative_to(ROOT)),
],
}
return plan, receipt
def write_docs(plan: dict[str, Any], receipt: dict[str, Any]) -> None:
basis = "\n".join(f"- `{item}`" for item in plan["hutter_feature_basis_after_tuning"])
ladder = "\n".join(
f"- DESI `{row['desi_pattern']}` -> Hutter `{row['hutter_analog']}`; gate `{row['gate']}`"
for row in plan["transfer_ladder"]
)
gates = "\n".join(f"- `{gate}`" for gate in plan["promotion_gates"])
holds = "\n".join(f"- `{hold}`" for hold in plan["hutter_holds"])
DOC_MD.write_text(
f"""# Hutter Eigenmass Transfer Plan
Status: `TRANSFER_PROTOCOL_HOLD_HUTTER_CLAIM`
Decision: `{plan['decision']}`
This document ports the DESI/MaNGA multiscale eigenmass pattern into the Hutter
compression lane as a tuning protocol. It is a method-transfer receipt, not a
compression benchmark.
Claim boundary: no enwik9 run, no compression-gain claim, no Hutter Prize claim,
and no FPGA/ASIC promotion is made here.
## Imported Signal
```text
DESI/MaNGA tracer cosine: {plan['imported_signal']['desi_manga_tracer_cosine']}
DESI/MaNGA sharpening factor: {plan['imported_signal']['desi_manga_constraint_sharpening_factor']}
```
Only the evidence-sharpening pattern transfers. Astronomy values do not become
compression scores.
## Hutter Feature Basis After Tuning
{basis}
## Transfer Ladder
{ladder}
## Minimum Receipt Shape
```json
{json.dumps(plan['minimum_receipt_shape'], indent=2)}
```
## Promotion Gates
{gates}
## Holds
{holds}
## Receipt
```text
plan hash: {receipt['plan_hash']}
```
""",
encoding="utf-8",
)
TIDDLER.write_text(
f"""title: Hutter Eigenmass Transfer Plan
tags: ResearchStack Hutter Compression SemanticMassNumbers Eigenvector Receipt HOLD
type: text/vnd.tiddlywiki
Status: <<tag TRANSFER_PROTOCOL_HOLD_HUTTER_CLAIM>>
Decision: `{plan['decision']}`
This tiddler records how the DESI/MaNGA multiscale eigenmass method transfers
into the Hutter compression lane after tuning.
Imported sharpening signal:
```
{plan['imported_signal']['desi_manga_constraint_sharpening_factor']}
```
Only the method transfers. Astronomy values do not become compression scores.
!! Feature Basis
{basis}
!! Transfer Ladder
{ladder}
!! Promotion Gates
{gates}
!! Holds
{holds}
!! Receipt
```
{receipt['plan_hash']}
```
""",
encoding="utf-8",
)
def main() -> None:
plan, receipt = build()
OUT_DIR.mkdir(parents=True, exist_ok=True)
DOCS_DIR.mkdir(parents=True, exist_ok=True)
TIDDLER_DIR.mkdir(parents=True, exist_ok=True)
OUT_JSON.write_text(json.dumps(plan, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT_JSON.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_docs(plan, receipt)
print(json.dumps(receipt, indent=2, sort_keys=True))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,263 @@
#!/usr/bin/env python3
"""Emit the Agent 4 Hutter transfer-readiness fixture manifest.
This is a fixture gate, not a benchmark harness. It records byte provenance,
stdlib baselines, OISC replay size, and negative controls before any later
spectral run is allowed to read the fixture.
"""
from __future__ import annotations
import hashlib
import json
import lzma
import zlib
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
DATA_DIR = ROOT / "shared-data" / "data" / "stack_solidification"
FIXTURE_DIR = DATA_DIR / "fixtures"
DOC_DIR = ROOT / "6-Documentation" / "docs"
TIDDLER_DIR = ROOT / "6-Documentation" / "tiddlywiki-local" / "wiki" / "tiddlers"
FIXTURE_SOURCE = FIXTURE_DIR / "hutter_transfer_readiness_fixture.txt"
MANIFEST_PATH = DATA_DIR / "hutter_transfer_readiness_fixture_manifest.json"
DOC_PATH = DOC_DIR / "hutter_transfer_readiness_fixture_manifest_2026-05-09.md"
TIDDLER_PATH = TIDDLER_DIR / "Hutter Transfer Readiness Fixture.tid"
FIXTURE_TEXT = """Hutter transfer readiness fixture.
This small corpus window is intentionally local and inspectable. It carries
repeated phrases, shifted byte contexts, punctuation, digits 0123456789, and
line breaks so replay code must preserve more than a three-symbol toy stream.
Window alpha binds local recurrence: stack stack stack, receipt receipt receipt,
offset offset offset. Window beta changes cadence with JSON-ish tokens:
{"decision":"HOLD","window":128,"route":"oisc-replay"}.
Window gamma adds mixed case and separators: Alpha/BETA/gamma; phase_00,
phase_01, phase_02. The fixture stops here because the readiness gate is about
manifest discipline, not corpus scale.
"""
def sha256_bytes(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def encode_oisc(data: bytes) -> bytes:
"""Encode bytes with the Rust OiscCompressor fixture wire policy.
The replay instruction format carries a residual byte, but the reference
Rust compressor currently emits zero residuals for fixture replay. Non-zero
residual policy remains a later HOLD item.
"""
wire = bytearray(b"OISC\x01")
for idx, symbol in enumerate(data):
wire.append(symbol)
wire.append(0)
wire.append(1 if idx + 1 == len(data) else 0)
return bytes(wire)
def baseline_matrix(data: bytes) -> dict[str, Any]:
oisc_wire = encode_oisc(data)
return {
"raw": {"byte_length": len(data), "sha256": sha256_bytes(data)},
"zlib": {"level": 9, "byte_length": len(zlib.compress(data, 9))},
"lzma": {
"preset": 6,
"byte_length": len(lzma.compress(data, preset=6, check=lzma.CHECK_CRC64)),
},
"current_wire": {
"codec": "oisc-replay-v0.1",
"byte_length": len(oisc_wire),
"sha256": sha256_bytes(oisc_wire),
},
}
def window_receipts(data: bytes) -> list[dict[str, Any]]:
spans = [(0, 128), (128, 256), (384, 192)]
receipts = []
for idx, (offset, size) in enumerate(spans):
chunk = data[offset : offset + size]
receipts.append(
{
"window_id": f"htrf_win_{idx:02d}",
"offset": offset,
"window_size": size,
"actual_byte_length": len(chunk),
"sha256": sha256_bytes(chunk),
"baseline_matrix": baseline_matrix(chunk),
}
)
return receipts
def negative_controls(data: bytes) -> list[dict[str, Any]]:
controls = [
("reverse_bytes", data[::-1], "order inversion control"),
("zero_bytes", bytes(len(data)), "symbol erasure control"),
("stride_permutation", data[1::2] + data[0::2], "local adjacency break control"),
]
return [
{
"control_id": control_id,
"purpose": purpose,
"byte_length": len(payload),
"sha256": sha256_bytes(payload),
"baseline_matrix": baseline_matrix(payload),
"decision": "HOLD",
}
for control_id, payload, purpose in controls
]
def build_manifest() -> dict[str, Any]:
FIXTURE_DIR.mkdir(parents=True, exist_ok=True)
fixture_bytes = FIXTURE_TEXT.encode("utf-8")
FIXTURE_SOURCE.write_bytes(fixture_bytes)
return {
"schema": "hutter_transfer_readiness_fixture_manifest_v0",
"agent": "Agent 4: Hutter Transfer Readiness",
"fixture_id": "htrf_local_text_window_2026_05_09_v0",
"source_path": str(FIXTURE_SOURCE.relative_to(ROOT)),
"byte_length": len(fixture_bytes),
"sha256": sha256_bytes(fixture_bytes),
"offsets_and_window_sizes": window_receipts(fixture_bytes),
"decision_boundary": {
"allowed_decisions": ["ADMIT_FIXTURE", "HOLD", "QUARANTINE"],
"admit_fixture_requires": [
"fixture_id",
"source_path",
"byte_length",
"sha256",
"offsets_and_window_sizes",
"raw_zlib_lzma_current_wire_baseline_matrix",
"negative_controls",
"byte_exact_oisc_replay_fixture",
],
"hold_when": [
"baseline matrix is incomplete",
"negative controls are absent",
"OISC replay fixture has not passed byte-exact replay",
"later spectral run is requested before control registration",
],
"quarantine_when": [
"fixture bytes do not match recorded sha256",
"claim text exceeds fixture-readiness scope",
"decompressor replay is not byte exact",
],
},
"baseline_matrix": baseline_matrix(fixture_bytes),
"negative_controls": negative_controls(fixture_bytes),
"oisc_replay_fixture": {
"rust_test": "non_toy_transfer_fixture_replays_byte_exact",
"source": "5-Applications/compression-core/src/oisc.rs",
"expected_decision": "ADMIT_FIXTURE",
"wire_codec": "oisc-replay-v0.1",
"wire_byte_length": len(encode_oisc(fixture_bytes)),
"wire_sha256": sha256_bytes(encode_oisc(fixture_bytes)),
},
"eigenmass_readiness": {
"decision": "HOLD",
"reason": "fixture admission has byte provenance, negative controls, and byte-exact Rust OISC replay; fixture-level eigenmass remains HOLD until residual policy and control accounting close",
},
"decision": "ADMIT_FIXTURE",
}
def write_markdown(manifest: dict[str, Any]) -> None:
DOC_DIR.mkdir(parents=True, exist_ok=True)
DOC_PATH.write_text(
"\n".join(
[
"# Hutter Transfer Readiness Fixture Manifest",
"",
"Decision: `ADMIT_FIXTURE`",
"",
"This is a small fixture-readiness manifest. It records byte provenance,",
"window receipts, baseline sizes, negative controls, and OISC replay",
"requirements before any later spectral analysis can be considered.",
"",
"## Fixture",
"",
f"- Fixture id: `{manifest['fixture_id']}`",
f"- Source path: `{manifest['source_path']}`",
f"- Byte length: `{manifest['byte_length']}`",
f"- SHA-256: `{manifest['sha256']}`",
"",
"## Baseline Matrix",
"",
"| route | bytes | note |",
"| --- | ---: | --- |",
f"| raw | {manifest['baseline_matrix']['raw']['byte_length']} | source bytes |",
f"| zlib | {manifest['baseline_matrix']['zlib']['byte_length']} | stdlib level 9 |",
f"| lzma | {manifest['baseline_matrix']['lzma']['byte_length']} | stdlib preset 6 |",
f"| current-wire | {manifest['baseline_matrix']['current_wire']['byte_length']} | OISC replay wire |",
"",
"## Controls",
"",
"Negative controls are registered as `HOLD` before any later result gate.",
"",
"## Receipt",
"",
f"- Manifest: `{MANIFEST_PATH.relative_to(ROOT)}`",
f"- OISC Rust test: `{manifest['oisc_replay_fixture']['rust_test']}`",
"",
]
),
encoding="utf-8",
)
def write_tiddler(manifest: dict[str, Any]) -> None:
TIDDLER_DIR.mkdir(parents=True, exist_ok=True)
TIDDLER_PATH.write_text(
"\n".join(
[
"created: 20260509235900000",
"modified: 20260509235900000",
"tags: ResearchStack Hutter Compression Fixture",
"title: Hutter Transfer Readiness Fixture",
"type: text/vnd.tiddlywiki",
"",
"! Hutter Transfer Readiness Fixture",
"",
f"* Decision: `ADMIT_FIXTURE`",
f"* Fixture id: `{manifest['fixture_id']}`",
f"* Source: `{manifest['source_path']}`",
f"* Byte length: `{manifest['byte_length']}`",
f"* SHA-256: `{manifest['sha256']}`",
f"* Manifest: `{MANIFEST_PATH.relative_to(ROOT)}`",
"",
"!! Boundary",
"",
"Allowed decisions are `ADMIT_FIXTURE`, `HOLD`, and `QUARANTINE`.",
"This tiddler admits only the local fixture manifest and keeps later",
"eigenmass or spectral analysis on `HOLD` until residual policy and",
"control accounting close. Negative controls and byte-exact Rust OISC",
"replay are registered here for fixture admission only.",
"",
]
),
encoding="utf-8",
)
def main() -> None:
manifest = build_manifest()
DATA_DIR.mkdir(parents=True, exist_ok=True)
MANIFEST_PATH.write_text(json.dumps(manifest, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_markdown(manifest)
write_tiddler(manifest)
print(json.dumps({"decision": manifest["decision"], "manifest": str(MANIFEST_PATH)}, indent=2))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,174 @@
#!/usr/bin/env python3
"""Create HOLD manifests for network topology coefficients and predictions."""
from __future__ import annotations
import json
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
DB = REPO / "shared-data" / "network_topology_database.json"
OUT_DIR = REPO / "shared-data" / "data" / "stack_solidification"
COEFF_OUT = OUT_DIR / "network_topology_coefficient_calibration_manifest.json"
PRED_OUT = OUT_DIR / "network_topology_prediction_hold_registry.json"
DOC = REPO / "6-Documentation" / "docs" / "network_topology_hold_manifests_2026-05-09.md"
def load_json(path: Path) -> Any:
return json.loads(path.read_text(encoding="utf-8"))
def coeff_rows(db: dict[str, Any]) -> list[dict[str, Any]]:
rows = []
raw = db.get("fundamental_equation_data", {}).get("methodology_weights", {})
reweighted = (
db.get("fundamental_equation_data", {})
.get("receipt_reweighted_methodology_weights", {})
.get("methodology_weights", {})
)
for key, item in sorted(raw.items()):
receipt = reweighted.get(key, {})
rows.append(
{
"methodology": key,
"raw_weight": item.get("weight"),
"alignment_score": item.get("alignment_score"),
"validation_status_label": item.get("validation_status"),
"receipt_multiplier": receipt.get("receipt_multiplier"),
"receipt_reweighted_weight": receipt.get("receipt_reweighted_weight"),
"decision": receipt.get("decision"),
"status": "HOLD_CALIBRATION",
"closure_gate": [
"source dataset receipt exists",
"calibration target and loss function are declared",
"sensitivity sweep passes",
"negative control shows coefficient is not arbitrary fit",
],
}
)
return rows
def prediction_rows(db: dict[str, Any]) -> list[dict[str, Any]]:
rows = []
for item in db.get("predicted_network_nodes", []):
rows.append(
{
"prediction_id": f"node::{item.get('provider')}::{item.get('location')}",
"kind": "predicted_network_node",
"provider": item.get("provider"),
"location": item.get("location"),
"prediction_reason": item.get("prediction_reason"),
"confidence": item.get("confidence"),
"validation_status_label": item.get("validation_status"),
"validation_score": item.get("validation_score"),
"status": "HOLD_PREDICTION_VALIDATION",
"closure_gate": [
"pre-registration timestamp and immutable hash exist",
"public/independent observation source is named",
"outcome comparison receipt exists",
"false-positive and null-baseline comparison are recorded",
],
}
)
for item in db.get("novel_network_paths", []):
rows.append(
{
"prediction_id": f"path::{item.get('source')}::{item.get('destination')}",
"kind": "novel_network_path",
"source": item.get("source"),
"destination": item.get("destination"),
"prediction_confidence": item.get("prediction_confidence"),
"path_quality": item.get("path_quality"),
"novelty_status_label": item.get("novelty_status"),
"status": "HOLD_PREDICTION_VALIDATION",
"closure_gate": [
"pre-registration timestamp and immutable hash exist",
"independent topology/source map is named",
"outcome comparison receipt exists",
"known-connection baseline comparison is recorded",
],
}
)
return rows
def build_doc(coeff: dict[str, Any], pred: dict[str, Any]) -> str:
lines = [
"# Network Topology HOLD Manifests",
"",
"**Date:** 2026-05-09",
"",
"These manifests separate hypothesis weights and predictions from calibrated or validated claims.",
"",
"## Coefficient Calibration",
"",
f"- Rows: `{coeff['summary']['row_count']}`",
f"- Status: `{coeff['summary']['status']}`",
f"- Receipt: `{COEFF_OUT.relative_to(REPO)}`",
"",
"| Methodology | Raw Weight | Receipt Weight | Decision |",
"| --- | ---: | ---: | --- |",
]
for row in coeff["rows"]:
lines.append(
f"| `{row['methodology']}` | {row['raw_weight']} | {row['receipt_reweighted_weight']} | `{row['decision']}` |"
)
lines.extend(
[
"",
"## Prediction HOLD Registry",
"",
f"- Rows: `{pred['summary']['row_count']}`",
f"- Status: `{pred['summary']['status']}`",
f"- Receipt: `{PRED_OUT.relative_to(REPO)}`",
"",
"| Prediction | Kind | Status |",
"| --- | --- | --- |",
]
)
for row in pred["rows"]:
lines.append(f"| `{row['prediction_id']}` | `{row['kind']}` | `{row['status']}` |")
lines.extend(
[
"",
"## Claim Boundary",
"",
"These rows are accounting and validation queues. They do not calibrate coefficients or validate topology predictions.",
]
)
return "\n".join(lines) + "\n"
def main() -> int:
db = load_json(DB)
now = datetime.now(timezone.utc).isoformat()
coeff = {
"schema": "network_topology_coefficient_calibration_manifest_v1",
"created_utc": now,
"source_database": str(DB.relative_to(REPO)),
"claim_boundary": "Coefficient HOLD manifest only. Raw and receipt-reweighted weights remain hypothesis/accounting surfaces until calibration gates close.",
"summary": {"row_count": len(coeff_rows(db)), "status": "HOLD_CALIBRATION"},
"rows": coeff_rows(db),
}
pred = {
"schema": "network_topology_prediction_hold_registry_v1",
"created_utc": now,
"source_database": str(DB.relative_to(REPO)),
"claim_boundary": "Prediction HOLD registry only. Entries are not validated claims until pre-registration and independent outcome receipts close.",
"summary": {"row_count": len(prediction_rows(db)), "status": "HOLD_PREDICTION_VALIDATION"},
"rows": prediction_rows(db),
}
OUT_DIR.mkdir(parents=True, exist_ok=True)
COEFF_OUT.write_text(json.dumps(coeff, indent=2, sort_keys=True), encoding="utf-8")
PRED_OUT.write_text(json.dumps(pred, indent=2, sort_keys=True), encoding="utf-8")
DOC.write_text(build_doc(coeff, pred), encoding="utf-8")
print(json.dumps({"coefficient_manifest": str(COEFF_OUT.relative_to(REPO)), "prediction_registry": str(PRED_OUT.relative_to(REPO)), "doc": str(DOC.relative_to(REPO))}, indent=2))
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,148 @@
#!/usr/bin/env python3
"""Generate closure checklists for Rainbow Raccoon Compiler HOLD objects."""
from __future__ import annotations
import json
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
COMPILER_RECEIPT = REPO / "4-Infrastructure" / "shim" / "rainbow_raccoon_compiler_receipt.json"
OUT_DIR = REPO / "shared-data" / "data" / "stack_solidification"
OUT = OUT_DIR / "rrc_hold_closure_checklist.json"
DOC = REPO / "6-Documentation" / "docs" / "rrc_hold_closure_checklist_2026-05-09.md"
AXIS_CLOSURE = {
"lean_or_independent_replay_gate": {
"gate": "Lean theorem, native_decide witness, or independent replay receipt exists",
"action": "add a replay harness or Lean theorem for the object-specific invariant",
},
"scale_band_declared": {
"gate": "scale/range band is explicit and checked",
"action": "declare numeric scale band, unit domain, and overflow behavior",
},
"projection_declared": {
"gate": "projection map from source object to manifold axes exists",
"action": "define projection axes and hash the projection payload",
},
"witness_declared": {
"gate": "witness payload and verifier are named",
"action": "attach witness schema, verifier command, and receipt path",
},
"decoder_declared": {
"gate": "decoder/replay route is named",
"action": "attach decoder route and exact replay hash",
},
}
def load_json(path: Path) -> Any:
return json.loads(path.read_text(encoding="utf-8"))
def checklist_item(obj: dict[str, Any]) -> dict[str, Any]:
source = obj["object"]
witness = obj["type_witness"]
missing = witness.get("missing_or_weak_axes", [])
closures = []
for axis in missing:
spec = AXIS_CLOSURE.get(
axis,
{
"gate": f"{axis} closed with explicit receipt",
"action": f"add closure evidence for {axis}",
},
)
closures.append({"axis": axis, **spec, "status": "OPEN"})
return {
"object_id": source["object_id"],
"label": source["label"],
"kind": source["kind"],
"source_path": source["source_path"],
"payload_sha256": source["payload_sha256"],
"nearest_shape": obj["nearest_lawful_shape"]["shape"],
"status": witness["status"],
"lean_boundary": witness.get("lean_boundary"),
"closures": closures,
"promotion_rule": "Remain HOLD until every closure status is CLOSED and the compiler rerun emits CANDIDATE.",
}
def build_doc(receipt: dict[str, Any]) -> str:
lines = [
"# RRC HOLD Closure Checklist",
"",
"**Date:** 2026-05-09",
"",
"This document gives each Rainbow Raccoon Compiler HOLD object a concrete closure checklist.",
"",
"## Summary",
"",
f"- Compiler receipt hash: `{receipt['compiler_receipt_hash']}`",
f"- HOLD objects: `{receipt['summary']['hold_count']}`",
f"- Candidate objects: `{receipt['summary']['candidate_count']}`",
f"- Open closure items: `{receipt['summary']['open_closure_count']}`",
"",
"## HOLD Objects",
"",
]
for item in receipt["hold_objects"]:
lines.append(f"### `{item['object_id']}` {item['label']}")
lines.append("")
lines.append(f"- Shape: `{item['nearest_shape']}`")
lines.append(f"- Source: `{item['source_path']}`")
lines.append(f"- Payload SHA-256: `{item['payload_sha256']}`")
lines.append(f"- Lean boundary: `{item['lean_boundary']}`")
for closure in item["closures"]:
lines.append(f"- OPEN `{closure['axis']}`: {closure['gate']}")
lines.append(f"- Action: {closure['action']}")
lines.append(f"- Promotion rule: {item['promotion_rule']}")
lines.append("")
lines.append("## Candidate Objects")
lines.append("")
for item in receipt["candidate_objects"]:
lines.append(f"- `{item['object_id']}` {item['label']} -> `{item['nearest_shape']}`")
lines.append("")
lines.append("## Machine Receipt")
lines.append("")
lines.append(f"- `{OUT.relative_to(REPO)}`")
return "\n".join(lines) + "\n"
def main() -> int:
compiler = load_json(COMPILER_RECEIPT)
hold_objects = []
candidate_objects = []
for obj in compiler.get("compiled_objects", []):
status = obj.get("type_witness", {}).get("status")
if status == "HOLD":
hold_objects.append(checklist_item(obj))
elif status == "CANDIDATE":
candidate_objects.append(checklist_item(obj))
receipt = {
"schema": "rrc_hold_closure_checklist_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"compiler_receipt": str(COMPILER_RECEIPT.relative_to(REPO)),
"compiler_receipt_hash": compiler.get("receipt_hash"),
"claim_boundary": "Checklist only. This does not close HOLD objects or promote compiler candidates.",
"summary": {
"hold_count": len(hold_objects),
"candidate_count": len(candidate_objects),
"open_closure_count": sum(len(item["closures"]) for item in hold_objects),
},
"hold_objects": hold_objects,
"candidate_objects": candidate_objects,
}
OUT_DIR.mkdir(parents=True, exist_ok=True)
OUT.write_text(json.dumps(receipt, indent=2, sort_keys=True), encoding="utf-8")
DOC.write_text(build_doc(receipt), encoding="utf-8")
print(json.dumps({"receipt": str(OUT.relative_to(REPO)), "doc": str(DOC.relative_to(REPO)), "open_closures": receipt["summary"]["open_closure_count"]}, indent=2))
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,495 @@
#!/usr/bin/env python3
"""Tri-cycle audit for prover, Rainbow Raccoon Compiler, and FPGA witness lanes.
The audit is intentionally conservative. It does not promote claims. It finds
HOLD/weak surfaces, reruns the available proof/compiler/witness gates, and emits
a receipt describing which parts are still blocked.
"""
from __future__ import annotations
import argparse
import json
import subprocess
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
SHIM = REPO / "4-Infrastructure" / "shim"
OUT_DIR = REPO / "shared-data" / "data" / "rrc_tri_cycle_audit"
OUT = OUT_DIR / "rrc_tri_cycle_audit_receipt.json"
DOC = REPO / "6-Documentation" / "docs" / "rrc_tri_cycle_audit_2026-05-09.md"
TARGET_FILES = [
"6-Documentation/wiki/Network-Topology-Theory.md",
"3-Mathematical-Models/fiber_optic_vibrational_tensor/Fundamental_Network_Topology_Equation.md",
"shared-data/network_topology_database.json",
"6-Documentation/docs/fpga_rrc_q16_accel_setup_2026-05-09.md",
]
MARKERS = [
"HOLD",
"HOLD_SECURITY_PROOF_DEBT",
"HOLD_COEFFICIENT_RECEIPT_DEBT",
"HOLD_ANALOGY_ADAPTER",
"HOLD_TOPOLOGY_PREDICTION_VALIDATION",
"not ready",
"unverified",
"USB-UART",
"CRC check : FAIL",
"does not validate",
"not a validation claim",
"receipt_path_exists",
]
GATE_REGISTER: dict[str, dict[str, Any]] = {
"security_proof_debt": {
"status": "BLOCK_PROMOTION",
"closure_required": [
"formal independence/freshness theorem for adaptive masks",
"secret-sharing non-reuse receipt",
"negative control showing adapted coefficients do not leak party inputs",
],
"allowed_use": "design hypothesis and simulation only",
},
"coefficient_or_calibration_debt": {
"status": "BLOCK_NUMERIC_CLAIMS",
"closure_required": [
"dataset provenance receipt",
"coefficient calibration receipt",
"negative controls and sensitivity sweep",
],
"allowed_use": "receipt-weighted prior accounting only",
},
"topology_prediction_debt": {
"status": "BLOCK_VALIDATION_CLAIMS",
"closure_required": [
"pre-registered prediction target",
"outcome receipt",
"independent public-map or measurement comparison",
],
"allowed_use": "HOLD topology hypothesis only",
},
"receipt_gate_debt": {
"status": "BLOCK_ROUTE_PROMOTION",
"closure_required": [
"validation receipt exists",
"rollback hash exists",
"exact replay or decode closure hash matches",
],
"allowed_use": "audit queue only",
},
"fpga_transport_or_witness_debt": {
"status": "BLOCK_HARDWARE_ACCELERATION_CLAIMS",
"closure_required": [
"simple UART loopback passes on fabric pins",
"Q16 accelerator hardware receipts match software receipts",
"durable flash readback passes or SRAM-only boundary remains explicit",
],
"allowed_use": "software witness and SRAM-loaded bitstream development",
},
"general_hold_surface": {
"status": "BLOCK_PUBLIC_PROMOTION",
"closure_required": [
"bucket-specific gate assigned",
"source receipt linked",
"negative-control or replay evidence attached",
],
"allowed_use": "internal research map",
},
}
@dataclass
class CmdResult:
command: list[str]
cwd: str
returncode: int
stdout_tail: str
stderr_tail: str
def run_cmd(command: list[str], cwd: Path, timeout: int = 120) -> CmdResult:
proc = subprocess.run(
command,
cwd=cwd,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=timeout,
check=False,
)
return CmdResult(
command=command,
cwd=str(cwd.relative_to(REPO)),
returncode=proc.returncode,
stdout_tail=proc.stdout[-4000:],
stderr_tail=proc.stderr[-4000:],
)
def load_json(path: Path) -> Any:
return json.loads(path.read_text(encoding="utf-8"))
def load_uart_beacon_diagnostics() -> dict[str, Any]:
receipts = [
SHIM / "tang9k_uart_beacon_probe_receipt.json",
SHIM / "tang9k_uart_beacon_swapped_probe_receipt.json",
]
diagnostics: list[dict[str, Any]] = []
for path in receipts:
if not path.exists():
diagnostics.append({"receipt": str(path.relative_to(REPO)), "present": False})
continue
data = load_json(path)
if isinstance(data.get("ports"), dict):
port_rows = [
{
"port": port,
"byte_count": item.get("byte_count"),
"contains_expected": item.get("contains_expected"),
"contains_q16_ascii": "513136" in item.get("raw_hex", ""),
}
for port, item in data["ports"].items()
]
else:
port_rows = [
{
"port": item.get("port"),
"byte_count": item.get("byte_count"),
"contains_expected": item.get("contains_expected"),
"contains_q16_ascii": item.get("contains_q16_ascii"),
}
for item in data.get("results", [])
]
conclusion = data.get("conclusion")
if conclusion is None and port_rows:
conclusion = (
"PASS"
if any(port.get("contains_expected") for port in port_rows)
else "FAIL_NO_BEACON_ON_FTDI_INTERFACES"
)
diagnostics.append(
{
"receipt": str(path.relative_to(REPO)),
"present": True,
"schema": data.get("schema"),
"constraints": data.get("constraints"),
"bitstream_sha256": data.get("bitstream_sha256"),
"conclusion": conclusion,
"ports": port_rows,
}
)
return {
"receipts": diagnostics,
"any_beacon_seen": any(
any(port.get("contains_expected") for port in item.get("ports", []))
for item in diagnostics
if item.get("present")
),
}
def scan_less_solid_surfaces() -> list[dict[str, Any]]:
findings: list[dict[str, Any]] = []
for rel in TARGET_FILES:
path = REPO / rel
if not path.exists():
continue
for lineno, line in enumerate(path.read_text(encoding="utf-8", errors="replace").splitlines(), 1):
marker_hits = [marker for marker in MARKERS if marker.lower() in line.lower()]
if not marker_hits:
continue
findings.append(
{
"path": rel,
"line": lineno,
"markers": marker_hits,
"text": line.strip()[:280],
"bucket": bucket_for_line(line),
}
)
return findings
def bucket_for_line(line: str) -> str:
lower = line.lower()
if "security" in lower or "privacy" in lower or "beaver" in lower:
return "security_proof_debt"
if "coefficient" in lower or "calibration" in lower or "weight" in lower:
return "coefficient_or_calibration_debt"
if "prediction" in lower or "validation" in lower or "outcome" in lower:
return "topology_prediction_debt"
if "fpga" in lower or "usb-uart" in lower or "crc" in lower or "q16" in lower:
return "fpga_transport_or_witness_debt"
if "receipt" in lower:
return "receipt_gate_debt"
return "general_hold_surface"
def run_q16_witnesses(include_hardware: bool, hardware_port: str | None) -> dict[str, Any]:
cases = [
(
"shift",
[
"python3",
"4-Infrastructure/shim/tang9k_rrc_q16_accel.py",
"--op",
"shift",
"--x",
"0x00038000",
],
),
(
"weighted",
[
"python3",
"4-Infrastructure/shim/tang9k_rrc_q16_accel.py",
"--op",
"weighted",
"--energy",
"0x000a0000",
"--alpha",
"0x00008000",
],
),
(
"monotone",
[
"python3",
"4-Infrastructure/shim/tang9k_rrc_q16_accel.py",
"--op",
"monotone",
"--a",
"0x00010000",
"--b",
"0x00030000",
],
),
]
results: dict[str, Any] = {"software": {}, "hardware": {}, "hardware_requested": include_hardware}
for name, cmd in cases:
software_out = SHIM / f"rrc_tri_cycle_{name}_software_receipt.json"
software_cmd = cmd + ["--out", str(software_out.relative_to(REPO))]
software = run_cmd(software_cmd, REPO)
results["software"][name] = {
"command": software.command,
"returncode": software.returncode,
"receipt": str(software_out.relative_to(REPO)),
"match": software.returncode == 0 and load_json(software_out).get("match") is True,
}
if include_hardware and hardware_port:
hardware_out = SHIM / f"rrc_tri_cycle_{name}_hardware_receipt.json"
hardware_cmd = cmd + [
"--port",
hardware_port,
"--retries",
"1",
"--out",
str(hardware_out.relative_to(REPO)),
]
hardware = run_cmd(hardware_cmd, REPO, timeout=20)
receipt = load_json(hardware_out) if hardware_out.exists() else {}
results["hardware"][name] = {
"command": hardware.command,
"returncode": hardware.returncode,
"receipt": str(hardware_out.relative_to(REPO)),
"match": hardware.returncode == 0 and receipt.get("match") is True,
"hardware_error": receipt.get("hardware_error"),
}
results["software_pass"] = all(v["match"] for v in results["software"].values())
results["hardware_pass"] = (
all(v["match"] for v in results["hardware"].values()) if results["hardware"] else None
)
return results
def summarize_rrc(receipt: dict[str, Any]) -> dict[str, Any]:
compiled = receipt.get("compiled_objects", [])
rows = []
for obj in compiled:
rows.append(
{
"object_id": obj["object"]["object_id"],
"label": obj["object"]["label"],
"shape": obj["nearest_lawful_shape"]["shape"],
"status": obj["type_witness"]["status"],
"missing_or_weak_axes": obj["type_witness"].get("missing_or_weak_axes", []),
"lean_boundary": obj["type_witness"].get("lean_boundary"),
}
)
return {
"receipt_hash": receipt.get("receipt_hash"),
"compiled_object_count": len(compiled),
"candidate_count": sum(1 for row in rows if row["status"] == "CANDIDATE"),
"hold_count": sum(1 for row in rows if row["status"] == "HOLD"),
"objects": rows,
}
def bucket_counts(findings: list[dict[str, Any]]) -> dict[str, int]:
counts: dict[str, int] = {}
for item in findings:
counts[item["bucket"]] = counts.get(item["bucket"], 0) + 1
return dict(sorted(counts.items(), key=lambda kv: (-kv[1], kv[0])))
def active_gate_register(findings: list[dict[str, Any]]) -> dict[str, Any]:
counts = bucket_counts(findings)
return {
bucket: GATE_REGISTER[bucket] | {"finding_count": count}
for bucket, count in counts.items()
if bucket in GATE_REGISTER
}
def build_doc(receipt: dict[str, Any]) -> str:
gates = receipt["gates"]
buckets = receipt["less_solid_surface_counts"]
lines = [
"# RRC Tri-Cycle Audit",
"",
"**Date:** 2026-05-09",
"",
"## Gates",
"",
f"- Prover gate: `{gates['prover']['status']}`",
f"- Compiler gate: `{gates['compiler']['status']}`",
f"- FPGA software witness gate: `{gates['fpga_witness']['software_status']}`",
f"- FPGA hardware witness gate: `{gates['fpga_witness']['hardware_status']}`",
f"- UART beacon diagnostic: `{'PASS' if gates['fpga_witness']['uart_beacon']['any_beacon_seen'] else 'FAIL_NO_BEACON'}`",
"",
"## Less Solid Buckets",
"",
]
for bucket, count in buckets.items():
gate = receipt["gate_register"].get(bucket, {})
lines.append(f"- `{bucket}`: {count} ({gate.get('status', 'NO_GATE')})")
lines.extend(
[
"",
"## Closure Gates",
"",
]
)
for bucket, gate in receipt["gate_register"].items():
lines.append(f"### `{bucket}`")
lines.append("")
lines.append(f"- Status: `{gate['status']}`")
lines.append(f"- Allowed use: {gate['allowed_use']}")
for req in gate["closure_required"]:
lines.append(f"- Requires: {req}")
lines.append("")
lines.extend(
[
"",
"## UART Beacon Diagnostics",
"",
]
)
for beacon in gates["fpga_witness"]["uart_beacon"]["receipts"]:
lines.append(f"- `{beacon['receipt']}`: `{beacon.get('conclusion', 'MISSING')}`")
for port in beacon.get("ports", []):
lines.append(
f"- Port `{port['port']}` byte_count={port['byte_count']} contains_expected={port['contains_expected']}"
)
lines.extend(
[
"",
"## Highest Priority Holds",
"",
]
)
for item in receipt["less_solid_surfaces"][:20]:
lines.append(
f"- `{item['bucket']}` [{item['path']}:{item['line']}](../../{item['path']}#L{item['line']}): {item['text']}"
)
lines.extend(
[
"",
"## Claim Boundary",
"",
"This audit does not promote any HOLD claim. It only checks which weak surfaces are currently covered by prover, compiler, and FPGA-witness receipts.",
]
)
return "\n".join(lines) + "\n"
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--include-hardware", action="store_true")
parser.add_argument("--hardware-port", default="/dev/ttyUSB1")
args = parser.parse_args()
OUT_DIR.mkdir(parents=True, exist_ok=True)
less_solid = scan_less_solid_surfaces()
prover = run_cmd(
["lake", "build", "Semantics.MetaManifoldProver"],
REPO / "0-Core-Formalism" / "lean" / "Semantics",
timeout=180,
)
compiler = run_cmd(["python3", "4-Infrastructure/shim/rainbow_raccoon_compiler.py"], REPO)
compiler_receipt = load_json(SHIM / "rainbow_raccoon_compiler_receipt.json")
q16 = run_q16_witnesses(args.include_hardware, args.hardware_port)
uart_beacon = load_uart_beacon_diagnostics()
receipt = {
"schema": "rrc_tri_cycle_audit_receipt_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"claim_boundary": (
"Tri-cycle audit only. Prover gate checks Lean build, compiler gate checks RRC HOLD/CANDIDATE "
"receipts, and FPGA gate checks Q16 witness harnesses. Hardware UART failures remain blockers."
),
"less_solid_surface_counts": bucket_counts(less_solid),
"gate_register": active_gate_register(less_solid),
"less_solid_surfaces": less_solid,
"gates": {
"prover": {
"status": "PASS" if prover.returncode == 0 else "FAIL",
"command": prover.command,
"returncode": prover.returncode,
"stdout_tail": prover.stdout_tail,
"stderr_tail": prover.stderr_tail,
},
"compiler": {
"status": "PASS_WITH_HOLDS" if compiler.returncode == 0 else "FAIL",
"command": compiler.command,
"returncode": compiler.returncode,
"stdout_tail": compiler.stdout_tail,
"stderr_tail": compiler.stderr_tail,
"summary": summarize_rrc(compiler_receipt),
},
"fpga_witness": {
"software_status": "PASS" if q16["software_pass"] else "FAIL",
"hardware_status": (
"PASS"
if q16["hardware_pass"] is True
else "FAIL"
if q16["hardware_pass"] is False
else "NOT_REQUESTED"
),
"q16": q16,
"uart_beacon": uart_beacon,
},
},
"promotion_decision": "NO_PROMOTION",
"next_actions": [
"Close or explicitly retain HOLD_SECURITY_PROOF_DEBT before treating adaptive Beaver coefficients as privacy-equivalent masks.",
"Treat coefficient and topology prediction rows as calibration debt until outcome receipts and negative controls close.",
"Fix Tang Nano 9K USB-UART fabric route before claiming live FPGA acceleration receipts; TX-only beacon receipts currently show no bytes on ttyUSB0 or ttyUSB1.",
"Use the Q16 lane as the first proof-backed compiler-to-FPGA witness once hardware transport responds.",
],
}
OUT.write_text(json.dumps(receipt, indent=2, sort_keys=True), encoding="utf-8")
DOC.write_text(build_doc(receipt), encoding="utf-8")
print(json.dumps({"receipt": str(OUT.relative_to(REPO)), "doc": str(DOC.relative_to(REPO)), "promotion_decision": "NO_PROMOTION"}, indent=2))
return 0 if prover.returncode == 0 and compiler.returncode == 0 and q16["software_pass"] else 1
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,205 @@
#!/usr/bin/env python3
"""Emit the tool-facing SMN naming boundary registry.
This keeps downstream shims from conflating:
- Semantic Mass: raw semantic/routing pressure
- SMN: Semantic Mass Number, a counted semantic-load index
- Mass Number: admissibility packet with residual and boundary guard
The registry is a routing/validation aid only. It does not promote SMN scores to
truth claims or Mass Number receipts.
"""
from __future__ import annotations
import hashlib
import json
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
OUT_DIR = REPO / "shared-data" / "data" / "stack_solidification"
REGISTRY = OUT_DIR / "smn_tool_awareness_registry.json"
RECEIPT = OUT_DIR / "smn_tool_awareness_receipt.json"
CANONICAL_FILES = [
REPO / "6-Documentation/docs/specs/SMN_SEMANTIC_MASS_NUMBERS.md",
REPO / "6-Documentation/tiddlywiki-local/wiki/tiddlers/Semantic Mass Numbers.tid",
REPO / "6-Documentation/tiddlywiki-local/wiki/tiddlers/Mass Number Theory.tid",
REPO / "6-Documentation/wiki/Concept-Archive.md",
REPO / "0-Core-Formalism/otom/docs/wiki/NotationNomenclatureRegistry.md",
REPO / "shared-data/data/stellar_gas_observation/sdss_manga_dr17_emission_line_channels.json",
]
def stable_json(obj: Any) -> str:
return json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True)
def sha256_bytes(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def hash_obj(obj: Any) -> str:
return sha256_bytes(stable_json(obj).encode("utf-8"))
def rel(path: Path) -> str:
return str(path.relative_to(REPO))
def file_entry(path: Path) -> dict[str, Any]:
return {
"path": rel(path),
"exists": path.exists(),
"sha256": sha256_bytes(path.read_bytes()) if path.exists() else None,
}
def load_json(path: Path) -> Any:
return json.loads(path.read_text(encoding="utf-8"))
def smn_data_gate() -> dict[str, Any]:
path = REPO / "shared-data/data/stellar_gas_observation/sdss_manga_dr17_emission_line_channels.json"
if not path.exists():
return {"status": "MISSING", "path": rel(path)}
data = load_json(path)
failures: list[dict[str, Any]] = []
channel_count = 0
ratio_count = 0
for kind, items in (("channel", data.get("channels", [])), ("diagnostic_ratio", data.get("diagnostic_ratios", []))):
for item in items:
payload = item.get("semantic_mass_number")
object_id = item.get("label") or item.get("id")
if not payload:
failures.append({"kind": kind, "object_id": object_id, "failure": "missing_semantic_mass_number"})
continue
components = payload.get("components", {})
expected = sum(int(value) for value in components.values())
if payload.get("smn") != expected:
failures.append({"kind": kind, "object_id": object_id, "failure": "component_sum_mismatch"})
if payload.get("not_atomic_mass_number") is not True:
failures.append({"kind": kind, "object_id": object_id, "failure": "missing_not_atomic_mass_number_guard"})
if payload.get("not_mass_number_receipt") is not True:
failures.append({"kind": kind, "object_id": object_id, "failure": "missing_not_mass_number_receipt_guard"})
if kind == "channel":
channel_count += 1
else:
ratio_count += 1
return {
"status": "PASS" if not failures else "FAIL",
"path": rel(path),
"channels_checked": channel_count,
"diagnostic_ratios_checked": ratio_count,
"failures": failures,
}
def nomenclature_gate() -> dict[str, Any]:
path = REPO / "0-Core-Formalism/otom/docs/wiki/NotationNomenclatureRegistry.md"
text = path.read_text(encoding="utf-8") if path.exists() else ""
required = [
"mass_number != SMN",
"SMN != atomic_mass_number",
"SMN != Mass_Number_receipt",
"| `smn` | Semantic Mass Number |",
"Do not use this as an alias for SMN",
]
missing = [item for item in required if item not in text]
forbidden = ["| `mass_number` | Mass Number | semantic mass number"]
forbidden_hits = [item for item in forbidden if item in text]
return {
"status": "PASS" if not missing and not forbidden_hits else "FAIL",
"path": rel(path),
"missing": missing,
"forbidden_hits": forbidden_hits,
}
def build_registry() -> dict[str, Any]:
registry = {
"schema": "smn_tool_awareness_registry_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"claim_boundary": "Tool awareness only. SMN routes attention; it is not physical mass, proof, truth, or a Mass Number admissibility receipt.",
"canonical_terms": [
{
"term": "Semantic Mass",
"tool_key": "semantic_mass",
"meaning": "raw semantic/routing pressure or burden",
"not_equal_to": ["SMN", "Mass Number", "physical mass"],
},
{
"term": "SMN",
"expanded": "Semantic Mass Number",
"tool_key": "smn",
"meaning": "countable project-local semantic-load number for a symbol, channel, ratio, route, or gate",
"formula": "SMN(x)=identity_load+relation_load+provenance_load+constraint_load+decision_load+repair_load",
"not_equal_to": ["atomic mass number", "isotope mass", "SI mass", "Mass Number admissibility packet", "proof", "truth"],
},
{
"term": "Mass Number",
"tool_key": "mass_number",
"meaning": "admissibility/accounting packet with residual and boundary guard",
"not_equal_to": ["SMN", "atomic mass number", "physical mass"],
},
],
"normalization_rules": [
{
"match": "semantic mass number",
"normalize_to": "SMN",
"reason": "Avoid aliasing Mass Number admissibility packets.",
},
{
"match": "mass_number",
"normalize_to": "Mass Number",
"reason": "Keep existing admissibility-gate code paths intact.",
},
{
"match": "semantic_mass_number",
"normalize_to": "SMN data payload",
"reason": "Machine-readable score object, not physical mass and not proof.",
},
],
"tool_rules": {
"search_index": "Index SMN as semantic-load terminology; do not merge with MassNumber gate entries.",
"tiddlywiki": "Link SMN references to [[Semantic Mass Numbers]] and Mass Number gate references to [[Mass Number Theory]].",
"equation_awareness": "Classify SMN formulas as semantic_load, not mass_number_transform.",
"stellar_gas_tools": "Read semantic_mass_number payloads as routing/audit priority metadata only.",
"promotion_gate": "High SMN may prioritize review but cannot admit without a receipt gate.",
},
"canonical_files": [file_entry(path) for path in CANONICAL_FILES],
"gates": {
"nomenclature": nomenclature_gate(),
"smn_data": smn_data_gate(),
},
}
registry["registry_hash"] = hash_obj({k: v for k, v in registry.items() if k != "registry_hash"})
return registry
def main() -> int:
OUT_DIR.mkdir(parents=True, exist_ok=True)
registry = build_registry()
status = "PASS" if all(gate.get("status") == "PASS" for gate in registry["gates"].values()) else "FAIL"
receipt = {
"schema": "smn_tool_awareness_receipt_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"decision": "ADMIT_SMN_TOOL_AWARENESS" if status == "PASS" else "HOLD_SMN_TOOL_AWARENESS",
"claim_boundary": registry["claim_boundary"],
"registry": rel(REGISTRY),
"registry_hash": registry["registry_hash"],
"status": status,
"gates": registry["gates"],
}
REGISTRY.write_text(json.dumps(registry, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print(json.dumps({"registry": rel(REGISTRY), "receipt": rel(RECEIPT), "status": status}, indent=2))
return 0 if status == "PASS" else 1
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,346 @@
#!/usr/bin/env python3
"""Create closure tickets for current stack failures and HOLD buckets."""
from __future__ import annotations
import json
import subprocess
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
TRI = REPO / "shared-data" / "data" / "rrc_tri_cycle_audit" / "rrc_tri_cycle_audit_receipt.json"
STACK = REPO / "shared-data" / "data" / "stack_solidification" / "stack_solidification_receipt.json"
OUT_DIR = REPO / "shared-data" / "data" / "stack_solidification"
OUT = OUT_DIR / "stack_fail_closure_register.json"
DOC = REPO / "6-Documentation" / "docs" / "stack_fail_closure_register_2026-05-09.md"
def load_json(path: Path) -> Any:
return json.loads(path.read_text(encoding="utf-8"))
def git_dirty_count() -> int | None:
proc = subprocess.run(
["git", "status", "--short", "--untracked-files=all"],
cwd=REPO,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
)
if proc.returncode != 0:
return None
return len([line for line in proc.stdout.splitlines() if line.strip()])
def ticket(
ticket_id: str,
title: str,
failure_class: str,
status: str,
evidence: list[str],
closure_gate: list[str],
next_action: list[str],
owner_surface: str,
) -> dict[str, Any]:
return {
"ticket_id": ticket_id,
"title": title,
"failure_class": failure_class,
"status": status,
"evidence": evidence,
"closure_gate": closure_gate,
"next_action": next_action,
"owner_surface": owner_surface,
}
def build_register() -> dict[str, Any]:
tri = load_json(TRI)
stack = load_json(STACK) if STACK.exists() else {}
sem_receipt_path = OUT_DIR / "external_sem_entity_diff_probe_receipt.json"
sem_receipt = load_json(sem_receipt_path) if sem_receipt_path.exists() else {}
hold_checklist_path = OUT_DIR / "rrc_hold_closure_checklist.json"
hold_checklist = load_json(hold_checklist_path) if hold_checklist_path.exists() else {}
coeff_manifest_path = OUT_DIR / "network_topology_coefficient_calibration_manifest.json"
coeff_manifest = load_json(coeff_manifest_path) if coeff_manifest_path.exists() else {}
pred_registry_path = OUT_DIR / "network_topology_prediction_hold_registry.json"
pred_registry = load_json(pred_registry_path) if pred_registry_path.exists() else {}
beaver_controls_path = OUT_DIR / "beaver_mask_freshness_negative_controls.json"
beaver_controls = load_json(beaver_controls_path) if beaver_controls_path.exists() else {}
virtual_serial_path = OUT_DIR / "tang9k_rrc_q16_virtual_serial_probe.json"
virtual_serial = load_json(virtual_serial_path) if virtual_serial_path.exists() else {}
transport_routes_path = OUT_DIR / "tang9k_uart_transport_routes.json"
transport_routes = load_json(transport_routes_path) if transport_routes_path.exists() else {}
less_solid = tri.get("less_solid_surface_counts", {})
gates = tri.get("gates", {})
fpga = gates.get("fpga_witness", {})
compiler_receipt_path = REPO / "4-Infrastructure" / "shim" / "rainbow_raccoon_compiler_receipt.json"
compiler_receipt = load_json(compiler_receipt_path) if compiler_receipt_path.exists() else {}
compiler = stack.get("gates", {}).get("compiler", {})
if not compiler:
objects = compiler_receipt.get("compiled_objects", [])
compiler = {
"receipt_hash": compiler_receipt.get("receipt_hash"),
"candidate_count": sum(
1 for obj in objects if obj.get("type_witness", {}).get("status") == "CANDIDATE"
),
"hold_count": sum(
1 for obj in objects if obj.get("type_witness", {}).get("status") == "HOLD"
),
}
worktree = stack.get("gates", {}).get("worktree", {})
if not worktree:
dirty_count = git_dirty_count()
worktree = {
"status": "DIRTY" if dirty_count else "CLEAN",
"tracked_or_untracked_count": dirty_count,
}
tickets = [
ticket(
"FAIL-FPGA-UART-001",
"Live fabric UART transport has no observable bytes",
"fpga_transport_or_witness_debt",
"BLOCKED",
[
"Q16 software witness passes",
"Q16 hardware witness returns short receipt frames",
"TX-only beacon standard pins produced zero bytes on ttyUSB0 and ttyUSB1",
"TX-only beacon swapped pins produced zero bytes on ttyUSB0 and ttyUSB1",
"Old faXX/ffXX direct-probe interpretation is superseded as bridge/MPSSE behavior, not fabric proof",
"FPGA UART route analysis identifies the onboard BL702 bridge route as blocked and recommends external USB-UART",
"Forced JTAG reset plus SRAM reload succeeds, but beacon/Q16 UART receipts remain empty",
"Loopback-after-JTAG-clear diagnostic produced faXX-style bytes on ttyUSB0, consistent with bridge/MPSSE behavior rather than a valid fabric receipt",
(
"PTY-backed virtual serial Q16 probe: "
f"{virtual_serial.get('summary', {}).get('status', 'not_run')} "
f"({virtual_serial.get('summary', {}).get('match_count', 'not_run')}/"
f"{virtual_serial.get('summary', {}).get('case_count', 'not_run')} matches)"
),
(
"UART transport router active route: "
f"{transport_routes.get('active_route', 'not_run')} "
f"({transport_routes.get('active_route_status', 'not_run')})"
),
],
[
"external USB-UART or verified onboard bridge captures beacon payload a6425131360a",
"loopback or beacon receipt passes before Q16 accelerator retry",
"Q16 hardware receipts match software receipts for shift, weighted, and monotone cases",
],
[
"Attach external USB-UART: adapter TX to fabric RX pin 18, adapter RX to fabric TX pin 17, and GND to GND",
"Probe the new adapter path, usually /dev/ttyUSB2 or /dev/ttyACM0, with the TX-only beacon before Q16",
"If external UART works, patch host default port or call scripts with --port and rerun Q16 hardware receipts",
"If external UART fails, inspect PNR pin placement and add LED-observed heartbeat fallback",
],
"6-Documentation/docs/fpga_uart_route_analysis_2026-05-09.md",
),
ticket(
"FAIL-FPGA-FLASH-002",
"Durable flash programming readback fails",
"fpga_transport_or_witness_debt",
"HELD_SRAM_ONLY",
[
"SRAM load passes CRC",
"Flash programming attempt had readback CRC failure",
],
[
"flash write and readback CRC pass for the exact Q16 bitstream",
"or documentation keeps SRAM-only boundary explicit",
],
[
"Keep SRAM-only claim boundary until flash command and board target are verified",
"Do not mark hardware install persistent",
],
"6-Documentation/docs/fpga_rrc_q16_accel_setup_2026-05-09.md",
),
ticket(
"FAIL-SECURITY-BEAVER-003",
"Adaptive Beaver coefficients are not privacy-equivalent masks yet",
"security_proof_debt",
"HOLD",
[
f"{less_solid.get('security_proof_debt', 0)} security proof debt surfaces found",
"tri-cycle audit blocks promotion for adaptive mask claims",
f"mask freshness negative controls: {beaver_controls.get('summary', {}).get('status', 'not_run')}",
f"mask freshness case count: {beaver_controls.get('summary', {}).get('case_count', 'not_run')}",
],
[
"formal independence/freshness theorem exists",
"secret-sharing non-reuse receipt exists",
"negative control shows adapted coefficients do not leak party inputs",
],
[
"Add a Lean-facing finite-state mask freshness model",
"Generate negative-control fixtures for repeated coefficients and topology-derived coefficients",
],
"Network-Topology-Theory.md + Fundamental_Network_Topology_Equation.md",
),
ticket(
"FAIL-COEFFICIENT-CALIBRATION-004",
"Numeric weights remain receipt-weighted priors, not calibrated coefficients",
"coefficient_or_calibration_debt",
"HOLD",
[
f"{less_solid.get('coefficient_or_calibration_debt', 0)} coefficient/calibration debt surfaces found",
"receipt reweighting exists, but coefficient calibration and negative controls remain open",
f"coefficient HOLD manifest rows: {coeff_manifest.get('summary', {}).get('row_count', 'not_run')}",
],
[
"dataset provenance receipt linked",
"coefficient calibration receipt linked",
"sensitivity sweep and negative controls pass",
],
[
"Create coefficient calibration fixture manifest",
"Separate hypothesis weights from calibrated weights in docs and JSON surfaces",
],
"shared-data/network_topology_database.json",
),
ticket(
"FAIL-TOPOLOGY-PREDICTION-005",
"Topology predictions are not validation claims",
"topology_prediction_debt",
"HOLD",
[
f"{less_solid.get('topology_prediction_debt', 0)} topology prediction debt surfaces found",
"tri-cycle audit requires pre-registered target and independent comparison",
f"prediction HOLD registry rows: {pred_registry.get('summary', {}).get('row_count', 'not_run')}",
],
[
"pre-registered prediction target exists",
"outcome receipt exists",
"independent public map or measurement comparison exists",
],
[
"Create prediction-target registry with timestamps and immutable receipt hashes",
"Move existing predicted nodes into HOLD prediction queue, not validation table",
],
"shared-data/network_topology_database.json",
),
ticket(
"FAIL-RECEIPT-GATE-006",
"Route promotion lacks complete receipt and rollback closure",
"receipt_gate_debt",
"HOLD",
[
f"{less_solid.get('receipt_gate_debt', 0)} receipt-gate debt surfaces found",
"compiler keeps 6 objects HOLD and only 1 candidate",
f"compiler receipt hash: {compiler.get('receipt_hash')}",
f"optional sem entity probe: {sem_receipt.get('decision', 'not_run')}",
(
"HOLD closure checklist: "
f"{hold_checklist.get('summary', {}).get('open_closure_count', 'not_run')} open items"
),
],
[
"validation receipt exists",
"rollback hash exists",
"exact replay or decode closure hash matches",
],
[
"Close every item in rrc_hold_closure_checklist.json before rerunning compiler promotion",
"Do not widen candidate admission beyond Q16 until Lean or independent replay closes",
],
"4-Infrastructure/shim/rainbow_raccoon_compiler.py",
),
ticket(
"FAIL-WORKTREE-SCOPE-007",
"Broad worktree is too dirty for safe sweep commit",
"release_hygiene_debt",
"BLOCKED_FOR_BROAD_STAGE",
[
f"worktree status: {worktree.get('status')}",
f"changed/untracked count: {worktree.get('tracked_or_untracked_count')}",
"staging manifest created for the solidification slice",
f"optional sem entity probe: {sem_receipt.get('decision', 'not_run')}",
],
[
"commit scope is explicit file list",
"generated artifacts are intentionally included or excluded",
"no unrelated modified Lean/doc/probe files are swept in",
"entity-level change list exists for staged or scoped files when sem is available",
],
[
"Use stack_solidification_staging_manifest_2026-05-09.md before any commit",
"Create separate manifests for CPU/logogram/wiki maturation slices if needed",
"Use /tmp/sem_probe/sem/crates/target/release/sem, not /usr/bin/sem, unless a durable sem binary is installed",
],
"6-Documentation/docs/stack_solidification_staging_manifest_2026-05-09.md",
),
]
return {
"schema": "stack_fail_closure_register_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"claim_boundary": "Closure register only. Tickets describe gates required to close failures; they do not claim closure.",
"source_receipts": [str(TRI.relative_to(REPO)), str(STACK.relative_to(REPO))],
"optional_tool_receipts": [str(sem_receipt_path.relative_to(REPO))] if sem_receipt_path.exists() else [],
"closure_receipts": [
str(path.relative_to(REPO))
for path in [hold_checklist_path, coeff_manifest_path, pred_registry_path, beaver_controls_path, virtual_serial_path, transport_routes_path]
if path.exists()
],
"summary": {
"ticket_count": len(tickets),
"blocked_or_hold_count": sum(1 for item in tickets if item["status"] in {"BLOCKED", "HOLD", "BLOCKED_FOR_BROAD_STAGE", "HELD_SRAM_ONLY"}),
"fpga_hardware_status": fpga.get("hardware_status"),
"fpga_software_status": fpga.get("software_status"),
"promotion_decision": tri.get("promotion_decision"),
},
"tickets": tickets,
}
def build_doc(register: dict[str, Any]) -> str:
lines = [
"# Stack Fail Closure Register",
"",
"**Date:** 2026-05-09",
"",
"This register turns current failures into closure gates. It does not mark them solved.",
"",
"## Summary",
"",
f"- Tickets: `{register['summary']['ticket_count']}`",
f"- Promotion decision: `{register['summary']['promotion_decision']}`",
f"- FPGA software status: `{register['summary']['fpga_software_status']}`",
f"- FPGA hardware status: `{register['summary']['fpga_hardware_status']}`",
"",
"## Tickets",
"",
]
for item in register["tickets"]:
lines.append(f"### `{item['ticket_id']}` {item['title']}")
lines.append("")
lines.append(f"- Class: `{item['failure_class']}`")
lines.append(f"- Status: `{item['status']}`")
lines.append(f"- Owner surface: `{item['owner_surface']}`")
for evidence in item["evidence"]:
lines.append(f"- Evidence: {evidence}")
for gate in item["closure_gate"]:
lines.append(f"- Closure gate: {gate}")
for action in item["next_action"]:
lines.append(f"- Next action: {action}")
lines.append("")
lines.append("## Machine Receipt")
lines.append("")
lines.append(f"- `{OUT.relative_to(REPO)}`")
return "\n".join(lines) + "\n"
def main() -> int:
OUT_DIR.mkdir(parents=True, exist_ok=True)
register = build_register()
OUT.write_text(json.dumps(register, indent=2, sort_keys=True), encoding="utf-8")
DOC.write_text(build_doc(register), encoding="utf-8")
print(json.dumps({"receipt": str(OUT.relative_to(REPO)), "doc": str(DOC.relative_to(REPO)), "tickets": len(register["tickets"])}, indent=2))
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,500 @@
#!/usr/bin/env python3
"""End-to-end stand-up audit for the Rainbow Raccoon stack.
This is a snapshot tool, not a promotion tool. It collects the current proof,
compiler, receipt, JSON, and hardware-boundary evidence into one receipt so the
stack can be stabilized without turning HOLD surfaces into claims.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import subprocess
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
OUT_DIR = REPO / "shared-data" / "data" / "stack_solidification"
OUT = OUT_DIR / "stack_solidification_receipt.json"
DOC = REPO / "6-Documentation" / "docs" / "stack_solidification_status_2026-05-09.md"
JSON_SURFACES = [
"shared-data/network_topology_database.json",
"4-Infrastructure/shim/rainbow_raccoon_compiler_receipt.json",
"shared-data/data/rrc_tri_cycle_audit/rrc_tri_cycle_audit_receipt.json",
"shared-data/data/stack_solidification/external_sem_entity_diff_probe_receipt.json",
"shared-data/data/stack_solidification/rrc_hold_closure_checklist.json",
"shared-data/data/stack_solidification/network_topology_coefficient_calibration_manifest.json",
"shared-data/data/stack_solidification/network_topology_prediction_hold_registry.json",
"shared-data/data/stack_solidification/beaver_mask_freshness_negative_controls.json",
"shared-data/data/stack_solidification/whitespace_zero_grammar_probe.json",
"shared-data/data/stack_solidification/tang9k_rrc_q16_virtual_serial_probe.json",
"shared-data/data/stack_solidification/tang9k_uart_transport_routes.json",
"shared-data/data/stack_solidification/stack_fail_closure_register.json",
"shared-data/data/stack_solidification/smn_tool_awareness_registry.json",
"shared-data/data/stack_solidification/smn_tool_awareness_receipt.json",
"4-Infrastructure/shim/tang9k_uart_beacon_probe_receipt.json",
"4-Infrastructure/shim/tang9k_uart_beacon_swapped_probe_receipt.json",
"4-Infrastructure/shim/tang9k_uart_loopback_after_jtag_clear_probe_receipt.json",
]
PYTHON_SURFACES = [
"4-Infrastructure/shim/external_sem_entity_diff_probe.py",
"4-Infrastructure/shim/beaver_mask_freshness_negative_controls.py",
"4-Infrastructure/shim/whitespace_zero_grammar_probe.py",
"4-Infrastructure/shim/network_topology_hold_manifests.py",
"4-Infrastructure/shim/rainbow_raccoon_compiler.py",
"4-Infrastructure/shim/rrc_hold_closure_checklist.py",
"4-Infrastructure/shim/rrc_tri_cycle_audit.py",
"4-Infrastructure/shim/stack_fail_closure_register.py",
"4-Infrastructure/shim/tang9k_rrc_q16_accel.py",
"4-Infrastructure/shim/tang9k_rrc_q16_virtual_serial_probe.py",
"4-Infrastructure/shim/tang9k_uart_transport_router.py",
"4-Infrastructure/shim/tang9k_uart_beacon_probe.py",
"4-Infrastructure/shim/smn_tool_awareness_registry.py",
"4-Infrastructure/shim/enwiki9_logogram_receipt_aggregation_probe.py",
"4-Infrastructure/shim/stack_solidification_audit.py",
]
@dataclass
class CmdResult:
command: list[str]
cwd: str
returncode: int
stdout_tail: str
stderr_tail: str
def rel(path: Path) -> str:
return str(path.relative_to(REPO))
def run_cmd(command: list[str], cwd: Path, timeout: int = 300) -> CmdResult:
proc = subprocess.run(
command,
cwd=cwd,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=timeout,
check=False,
)
return CmdResult(
command=command,
cwd=rel(cwd),
returncode=proc.returncode,
stdout_tail=proc.stdout[-6000:],
stderr_tail=proc.stderr[-6000:],
)
def load_json(path: Path) -> Any:
return json.loads(path.read_text(encoding="utf-8"))
def file_sha256(path: Path) -> str | None:
if not path.exists():
return None
h = hashlib.sha256()
with path.open("rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def json_gate() -> dict[str, Any]:
rows = []
for item in JSON_SURFACES:
path = REPO / item
try:
data = load_json(path)
rows.append(
{
"path": item,
"status": "PASS",
"top_level_type": type(data).__name__,
"sha256": file_sha256(path),
}
)
except Exception as exc: # pragma: no cover - diagnostic path
rows.append({"path": item, "status": "FAIL", "error": repr(exc)})
return {"status": "PASS" if all(row["status"] == "PASS" for row in rows) else "FAIL", "rows": rows}
def py_compile_gate() -> dict[str, Any]:
cmd = ["python3", "-m", "py_compile", *PYTHON_SURFACES]
result = run_cmd(cmd, REPO, timeout=120)
return {"status": "PASS" if result.returncode == 0 else "FAIL", "result": result.__dict__}
def refresh_support_receipts_gate() -> dict[str, Any]:
scripts = [
"4-Infrastructure/shim/network_topology_hold_manifests.py",
"4-Infrastructure/shim/beaver_mask_freshness_negative_controls.py",
"4-Infrastructure/shim/whitespace_zero_grammar_probe.py",
"4-Infrastructure/shim/tang9k_rrc_q16_virtual_serial_probe.py",
"4-Infrastructure/shim/tang9k_uart_transport_router.py",
"4-Infrastructure/shim/rrc_hold_closure_checklist.py",
]
rows = []
for script in scripts:
result = run_cmd(["python3", script], REPO, timeout=300)
rows.append(
{
"script": script,
"status": "PASS" if result.returncode == 0 else "FAIL",
"result": result.__dict__,
}
)
return {"status": "PASS" if all(row["status"] == "PASS" for row in rows) else "FAIL", "rows": rows}
def compiler_gate() -> dict[str, Any]:
result = run_cmd(["python3", "4-Infrastructure/shim/rainbow_raccoon_compiler.py"], REPO, timeout=120)
receipt_path = REPO / "4-Infrastructure" / "shim" / "rainbow_raccoon_compiler_receipt.json"
receipt = load_json(receipt_path) if receipt_path.exists() else {}
objects = receipt.get("compiled_objects", [])
return {
"status": "PASS_WITH_HOLDS" if result.returncode == 0 else "FAIL",
"result": result.__dict__,
"receipt": rel(receipt_path),
"receipt_hash": receipt.get("receipt_hash"),
"compiled_object_count": len(objects),
"candidate_count": sum(1 for obj in objects if obj.get("type_witness", {}).get("status") == "CANDIDATE"),
"hold_count": sum(1 for obj in objects if obj.get("type_witness", {}).get("status") == "HOLD"),
}
def tri_cycle_gate(include_hardware: bool, hardware_port: str) -> dict[str, Any]:
cmd = ["python3", "4-Infrastructure/shim/rrc_tri_cycle_audit.py"]
if include_hardware:
cmd.extend(["--include-hardware", "--hardware-port", hardware_port])
result = run_cmd(cmd, REPO, timeout=300)
receipt_path = REPO / "shared-data" / "data" / "rrc_tri_cycle_audit" / "rrc_tri_cycle_audit_receipt.json"
receipt = load_json(receipt_path) if receipt_path.exists() else {}
gates = receipt.get("gates", {})
fpga = gates.get("fpga_witness", {})
return {
"status": "PASS_WITH_BLOCKED_HARDWARE" if result.returncode == 0 and fpga.get("hardware_status") == "FAIL" else "PASS" if result.returncode == 0 else "FAIL",
"result": result.__dict__,
"receipt": rel(receipt_path),
"promotion_decision": receipt.get("promotion_decision"),
"prover": gates.get("prover", {}).get("status"),
"compiler": gates.get("compiler", {}).get("status"),
"fpga_software": fpga.get("software_status"),
"fpga_hardware": fpga.get("hardware_status"),
"uart_beacon_any": fpga.get("uart_beacon", {}).get("any_beacon_seen"),
"less_solid_surface_counts": receipt.get("less_solid_surface_counts", {}),
}
def lean_gate(run_full_lean: bool) -> dict[str, Any]:
if not run_full_lean:
return {"status": "SKIPPED", "reason": "run with --full-lean to refresh this gate"}
result = run_cmd(["lake", "build"], REPO / "0-Core-Formalism" / "lean" / "Semantics", timeout=600)
return {"status": "PASS" if result.returncode == 0 else "FAIL", "result": result.__dict__}
def hardware_bitstream_gate() -> dict[str, Any]:
surfaces = [
"4-Infrastructure/hardware/tangnano9k_rrc_q16_accel.fs",
"4-Infrastructure/hardware/tangnano9k_uart_beacon.fs",
]
return {
"status": "PASS" if all((REPO / p).exists() for p in surfaces) else "FAIL",
"bitstreams": [{"path": p, "sha256": file_sha256(REPO / p), "present": (REPO / p).exists()} for p in surfaces],
}
def external_sem_gate() -> dict[str, Any]:
path = REPO / "shared-data" / "data" / "stack_solidification" / "external_sem_entity_diff_probe_receipt.json"
if not path.exists():
return {"status": "NOT_RUN", "receipt": str(path.relative_to(REPO))}
receipt = load_json(path)
return {
"status": receipt.get("decision", "UNKNOWN"),
"receipt": str(path.relative_to(REPO)),
"gnu_parallel_collision": receipt.get("gnu_parallel_collision", {}).get("detected"),
"mapped_fail_tickets": receipt.get("mapped_fail_tickets", []),
}
def hold_checklist_gate() -> dict[str, Any]:
path = REPO / "shared-data" / "data" / "stack_solidification" / "rrc_hold_closure_checklist.json"
if not path.exists():
return {"status": "NOT_RUN", "receipt": str(path.relative_to(REPO))}
receipt = load_json(path)
return {
"status": "PASS_OPEN_ITEMS_TRACKED",
"receipt": str(path.relative_to(REPO)),
"hold_count": receipt.get("summary", {}).get("hold_count"),
"open_closure_count": receipt.get("summary", {}).get("open_closure_count"),
}
def network_hold_manifest_gate() -> dict[str, Any]:
coeff = REPO / "shared-data" / "data" / "stack_solidification" / "network_topology_coefficient_calibration_manifest.json"
pred = REPO / "shared-data" / "data" / "stack_solidification" / "network_topology_prediction_hold_registry.json"
if not coeff.exists() or not pred.exists():
return {"status": "NOT_RUN", "coefficient_manifest": str(coeff.relative_to(REPO)), "prediction_registry": str(pred.relative_to(REPO))}
coeff_data = load_json(coeff)
pred_data = load_json(pred)
return {
"status": "PASS_HOLD_QUEUES_DECLARED",
"coefficient_manifest": str(coeff.relative_to(REPO)),
"prediction_registry": str(pred.relative_to(REPO)),
"coefficient_rows": coeff_data.get("summary", {}).get("row_count"),
"prediction_rows": pred_data.get("summary", {}).get("row_count"),
}
def beaver_mask_freshness_gate() -> dict[str, Any]:
path = REPO / "shared-data" / "data" / "stack_solidification" / "beaver_mask_freshness_negative_controls.json"
if not path.exists():
return {"status": "NOT_RUN", "receipt": str(path.relative_to(REPO))}
receipt = load_json(path)
return {
"status": receipt.get("summary", {}).get("status", "UNKNOWN"),
"receipt": str(path.relative_to(REPO)),
"lean_build": receipt.get("lean_build", {}).get("status"),
"case_count": receipt.get("summary", {}).get("case_count"),
"negative_control_count": receipt.get("summary", {}).get("negative_control_count"),
"promotion_effect": receipt.get("promotion_effect"),
}
def whitespace_zero_grammar_gate() -> dict[str, Any]:
path = REPO / "shared-data" / "data" / "stack_solidification" / "whitespace_zero_grammar_probe.json"
if not path.exists():
return {"status": "NOT_RUN", "receipt": str(path.relative_to(REPO))}
receipt = load_json(path)
return {
"status": receipt.get("summary", {}).get("status", "UNKNOWN"),
"receipt": str(path.relative_to(REPO)),
"lean_build": receipt.get("lean_build", {}).get("status"),
"case_count": receipt.get("summary", {}).get("case_count"),
"admit_count": receipt.get("summary", {}).get("admit_count"),
"hold_count": receipt.get("summary", {}).get("hold_count"),
"stored_whitespace_codes_total": receipt.get("summary", {}).get("stored_whitespace_codes_total"),
}
def virtual_serial_gate() -> dict[str, Any]:
path = REPO / "shared-data" / "data" / "stack_solidification" / "tang9k_rrc_q16_virtual_serial_probe.json"
if not path.exists():
return {"status": "NOT_RUN", "receipt": str(path.relative_to(REPO))}
receipt = load_json(path)
return {
"status": receipt.get("summary", {}).get("status", "UNKNOWN"),
"receipt": str(path.relative_to(REPO)),
"case_count": receipt.get("summary", {}).get("case_count"),
"match_count": receipt.get("summary", {}).get("match_count"),
"frames_seen": receipt.get("summary", {}).get("frames_seen"),
}
def uart_transport_routes_gate() -> dict[str, Any]:
path = REPO / "shared-data" / "data" / "stack_solidification" / "tang9k_uart_transport_routes.json"
if not path.exists():
return {"status": "NOT_RUN", "receipt": str(path.relative_to(REPO))}
receipt = load_json(path)
return {
"status": receipt.get("active_route_status", "UNKNOWN"),
"receipt": str(path.relative_to(REPO)),
"active_route": receipt.get("active_route"),
"route_count": len(receipt.get("route_table", [])),
"virtual_status": receipt.get("virtual_probe", {}).get("status"),
}
def stack_fail_closure_gate() -> dict[str, Any]:
result = run_cmd(["python3", "4-Infrastructure/shim/stack_fail_closure_register.py"], REPO, timeout=120)
path = REPO / "shared-data" / "data" / "stack_solidification" / "stack_fail_closure_register.json"
receipt = load_json(path) if path.exists() else {}
return {
"status": "PASS_TICKETS_DECLARED" if result.returncode == 0 else "FAIL",
"receipt": str(path.relative_to(REPO)),
"ticket_count": receipt.get("summary", {}).get("ticket_count"),
"blocked_or_hold_count": receipt.get("summary", {}).get("blocked_or_hold_count"),
"fpga_hardware_status": receipt.get("summary", {}).get("fpga_hardware_status"),
"promotion_decision": receipt.get("summary", {}).get("promotion_decision"),
"result": result.__dict__,
}
def smn_tool_awareness_gate() -> dict[str, Any]:
result = run_cmd(["python3", "4-Infrastructure/shim/smn_tool_awareness_registry.py"], REPO, timeout=120)
path = REPO / "shared-data" / "data" / "stack_solidification" / "smn_tool_awareness_receipt.json"
receipt = load_json(path) if path.exists() else {}
return {
"status": receipt.get("status", "FAIL" if result.returncode else "UNKNOWN"),
"decision": receipt.get("decision"),
"receipt": str(path.relative_to(REPO)),
"nomenclature": receipt.get("gates", {}).get("nomenclature", {}).get("status"),
"smn_data": receipt.get("gates", {}).get("smn_data", {}).get("status"),
"result": result.__dict__,
}
def worktree_gate() -> dict[str, Any]:
proc = subprocess.run(
["git", "status", "--short"],
cwd=REPO,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=120,
check=False,
)
counts: dict[str, int] = {}
total = 0
for line in proc.stdout.splitlines():
if not line.strip():
continue
total += 1
key = line[:2]
counts[key] = counts.get(key, 0) + 1
return {
"status": "DIRTY" if total else "CLEAN",
"tracked_or_untracked_count": total,
"status_prefix_counts": dict(sorted(counts.items())),
"note": "Broad working tree is dirty; stage only scoped artifacts.",
}
def build_doc(receipt: dict[str, Any]) -> str:
gates = receipt["gates"]
tri = gates["tri_cycle"]
lines = [
"# Stack Solidification Status",
"",
"**Date:** 2026-05-09",
"",
"## Bottom Line",
"",
"The stack is buildable and internally gateable, but not promotable as a live hardware-accelerated system yet.",
"",
"## Gates",
"",
f"- Full Lean/Semantics build: `{gates['lean']['status']}`",
f"- JSON integrity: `{gates['json']['status']}`",
f"- Python shim compile: `{gates['python_compile']['status']}`",
f"- Support receipt refresh: `{gates['support_receipts']['status']}`",
f"- Rainbow Raccoon compiler: `{gates['compiler']['status']}` ({gates['compiler']['candidate_count']} candidate, {gates['compiler']['hold_count']} HOLD)",
f"- Tri-cycle audit: `{tri['status']}`; promotion decision `{tri['promotion_decision']}`",
f"- FPGA software witness: `{tri['fpga_software']}`",
f"- FPGA hardware witness: `{tri['fpga_hardware']}`",
f"- UART beacon seen: `{tri['uart_beacon_any']}`",
f"- Hardware bitstreams present: `{gates['hardware_bitstreams']['status']}`",
f"- Optional sem entity-diff aid: `{gates['external_sem']['status']}`",
f"- RRC HOLD closure checklist: `{gates['hold_checklist']['status']}` ({gates['hold_checklist'].get('open_closure_count')} open)",
f"- Network HOLD manifests: `{gates['network_hold_manifests']['status']}` ({gates['network_hold_manifests'].get('coefficient_rows')} coefficient rows, {gates['network_hold_manifests'].get('prediction_rows')} prediction rows)",
f"- Beaver mask freshness controls: `{gates['beaver_mask_freshness']['status']}` ({gates['beaver_mask_freshness'].get('case_count')} cases)",
f"- Whitespace-zero grammar: `{gates['whitespace_zero_grammar']['status']}` ({gates['whitespace_zero_grammar'].get('admit_count')} admitted, {gates['whitespace_zero_grammar'].get('hold_count')} HOLD)",
f"- Q16 virtual serial probe: `{gates['virtual_serial']['status']}` ({gates['virtual_serial'].get('match_count')}/{gates['virtual_serial'].get('case_count')} matches)",
f"- UART transport routes: `{gates['uart_transport_routes']['status']}` (active `{gates['uart_transport_routes'].get('active_route')}`)",
f"- Stack fail closure register: `{gates['stack_fail_closure']['status']}` ({gates['stack_fail_closure'].get('ticket_count')} tickets)",
f"- SMN tool awareness: `{gates['smn_tool_awareness']['status']}` ({gates['smn_tool_awareness'].get('decision')})",
f"- Worktree: `{gates['worktree']['status']}`",
"- Staging manifest: `6-Documentation/docs/stack_solidification_staging_manifest_2026-05-09.md`",
"",
"## Current Solid Core",
"",
"- Lean/Semantics builds end to end.",
"- Core JSON receipts and network topology database parse.",
"- Compiler gate admits only the Q16 fixed-point lowering certificate as candidate.",
"- Q16 software witness lane passes.",
"- Q16 host UART framing and parser pass over a PTY-backed virtual serial device.",
"- UART route table now selects the PTY-backed Q16 route while keeping blocked physical routes visible.",
"- HOLD buckets are explicit rather than silently promoted.",
"- Optional sem entity extraction is available for scoped Python audit files.",
"- SMN is tool-visible as Semantic Mass Number and explicitly separated from Mass Number admissibility packets.",
"- Every compiler HOLD object now has an explicit closure checklist.",
"- Current failures and broad HOLD buckets have closure tickets in a stack fail register.",
"- Network topology coefficients and predictions are split into HOLD queues.",
"- Beaver mask freshness has Lean-backed finite negative controls; full MPC security remains HOLD.",
"- Canonical logogram grammar can derive ordinary spaces from symbol count/order with zero stored whitespace codes.",
"- Agent routing now has repo-root, Lean/Semantics, and Infrastructure contracts.",
"- Stack receipts and the network topology database are visible through narrow `.gitignore` exceptions instead of broad `shared-data/` exposure.",
"",
"## Current Blockers",
"",
"- Live FPGA UART transport remains blocked: beacon receipts show no bytes on `/dev/ttyUSB0` or `/dev/ttyUSB1`.",
"- Hardware acceleration claims remain blocked until the UART route or external adapter path produces matching receipts.",
"- Security, coefficient, topology-prediction, and receipt-gate debts remain HOLD surfaces.",
"- The worktree is broad and dirty; do not stage by directory sweep.",
"- `/usr/bin/sem` is GNU Parallel on this machine; use the isolated sem binary path if sem is needed.",
"",
"## Less Solid Surface Counts",
"",
]
for bucket, count in sorted(tri.get("less_solid_surface_counts", {}).items()):
lines.append(f"- `{bucket}`: {count}")
lines.extend(
[
"",
"## Next Stabilization Moves",
"",
"1. Resolve fabric UART transport with board bridge docs or an external USB-UART adapter.",
"2. Work the closure register tickets in order: UART transport, flash persistence, adaptive-mask security, coefficient calibration, topology predictions, receipt gates, then worktree scope.",
"3. Keep Q16 as the first narrow candidate lane; do not widen compiler promotion until more Lean-backed certificates exist.",
"4. Produce a scoped staging manifest before any commit because the working tree contains many unrelated/generated surfaces.",
"",
"## Receipt",
"",
f"- Machine receipt: `{rel(OUT)}`",
"- Staging manifest: `6-Documentation/docs/stack_solidification_staging_manifest_2026-05-09.md`",
]
)
return "\n".join(lines) + "\n"
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--full-lean", action="store_true")
parser.add_argument("--include-hardware", action="store_true")
parser.add_argument("--hardware-port", default="/dev/ttyUSB1")
args = parser.parse_args()
OUT_DIR.mkdir(parents=True, exist_ok=True)
receipt: dict[str, Any] = {
"schema": "stack_solidification_receipt_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"claim_boundary": "Stack status audit only. This does not promote HOLD surfaces or hardware acceleration claims.",
"gates": {},
}
receipt["gates"]["lean"] = lean_gate(args.full_lean)
receipt["gates"]["python_compile"] = py_compile_gate()
receipt["gates"]["compiler"] = compiler_gate()
receipt["gates"]["support_receipts"] = refresh_support_receipts_gate()
receipt["gates"]["tri_cycle"] = tri_cycle_gate(args.include_hardware, args.hardware_port)
receipt["gates"]["stack_fail_closure"] = stack_fail_closure_gate()
receipt["gates"]["smn_tool_awareness"] = smn_tool_awareness_gate()
receipt["gates"]["json"] = json_gate()
receipt["gates"]["hardware_bitstreams"] = hardware_bitstream_gate()
receipt["gates"]["external_sem"] = external_sem_gate()
receipt["gates"]["hold_checklist"] = hold_checklist_gate()
receipt["gates"]["network_hold_manifests"] = network_hold_manifest_gate()
receipt["gates"]["beaver_mask_freshness"] = beaver_mask_freshness_gate()
receipt["gates"]["whitespace_zero_grammar"] = whitespace_zero_grammar_gate()
receipt["gates"]["virtual_serial"] = virtual_serial_gate()
receipt["gates"]["uart_transport_routes"] = uart_transport_routes_gate()
receipt["gates"]["worktree"] = worktree_gate()
OUT.write_text(json.dumps(receipt, indent=2, sort_keys=True), encoding="utf-8")
DOC.write_text(build_doc(receipt), encoding="utf-8")
print(json.dumps({"receipt": rel(OUT), "doc": rel(DOC), "status": "WRITTEN"}, indent=2))
hard_fail = any(
receipt["gates"][gate]["status"] == "FAIL"
for gate in ["lean", "json", "python_compile", "compiler", "support_receipts", "stack_fail_closure", "hardware_bitstreams"]
)
return 1 if hard_fail else 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,324 @@
#!/usr/bin/env python3
"""Treat stellar-gas eigenmass cells as an Abelian-sandpile-style diagnostic.
The metaphor is operationalized carefully:
* "grains" are normalized SMN/evidence eigenmass in a sky/redshift cell.
* "toppling pressure" is a standardized mix of gas/shock propagation channels.
* "avalanche candidates" are cells with both high eigenmass and high pressure.
Boundary: this is a routing/diagnostic model over observational proxies. It is
not a physical sandpile simulation, not stellar mass, and not cosmology.
"""
from __future__ import annotations
import json
import math
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
MASS_JSON = ROOT / "shared-data/data/stellar_gas_observation/stellar_gas_eigenvector_mass_probe.json"
GROUP_JSON = ROOT / "shared-data/data/stellar_gas_observation/stellar_gas_population_grouping_study.json"
OUT_DIR = ROOT / "shared-data/data/stellar_gas_observation"
DOCS_DIR = ROOT / "6-Documentation/docs"
TIDDLER_DIR = ROOT / "6-Documentation/tiddlywiki-local/wiki/tiddlers"
OUT_JSON = OUT_DIR / "stellar_gas_abelian_sandpile_probe.json"
RECEIPT_JSON = OUT_DIR / "stellar_gas_abelian_sandpile_probe_receipt.json"
DOC_MD = DOCS_DIR / "stellar_gas_abelian_sandpile_probe_2026-05-09.md"
TIDDLER = TIDDLER_DIR / "Stellar Gas Abelian Sandpile Probe.tid"
CHANNELS = [
"log_desi_count",
"log_manga_count",
"partial_full_shock_fraction",
"shock_lier_fraction",
"shock_score_mean",
"gas_sigma_mean",
"gas_sigma_p90",
"stellar_sigma_mean",
"snr_mean",
"agn_liner_or_shock_fraction",
"star_forming_fraction",
]
def load_json(path: Path) -> dict[str, Any]:
with path.open() as f:
return json.load(f)
def safe_div(a: float, b: float) -> float:
return a / b if b else 0.0
def pearson(a: list[float], b: list[float]) -> float:
if len(a) != len(b) or len(a) < 2:
return 0.0
ma = sum(a) / len(a)
mb = sum(b) / len(b)
va = [x - ma for x in a]
vb = [x - mb for x in b]
den = math.sqrt(sum(x * x for x in va) * sum(y * y for y in vb))
return sum(x * y for x, y in zip(va, vb)) / den if den else 0.0
def mean_std(values: list[float]) -> tuple[float, float]:
if not values:
return 0.0, 1.0
mean = sum(values) / len(values)
var = sum((x - mean) ** 2 for x in values) / len(values)
std = math.sqrt(var)
return mean, std if std else 1.0
def zscore(values: list[float]) -> list[float]:
mean, std = mean_std(values)
return [(x - mean) / std for x in values]
def round9(x: float) -> float:
return round(x, 9)
def cell_channels(mass_row: dict[str, Any], group_row: dict[str, Any]) -> dict[str, float]:
count = float(group_row["count"])
bpt = group_row.get("bpt_proxy_classes", {})
gas = group_row.get("gas_sigma_summary", {})
stellar = group_row.get("stellar_sigma_summary", {})
snr = group_row.get("snr_summary", {})
shock = group_row.get("shock_score_summary", {})
return {
"log_desi_count": math.log1p(float(mass_row["desi_count"])),
"log_manga_count": math.log1p(float(mass_row["manga_count"])),
"partial_full_shock_fraction": float(group_row.get("partial_or_full_shock_fraction") or 0.0),
"shock_lier_fraction": float(group_row.get("shock_lier_fraction") or 0.0),
"shock_score_mean": float(shock.get("mean") or 0.0),
"gas_sigma_mean": float(gas.get("mean") or 0.0),
"gas_sigma_p90": float(gas.get("p90") or 0.0),
"stellar_sigma_mean": float(stellar.get("mean") or 0.0),
"snr_mean": float(snr.get("mean") or 0.0),
"agn_liner_or_shock_fraction": safe_div(float(bpt.get("agn_liner_or_shock_proxy", 0)), count),
"star_forming_fraction": safe_div(float(bpt.get("star_forming_proxy", 0)), count),
}
def build() -> tuple[dict[str, Any], dict[str, Any]]:
mass = load_json(MASS_JSON)
groups = load_json(GROUP_JSON)["groups"]["by_sky_z_cell"]
rows: list[dict[str, Any]] = []
for mass_row in mass["top_cell_masses"]:
cell = mass_row["cell"]
if cell not in groups:
continue
channels = cell_channels(mass_row, groups[cell])
rows.append(
{
"cell": cell,
"eigenmass": float(mass_row["normalized_eigenvector_mass"]),
"eigen_score": float(mass_row["eigen_score"]),
"channels": channels,
}
)
eigenmass_values = [row["eigenmass"] for row in rows]
channel_values = {name: [row["channels"][name] for row in rows] for name in CHANNELS}
channel_correlations = {
name: round9(pearson(eigenmass_values, values))
for name, values in channel_values.items()
}
pressure_components = [
"partial_full_shock_fraction",
"shock_lier_fraction",
"shock_score_mean",
"gas_sigma_mean",
"gas_sigma_p90",
"agn_liner_or_shock_fraction",
]
z_components = {name: zscore(channel_values[name]) for name in pressure_components}
mass_z = zscore(eigenmass_values)
pressure_scores = []
for i, row in enumerate(rows):
pressure = sum(z_components[name][i] for name in pressure_components) / len(pressure_components)
toppling_index = 0.5 * mass_z[i] + 0.5 * pressure
pressure_scores.append(pressure)
row["sandpile"] = {
"grains": round9(row["eigenmass"]),
"toppling_pressure": round9(pressure),
"toppling_index": round9(toppling_index),
}
pressure_mean, pressure_std = mean_std(pressure_scores)
index_values = [row["sandpile"]["toppling_index"] for row in rows]
index_mean, index_std = mean_std(index_values)
for row in rows:
row["sandpile"]["state"] = (
"AVALANCHE_CANDIDATE"
if row["sandpile"]["toppling_index"] >= index_mean + index_std
else "LOADED"
if row["sandpile"]["toppling_index"] >= index_mean
else "STABLE"
)
rows.sort(key=lambda item: item["sandpile"]["toppling_index"], reverse=True)
created = datetime.now(timezone.utc).isoformat(timespec="seconds")
result = {
"schema": "stellar_gas_abelian_sandpile_probe_v0",
"created": created,
"decision": "ADMIT_SANDPILE_DIAGNOSTIC_HOLD_PHYSICAL_SANDPILE",
"claim_boundary": (
"Uses an Abelian-sandpile metaphor as a diagnostic over SMN/evidence "
"mass and gas/shock observational proxies. It is not a physical "
"sandpile simulation, not stellar mass, and not cosmology."
),
"sources": {
"eigenmass": str(MASS_JSON.relative_to(ROOT)),
"population_groups": str(GROUP_JSON.relative_to(ROOT)),
},
"cell_count": len(rows),
"channel_correlations_with_eigenmass": channel_correlations,
"pressure_components": pressure_components,
"pressure_summary": {
"mean": round9(pressure_mean),
"std": round9(pressure_std),
},
"toppling_index_summary": {
"mean": round9(index_mean),
"std": round9(index_std),
},
"top_cells": rows[:25],
"interpretation": (
"High eigenmass plus high gas/shock pressure marks cells that deserve "
"fine-grained follow-up. Negative or weak channel correlation marks "
"channels that may be less explanatory for the current eigenmass surface."
),
"holds": [
"HOLD_PHYSICAL_SANDPILE_SIMULATION",
"HOLD_DIRECT_STELLAR_MASS",
"HOLD_DIRECT_GAS_DENSITY_INFERENCE",
"HOLD_OBJECT_LEVEL_CROSSMATCH",
"HOLD_COSMOLOGY_FIT",
],
}
receipt = {
"receipt_type": "stellar_gas_abelian_sandpile_probe_receipt",
"created": created,
"cell_count": len(rows),
"avalanche_candidate_count": sum(1 for row in rows if row["sandpile"]["state"] == "AVALANCHE_CANDIDATE"),
"decision": result["decision"],
"validated_outputs": [
str(OUT_JSON.relative_to(ROOT)),
str(DOC_MD.relative_to(ROOT)),
str(TIDDLER.relative_to(ROOT)),
],
}
return result, receipt
def write_docs(result: dict[str, Any]) -> None:
corr_lines = "\n".join(
f"- `{name}`: {value}"
for name, value in sorted(
result["channel_correlations_with_eigenmass"].items(),
key=lambda item: abs(item[1]),
reverse=True,
)
)
top_lines = "\n".join(
f"- `{row['cell']}`: state `{row['sandpile']['state']}`, grains `{row['sandpile']['grains']}`, "
f"pressure `{row['sandpile']['toppling_pressure']}`, index `{row['sandpile']['toppling_index']}`"
for row in result["top_cells"][:10]
)
holds = "\n".join(f"- `{hold}`" for hold in result["holds"])
DOC_MD.write_text(
f"""# Stellar Gas Abelian Sandpile Probe
Status: `SANDPILE_DIAGNOSTIC`
Decision: `{result['decision']}`
This probe treats the stellar-gas eigenmass surface as an Abelian-sandpile-style
diagnostic. Cells carry normalized eigenmass as "grains"; gas/shock observables
act as toppling pressure; high grain/high-pressure cells become avalanche
candidates for fine-grained follow-up.
Claim boundary: this is a metaphor-backed diagnostic over observational proxies.
It is not a physical sandpile simulation, not stellar mass, not direct gas
density inference, and not a cosmology fit.
## Channel Correlations With Eigenmass
{corr_lines}
## Toppling Candidates
{top_lines}
## Pressure Components
```json
{json.dumps(result['pressure_components'], indent=2)}
```
## Holds
{holds}
""",
encoding="utf-8",
)
TIDDLER.write_text(
f"""title: Stellar Gas Abelian Sandpile Probe
tags: StellarGasObservation SemanticMassNumbers Eigenvector Physics Sandpile Receipts
type: text/vnd.tiddlywiki
Status: <<tag SANDPILE_DIAGNOSTIC>>
Decision: `{result['decision']}`
This tiddler operationalizes the "stars as Abelian sand piles" metaphor as a
diagnostic over the stellar-gas eigenmass surface.
```
eigenmass grains + gas/shock pressure -> toppling candidates
```
!! Channel Correlations With Eigenmass
{corr_lines}
!! Toppling Candidates
{top_lines}
!! Boundary
This is not a physical sandpile simulation, not stellar mass, not direct gas
density inference, and not a cosmology fit.
""",
encoding="utf-8",
)
def main() -> None:
result, receipt = build()
OUT_DIR.mkdir(parents=True, exist_ok=True)
DOCS_DIR.mkdir(parents=True, exist_ok=True)
TIDDLER_DIR.mkdir(parents=True, exist_ok=True)
OUT_JSON.write_text(json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT_JSON.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_docs(result)
print(json.dumps(receipt, indent=2, sort_keys=True))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,409 @@
#!/usr/bin/env python3
"""Infer an SMN/evidence eigenvector mass over DESI-MaNGA population cells.
This probe treats the coarse DESI epoviz to MaNGA cell join as an evidence
matrix. It computes the dominant covariance eigenvector with deterministic
pure-Python Jacobi iteration, then emits a receipt-bearing semantic mass surface.
Boundary: this is not physical mass, not stellar mass, and not a cosmology fit.
It is a semantic/evidence-load direction over the current joined data surface.
"""
from __future__ import annotations
import json
import math
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
JOIN_JSON = ROOT / "shared-data/data/stellar_gas_observation/desi_epoviz_manga_population_cell_join.json"
OUT_DIR = ROOT / "shared-data/data/stellar_gas_observation"
DOCS_DIR = ROOT / "6-Documentation/docs"
TIDDLER_DIR = ROOT / "6-Documentation/tiddlywiki-local/wiki/tiddlers"
OUT_JSON = OUT_DIR / "stellar_gas_eigenvector_mass_probe.json"
RECEIPT_JSON = OUT_DIR / "stellar_gas_eigenvector_mass_probe_receipt.json"
DOC_MD = DOCS_DIR / "stellar_gas_eigenvector_mass_probe_2026-05-09.md"
TIDDLER = TIDDLER_DIR / "Stellar Gas Eigenvector Mass Probe.tid"
FEATURES = [
"log_desi_count",
"log_manga_count",
"partial_full_shock_fraction",
"shock_lier_fraction",
"BGS_share",
"ELG_share",
"LRG_share",
"QSO_share",
]
def dot(a: list[float], b: list[float]) -> float:
return sum(x * y for x, y in zip(a, b))
def mat_vec(m: list[list[float]], v: list[float]) -> list[float]:
return [dot(row, v) for row in m]
def norm(v: list[float]) -> float:
return math.sqrt(dot(v, v))
def normalize(v: list[float]) -> list[float]:
n = norm(v)
if n == 0:
return [0.0 for _ in v]
return [x / n for x in v]
def transpose(matrix: list[list[float]]) -> list[list[float]]:
return [list(col) for col in zip(*matrix)]
def jacobi_eigen_symmetric(matrix: list[list[float]], max_iter: int = 200, eps: float = 1e-12) -> tuple[list[float], list[list[float]]]:
"""Return eigenvalues and eigenvectors for a small symmetric matrix.
Eigenvectors are returned as columns in the second return value.
"""
n = len(matrix)
a = [row[:] for row in matrix]
v = [[1.0 if i == j else 0.0 for j in range(n)] for i in range(n)]
iterations = 0
final_max_offdiag = 0.0
converged = False
for iteration in range(1, max_iter + 1):
p, q = 0, 1
max_off = 0.0
for i in range(n):
for j in range(i + 1, n):
val = abs(a[i][j])
if val > max_off:
max_off = val
p, q = i, j
iterations = iteration
final_max_offdiag = max_off
if max_off < eps:
converged = True
break
if abs(a[p][p] - a[q][q]) < eps:
angle = math.pi / 4
else:
angle = 0.5 * math.atan2(2.0 * a[p][q], a[q][q] - a[p][p])
c = math.cos(angle)
s = math.sin(angle)
app = c * c * a[p][p] - 2.0 * s * c * a[p][q] + s * s * a[q][q]
aqq = s * s * a[p][p] + 2.0 * s * c * a[p][q] + c * c * a[q][q]
a[p][p] = app
a[q][q] = aqq
a[p][q] = 0.0
a[q][p] = 0.0
for k in range(n):
if k == p or k == q:
continue
akp = c * a[k][p] - s * a[k][q]
akq = s * a[k][p] + c * a[k][q]
a[k][p] = akp
a[p][k] = akp
a[k][q] = akq
a[q][k] = akq
for k in range(n):
vkp = c * v[k][p] - s * v[k][q]
vkq = s * v[k][p] + c * v[k][q]
v[k][p] = vkp
v[k][q] = vkq
eigenvalues = [a[i][i] for i in range(n)]
return eigenvalues, v, {
"method": "jacobi_symmetric",
"max_iter": max_iter,
"eps": eps,
"iterations": iterations,
"converged": converged,
"final_max_offdiag": final_max_offdiag,
}
def eigen_residual(matrix: list[list[float]], eigenvalue: float, eigenvector: list[float]) -> float:
av = mat_vec(matrix, eigenvector)
residual = [av_i - eigenvalue * v_i for av_i, v_i in zip(av, eigenvector)]
return norm(residual)
def build_feature_rows(join: dict[str, Any]) -> tuple[list[str], list[list[float]], list[dict[str, Any]]]:
labels: list[str] = []
rows: list[list[float]] = []
payloads: list[dict[str, Any]] = []
for cell in join["manga_join"]["top_joined_cells"]:
mix = cell["desi_tracer_mix"]
total = sum(float(v) for v in mix.values()) or 1.0
labels.append(cell["cell"])
payloads.append(cell)
rows.append(
[
math.log1p(float(cell["desi_count"])),
math.log1p(float(cell["manga_count"])),
float(cell.get("manga_partial_or_full_shock_fraction") or 0.0),
float(cell.get("manga_shock_lier_fraction") or 0.0),
float(mix.get("BGS", 0.0)) / total,
float(mix.get("ELG", 0.0)) / total,
float(mix.get("LRG", 0.0)) / total,
float(mix.get("QSO", 0.0)) / total,
]
)
return labels, rows, payloads
def zscore(rows: list[list[float]]) -> tuple[list[list[float]], list[float], list[float]]:
cols = transpose(rows)
means = [sum(col) / len(col) for col in cols]
stds = []
for col, mean in zip(cols, means):
var = sum((x - mean) ** 2 for x in col) / len(col)
std = math.sqrt(var)
stds.append(std if std > 0 else 1.0)
scaled = [[(x - means[i]) / stds[i] for i, x in enumerate(row)] for row in rows]
return scaled, means, stds
def covariance(rows: list[list[float]]) -> list[list[float]]:
n = len(rows)
cols = len(rows[0])
return [
[sum(row[i] * row[j] for row in rows) / (n - 1) for j in range(cols)]
for i in range(cols)
]
def build() -> tuple[dict[str, Any], dict[str, Any]]:
with JOIN_JSON.open() as f:
join = json.load(f)
labels, raw_rows, payloads = build_feature_rows(join)
scaled_rows, means, stds = zscore(raw_rows)
cov = covariance(scaled_rows)
values, vectors_as_columns, solver = jacobi_eigen_symmetric(cov)
order = sorted(range(len(values)), key=lambda i: values[i], reverse=True)
eigenvalues = [values[i] for i in order]
eigenvectors_by_rank = [[vectors_as_columns[row][i] for row in range(len(FEATURES))] for i in order]
dominant = normalize(eigenvectors_by_rank[0])
shock_index = FEATURES.index("partial_full_shock_fraction")
if dominant[shock_index] < 0:
dominant = [-x for x in dominant]
residual = eigen_residual(cov, eigenvalues[0], dominant)
scores = [dot(row, dominant) for row in scaled_rows]
min_score = min(scores)
shifted = [score - min_score for score in scores]
total_shifted = sum(shifted)
masses = [x / total_shifted if total_shifted > 0 else 1.0 / len(shifted) for x in shifted]
cell_masses = []
for label, payload, score, mass in zip(labels, payloads, scores, masses):
cell_masses.append(
{
"cell": label,
"eigen_score": round(score, 6),
"normalized_eigenvector_mass": round(mass, 6),
"desi_count": payload["desi_count"],
"manga_count": payload["manga_count"],
"manga_partial_or_full_shock_fraction": payload.get("manga_partial_or_full_shock_fraction"),
"manga_shock_lier_fraction": payload.get("manga_shock_lier_fraction"),
"desi_tracer_mix": payload["desi_tracer_mix"],
}
)
cell_masses.sort(key=lambda row: row["normalized_eigenvector_mass"], reverse=True)
total_eigen = sum(x for x in eigenvalues if x > 0)
explained = [(x / total_eigen if total_eigen > 0 else 0.0) for x in eigenvalues]
created = datetime.now(timezone.utc).isoformat(timespec="seconds")
result = {
"schema": "stellar_gas_eigenvector_mass_probe_v0",
"created": created,
"decision": "ADMIT_SMN_EIGENVECTOR_MASS_HOLD_PHYSICAL_MASS",
"claim_boundary": (
"Eigenvector mass is an SMN/evidence-load direction over the coarse "
"DESI epoviz to MaNGA population-cell join. It is not physical mass, "
"not stellar mass, not a gas-density map, and not a cosmology fit."
),
"source_join": str(JOIN_JSON.relative_to(ROOT)),
"feature_basis": FEATURES,
"feature_means": {name: round(means[i], 9) for i, name in enumerate(FEATURES)},
"feature_stds": {name: round(stds[i], 9) for i, name in enumerate(FEATURES)},
"cell_count": len(labels),
"eigenvalues": [round(x, 9) for x in eigenvalues],
"explained_mass_share": [round(x, 9) for x in explained],
"dominant_eigenvector": {name: round(dominant[i], 9) for i, name in enumerate(FEATURES)},
"dominant_eigenvalue": round(eigenvalues[0], 9),
"dominant_explained_mass_share": round(explained[0], 9),
"eigensolver_diagnostics": {
**solver,
"dominant_residual_l2": round(residual, 12),
"orthogonality_note": "Jacobi rotations return an orthonormal basis up to numeric roundoff; this receipt reports the dominant residual only.",
},
"top_cell_masses": cell_masses[:25],
"holds": [
"HOLD_PHYSICAL_MASS_INTERPRETATION",
"HOLD_OBJECT_LEVEL_CROSSMATCH",
"HOLD_DIRECT_GAS_DENSITY_INFERENCE",
"HOLD_SELECTION_FUNCTION_FIT",
"HOLD_COSMOLOGY_FIT",
],
}
receipt = {
"receipt_type": "stellar_gas_eigenvector_mass_probe_receipt",
"created": created,
"source_join": str(JOIN_JSON.relative_to(ROOT)),
"cell_count": len(labels),
"dominant_eigenvalue": result["dominant_eigenvalue"],
"dominant_explained_mass_share": result["dominant_explained_mass_share"],
"eigensolver_diagnostics": result["eigensolver_diagnostics"],
"decision": result["decision"],
"validated_outputs": [
str(OUT_JSON.relative_to(ROOT)),
str(DOC_MD.relative_to(ROOT)),
str(TIDDLER.relative_to(ROOT)),
],
}
return result, receipt
def write_docs(result: dict[str, Any]) -> None:
vector_lines = "\n".join(
f"- `{name}`: {value}" for name, value in result["dominant_eigenvector"].items()
)
cell_lines = "\n".join(
f"- `{row['cell']}`: mass `{row['normalized_eigenvector_mass']}`, "
f"score `{row['eigen_score']}`, DESI `{row['desi_count']}`, MaNGA `{row['manga_count']}`"
for row in result["top_cell_masses"][:10]
)
holds = "\n".join(f"- `{hold}`" for hold in result["holds"])
diag = result["eigensolver_diagnostics"]
DOC_MD.write_text(
f"""# Stellar Gas Eigenvector Mass Probe
Status: `SMN_EIGENVECTOR_MASS`
Decision: `{result['decision']}`
This probe computes the dominant covariance eigenvector over the coarse DESI
epoviz to MaNGA population-cell join. The output is an SMN/evidence-load mass
direction: it ranks the current coarse joined cells by this diagnostic score so
later zoom work can choose explicit follow-up targets.
Claim boundary: this is not physical mass, not stellar mass, not a direct gas
density map, and not a cosmology fit.
## Result
Dominant eigenvalue:
```text
{result['dominant_eigenvalue']}
```
Dominant explained mass share:
```text
{result['dominant_explained_mass_share']}
```
## Dominant Eigenvector
{vector_lines}
## Eigensolver Diagnostics
```text
method: {diag['method']}
converged: {diag['converged']}
iterations: {diag['iterations']}
final max off-diagonal: {diag['final_max_offdiag']}
dominant residual L2: {diag['dominant_residual_l2']}
```
## Top Cell Masses
{cell_lines}
## Holds
{holds}
""",
encoding="utf-8",
)
TIDDLER.write_text(
f"""title: Stellar Gas Eigenvector Mass Probe
tags: StellarGasObservation SemanticMassNumbers DESI MaNGA Eigenvector Receipts
type: text/vnd.tiddlywiki
Status: <<tag SMN_EIGENVECTOR_MASS>>
Decision: `{result['decision']}`
The inferred eigenvector mass is the dominant SMN/evidence-load direction over
the coarse DESI epoviz to MaNGA population-cell join.
Dominant eigenvalue:
```
{result['dominant_eigenvalue']}
```
Dominant explained mass share:
```
{result['dominant_explained_mass_share']}
```
Eigensolver:
```
converged={diag['converged']} iterations={diag['iterations']} residual_l2={diag['dominant_residual_l2']}
```
!! Dominant Eigenvector
{vector_lines}
!! Top Cell Masses
{cell_lines}
!! Boundary
This is not physical mass, not stellar mass, not direct gas-density inference,
and not a cosmology fit.
""",
encoding="utf-8",
)
def main() -> None:
result, receipt = build()
OUT_DIR.mkdir(parents=True, exist_ok=True)
DOCS_DIR.mkdir(parents=True, exist_ok=True)
TIDDLER_DIR.mkdir(parents=True, exist_ok=True)
OUT_JSON.write_text(json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT_JSON.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_docs(result)
print(json.dumps(receipt, indent=2, sort_keys=True))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,586 @@
#!/usr/bin/env python3
"""Full-cell eigenmass stability and ablation controls.
This probe reuses the existing DESI epoviz to MaNGA population-cell join and
checks whether the 25-cell SMN/evidence-load eigenvector remains stable across
all joined cells, leave-one-cell-out slices, deterministic null shuffles, and
feature ablations.
Boundary: this is evidence-geometry quality control. It is not physical mass,
not gas density, not shock proof, and not cosmology.
"""
from __future__ import annotations
import json
import math
import random
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
JOIN_JSON = ROOT / "shared-data/data/stellar_gas_observation/desi_epoviz_manga_population_cell_join.json"
BASELINE_JSON = ROOT / "shared-data/data/stellar_gas_observation/stellar_gas_eigenvector_mass_probe.json"
OUT_DIR = ROOT / "shared-data/data/stellar_gas_observation"
DOCS_DIR = ROOT / "6-Documentation/docs"
TIDDLER_DIR = ROOT / "6-Documentation/tiddlywiki-local/wiki/tiddlers"
OUT_JSON = OUT_DIR / "stellar_gas_full_cell_eigenmass_stability.json"
RECEIPT_JSON = OUT_DIR / "stellar_gas_full_cell_eigenmass_stability_receipt.json"
DOC_MD = DOCS_DIR / "stellar_gas_full_cell_eigenmass_stability_2026-05-09.md"
TIDDLER = TIDDLER_DIR / "Stellar Gas Full Cell Eigenmass Stability.tid"
FEATURES = [
"log_desi_count",
"log_manga_count",
"partial_full_shock_fraction",
"shock_lier_fraction",
"BGS_share",
"ELG_share",
"LRG_share",
"QSO_share",
]
SHOCK_FEATURES = {"partial_full_shock_fraction", "shock_lier_fraction"}
TRACER_FEATURES = {"BGS_share", "ELG_share", "LRG_share", "QSO_share"}
def dot(a: list[float], b: list[float]) -> float:
return sum(x * y for x, y in zip(a, b))
def norm(v: list[float]) -> float:
return math.sqrt(dot(v, v))
def normalize(v: list[float]) -> list[float]:
n = norm(v)
if n == 0:
return [0.0 for _ in v]
return [x / n for x in v]
def cosine(a: list[float], b: list[float]) -> float:
denom = norm(a) * norm(b)
if denom == 0:
return 0.0
return dot(a, b) / denom
def transpose(matrix: list[list[float]]) -> list[list[float]]:
return [list(col) for col in zip(*matrix)]
def round9(value: float) -> float:
return round(value, 9)
def jacobi_eigen_symmetric(matrix: list[list[float]], max_iter: int = 240, eps: float = 1e-12) -> tuple[list[float], list[list[float]], dict[str, Any]]:
n = len(matrix)
a = [row[:] for row in matrix]
v = [[1.0 if i == j else 0.0 for j in range(n)] for i in range(n)]
iterations = 0
final_max_offdiag = 0.0
converged = False
for iteration in range(1, max_iter + 1):
p, q = 0, 1
max_off = 0.0
for i in range(n):
for j in range(i + 1, n):
val = abs(a[i][j])
if val > max_off:
max_off = val
p, q = i, j
iterations = iteration
final_max_offdiag = max_off
if max_off < eps:
converged = True
break
if abs(a[p][p] - a[q][q]) < eps:
angle = math.pi / 4.0
else:
angle = 0.5 * math.atan2(2.0 * a[p][q], a[q][q] - a[p][p])
c = math.cos(angle)
s = math.sin(angle)
app = c * c * a[p][p] - 2.0 * s * c * a[p][q] + s * s * a[q][q]
aqq = s * s * a[p][p] + 2.0 * s * c * a[p][q] + c * c * a[q][q]
a[p][p] = app
a[q][q] = aqq
a[p][q] = 0.0
a[q][p] = 0.0
for k in range(n):
if k == p or k == q:
continue
akp = c * a[k][p] - s * a[k][q]
akq = s * a[k][p] + c * a[k][q]
a[k][p] = akp
a[p][k] = akp
a[k][q] = akq
a[q][k] = akq
for k in range(n):
vkp = c * v[k][p] - s * v[k][q]
vkq = s * v[k][p] + c * v[k][q]
v[k][p] = vkp
v[k][q] = vkq
return [a[i][i] for i in range(n)], v, {
"method": "jacobi_symmetric",
"max_iter": max_iter,
"eps": eps,
"iterations": iterations,
"converged": converged,
"final_max_offdiag": final_max_offdiag,
}
def mat_vec(m: list[list[float]], v: list[float]) -> list[float]:
return [dot(row, v) for row in m]
def eigen_residual(matrix: list[list[float]], eigenvalue: float, eigenvector: list[float]) -> float:
av = mat_vec(matrix, eigenvector)
residual = [av_i - eigenvalue * v_i for av_i, v_i in zip(av, eigenvector)]
return norm(residual)
def zscore(rows: list[list[float]]) -> tuple[list[list[float]], list[float], list[float]]:
cols = transpose(rows)
means = [sum(col) / len(col) for col in cols]
stds = []
for col, mean in zip(cols, means):
var = sum((x - mean) ** 2 for x in col) / len(col)
std = math.sqrt(var)
stds.append(std if std > 0 else 1.0)
return [[(x - means[i]) / stds[i] for i, x in enumerate(row)] for row in rows], means, stds
def covariance(rows: list[list[float]]) -> list[list[float]]:
n = len(rows)
cols = len(rows[0])
return [
[sum(row[i] * row[j] for row in rows) / (n - 1) for j in range(cols)]
for i in range(cols)
]
def load_json(path: Path) -> dict[str, Any]:
with path.open() as f:
return json.load(f)
def source_rows(join: dict[str, Any]) -> list[dict[str, Any]]:
rows = []
for cell in join["manga_join"]["top_joined_cells"]:
mix = cell["desi_tracer_mix"]
total = sum(float(v) for v in mix.values()) or 1.0
rows.append(
{
"cell": cell["cell"],
"payload": cell,
"features": {
"log_desi_count": math.log1p(float(cell["desi_count"])),
"log_manga_count": math.log1p(float(cell["manga_count"])),
"partial_full_shock_fraction": float(cell.get("manga_partial_or_full_shock_fraction") or 0.0),
"shock_lier_fraction": float(cell.get("manga_shock_lier_fraction") or 0.0),
"BGS_share": float(mix.get("BGS", 0.0)) / total,
"ELG_share": float(mix.get("ELG", 0.0)) / total,
"LRG_share": float(mix.get("LRG", 0.0)) / total,
"QSO_share": float(mix.get("QSO", 0.0)) / total,
},
}
)
return rows
def permuted(values: list[Any], seed: int) -> list[Any]:
out = values[:]
random.Random(seed).shuffle(out)
return out
def vector_for_features(row: dict[str, Any], features: list[str]) -> list[float]:
return [float(row["features"][feature]) for feature in features]
def fit_eigenmass(rows: list[dict[str, Any]], features: list[str], baseline_vector: dict[str, float] | None = None) -> dict[str, Any]:
labels = [row["cell"] for row in rows]
raw_rows = [vector_for_features(row, features) for row in rows]
scaled_rows, means, stds = zscore(raw_rows)
cov = covariance(scaled_rows)
values, vectors_as_columns, solver = jacobi_eigen_symmetric(cov)
order = sorted(range(len(values)), key=lambda i: values[i], reverse=True)
eigenvalues = [values[i] for i in order]
dominant = normalize([vectors_as_columns[row][order[0]] for row in range(len(features))])
if baseline_vector is None:
if "partial_full_shock_fraction" in features:
anchor = dominant[features.index("partial_full_shock_fraction")]
else:
anchor = sum(dominant)
if anchor < 0:
dominant = [-x for x in dominant]
else:
common = [feature for feature in features if feature in baseline_vector]
candidate_common = [dominant[features.index(feature)] for feature in common]
baseline_common = [baseline_vector[feature] for feature in common]
if dot(candidate_common, baseline_common) < 0:
dominant = [-x for x in dominant]
residual = eigen_residual(cov, eigenvalues[0], dominant)
scores = [dot(row, dominant) for row in scaled_rows]
min_score = min(scores)
shifted = [score - min_score for score in scores]
total_shifted = sum(shifted)
masses = [x / total_shifted if total_shifted > 0 else 1.0 / len(shifted) for x in shifted]
cell_masses = [
{
"cell": label,
"eigen_score": round(score, 6),
"normalized_eigenvector_mass": round(mass, 6),
}
for label, score, mass in zip(labels, scores, masses)
]
cell_masses.sort(key=lambda row: row["normalized_eigenvector_mass"], reverse=True)
total_eigen = sum(x for x in eigenvalues if x > 0)
explained = [(x / total_eigen if total_eigen > 0 else 0.0) for x in eigenvalues]
return {
"cell_count": len(rows),
"feature_basis": features,
"feature_means": {name: round9(means[i]) for i, name in enumerate(features)},
"feature_stds": {name: round9(stds[i]) for i, name in enumerate(features)},
"dominant_eigenvalue": round9(eigenvalues[0]),
"dominant_explained_mass_share": round9(explained[0]),
"eigensolver_diagnostics": {
**solver,
"dominant_residual_l2": round(residual, 12),
"orthogonality_note": "Jacobi rotations return an orthonormal basis up to numeric roundoff; this receipt reports the dominant residual only.",
},
"dominant_eigenvector": {name: round9(dominant[i]) for i, name in enumerate(features)},
"eigenvalues": [round9(x) for x in eigenvalues],
"explained_mass_share": [round9(x) for x in explained],
"top_cell_masses": cell_masses,
}
def compare_to_baseline(candidate: dict[str, Any], baseline: dict[str, Any], top_n: int = 5) -> dict[str, Any]:
common = [feature for feature in FEATURES if feature in candidate["dominant_eigenvector"] and feature in baseline["dominant_eigenvector"]]
cand_vec = [candidate["dominant_eigenvector"][feature] for feature in common]
base_vec = [baseline["dominant_eigenvector"][feature] for feature in common]
base_top = {row["cell"] for row in baseline["top_cell_masses"][:top_n]}
cand_top = {row["cell"] for row in candidate["top_cell_masses"][:top_n]}
return {
"common_feature_basis": common,
"common_basis_cosine_to_original": round9(cosine(cand_vec, base_vec)),
"dominant_explained_share_delta": round9(candidate["dominant_explained_mass_share"] - baseline["dominant_explained_mass_share"]),
"top_cell_overlap_at_5": len(base_top & cand_top),
"top_cell_overlap_fraction_at_5": round9(len(base_top & cand_top) / top_n),
}
def summarize_leave_one_out(values: list[dict[str, Any]]) -> dict[str, Any]:
cosines = sorted(row["common_basis_cosine_to_original"] for row in values)
overlaps = [row["top_cell_overlap_fraction_at_5"] for row in values]
return {
"loo_count": len(values),
"min_cosine_to_original": round9(cosines[0]),
"median_cosine_to_original": round9(cosines[len(cosines) // 2]),
"mean_cosine_to_original": round9(sum(cosines) / len(cosines)),
"mean_top5_overlap_fraction": round9(sum(overlaps) / len(overlaps)),
}
def with_shuffled_feature_columns(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
shuffled_columns = {
feature: permuted([row["features"][feature] for row in rows], seed=2026050901 + idx)
for idx, feature in enumerate(FEATURES)
}
out = []
for idx, row in enumerate(rows):
clone = {"cell": row["cell"], "payload": row["payload"], "features": dict(row["features"])}
for feature in FEATURES:
clone["features"][feature] = shuffled_columns[feature][idx]
out.append(clone)
return out
def with_shuffled_shocks(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
partial = permuted([row["features"]["partial_full_shock_fraction"] for row in rows], seed=2026050902)
lier = permuted([row["features"]["shock_lier_fraction"] for row in rows], seed=2026050903)
out = []
for idx, row in enumerate(rows):
clone = {"cell": row["cell"], "payload": row["payload"], "features": dict(row["features"])}
clone["features"]["partial_full_shock_fraction"] = partial[idx]
clone["features"]["shock_lier_fraction"] = lier[idx]
out.append(clone)
return out
def control_result(name: str, rows: list[dict[str, Any]], features: list[str], baseline: dict[str, Any]) -> dict[str, Any]:
fit = fit_eigenmass(rows, features, baseline["dominant_eigenvector"])
return {
"control": name,
"cell_count": fit["cell_count"],
"feature_basis": fit["feature_basis"],
"dominant_eigenvalue": fit["dominant_eigenvalue"],
"dominant_explained_mass_share": fit["dominant_explained_mass_share"],
"dominant_eigenvector": fit["dominant_eigenvector"],
"comparison_to_original": compare_to_baseline(fit, baseline),
"top_cell_masses": fit["top_cell_masses"][:10],
}
def build() -> tuple[dict[str, Any], dict[str, Any]]:
join = load_json(JOIN_JSON)
stored = load_json(BASELINE_JSON)
rows = source_rows(join)
baseline = fit_eigenmass(rows, FEATURES)
stored_compare = compare_to_baseline(baseline, stored, top_n=5)
stored_vector_abs_delta = {
feature: round9(abs(baseline["dominant_eigenvector"][feature] - stored["dominant_eigenvector"][feature]))
for feature in FEATURES
}
loo_rows = []
for cell in [row["cell"] for row in rows]:
subset = [row for row in rows if row["cell"] != cell]
fit = fit_eigenmass(subset, FEATURES, baseline["dominant_eigenvector"])
comparison = compare_to_baseline(fit, baseline)
loo_rows.append(
{
"held_out_cell": cell,
"dominant_explained_mass_share": fit["dominant_explained_mass_share"],
**comparison,
}
)
controls = [
control_result("shuffled_feature_columns", with_shuffled_feature_columns(rows), FEATURES, baseline),
control_result("shuffled_shock_channels", with_shuffled_shocks(rows), FEATURES, baseline),
control_result("desi_count_removed", rows, [feature for feature in FEATURES if feature != "log_desi_count"], baseline),
control_result("shock_proxy_removed", rows, [feature for feature in FEATURES if feature not in SHOCK_FEATURES], baseline),
control_result("tracer_mix_removed", rows, [feature for feature in FEATURES if feature not in TRACER_FEATURES], baseline),
]
created = datetime.now(timezone.utc).isoformat(timespec="seconds")
result = {
"schema": "stellar_gas_full_cell_eigenmass_stability_v0",
"created": created,
"decision": "REPORT_FULL_CELL_EIGENMASS_STABILITY_WITH_NULL_CONTROLS_HOLD_PHYSICAL_CLAIMS",
"claim_boundary": (
"Full-cell stability and ablation controls for the joined DESI/MaNGA "
"SMN/evidence-load eigenvector. This does not promote physical mass, "
"gas density, shock proof, or cosmology."
),
"sources": {
"join": str(JOIN_JSON.relative_to(ROOT)),
"stored_25_cell_probe": str(BASELINE_JSON.relative_to(ROOT)),
},
"full_cell_baseline": baseline,
"stored_25_cell_comparison": {
"stored_cell_count": stored["cell_count"],
"recomputed_cell_count": baseline["cell_count"],
"common_basis_cosine_to_stored": stored_compare["common_basis_cosine_to_original"],
"top_cell_overlap_at_5": stored_compare["top_cell_overlap_at_5"],
"max_abs_eigenvector_component_delta": round9(max(stored_vector_abs_delta.values())),
"abs_eigenvector_component_delta": stored_vector_abs_delta,
},
"leave_one_cell_out_stability": {
"summary": summarize_leave_one_out(loo_rows),
"rows": loo_rows,
},
"null_and_ablation_controls": controls,
"holds": [
"HOLD_PHYSICAL_MASS_INTERPRETATION",
"HOLD_DIRECT_GAS_DENSITY_INFERENCE",
"HOLD_SHOCK_PROOF",
"HOLD_OBJECT_LEVEL_CROSSMATCH",
"HOLD_SELECTION_FUNCTION_FIT",
"HOLD_COSMOLOGY_FIT",
],
}
receipt = {
"receipt_type": "stellar_gas_full_cell_eigenmass_stability_receipt",
"created": created,
"source_join": str(JOIN_JSON.relative_to(ROOT)),
"stored_25_cell_probe": str(BASELINE_JSON.relative_to(ROOT)),
"full_cell_count": baseline["cell_count"],
"stored_25_cell_count": stored["cell_count"],
"stored_comparison_cosine": result["stored_25_cell_comparison"]["common_basis_cosine_to_stored"],
"leave_one_out_min_cosine": result["leave_one_cell_out_stability"]["summary"]["min_cosine_to_original"],
"control_names": [control["control"] for control in controls],
"eigensolver_diagnostics": baseline["eigensolver_diagnostics"],
"decision": result["decision"],
"validated_outputs": [
str(OUT_JSON.relative_to(ROOT)),
str(DOC_MD.relative_to(ROOT)),
str(TIDDLER.relative_to(ROOT)),
],
}
return result, receipt
def write_docs(result: dict[str, Any]) -> None:
baseline = result["full_cell_baseline"]
stored = result["stored_25_cell_comparison"]
loo = result["leave_one_cell_out_stability"]["summary"]
controls = result["null_and_ablation_controls"]
diag = baseline["eigensolver_diagnostics"]
vector_lines = "\n".join(
f"- `{name}`: {value}" for name, value in baseline["dominant_eigenvector"].items()
)
control_lines = "\n".join(
"- `{control}`: cosine `{cosine}`, explained share `{share}`, top5 overlap `{overlap}`".format(
control=control["control"],
cosine=control["comparison_to_original"]["common_basis_cosine_to_original"],
share=control["dominant_explained_mass_share"],
overlap=control["comparison_to_original"]["top_cell_overlap_fraction_at_5"],
)
for control in controls
)
holds = "\n".join(f"- `{hold}`" for hold in result["holds"])
DOC_MD.write_text(
f"""# Stellar Gas Full Cell Eigenmass Stability
Status: `FULL_CELL_EIGENMASS_STABILITY`
Decision: `{result['decision']}`
This probe checks the 25-cell DESI/MaNGA joined-cell eigenmass against all
available joined cells, leave-one-cell-out slices, deterministic null shuffles,
and feature ablations.
Claim boundary: this is evidence-geometry quality control only. It does not
promote physical mass, gas density, shock proof, object-level crossmatch, or
cosmology.
## Full-Cell Baseline
Cell count: `{baseline['cell_count']}`
Dominant eigenvalue:
```text
{baseline['dominant_eigenvalue']}
```
Dominant explained mass share:
```text
{baseline['dominant_explained_mass_share']}
```
Dominant eigenvector:
{vector_lines}
## Stored 25-Cell Comparison
```text
common-basis cosine to stored probe: {stored['common_basis_cosine_to_stored']}
top-cell overlap at 5: {stored['top_cell_overlap_at_5']}
max abs component delta: {stored['max_abs_eigenvector_component_delta']}
```
## Eigensolver Diagnostics
```text
method: {diag['method']}
converged: {diag['converged']}
iterations: {diag['iterations']}
final max off-diagonal: {diag['final_max_offdiag']}
dominant residual L2: {diag['dominant_residual_l2']}
```
## Leave-One-Cell-Out Stability
```text
loo count: {loo['loo_count']}
min cosine to original: {loo['min_cosine_to_original']}
median cosine to original: {loo['median_cosine_to_original']}
mean cosine to original: {loo['mean_cosine_to_original']}
mean top5 overlap: {loo['mean_top5_overlap_fraction']}
```
## Null And Ablation Controls
{control_lines}
## Holds
{holds}
""",
encoding="utf-8",
)
TIDDLER.write_text(
f"""title: Stellar Gas Full Cell Eigenmass Stability
tags: StellarGasObservation DESI MaNGA Eigenvector Controls Receipts
type: text/vnd.tiddlywiki
Status: <<tag FULL_CELL_EIGENMASS_STABILITY>>
Decision: `{result['decision']}`
This tiddler records a full-cell stability and null-control check for the
DESI/MaNGA joined-cell SMN/evidence-load eigenvector.
Cell count: `{baseline['cell_count']}`
Stored 25-cell cosine:
```
{stored['common_basis_cosine_to_stored']}
```
Leave-one-cell-out minimum cosine:
```
{loo['min_cosine_to_original']}
```
Eigensolver:
```
converged={diag['converged']} iterations={diag['iterations']} residual_l2={diag['dominant_residual_l2']}
```
!! Original Dominant Eigenvector
{vector_lines}
!! Null And Ablation Controls
{control_lines}
!! Boundary
This is evidence-geometry quality control only. It is not physical mass, gas
density, shock proof, or cosmology.
""",
encoding="utf-8",
)
def main() -> None:
result, receipt = build()
OUT_DIR.mkdir(parents=True, exist_ok=True)
DOCS_DIR.mkdir(parents=True, exist_ok=True)
TIDDLER_DIR.mkdir(parents=True, exist_ok=True)
OUT_JSON.write_text(json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT_JSON.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_docs(result)
print(json.dumps(receipt, indent=2, sort_keys=True))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,291 @@
#!/usr/bin/env python3
"""Compare row-level DESI eigenmass with DESI/MaNGA joined-cell eigenmass.
This probe measures whether the SMN/evidence-load direction survives the zoom
from the literal DESI row surface into the gas/shock-constrained MaNGA overlap
surface. It reports a tracer-subspace cosine alignment and a sharpening factor.
Boundary: this is an evidence-geometry comparison. It is not physical mass, not
stellar mass, not a gas-density map, and not a cosmology fit.
"""
from __future__ import annotations
import json
import math
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
ROW_JSON = ROOT / "shared-data/data/stellar_gas_observation/desi_epoviz_row_eigenmass_probe.json"
CELL_JSON = ROOT / "shared-data/data/stellar_gas_observation/stellar_gas_eigenvector_mass_probe.json"
OUT_DIR = ROOT / "shared-data/data/stellar_gas_observation"
DOCS_DIR = ROOT / "6-Documentation/docs"
TIDDLER_DIR = ROOT / "6-Documentation/tiddlywiki-local/wiki/tiddlers"
OUT_JSON = OUT_DIR / "stellar_gas_multiscale_eigenmass_alignment.json"
RECEIPT_JSON = OUT_DIR / "stellar_gas_multiscale_eigenmass_alignment_receipt.json"
DOC_MD = DOCS_DIR / "stellar_gas_multiscale_eigenmass_alignment_2026-05-09.md"
TIDDLER = TIDDLER_DIR / "Stellar Gas Multiscale Eigenmass Alignment.tid"
TRACER_ORDER = ["QSO", "ELG", "LRG", "BGS"]
def dot(a: list[float], b: list[float]) -> float:
return sum(x * y for x, y in zip(a, b))
def norm(v: list[float]) -> float:
return math.sqrt(dot(v, v))
def cosine(a: list[float], b: list[float]) -> float:
denom = norm(a) * norm(b)
if denom == 0:
return 0.0
return dot(a, b) / denom
def round9(x: float) -> float:
return round(x, 9)
def load_json(path: Path) -> dict[str, Any]:
with path.open() as f:
return json.load(f)
def tracer_vector_from_row(row: dict[str, float]) -> list[float]:
return [
row["tracer_QSO"],
row["tracer_ELG"],
row["tracer_LRG"],
row["tracer_BGS"],
]
def tracer_vector_from_cell(cell: dict[str, float]) -> list[float]:
return [
cell["QSO_share"],
cell["ELG_share"],
cell["LRG_share"],
cell["BGS_share"],
]
def classify_alignment(value: float) -> str:
if value >= 0.85:
return "STRONG_ALIGNMENT"
if value >= 0.65:
return "MODERATE_ALIGNMENT"
if value >= 0.35:
return "WEAK_ALIGNMENT"
if value > -0.35:
return "ORTHOGONAL_OR_MIXED"
return "ANTI_ALIGNMENT"
def build() -> tuple[dict[str, Any], dict[str, Any]]:
row = load_json(ROW_JSON)
cell = load_json(CELL_JSON)
row_vec = tracer_vector_from_row(row["dominant_eigenvector"])
cell_vec = tracer_vector_from_cell(cell["dominant_eigenvector"])
tracer_alignment = cosine(row_vec, cell_vec)
row_share = float(row["dominant_explained_mass_share"])
cell_share = float(cell["dominant_explained_mass_share"])
sharpening_factor = cell_share / row_share if row_share else 0.0
eigenvalue_ratio = float(cell["dominant_eigenvalue"]) / float(row["dominant_eigenvalue"])
created = datetime.now(timezone.utc).isoformat(timespec="seconds")
result = {
"schema": "stellar_gas_multiscale_eigenmass_alignment_v0",
"created": created,
"decision": "ADMIT_MULTISCALE_EIGENMASS_ALIGNMENT_HOLD_PHYSICAL_MASS",
"claim_boundary": (
"Compares SMN/evidence-load eigenvectors across DESI row level and "
"DESI/MaNGA joined-cell level. It does not infer physical mass, "
"stellar mass, gas density, or cosmology."
),
"sources": {
"row_eigenmass": str(ROW_JSON.relative_to(ROOT)),
"cell_eigenmass": str(CELL_JSON.relative_to(ROOT)),
},
"row_level": {
"cell_or_row_count": row["row_count"],
"dominant_eigenvalue": row["dominant_eigenvalue"],
"dominant_explained_mass_share": row_share,
"tracer_subvector_order": TRACER_ORDER,
"tracer_subvector": [round9(x) for x in row_vec],
},
"cell_level": {
"cell_or_row_count": cell["cell_count"],
"dominant_eigenvalue": cell["dominant_eigenvalue"],
"dominant_explained_mass_share": cell_share,
"tracer_subvector_order": TRACER_ORDER,
"tracer_subvector": [round9(x) for x in cell_vec],
},
"alignment": {
"tracer_subspace_cosine": round9(tracer_alignment),
"alignment_class": classify_alignment(tracer_alignment),
"constraint_sharpening_factor": round9(sharpening_factor),
"dominant_eigenvalue_ratio_cell_over_row": round9(eigenvalue_ratio),
"interpretation": (
"The cell-level explained share is larger than the row-level share "
"under this diagnostic ratio. This is an accounting comparison, not "
"a causal gas/shock mechanism."
),
},
"holds": [
"HOLD_PHYSICAL_MASS_INTERPRETATION",
"HOLD_DIRECT_GAS_DENSITY_INFERENCE",
"HOLD_OBJECT_LEVEL_CROSSMATCH",
"HOLD_SELECTION_FUNCTION_FIT",
"HOLD_COSMOLOGY_FIT",
],
}
receipt = {
"receipt_type": "stellar_gas_multiscale_eigenmass_alignment_receipt",
"created": created,
"row_rows": row["row_count"],
"cell_count": cell["cell_count"],
"tracer_subspace_cosine": result["alignment"]["tracer_subspace_cosine"],
"constraint_sharpening_factor": result["alignment"]["constraint_sharpening_factor"],
"decision": result["decision"],
"validated_outputs": [
str(OUT_JSON.relative_to(ROOT)),
str(DOC_MD.relative_to(ROOT)),
str(TIDDLER.relative_to(ROOT)),
],
}
return result, receipt
def write_docs(result: dict[str, Any]) -> None:
align = result["alignment"]
row = result["row_level"]
cell = result["cell_level"]
holds = "\n".join(f"- `{hold}`" for hold in result["holds"])
tracer_lines = "\n".join(
f"- `{name}`: row `{row['tracer_subvector'][i]}`, cell `{cell['tracer_subvector'][i]}`"
for i, name in enumerate(TRACER_ORDER)
)
DOC_MD.write_text(
f"""# Stellar Gas Multiscale Eigenmass Alignment
Status: `MULTISCALE_EIGENMASS_ALIGNMENT`
Decision: `{result['decision']}`
This probe compares the row-level DESI epoviz eigenmass with the DESI/MaNGA
joined-cell eigenmass. It reports a tracer-subspace cosine and explained-share
ratio between the literal row data and the coarse joined-cell overlap surface.
Claim boundary: this is not physical mass, not stellar mass, not gas-density
inference, and not a cosmology fit.
## Alignment Result
Tracer-subspace cosine:
```text
{align['tracer_subspace_cosine']}
```
Alignment class:
```text
{align['alignment_class']}
```
Constraint sharpening factor:
```text
{align['constraint_sharpening_factor']}
```
Dominant eigenvalue ratio, cell over row:
```text
{align['dominant_eigenvalue_ratio_cell_over_row']}
```
## Tracer Subvectors
{tracer_lines}
## Scale Comparison
```text
row level rows: {row['cell_or_row_count']}
row explained share: {row['dominant_explained_mass_share']}
cell level cells: {cell['cell_or_row_count']}
cell explained share: {cell['dominant_explained_mass_share']}
```
## Holds
{holds}
""",
encoding="utf-8",
)
TIDDLER.write_text(
f"""title: Stellar Gas Multiscale Eigenmass Alignment
tags: StellarGasObservation DESI MaNGA SemanticMassNumbers Eigenvector Receipts
type: text/vnd.tiddlywiki
Status: <<tag MULTISCALE_EIGENMASS_ALIGNMENT>>
Decision: `{result['decision']}`
This tiddler compares the row-level DESI epoviz eigenmass with the DESI/MaNGA
joined-cell eigenmass.
Tracer-subspace cosine:
```
{align['tracer_subspace_cosine']}
```
Alignment class:
```
{align['alignment_class']}
```
Constraint sharpening factor:
```
{align['constraint_sharpening_factor']}
```
!! Tracer Subvectors
{tracer_lines}
!! Boundary
This is SMN/evidence-load alignment, not physical mass or cosmology inference.
""",
encoding="utf-8",
)
def main() -> None:
result, receipt = build()
OUT_DIR.mkdir(parents=True, exist_ok=True)
DOCS_DIR.mkdir(parents=True, exist_ok=True)
TIDDLER_DIR.mkdir(parents=True, exist_ok=True)
OUT_JSON.write_text(json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT_JSON.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_docs(result)
print(json.dumps(receipt, indent=2, sort_keys=True))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,620 @@
#!/usr/bin/env python3
"""Population study for MaNGA stellar-gas groupings.
This is the first local population layer for the DESI -> environment prior ->
stellar-gas distribution bridge. It groups MaNGA DAPall galaxies by redshift,
sky cell, BPT proxy, shock/LIER proxy, gas sigma, and stellar sigma. DESI is
kept as a join target only: no direct DESI gas-map claim is made here.
"""
from __future__ import annotations
import argparse
import importlib.util
import json
import math
import statistics
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
SEED_SCRIPT = REPO / "4-Infrastructure/shim/sdss_manga_dapall_observation_seed.py"
DATA_DIR = REPO / "shared-data/data/stellar_gas_observation"
CHANNELS = DATA_DIR / "sdss_manga_dr17_emission_line_channels.json"
DEFAULT_FITS = REPO / "shared-data/artifacts/stellar_gas_observation/dapall-v3_1_1-3.1.0.fits"
DESTINATION = "Gdrive:topological_storage/research-stack/stellar-gas-observation/seed-2026-05-09"
OUT = DATA_DIR / "stellar_gas_population_grouping_study.json"
DOC = REPO / "6-Documentation/docs/stellar_gas_population_grouping_study_2026-05-09.md"
TIDDLER = REPO / "6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Population Grouping Study.tid"
PREFERRED_DAPTYPE = "HYB10-MILESHC-MASTARSSP"
TARGET_COLUMNS = [
"PLATEIFU",
"MANGAID",
"DAPTYPE",
"OBJRA",
"OBJDEC",
"Z",
"BINSNR",
"SNR_MED",
"STELLAR_SIGMA_1RE",
"HA_GSIGMA_1RE",
"EMLINE_GFLUX_1RE",
]
def now_iso() -> str:
return datetime.now(timezone.utc).astimezone().isoformat(timespec="seconds")
def load_seed_module():
spec = importlib.util.spec_from_file_location("sdss_manga_dapall_observation_seed", SEED_SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"cannot load {SEED_SCRIPT}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def load_channels() -> dict[str, int]:
payload = json.loads(CHANNELS.read_text(encoding="utf-8"))
return {row["label"]: row["index0"] for row in payload["channels"]}
def run(cmd: list[str]) -> subprocess.CompletedProcess:
return subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False)
def rclone_copyto(local: Path, remote: str) -> tuple[bool, str]:
proc = run(["rclone", "copyto", str(local), remote, "--checksum"])
message = (proc.stderr or proc.stdout).decode(errors="replace").strip()
return proc.returncode == 0, message
def finite(value: Any) -> bool:
return isinstance(value, (int, float)) and math.isfinite(value) and value > -900
def pos(value: Any) -> float | None:
if finite(value) and value > 0:
return float(value)
return None
def scalar(value: Any) -> float | None:
if finite(value):
return float(value)
return None
def mean_list(value: Any) -> float | None:
if not isinstance(value, list):
return None
vals = [float(v) for v in value if finite(v)]
return statistics.fmean(vals) if vals else None
def log_ratio(num: float | None, den: float | None) -> float | None:
if num is None or den is None or num <= 0 or den <= 0:
return None
return math.log10(num / den)
def ratio(num: float | None, den: float | None) -> float | None:
if num is None or den is None or den <= 0:
return None
return num / den
def classify_bpt(log_nii_ha: float | None, log_oiii_hb: float | None) -> str:
if log_nii_ha is None or log_oiii_hb is None:
return "unclassified"
if log_nii_ha >= 0.47:
return "agn_liner_or_shock_proxy"
kewley = 0.61 / (log_nii_ha - 0.47) + 1.19
kauffmann = 0.61 / (log_nii_ha - 0.05) + 1.3
if log_oiii_hb > kewley:
return "agn_liner_or_shock_proxy"
if log_oiii_hb > kauffmann:
return "composite_proxy"
return "star_forming_proxy"
def classify_shock(log_sii_ha: float | None, log_oi_ha: float | None, gas_sigma: float | None) -> tuple[float, str]:
score = 0.0
if log_sii_ha is not None and log_sii_ha > -0.4:
score += 0.35
if log_oi_ha is not None and log_oi_ha > -1.1:
score += 0.35
if gas_sigma is not None and gas_sigma > 120:
score += 0.30
if score >= 0.65:
return min(1.0, score), "shock_lier_proxy"
if score > 0:
return score, "partial_shock_proxy"
return 0.0, "no_shock_proxy"
def z_bin(z: float | None) -> str:
if z is None:
return "z_missing"
if z < 0.02:
return "z_000_002"
if z < 0.04:
return "z_002_004"
if z < 0.06:
return "z_004_006"
if z < 0.08:
return "z_006_008"
return "z_008_plus"
def sigma_bin(value: float | None, prefix: str) -> str:
if value is None:
return f"{prefix}_missing"
if value < 50:
return f"{prefix}_000_050"
if value < 100:
return f"{prefix}_050_100"
if value < 150:
return f"{prefix}_100_150"
if value < 250:
return f"{prefix}_150_250"
return f"{prefix}_250_plus"
def sky_bin(ra: float | None, dec: float | None) -> str:
if ra is None or dec is None:
return "sky_missing"
ra_bin = int(max(0, min(5, math.floor((ra % 360.0) / 60.0))))
dec_band = "south" if dec < 0 else "north"
return f"ra{ra_bin:02d}_{dec_band}"
def sky_z_cell(ra: float | None, dec: float | None, z: float | None) -> str:
return f"{sky_bin(ra, dec)}__{z_bin(z)}"
def summarize(vals: list[float]) -> dict[str, Any]:
values = sorted(v for v in vals if math.isfinite(v))
if not values:
return {"count": 0}
return {
"count": len(values),
"min": round(values[0], 6),
"max": round(values[-1], 6),
"mean": round(statistics.fmean(values), 6),
"median": round(statistics.median(values), 6),
"p90": round(values[int(0.9 * (len(values) - 1))], 6),
}
def iter_rows(fits_path: Path):
seed = load_seed_module()
with fits_path.open("rb") as f:
hdu_index = 0
while True:
try:
header, _ = seed.read_header(f)
except EOFError:
break
data_start = f.tell()
if str(header.get("XTENSION", "PRIMARY")) == "BINTABLE":
row_len = int(header["NAXIS1"])
row_count = int(header["NAXIS2"])
pcount = int(header.get("PCOUNT", 0))
columns = seed.build_columns(header)
by_name = {col["name"]: col for col in columns}
selected = [by_name[name] for name in TARGET_COLUMNS if name in by_name]
hdu_name = str(header.get("EXTNAME", f"HDU{hdu_index}"))
for row_idx in range(row_count):
f.seek(data_start + row_idx * row_len)
row = f.read(row_len)
fields: dict[str, Any] = {}
for col in selected:
raw = row[col["offset"] : col["offset"] + col["width"]]
value = seed.decode_value(raw, col)
if isinstance(value, str):
value = value.replace("\u0000", "").strip()
fields[col["name"]] = value
yield hdu_index, hdu_name, row_idx, fields
f.seek(data_start + seed.padded_size(row_len * row_count + pcount))
else:
bitpix = int(header.get("BITPIX", 8))
naxis = int(header.get("NAXIS", 0))
data_size = 0
if naxis:
pixels = 1
for axis in range(1, naxis + 1):
pixels *= int(header.get(f"NAXIS{axis}", 0))
data_size = abs(bitpix) // 8 * pixels
f.seek(data_start + seed.padded_size(data_size))
hdu_index += 1
def add_count(bucket: dict[str, int], key: str) -> None:
bucket[key] = bucket.get(key, 0) + 1
def group_template() -> dict[str, Any]:
return {
"count": 0,
"bpt_proxy_classes": {},
"shock_lier_proxy_classes": {},
"z_bins": {},
"gas_sigma_bins": {},
"stellar_sigma_bins": {},
"sky_bins": {},
"shock_scores": [],
"gas_sigma_values": [],
"stellar_sigma_values": [],
"snr_values": [],
}
def update_group(group: dict[str, Any], row: dict[str, Any]) -> None:
group["count"] += 1
add_count(group["bpt_proxy_classes"], row["bpt_proxy_class"])
add_count(group["shock_lier_proxy_classes"], row["shock_lier_proxy_class"])
add_count(group["z_bins"], row["z_bin"])
add_count(group["gas_sigma_bins"], row["gas_sigma_bin"])
add_count(group["stellar_sigma_bins"], row["stellar_sigma_bin"])
add_count(group["sky_bins"], row["sky_bin"])
group["shock_scores"].append(row["shock_lier_score"])
if row["gas_sigma_1re_kms"] is not None:
group["gas_sigma_values"].append(row["gas_sigma_1re_kms"])
if row["stellar_sigma_1re_kms"] is not None:
group["stellar_sigma_values"].append(row["stellar_sigma_1re_kms"])
if row["snr_mean"] is not None:
group["snr_values"].append(row["snr_mean"])
def finalize_group(group: dict[str, Any]) -> dict[str, Any]:
count = group["count"] or 1
shock = group["shock_lier_proxy_classes"]
group["shock_lier_fraction"] = round(shock.get("shock_lier_proxy", 0) / count, 6)
group["partial_or_full_shock_fraction"] = round(
(shock.get("shock_lier_proxy", 0) + shock.get("partial_shock_proxy", 0)) / count,
6,
)
group["shock_score_summary"] = summarize(group.pop("shock_scores"))
group["gas_sigma_summary"] = summarize(group.pop("gas_sigma_values"))
group["stellar_sigma_summary"] = summarize(group.pop("stellar_sigma_values"))
group["snr_summary"] = summarize(group.pop("snr_values"))
return group
def row_payload(fields: dict[str, Any], channel_index: dict[str, int]) -> dict[str, Any] | None:
flux = fields.get("EMLINE_GFLUX_1RE")
if not isinstance(flux, list) or len(flux) < 35:
return None
ha = pos(flux[channel_index["Ha-6564"]])
hb = pos(flux[channel_index["Hb-4862"]])
oiii = pos(flux[channel_index["OIII-5008"]])
nii = pos(flux[channel_index["NII-6585"]])
sii_1 = pos(flux[channel_index["SII-6718"]])
sii_2 = pos(flux[channel_index["SII-6732"]])
sii = sii_1 + sii_2 if sii_1 is not None and sii_2 is not None else None
oi = pos(flux[channel_index["OI-6302"]])
gas_sigma = pos(fields.get("HA_GSIGMA_1RE"))
stellar_sigma = pos(fields.get("STELLAR_SIGMA_1RE"))
z = scalar(fields.get("Z"))
ra = scalar(fields.get("OBJRA"))
dec = scalar(fields.get("OBJDEC"))
log_nii_ha = log_ratio(nii, ha)
log_sii_ha = log_ratio(sii, ha)
log_oi_ha = log_ratio(oi, ha)
log_oiii_hb = log_ratio(oiii, hb)
balmer = ratio(ha, hb)
shock_score, shock_class = classify_shock(log_sii_ha, log_oi_ha, gas_sigma)
bpt = classify_bpt(log_nii_ha, log_oiii_hb)
return {
"plateifu": fields.get("PLATEIFU"),
"mangaid": fields.get("MANGAID"),
"daptype": fields.get("DAPTYPE"),
"ra": ra,
"dec": dec,
"z": z,
"z_bin": z_bin(z),
"sky_bin": sky_bin(ra, dec),
"sky_z_cell": sky_z_cell(ra, dec, z),
"snr_mean": mean_list(fields.get("SNR_MED")) or scalar(fields.get("BINSNR")),
"gas_sigma_1re_kms": gas_sigma,
"stellar_sigma_1re_kms": stellar_sigma,
"gas_sigma_bin": sigma_bin(gas_sigma, "gas_sigma"),
"stellar_sigma_bin": sigma_bin(stellar_sigma, "stellar_sigma"),
"line_ratios": {
"log_NII6585_Ha": log_nii_ha,
"log_SII6718_6732_Ha": log_sii_ha,
"log_OI6302_Ha": log_oi_ha,
"log_OIII5008_Hb": log_oiii_hb,
"Ha_Hb": balmer,
},
"bpt_proxy_class": bpt,
"shock_lier_proxy_class": shock_class,
"shock_lier_score": shock_score,
}
def build_population_study(fits_path: Path, preferred_daptype: str) -> dict[str, Any]:
channel_index = load_channels()
required = ["Ha-6564", "Hb-4862", "OIII-5008", "NII-6585", "SII-6718", "SII-6732", "OI-6302"]
missing = [name for name in required if name not in channel_index]
if missing:
raise RuntimeError(f"missing channel labels: {missing}")
all_rows = 0
selected_rows: list[dict[str, Any]] = []
by_plateifu: dict[str, dict[str, Any]] = {}
daptype_counts: dict[str, int] = {}
for _, _, _, fields in iter_rows(fits_path):
all_rows += 1
payload = row_payload(fields, channel_index)
if not payload:
continue
daptype = str(payload.get("daptype") or "unknown")
add_count(daptype_counts, daptype)
plateifu = str(payload.get("plateifu") or "")
current = by_plateifu.get(plateifu)
if current is None or daptype == preferred_daptype:
by_plateifu[plateifu] = payload
selected_rows = list(by_plateifu.values())
groups = {
"by_bpt_proxy_class": {},
"by_shock_lier_proxy_class": {},
"by_redshift_bin": {},
"by_sky_bin": {},
"by_sky_z_cell": {},
"by_gas_sigma_bin": {},
"by_stellar_sigma_bin": {},
}
aggregate = group_template()
for row in selected_rows:
update_group(aggregate, row)
for group_name, key_name in [
("by_bpt_proxy_class", "bpt_proxy_class"),
("by_shock_lier_proxy_class", "shock_lier_proxy_class"),
("by_redshift_bin", "z_bin"),
("by_sky_bin", "sky_bin"),
("by_sky_z_cell", "sky_z_cell"),
("by_gas_sigma_bin", "gas_sigma_bin"),
("by_stellar_sigma_bin", "stellar_sigma_bin"),
]:
key = row[key_name]
groups[group_name].setdefault(key, group_template())
update_group(groups[group_name][key], row)
finalized_groups = {
group_name: {
key: finalize_group(value)
for key, value in sorted(group.items(), key=lambda item: (-item[1]["count"], item[0]))
}
for group_name, group in groups.items()
}
top_cells = [
{"cell": key, **value}
for key, value in list(finalized_groups["by_sky_z_cell"].items())[:20]
]
examples = sorted(
selected_rows,
key=lambda row: (row["shock_lier_score"], row["gas_sigma_1re_kms"] or 0.0),
reverse=True,
)[:20]
return {
"schema": "stellar_gas_population_grouping_study_v1",
"created": now_iso(),
"decision": "ADMIT_POPULATION_GROUPING_SURFACE",
"claim_boundary": "MaNGA stellar-gas population grouping only. A coarse DESI/MaNGA cell join exists; object-level crossmatch remains HOLD. Proxy classes do not prove physical shock, AGN, or gas mechanism.",
"source_fits": str(fits_path.relative_to(REPO)) if fits_path.is_relative_to(REPO) else str(fits_path),
"channel_map": str(CHANNELS.relative_to(REPO)),
"preferred_daptype": preferred_daptype,
"rows_seen_all_daptypes": all_rows,
"daptype_counts": daptype_counts,
"unique_plateifu_count": len(by_plateifu),
"selected_population_count": len(selected_rows),
"aggregate": finalize_group(aggregate),
"groups": finalized_groups,
"top_sky_z_cells_for_desi_join": top_cells,
"top_shock_lier_examples": examples,
"desi_bridge": {
"status": "COARSE_CELL_JOIN_EXISTS_OBJECT_CROSSMATCH_HOLD",
"source_prior": "shared-data/data/stack_solidification/desi_stellar_gas_distribution_prior.json",
"join_key_shape": "coarse sky/redshift population cell; object-level cone/crossmatch remains HOLD",
"required_fields": ["ra", "dec", "z", "tracer_type", "selection_flags"],
},
}
def write_doc(result: dict[str, Any]) -> None:
agg = result["aggregate"]
groups = result["groups"]
lines = [
"# Stellar Gas Population Grouping Study",
"",
"**Date:** 2026-05-09",
"",
f"**Decision:** `{result['decision']}`",
"",
"**Claim boundary:** MaNGA stellar-gas population grouping only. DESI",
"environment inference remains a prior bridge until a DESI-MaNGA join",
"receipt exists. Proxy classes do not prove physical shock, AGN, or gas",
"mechanism.",
"",
"## Population Surface",
"",
"```text",
f"rows seen, all DAP types: {result['rows_seen_all_daptypes']}",
f"unique Plate-IFU count: {result['unique_plateifu_count']}",
f"selected population: {result['selected_population_count']}",
f"preferred DAPTYPE: {result['preferred_daptype']}",
"```",
"",
"## Aggregate",
"",
"```json",
json.dumps(
{
"bpt_proxy_classes": agg["bpt_proxy_classes"],
"shock_lier_proxy_classes": agg["shock_lier_proxy_classes"],
"partial_or_full_shock_fraction": agg["partial_or_full_shock_fraction"],
"gas_sigma_summary": agg["gas_sigma_summary"],
"stellar_sigma_summary": agg["stellar_sigma_summary"],
},
indent=2,
),
"```",
"",
"## Redshift Bins",
"",
"| Bin | Count | Shock+Partial Fraction | Gas Sigma Median | Stellar Sigma Median |",
"|---|---:|---:|---:|---:|",
]
for key, value in groups["by_redshift_bin"].items():
lines.append(
f"| `{key}` | {value['count']} | {value['partial_or_full_shock_fraction']} | "
f"{value['gas_sigma_summary'].get('median', '')} | {value['stellar_sigma_summary'].get('median', '')} |"
)
lines += [
"",
"## DESI-Ready Sky/Redshift Cells",
"",
"These are population cells for a later DESI join. They are not DESI",
"environment classes yet.",
"",
"| Cell | Count | Shock+Partial Fraction | Main BPT Counts |",
"|---|---:|---:|---|",
]
for row in result["top_sky_z_cells_for_desi_join"][:12]:
lines.append(
f"| `{row['cell']}` | {row['count']} | {row['partial_or_full_shock_fraction']} | "
f"`{row['bpt_proxy_classes']}` |"
)
lines += [
"",
"## What This Gives Us",
"",
"- A population baseline over unique MaNGA Plate-IFU rows.",
"- Grouped shock/LIER and BPT proxy counts by redshift and sky cell.",
"- DESI-ready cells for a future sky-cone plus redshift-window join.",
"- Residual target: cells whose local gas state diverges from later DESI environment priors.",
"",
"## Receipt",
"",
"`shared-data/data/stellar_gas_observation/stellar_gas_population_grouping_study_receipt_*.json`",
"",
]
DOC.write_text("\n".join(lines) + "\n", encoding="utf-8")
def write_tiddler(result: dict[str, Any]) -> None:
agg = result["aggregate"]
TIDDLER.write_text(
f"""created: 20260509224000000
modified: 20260509224000000
tags: ResearchStack StellarGas MaNGA PopulationStudy DESI Calibration
title: Stellar Gas Population Grouping Study
type: text/vnd.tiddlywiki
! Stellar Gas Population Grouping Study
Status: `ADMIT_POPULATION_GROUPING_SURFACE`
This page records the first local population grouping pass over MaNGA DAPall
stellar-gas diagnostics.
```text
unique Plate-IFU count: {result['unique_plateifu_count']}
selected population: {result['selected_population_count']}
preferred DAPTYPE: {result['preferred_daptype']}
```
!! Aggregate
```json
{json.dumps({
'bpt_proxy_classes': agg['bpt_proxy_classes'],
'shock_lier_proxy_classes': agg['shock_lier_proxy_classes'],
'partial_or_full_shock_fraction': agg['partial_or_full_shock_fraction'],
}, indent=2)}
```
!! DESI Bridge
The sky/redshift cells are DESI-ready join buckets, not DESI environment classes
yet.
```text
MaNGA population cell
-> future DESI sky-cone/redshift join
-> environment prior
-> gas-state residual map
```
!! Boundary
Proxy classes do not prove physical shock, AGN, or gas mechanism. DESI
environment inference remains HOLD until the join receipt exists.
""",
encoding="utf-8",
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--fits", type=Path, default=DEFAULT_FITS)
parser.add_argument("--preferred-daptype", default=PREFERRED_DAPTYPE)
parser.add_argument("--destination", default=DESTINATION)
parser.add_argument("--no-upload", action="store_true")
args = parser.parse_args()
result = build_population_study(args.fits, args.preferred_daptype)
OUT.write_text(json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_doc(result)
write_tiddler(result)
receipt_path = DATA_DIR / f"stellar_gas_population_grouping_study_receipt_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json"
receipt: dict[str, Any] = {
"schema": "stellar_gas_population_grouping_study_receipt_v1",
"created": now_iso(),
"claim_boundary": result["claim_boundary"],
"study_file": str(OUT.relative_to(REPO)),
"doc_file": str(DOC.relative_to(REPO)),
"tiddler_file": str(TIDDLER.relative_to(REPO)),
"source_fits": result["source_fits"],
"decision": result["decision"],
"summary": {
"unique_plateifu_count": result["unique_plateifu_count"],
"selected_population_count": result["selected_population_count"],
"partial_or_full_shock_fraction": result["aggregate"]["partial_or_full_shock_fraction"],
"desi_bridge_status": result["desi_bridge"]["status"],
},
"uploads": {},
}
receipt_path.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
if not args.no_upload:
uploads = {
"study": (OUT, f"{args.destination}/derived/{OUT.name}"),
"doc": (DOC, f"{args.destination}/docs/{DOC.name}"),
"tiddler": (TIDDLER, f"{args.destination}/docs/{TIDDLER.name.replace(' ', '_')}"),
"receipt": (receipt_path, f"{args.destination}/receipts/{receipt_path.name}"),
}
for key, (local, remote) in uploads.items():
ok, message = rclone_copyto(local, remote)
receipt["uploads"][key] = {"drive_path": remote, "ok": ok, "message": message}
receipt_path.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
if receipt["uploads"].get("receipt", {}).get("ok"):
rclone_copyto(receipt_path, receipt["uploads"]["receipt"]["drive_path"])
print(json.dumps(receipt, indent=2, sort_keys=True))
return 0
if __name__ == "__main__":
sys.exit(main())

View file

@ -0,0 +1,281 @@
#!/usr/bin/env python3
"""Fine-zoom object examples under the stellar-gas sandpile candidates.
This script drills from the sandpile cell diagnostic down to MaNGA Plate-IFU
examples in the avalanche-candidate cells. It keeps the same proxy boundary as
the population study: these are observable gas/shock routing candidates, not
proof of a physical shock mechanism.
"""
from __future__ import annotations
import importlib.util
import json
import math
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
POP_SCRIPT = ROOT / "4-Infrastructure/shim/stellar_gas_population_grouping_study.py"
SANDPILE_JSON = ROOT / "shared-data/data/stellar_gas_observation/stellar_gas_abelian_sandpile_probe.json"
OUT_DIR = ROOT / "shared-data/data/stellar_gas_observation"
DOCS_DIR = ROOT / "6-Documentation/docs"
TIDDLER_DIR = ROOT / "6-Documentation/tiddlywiki-local/wiki/tiddlers"
OUT_JSON = OUT_DIR / "stellar_gas_sandpile_fine_zoom.json"
RECEIPT_JSON = OUT_DIR / "stellar_gas_sandpile_fine_zoom_receipt.json"
DOC_MD = DOCS_DIR / "stellar_gas_sandpile_fine_zoom_2026-05-09.md"
TIDDLER = TIDDLER_DIR / "Stellar Gas Sandpile Fine Zoom.tid"
EXAMPLES_PER_CELL = 8
def load_population_module():
spec = importlib.util.spec_from_file_location("stellar_gas_population_grouping_study", POP_SCRIPT)
if spec is None or spec.loader is None:
raise RuntimeError(f"cannot load {POP_SCRIPT}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def load_json(path: Path) -> dict[str, Any]:
with path.open() as f:
return json.load(f)
def now_iso() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def finite(value: Any) -> bool:
return isinstance(value, (int, float)) and math.isfinite(value)
def round6(value: float | None) -> float | None:
if value is None or not math.isfinite(value):
return None
return round(value, 6)
def object_pressure(row: dict[str, Any]) -> float:
gas_sigma = row.get("gas_sigma_1re_kms") or 0.0
stellar_sigma = row.get("stellar_sigma_1re_kms") or 0.0
snr = row.get("snr_mean") or 0.0
pressure = float(row.get("shock_lier_score") or 0.0)
pressure += min(float(gas_sigma) / 250.0, 2.0)
pressure += min(float(stellar_sigma) / 250.0, 2.0) * 0.5
pressure += min(max(float(snr), 0.0) / 50.0, 1.0) * 0.25
if row.get("bpt_proxy_class") == "agn_liner_or_shock_proxy":
pressure += 0.5
if row.get("shock_lier_proxy_class") == "shock_lier_proxy":
pressure += 0.5
return pressure
def compact_row(row: dict[str, Any]) -> dict[str, Any]:
ratios = row.get("line_ratios", {})
return {
"plateifu": row.get("plateifu"),
"mangaid": row.get("mangaid"),
"ra": round6(row.get("ra")),
"dec": round6(row.get("dec")),
"z": round6(row.get("z")),
"sky_z_cell": row.get("sky_z_cell"),
"bpt_proxy_class": row.get("bpt_proxy_class"),
"shock_lier_proxy_class": row.get("shock_lier_proxy_class"),
"shock_lier_score": round6(row.get("shock_lier_score")),
"gas_sigma_1re_kms": round6(row.get("gas_sigma_1re_kms")),
"stellar_sigma_1re_kms": round6(row.get("stellar_sigma_1re_kms")),
"snr_mean": round6(row.get("snr_mean")),
"object_pressure": round6(object_pressure(row)),
"line_ratios": {key: round6(value) for key, value in ratios.items()},
}
def build() -> tuple[dict[str, Any], dict[str, Any]]:
pop = load_population_module()
sandpile = load_json(SANDPILE_JSON)
candidate_cells = [
row["cell"]
for row in sandpile["top_cells"]
if row["sandpile"]["state"] == "AVALANCHE_CANDIDATE"
]
candidate_set = set(candidate_cells)
channel_index = pop.load_channels()
by_plateifu: dict[str, dict[str, Any]] = {}
rows_seen = 0
rows_in_candidate_cells = 0
for _, _, _, fields in pop.iter_rows(pop.DEFAULT_FITS):
rows_seen += 1
payload = pop.row_payload(fields, channel_index)
if not payload:
continue
daptype = str(payload.get("daptype") or "")
plateifu = str(payload.get("plateifu") or "")
current = by_plateifu.get(plateifu)
if current is None or daptype == pop.PREFERRED_DAPTYPE:
by_plateifu[plateifu] = payload
examples_by_cell: dict[str, list[dict[str, Any]]] = {cell: [] for cell in candidate_cells}
for row in by_plateifu.values():
cell = row.get("sky_z_cell")
if cell not in candidate_set:
continue
rows_in_candidate_cells += 1
examples_by_cell[cell].append(row)
output_cells = []
for sand_row in sandpile["top_cells"]:
cell = sand_row["cell"]
if cell not in candidate_set:
continue
examples = sorted(
examples_by_cell[cell],
key=lambda row: object_pressure(row),
reverse=True,
)[:EXAMPLES_PER_CELL]
output_cells.append(
{
"cell": cell,
"sandpile": sand_row["sandpile"],
"channel_summary": sand_row["channels"],
"candidate_count": len(examples_by_cell[cell]),
"top_examples": [compact_row(row) for row in examples],
}
)
created = now_iso()
result = {
"schema": "stellar_gas_sandpile_fine_zoom_v0",
"created": created,
"decision": "ADMIT_FINE_ZOOM_OBJECT_EXAMPLES_HOLD_MECHANISM_PROOF",
"claim_boundary": (
"Fine-zoom object examples under sandpile avalanche-candidate cells. "
"Proxy classes and object-pressure scores route follow-up; they do "
"not prove physical shock, AGN, stellar mass, or gas mechanism."
),
"sources": {
"sandpile_probe": str(SANDPILE_JSON.relative_to(ROOT)),
"manga_fits": str(pop.DEFAULT_FITS.relative_to(ROOT)),
"population_script": str(POP_SCRIPT.relative_to(ROOT)),
},
"rows_seen_all_daptypes": rows_seen,
"unique_plateifu_count": len(by_plateifu),
"candidate_cell_count": len(candidate_cells),
"rows_in_candidate_cells": rows_in_candidate_cells,
"examples_per_cell": EXAMPLES_PER_CELL,
"candidate_cells": output_cells,
"holds": [
"HOLD_PHYSICAL_SHOCK_PROOF",
"HOLD_DIRECT_STELLAR_MASS",
"HOLD_DIRECT_GAS_DENSITY_INFERENCE",
"HOLD_OBJECT_LEVEL_DESI_CROSSMATCH",
"HOLD_COSMOLOGY_FIT",
],
}
receipt = {
"receipt_type": "stellar_gas_sandpile_fine_zoom_receipt",
"created": created,
"candidate_cell_count": len(candidate_cells),
"rows_in_candidate_cells": rows_in_candidate_cells,
"examples_written": sum(len(cell["top_examples"]) for cell in output_cells),
"decision": result["decision"],
"validated_outputs": [
str(OUT_JSON.relative_to(ROOT)),
str(DOC_MD.relative_to(ROOT)),
str(TIDDLER.relative_to(ROOT)),
],
}
return result, receipt
def write_docs(result: dict[str, Any]) -> None:
cell_lines = []
for cell in result["candidate_cells"]:
top = cell["top_examples"][0] if cell["top_examples"] else {}
cell_lines.append(
f"- `{cell['cell']}`: candidates `{cell['candidate_count']}`, "
f"top `{top.get('plateifu')}`, pressure `{top.get('object_pressure')}`, "
f"class `{top.get('shock_lier_proxy_class')}`"
)
holds = "\n".join(f"- `{hold}`" for hold in result["holds"])
DOC_MD.write_text(
f"""# Stellar Gas Sandpile Fine Zoom
Status: `FINE_ZOOM_OBJECT_EXAMPLES`
Decision: `{result['decision']}`
This fine-zoom pass drills from sandpile avalanche-candidate cells down to MaNGA
Plate-IFU examples. It is meant to show which concrete objects carry the
strongest local proxy pressure under the cell-level eigenmass surface.
Claim boundary: proxy classes and object-pressure scores route follow-up; they
do not prove physical shock, AGN, stellar mass, gas density, or cosmology.
## Summary
```text
candidate cells: {result['candidate_cell_count']}
candidate-cell objects: {result['rows_in_candidate_cells']}
examples per cell: {result['examples_per_cell']}
```
## Top Object Per Candidate Cell
{chr(10).join(cell_lines)}
## Holds
{holds}
""",
encoding="utf-8",
)
TIDDLER.write_text(
f"""title: Stellar Gas Sandpile Fine Zoom
tags: StellarGasObservation MaNGA SemanticMassNumbers Sandpile Receipts
type: text/vnd.tiddlywiki
Status: <<tag FINE_ZOOM_OBJECT_EXAMPLES>>
Decision: `{result['decision']}`
This tiddler drills from the Abelian-sandpile candidate cells into MaNGA
Plate-IFU examples.
```
candidate cells: {result['candidate_cell_count']}
candidate-cell objects: {result['rows_in_candidate_cells']}
examples per cell: {result['examples_per_cell']}
```
!! Top Object Per Candidate Cell
{chr(10).join(cell_lines)}
!! Boundary
These are proxy-ranked follow-up candidates, not proof of physical shock, AGN,
stellar mass, gas density, or cosmology.
""",
encoding="utf-8",
)
def main() -> None:
result, receipt = build()
OUT_JSON.write_text(json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT_JSON.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_docs(result)
print(json.dumps(receipt, indent=2, sort_keys=True))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,444 @@
#!/usr/bin/env python3
"""Graph replay hardening for the stellar-gas sandpile diagnostic.
This converts the existing sandpile metaphor into a reproducible graph
diagnostic. It is a toppling proxy over sky/redshift cells, not a physical
sandpile simulation and not a claim about stellar gas mechanics.
"""
from __future__ import annotations
import hashlib
import json
import math
from collections import deque
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[2]
SANDPILE_JSON = ROOT / "shared-data/data/stellar_gas_observation/stellar_gas_abelian_sandpile_probe.json"
FINE_ZOOM_JSON = ROOT / "shared-data/data/stellar_gas_observation/stellar_gas_sandpile_fine_zoom.json"
OUT_DIR = ROOT / "shared-data/data/stellar_gas_observation"
DOCS_DIR = ROOT / "6-Documentation/docs"
TIDDLER_DIR = ROOT / "6-Documentation/tiddlywiki-local/wiki/tiddlers"
OUT_JSON = OUT_DIR / "stellar_gas_sandpile_graph_replay.json"
RECEIPT_JSON = OUT_DIR / "stellar_gas_sandpile_graph_replay_receipt.json"
DOC_MD = DOCS_DIR / "stellar_gas_sandpile_graph_replay_2026-05-09.md"
TIDDLER = TIDDLER_DIR / "Stellar Gas Sandpile Graph Replay.tid"
Z_BIN_ORDER = {
"z_000_002": 0,
"z_002_004": 1,
"z_004_006": 2,
"z_006_008": 3,
"z_008_plus": 4,
}
def load_json(path: Path) -> dict[str, Any]:
with path.open(encoding="utf-8") as f:
return json.load(f)
def sha256_file(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def sha256_payload(payload: Any) -> str:
raw = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode("utf-8")
return hashlib.sha256(raw).hexdigest()
def now_iso() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def parse_cell(cell: str) -> dict[str, Any]:
sky_sector, z_bin = cell.split("__", 1)
ra_sector, hemisphere = sky_sector.split("_", 1)
return {
"cell": cell,
"sky_sector": sky_sector,
"ra_sector": int(ra_sector.removeprefix("ra")),
"hemisphere": hemisphere,
"z_bin": z_bin,
"z_bin_index": Z_BIN_ORDER[z_bin],
}
def are_adjacent(a: dict[str, Any], b: dict[str, Any]) -> bool:
same_z = a["z_bin_index"] == b["z_bin_index"]
same_ra = a["ra_sector"] == b["ra_sector"]
same_hemisphere = a["hemisphere"] == b["hemisphere"]
ra_neighbor = same_z and same_hemisphere and abs(a["ra_sector"] - b["ra_sector"]) == 1
hemisphere_neighbor = same_z and same_ra and a["hemisphere"] != b["hemisphere"]
redshift_neighbor = same_ra and same_hemisphere and abs(a["z_bin_index"] - b["z_bin_index"]) == 1
return ra_neighbor or hemisphere_neighbor or redshift_neighbor
def round9(value: float) -> float:
return round(value, 9)
def seed_counts(fine_zoom: dict[str, Any]) -> dict[str, int]:
return {
cell["cell"]: int(cell["candidate_count"])
for cell in fine_zoom["candidate_cells"]
}
def node_rows(sandpile: dict[str, Any], fine_zoom: dict[str, Any]) -> list[dict[str, Any]]:
counts = seed_counts(fine_zoom)
index_std = float(sandpile["toppling_index_summary"]["std"]) or 1.0
rows = []
for row in sorted(sandpile["top_cells"], key=lambda item: item["cell"]):
parsed = parse_cell(row["cell"])
proxy_z = float(row["sandpile"]["toppling_index"]) / index_std
rows.append(
{
**parsed,
"state": row["sandpile"]["state"],
"grains": float(row["sandpile"]["grains"]),
"toppling_pressure": float(row["sandpile"]["toppling_pressure"]),
"toppling_index": float(row["sandpile"]["toppling_index"]),
"toppling_index_z_proxy": round9(proxy_z),
"candidate_object_count": counts.get(row["cell"], 0),
}
)
return rows
def graph_edges(rows: list[dict[str, Any]]) -> list[dict[str, str]]:
edges = []
for i, left in enumerate(rows):
for right in rows[i + 1 :]:
if are_adjacent(left, right):
edges.append({"source": left["cell"], "target": right["cell"]})
return edges
def adjacency(nodes: list[str], edges: list[dict[str, str]]) -> dict[str, list[str]]:
out = {node: [] for node in nodes}
for edge in edges:
out[edge["source"]].append(edge["target"])
out[edge["target"]].append(edge["source"])
return {node: sorted(neighbors) for node, neighbors in out.items()}
def threshold_and_grain_tables(rows: list[dict[str, Any]], adj: dict[str, list[str]]) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
threshold_table = []
grain_table = []
for row in rows:
cell = row["cell"]
degree = len(adj[cell])
threshold = max(1, degree + 1)
proxy_z = max(0.0, row["toppling_index_z_proxy"])
initial_grains = max(0, math.ceil(proxy_z * threshold))
threshold_table.append(
{
"cell": cell,
"degree": degree,
"toppling_threshold": threshold,
"threshold_rule": "degree_plus_one_toppling_proxy",
}
)
grain_table.append(
{
"cell": cell,
"initial_grains": initial_grains,
"grain_rule": "ceil(max(0, toppling_index_z_proxy) * threshold)",
"toppling_index_z_proxy": row["toppling_index_z_proxy"],
"seed_object_count": row["candidate_object_count"],
"source_state": row["state"],
}
)
return threshold_table, grain_table
def replay_one(seed: str, adj: dict[str, list[str]], thresholds: dict[str, int], grains: dict[str, int]) -> dict[str, Any]:
working = dict(grains)
topple_count = {cell: 0 for cell in working}
touched = set()
q: deque[str] = deque([seed])
max_steps = max(1, len(working) * 20)
steps = 0
while q and steps < max_steps:
cell = q.popleft()
if working[cell] < thresholds[cell]:
continue
steps += 1
touched.add(cell)
topple_count[cell] += 1
working[cell] -= thresholds[cell]
for neighbor in adj[cell]:
working[neighbor] += 1
if working[neighbor] >= thresholds[neighbor]:
q.append(neighbor)
toppled_cells = sorted(cell for cell, count in topple_count.items() if count)
return {
"seed_cell": seed,
"terminated": not q,
"step_limit": max_steps,
"topple_events": sum(topple_count.values()),
"avalanche_size_cells": len(toppled_cells),
"toppled_cells": toppled_cells,
"final_seed_grains": working[seed],
}
def build() -> tuple[dict[str, Any], dict[str, Any]]:
sandpile = load_json(SANDPILE_JSON)
fine_zoom = load_json(FINE_ZOOM_JSON)
rows = node_rows(sandpile, fine_zoom)
nodes = [row["cell"] for row in rows]
edges = graph_edges(rows)
adj = adjacency(nodes, edges)
threshold_table, grain_table = threshold_and_grain_tables(rows, adj)
thresholds = {row["cell"]: int(row["toppling_threshold"]) for row in threshold_table}
grains = {row["cell"]: int(row["initial_grains"]) for row in grain_table}
seed_cells = sorted(
row["cell"]
for row in rows
if row["state"] == "AVALANCHE_CANDIDATE" and row["candidate_object_count"] > 0
)
avalanches = [replay_one(seed, adj, thresholds, grains) for seed in seed_cells]
canonical_graph = {
"adjacency_rule": (
"Edges join same-redshift neighboring RA sectors, same-RA north/south sectors, "
"or same-sky-sector adjacent redshift bins."
),
"nodes": [
{
"cell": row["cell"],
"ra_sector": row["ra_sector"],
"hemisphere": row["hemisphere"],
"z_bin": row["z_bin"],
}
for row in rows
],
"edges": edges,
"threshold_table": threshold_table,
"initial_grain_table": grain_table,
}
graph_hash = sha256_payload(canonical_graph)
replay_payload = {
"graph_hash": graph_hash,
"seed_cells": seed_cells,
"avalanche_replay": avalanches,
}
replay_hash = sha256_payload(replay_payload)
created = now_iso()
result = {
"schema": "stellar_gas_sandpile_graph_replay_v0",
"created": created,
"decision": "ADMIT_GRAPH_TOPPLING_PROXY_HOLD_PHYSICAL_SANDPILE_SIMULATION",
"claim_boundary": (
"This is a reproducible graph diagnostic and toppling proxy over existing "
"stellar-gas evidence cells. It is not a physical sandpile simulation, "
"not a stellar-gas mechanism proof, and not a cosmology fit."
),
"sources": {
"sandpile_probe": str(SANDPILE_JSON.relative_to(ROOT)),
"fine_zoom_examples": str(FINE_ZOOM_JSON.relative_to(ROOT)),
},
"source_hashes": {
"sandpile_probe_sha256": sha256_file(SANDPILE_JSON),
"fine_zoom_examples_sha256": sha256_file(FINE_ZOOM_JSON),
},
"seed_evidence": {
"avalanche_cell_count": len(seed_cells),
"candidate_object_count": int(fine_zoom["rows_in_candidate_cells"]),
"candidate_counts_by_cell": seed_counts(fine_zoom),
},
"adjacency_rule": canonical_graph["adjacency_rule"],
"graph_hash": graph_hash,
"replay_hash": replay_hash,
"node_count": len(nodes),
"edge_count": len(edges),
"nodes": rows,
"edges": edges,
"adjacency": adj,
"toppling_threshold_table": threshold_table,
"initial_grain_table": grain_table,
"avalanche_sizes": [
{
"seed_cell": row["seed_cell"],
"avalanche_size_cells": row["avalanche_size_cells"],
"topple_events": row["topple_events"],
"terminated": row["terminated"],
}
for row in avalanches
],
"avalanche_replay": avalanches,
"holds": [
"HOLD_PHYSICAL_SANDPILE_SIMULATION",
"HOLD_STELLAR_GAS_MECHANISM_PROOF",
"HOLD_DIRECT_STELLAR_MASS",
"HOLD_DIRECT_GAS_DENSITY_INFERENCE",
"HOLD_COSMOLOGY_FIT",
],
}
receipt = {
"receipt_type": "stellar_gas_sandpile_graph_replay_receipt",
"created": created,
"decision": result["decision"],
"graph_hash": graph_hash,
"replay_hash": replay_hash,
"node_count": len(nodes),
"edge_count": len(edges),
"seed_avalanche_cell_count": len(seed_cells),
"seed_candidate_object_count": int(fine_zoom["rows_in_candidate_cells"]),
"avalanche_sizes": result["avalanche_sizes"],
"validated_outputs": [
str(OUT_JSON.relative_to(ROOT)),
str(RECEIPT_JSON.relative_to(ROOT)),
str(DOC_MD.relative_to(ROOT)),
str(TIDDLER.relative_to(ROOT)),
],
}
return result, receipt
def write_docs(result: dict[str, Any], receipt: dict[str, Any]) -> None:
threshold_lines = "\n".join(
f"| `{row['cell']}` | {row['degree']} | {row['toppling_threshold']} |"
for row in result["toppling_threshold_table"]
)
grain_lines = "\n".join(
f"| `{row['cell']}` | {row['initial_grains']} | {row['toppling_index_z_proxy']} | {row['seed_object_count']} |"
for row in result["initial_grain_table"]
)
avalanche_lines = "\n".join(
f"| `{row['seed_cell']}` | {row['avalanche_size_cells']} | {row['topple_events']} | `{row['terminated']}` |"
for row in result["avalanche_sizes"]
)
holds = "\n".join(f"- `{hold}`" for hold in result["holds"])
DOC_MD.write_text(
f"""# Stellar Gas Sandpile Graph Replay
Status: `GRAPH_TOPPLING_PROXY`
Decision: `{result['decision']}`
This hardening pass turns the sandpile metaphor into a reproducible graph
diagnostic. Nodes are sky/redshift cells. Edges are defined by sky-sector
neighbors plus adjacent redshift bins. Grain and toppling values are diagnostic
proxies derived from the existing cell-level toppling index.
Claim boundary: this is not a physical sandpile simulation, not a stellar-gas
mechanism proof, not direct stellar mass, not gas density inference, and not a
cosmology fit.
## Replay Receipt
```json
{json.dumps(receipt, indent=2, sort_keys=True)}
```
## Seed Evidence
```text
avalanche cells: {result['seed_evidence']['avalanche_cell_count']}
candidate objects: {result['seed_evidence']['candidate_object_count']}
graph nodes: {result['node_count']}
graph edges: {result['edge_count']}
graph hash: {result['graph_hash']}
replay hash: {result['replay_hash']}
```
## Toppling Threshold Table
| cell | degree | threshold |
| --- | ---: | ---: |
{threshold_lines}
## Initial Grain Table
| cell | initial grains | index z proxy | seed objects |
| --- | ---: | ---: | ---: |
{grain_lines}
## Avalanche Sizes
| seed cell | size cells | topple events | terminated |
| --- | ---: | ---: | --- |
{avalanche_lines}
## Holds
{holds}
""",
encoding="utf-8",
)
TIDDLER.write_text(
f"""title: Stellar Gas Sandpile Graph Replay
tags: StellarGasObservation SemanticMassNumbers Sandpile GraphReplay Receipts
type: text/vnd.tiddlywiki
Status: <<tag GRAPH_TOPPLING_PROXY>>
Decision: `{result['decision']}`
This tiddler records the graph/replay hardening of the stellar-gas sandpile
diagnostic. It is a toppling proxy over evidence cells, not a physical sandpile
simulation or mechanism proof.
```
avalanche cells: {result['seed_evidence']['avalanche_cell_count']}
candidate objects: {result['seed_evidence']['candidate_object_count']}
graph nodes: {result['node_count']}
graph edges: {result['edge_count']}
graph hash: {result['graph_hash']}
replay hash: {result['replay_hash']}
```
!! Toppling Threshold Table
|cell | degree | threshold |
|---|---:|---:|
{threshold_lines}
!! Initial Grain Table
|cell | initial grains | index z proxy | seed objects |
|---|---:|---:|---:|
{grain_lines}
!! Avalanche Sizes
|seed cell | size cells | topple events | terminated |
|---|---:|---:|---|
{avalanche_lines}
!! Boundary
Diagnostic/toppling proxy only. Holds: {", ".join(result["holds"])}.
""",
encoding="utf-8",
)
def main() -> None:
result, receipt = build()
OUT_DIR.mkdir(parents=True, exist_ok=True)
DOCS_DIR.mkdir(parents=True, exist_ok=True)
TIDDLER_DIR.mkdir(parents=True, exist_ok=True)
OUT_JSON.write_text(json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
RECEIPT_JSON.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
write_docs(result, receipt)
print(json.dumps(receipt, indent=2, sort_keys=True))
if __name__ == "__main__":
main()

View file

@ -0,0 +1,246 @@
#!/usr/bin/env python3
"""PTY-backed virtual serial probe for the Tang Nano 9K Q16 host protocol.
This creates a local pseudo-terminal, runs a small responder that speaks the
same framed Q16 receipt protocol as the FPGA surface, then drives the existing
host harness through the PTY path. It proves host serial framing and parsing,
not live FPGA fabric behavior.
"""
from __future__ import annotations
import argparse
import json
import os
import pty
import select
import struct
import subprocess
import threading
import time
import tty
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from tang9k_rrc_q16_accel import (
MAGIC_IN,
MAGIC_OUT,
OP_MONOTONE,
OP_SHIFT_DIV,
OP_WEIGHTED,
VERSION,
xor_crc,
)
REPO = Path(__file__).resolve().parents[2]
OUT = REPO / "shared-data" / "data" / "stack_solidification" / "tang9k_rrc_q16_virtual_serial_probe.json"
DOC = REPO / "6-Documentation" / "docs" / "tang9k_rrc_q16_virtual_serial_probe_2026-05-09.md"
def i32(value: int) -> bytes:
return struct.pack(">i", value)
def from_i32(data: bytes) -> int:
return struct.unpack(">i", data)[0]
def response_frame(seq: int, status: int, payload: bytes) -> bytes:
frame = bytearray([MAGIC_OUT, VERSION, seq & 0xFF, status & 0xFF, len(payload)])
frame.extend(payload)
frame.append(xor_crc(frame))
return bytes(frame)
def emulate_frame(frame: bytes) -> bytes:
if len(frame) < 6 or frame[0] != MAGIC_IN or frame[1] != VERSION:
return response_frame(0, 0xE1, b"\x00")
seq = frame[2]
opcode = frame[3]
payload_len = frame[4]
payload = frame[5 : 5 + payload_len]
crc = frame[5 + payload_len] if len(frame) > 5 + payload_len else None
if len(payload) != payload_len or crc is None or xor_crc(frame[:-1]) != crc:
return response_frame(seq, 0xE2, bytes([opcode]))
if opcode == OP_SHIFT_DIV and payload_len == 4:
x = from_i32(payload)
result0 = x >> 16
return response_frame(seq, 0, bytes([opcode]) + i32(result0) + b"\x01")
if opcode == OP_WEIGHTED and payload_len == 8:
energy = from_i32(payload[:4])
alpha = from_i32(payload[4:])
result0 = (energy * alpha) >> 16
passed = energy >= 0 and alpha >= 0 and alpha <= 65536 and result0 <= energy
return response_frame(seq, 0, bytes([opcode]) + i32(result0) + bytes([1 if passed else 0]))
if opcode == OP_MONOTONE and payload_len == 8:
a = from_i32(payload[:4])
b = from_i32(payload[4:])
result0 = a >> 16
result1 = b >> 16
passed = a <= b and result0 <= result1
return response_frame(seq, 0, bytes([opcode]) + i32(result0) + i32(result1) + bytes([1 if passed else 0]))
return response_frame(seq, 0xE3, bytes([opcode]))
class VirtualResponder:
def __init__(self, master_fd: int) -> None:
self.master_fd = master_fd
self.stop = threading.Event()
self.frames_seen: list[dict[str, Any]] = []
self.thread = threading.Thread(target=self._run, daemon=True)
def start(self) -> None:
self.thread.start()
def close(self) -> None:
self.stop.set()
self.thread.join(timeout=1.0)
def _read_exactish(self, size: int, timeout_s: float = 1.0) -> bytes:
deadline = time.monotonic() + timeout_s
data = bytearray()
while len(data) < size and time.monotonic() < deadline and not self.stop.is_set():
ready, _, _ = select.select([self.master_fd], [], [], 0.05)
if not ready:
continue
chunk = os.read(self.master_fd, size - len(data))
if chunk:
data.extend(chunk)
return bytes(data)
def _run(self) -> None:
buf = bytearray()
while not self.stop.is_set():
ready, _, _ = select.select([self.master_fd], [], [], 0.05)
if not ready:
continue
chunk = os.read(self.master_fd, 256)
if not chunk:
continue
buf.extend(chunk)
while buf:
while buf and buf[0] != MAGIC_IN:
del buf[0]
if len(buf) < 5:
break
need = 6 + buf[4]
if len(buf) < need:
break
frame = bytes(buf[:need])
del buf[:need]
reply = emulate_frame(frame)
os.write(self.master_fd, reply)
self.frames_seen.append({"request_hex": frame.hex(), "response_hex": reply.hex()})
def run_case(port: str, name: str, args: list[str]) -> dict[str, Any]:
out_path = REPO / "4-Infrastructure" / "shim" / f"tang9k_rrc_q16_virtual_{name}_receipt.json"
cmd = [
"python3",
"4-Infrastructure/shim/tang9k_rrc_q16_accel.py",
*args,
"--port",
port,
"--retries",
"1",
"--out",
str(out_path.relative_to(REPO)),
]
proc = subprocess.run(
cmd,
cwd=REPO,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=30,
check=False,
)
receipt = json.loads(out_path.read_text(encoding="utf-8")) if out_path.exists() else {}
return {
"case": name,
"command": cmd,
"returncode": proc.returncode,
"stdout_tail": proc.stdout[-4000:],
"stderr_tail": proc.stderr[-4000:],
"receipt": str(out_path.relative_to(REPO)),
"match": receipt.get("match"),
"hardware": receipt.get("hardware"),
}
def build_doc(receipt: dict[str, Any]) -> str:
lines = [
"# Tang Nano 9K Q16 Virtual Serial Probe",
"",
"**Date:** 2026-05-09",
"",
receipt["claim_boundary"],
"",
"## Status",
"",
f"- Status: `{receipt['summary']['status']}`",
f"- Cases: `{receipt['summary']['case_count']}`",
f"- Matches: `{receipt['summary']['match_count']}`",
"",
"## Cases",
"",
]
for case in receipt["cases"]:
lines.append(f"- `{case['case']}`: match `{case['match']}`, receipt `{case['receipt']}`")
lines.extend(["", "## Machine Receipt", "", f"- `{OUT.relative_to(REPO)}`"])
return "\n".join(lines) + "\n"
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--out", type=Path, default=OUT)
args = parser.parse_args()
master_fd, slave_fd = pty.openpty()
tty.setraw(master_fd)
tty.setraw(slave_fd)
slave_path = os.ttyname(slave_fd)
responder = VirtualResponder(master_fd)
responder.start()
try:
cases = [
run_case(slave_path, "shift", ["--op", "shift", "--x", "0x00038000"]),
run_case(slave_path, "weighted", ["--op", "weighted", "--energy", "0x000a0000", "--alpha", "0x00008000"]),
run_case(slave_path, "monotone", ["--op", "monotone", "--a", "0x00010000", "--b", "0x00030000"]),
]
finally:
responder.close()
os.close(master_fd)
os.close(slave_fd)
match_count = sum(1 for case in cases if case.get("match") is True)
receipt = {
"schema": "tang9k_rrc_q16_virtual_serial_probe_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"claim_boundary": (
"Virtual serial probe only. This validates the host Q16 UART framing, "
"receipt parser, and opcode semantics over a PTY-backed serial device; "
"it does not validate live FPGA fabric or the Tang Nano UART route."
),
"virtual_port": slave_path,
"summary": {
"status": "PASS_VIRTUAL_SERIAL" if match_count == len(cases) else "FAIL",
"case_count": len(cases),
"match_count": match_count,
"frames_seen": len(responder.frames_seen),
},
"cases": cases,
"virtual_frames": responder.frames_seen,
}
args.out.parent.mkdir(parents=True, exist_ok=True)
args.out.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
DOC.write_text(build_doc(receipt), encoding="utf-8")
print(json.dumps({"receipt": str(args.out.relative_to(REPO)), "doc": str(DOC.relative_to(REPO)), "status": receipt["summary"]["status"]}, indent=2))
return 0 if receipt["summary"]["status"] == "PASS_VIRTUAL_SERIAL" else 1
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,87 @@
#!/usr/bin/env python3
"""Probe a Tang Nano 9K TX-only UART beacon on one or more serial ports."""
from __future__ import annotations
import argparse
import json
import time
from datetime import datetime, timezone
from pathlib import Path
import serial
DEFAULT_EXPECTED = "a6425131360a"
def read_port(port: str, baud: int, duration_s: float, expected: bytes) -> dict[str, object]:
entry: dict[str, object] = {"port": port, "baud": baud, "duration_s": duration_s}
try:
with serial.Serial(port, baud, timeout=0.05) as ser:
ser.reset_input_buffer()
start = time.monotonic()
data = bytearray()
while time.monotonic() - start < duration_s:
chunk = ser.read(256)
if chunk:
data.extend(chunk)
payload = bytes(data)
entry.update(
{
"byte_count": len(payload),
"hex_prefix": payload[:128].hex(),
"ascii_prefix": payload[:128].decode("ascii", "replace"),
"contains_expected": expected in payload,
"contains_q16_ascii": b"Q16" in payload,
}
)
except Exception as exc: # pragma: no cover - hardware diagnostic path
entry.update(
{
"byte_count": 0,
"hex_prefix": "",
"ascii_prefix": "",
"contains_expected": False,
"contains_q16_ascii": False,
"error": repr(exc),
}
)
return entry
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--port", action="append", dest="ports", help="serial port to probe; repeatable")
parser.add_argument("--baud", type=int, default=115200)
parser.add_argument("--duration", type=float, default=2.0)
parser.add_argument("--expected-hex", default=DEFAULT_EXPECTED)
parser.add_argument("--constraints", default="")
parser.add_argument("--bitstream", default="4-Infrastructure/hardware/tangnano9k_uart_beacon.fs")
parser.add_argument("--bitstream-sha256", default="")
parser.add_argument("--out", default="4-Infrastructure/shim/tang9k_uart_beacon_probe_receipt.json")
args = parser.parse_args()
ports = args.ports or ["/dev/ttyUSB0", "/dev/ttyUSB1"]
expected = bytes.fromhex(args.expected_hex)
results = [read_port(port, args.baud, args.duration, expected) for port in ports]
receipt = {
"schema": "tangnano9k_uart_beacon_probe_v3",
"created_utc": datetime.now(timezone.utc).isoformat(),
"purpose": "Probe TX-only UART beacon bytes on host-visible serial ports.",
"bitstream": args.bitstream,
"bitstream_sha256": args.bitstream_sha256,
"constraints": args.constraints,
"expected_payload_hex": expected.hex(),
"results": results,
"conclusion": "PASS" if any(item.get("contains_expected") for item in results) else "FAIL_NO_BEACON_ON_SERIAL_PORTS",
}
out = Path(args.out)
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(receipt, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print(json.dumps(receipt, indent=2, sort_keys=True))
return 0 if receipt["conclusion"] == "PASS" else 2
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,196 @@
#!/usr/bin/env python3
"""Route table for Tang Nano 9K UART-like transports.
The onboard FTDI/BL702 route is currently blocked for fabric UART. This router
subsumes those physical UART entries under one manifest and selects a
PTY-backed virtual Q16 route as the active non-hardware transport.
"""
from __future__ import annotations
import json
import subprocess
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
OUT = REPO / "shared-data" / "data" / "stack_solidification" / "tang9k_uart_transport_routes.json"
DOC = REPO / "6-Documentation" / "docs" / "tang9k_uart_transport_routes_2026-05-09.md"
def rel(path: Path) -> str:
return str(path.relative_to(REPO))
def load_json(path: Path) -> dict[str, Any]:
if not path.exists():
return {}
return json.loads(path.read_text(encoding="utf-8"))
def run_virtual_probe() -> dict[str, Any]:
proc = subprocess.run(
["python3", "4-Infrastructure/shim/tang9k_rrc_q16_virtual_serial_probe.py"],
cwd=REPO,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=60,
check=False,
)
receipt_path = REPO / "shared-data" / "data" / "stack_solidification" / "tang9k_rrc_q16_virtual_serial_probe.json"
receipt = load_json(receipt_path)
return {
"command": ["python3", "4-Infrastructure/shim/tang9k_rrc_q16_virtual_serial_probe.py"],
"returncode": proc.returncode,
"stdout_tail": proc.stdout[-4000:],
"stderr_tail": proc.stderr[-4000:],
"receipt": rel(receipt_path),
"status": receipt.get("summary", {}).get("status", "UNKNOWN"),
"match_count": receipt.get("summary", {}).get("match_count"),
"case_count": receipt.get("summary", {}).get("case_count"),
}
def serial_receipt_summary(path: str) -> dict[str, Any]:
receipt = load_json(REPO / path)
results = receipt.get("results", [])
return {
"receipt": path,
"present": bool(receipt),
"conclusion": receipt.get("conclusion"),
"ports": [
{
"port": row.get("port"),
"byte_count": row.get("byte_count"),
"contains_expected": row.get("contains_expected"),
"hex_prefix": row.get("hex_prefix"),
}
for row in results
],
}
def build_manifest() -> dict[str, Any]:
virtual = run_virtual_probe()
onboard_beacon = serial_receipt_summary("4-Infrastructure/shim/tang9k_uart_beacon_probe_receipt.json")
swapped_beacon = serial_receipt_summary("4-Infrastructure/shim/tang9k_uart_beacon_swapped_probe_receipt.json")
loopback_after_clear = serial_receipt_summary("4-Infrastructure/shim/tang9k_uart_loopback_after_jtag_clear_probe_receipt.json")
active_status = (
"PASS_ACTIVE_VIRTUAL_ROUTE"
if virtual["status"] == "PASS_VIRTUAL_SERIAL"
and virtual.get("match_count") == virtual.get("case_count") == 3
else "FAIL"
)
return {
"schema": "tang9k_uart_transport_routes_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"claim_boundary": (
"Transport route table only. The active virtual route validates host Q16 serial "
"framing and parser behavior. It does not validate live FPGA fabric or the "
"Tang Nano onboard UART bridge."
),
"active_route": "virtual://q16-pty",
"active_route_status": active_status,
"route_table": [
{
"route_id": "onboard-ftdi-a",
"device": "/dev/ttyUSB0",
"kind": "physical_ftdi_mpsse_or_bridge",
"status": "BLOCKED_FOR_FABRIC_UART",
"evidence": "faXX/MPSSE-style bytes or zero beacon bytes; no valid fabric receipt",
"receipts": [onboard_beacon["receipt"], swapped_beacon["receipt"], loopback_after_clear["receipt"]],
},
{
"route_id": "onboard-ftdi-b",
"device": "/dev/ttyUSB1",
"kind": "physical_ftdi_secondary_endpoint",
"status": "BLOCKED_FOR_FABRIC_UART",
"evidence": "zero beacon bytes and empty Q16 hardware receipts",
"receipts": [onboard_beacon["receipt"], swapped_beacon["receipt"], loopback_after_clear["receipt"]],
},
{
"route_id": "external-usb-uart",
"device": "/dev/ttyUSB2_or_/dev/ttyACM0",
"kind": "physical_external_adapter",
"status": "PENDING_HARDWARE",
"evidence": "recommended live-hardware closure route; adapter not present in this probe",
"receipts": [],
},
{
"route_id": "virtual-q16-pty",
"device": "virtual://q16-pty",
"kind": "pty_backed_virtual_serial",
"status": virtual["status"],
"evidence": "shift, weighted, and monotone Q16 receipt frames match through PTY-backed serial route",
"receipts": [virtual["receipt"]],
},
],
"physical_receipt_summaries": {
"onboard_beacon": onboard_beacon,
"swapped_beacon": swapped_beacon,
"loopback_after_jtag_clear": loopback_after_clear,
},
"virtual_probe": virtual,
"routing_policy": {
"default_non_hardware_route": "virtual://q16-pty",
"live_hardware_promotion_requires": [
"external or verified onboard route captures beacon payload a6425131360a",
"Q16 shift, weighted, and monotone hardware receipts match software expectations",
"stack audit reports FPGA hardware witness PASS",
],
"blocked_routes_remain_visible": True,
},
}
def build_doc(manifest: dict[str, Any]) -> str:
lines = [
"# Tang Nano 9K UART Transport Routes",
"",
"**Date:** 2026-05-09",
"",
manifest["claim_boundary"],
"",
"## Active Route",
"",
f"- Active route: `{manifest['active_route']}`",
f"- Status: `{manifest['active_route_status']}`",
"",
"## Route Table",
"",
"| Route | Device | Kind | Status |",
"| --- | --- | --- | --- |",
]
for route in manifest["route_table"]:
lines.append(
f"| `{route['route_id']}` | `{route['device']}` | `{route['kind']}` | `{route['status']}` |"
)
lines.extend(
[
"",
"## Routing Policy",
"",
f"- Default non-hardware route: `{manifest['routing_policy']['default_non_hardware_route']}`",
"- Live hardware promotion requires:",
]
)
for gate in manifest["routing_policy"]["live_hardware_promotion_requires"]:
lines.append(f" - {gate}")
lines.extend(["", "## Machine Receipt", "", f"- `{OUT.relative_to(REPO)}`"])
return "\n".join(lines) + "\n"
def main() -> int:
manifest = build_manifest()
OUT.parent.mkdir(parents=True, exist_ok=True)
OUT.write_text(json.dumps(manifest, indent=2, sort_keys=True) + "\n", encoding="utf-8")
DOC.write_text(build_doc(manifest), encoding="utf-8")
print(json.dumps({"receipt": rel(OUT), "doc": rel(DOC), "status": manifest["active_route_status"]}, indent=2))
return 0 if manifest["active_route_status"] == "PASS_ACTIVE_VIRTUAL_ROUTE" else 1
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,191 @@
#!/usr/bin/env python3
"""Receipt probe for zero-whitespace-code logogram grammar.
The grammar does not store ordinary spaces as payload atoms. For canonical
single-space token streams, spacing is reconstructed from symbol count/order.
Non-canonical whitespace remains HOLD unless a residual policy is declared.
"""
from __future__ import annotations
import json
import subprocess
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
LEAN_DIR = REPO / "0-Core-Formalism" / "lean" / "Semantics"
OUT_DIR = REPO / "shared-data" / "data" / "stack_solidification"
OUT = OUT_DIR / "whitespace_zero_grammar_probe.json"
DOC = REPO / "6-Documentation" / "docs" / "whitespace_zero_grammar_2026-05-09.md"
CANONICAL_CASES = [
"structure transformation receipt replay repair",
"braid rope ammr leaf peak",
"token order symbol identity transform rule",
]
HOLD_CASES = [
"two spaces need residual",
" leading space needs residual",
"tabs\tneed\tresidual",
]
def rel(path: Path) -> str:
return str(path.relative_to(REPO))
def run_lean_build() -> dict[str, Any]:
proc = subprocess.run(
["lake", "build", "Semantics.WhitespaceFreeGrammar"],
cwd=LEAN_DIR,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=300,
check=False,
)
return {
"command": ["lake", "build", "Semantics.WhitespaceFreeGrammar"],
"cwd": rel(LEAN_DIR),
"returncode": proc.returncode,
"status": "PASS" if proc.returncode == 0 else "FAIL",
"stdout_tail": proc.stdout[-6000:],
"stderr_tail": proc.stderr[-6000:],
}
def is_canonical_single_space(text: str) -> bool:
return bool(text) and text == " ".join(text.split(" ")) and "\t" not in text and "\n" not in text and not text.startswith(" ") and not text.endswith(" ")
def encode_symbols(text: str) -> list[str]:
return text.split()
def run_case(case_id: str, text: str, expect_exact: bool) -> dict[str, Any]:
symbols = encode_symbols(text)
reconstructed = " ".join(symbols)
raw_bytes = len(text.encode("utf-8"))
payload_bytes = sum(len(symbol.encode("utf-8")) for symbol in symbols)
derived_boundaries = max(0, len(symbols) - 1)
stored_whitespace_codes = 0
exact_replay = reconstructed == text
canonical = is_canonical_single_space(text)
status = "ADMIT_FIXTURE" if exact_replay and canonical and expect_exact else "HOLD_NEEDS_WHITESPACE_RESIDUAL"
return {
"case_id": case_id,
"raw": text,
"symbols": symbols,
"symbol_count": len(symbols),
"raw_bytes": raw_bytes,
"payload_bytes": payload_bytes,
"stored_whitespace_codes": stored_whitespace_codes,
"stored_whitespace_bytes": 0,
"derived_boundary_count": derived_boundaries,
"reconstructed": reconstructed,
"exact_replay": exact_replay,
"canonical_single_space": canonical,
"delta_vs_raw_without_receipt": raw_bytes - payload_bytes,
"status": status,
}
def build_receipt() -> dict[str, Any]:
lean_build = run_lean_build()
cases = [
run_case(f"canonical_{idx}", text, True)
for idx, text in enumerate(CANONICAL_CASES, start=1)
] + [
run_case(f"hold_{idx}", text, False)
for idx, text in enumerate(HOLD_CASES, start=1)
]
admitted = [case for case in cases if case["status"] == "ADMIT_FIXTURE"]
holds = [case for case in cases if case["status"].startswith("HOLD")]
return {
"schema": "whitespace_zero_grammar_probe_v1",
"created_utc": datetime.now(timezone.utc).isoformat(),
"claim_boundary": (
"Zero whitespace-code grammar for canonical single-space token streams. "
"Whitespace is reconstructed from symbol count/order. Non-canonical "
"spacing requires an explicit residual and is not admitted by this gate."
),
"lean_module": "Semantics.WhitespaceFreeGrammar",
"lean_build": lean_build,
"grammar_rule": {
"stored_whitespace_codes": 0,
"boundary_rule": "insert one display space between adjacent symbols during canonical replay",
"payload_rule": "store symbol payloads only",
"residual_rule": "non-canonical whitespace requires residual",
},
"summary": {
"status": "PASS_ZERO_WHITESPACE_CANONICAL" if lean_build["status"] == "PASS" and len(admitted) == len(CANONICAL_CASES) and len(holds) == len(HOLD_CASES) else "FAIL",
"case_count": len(cases),
"admit_count": len(admitted),
"hold_count": len(holds),
"stored_whitespace_codes_total": sum(case["stored_whitespace_codes"] for case in cases),
"canonical_delta_bytes_total": sum(case["delta_vs_raw_without_receipt"] for case in admitted),
},
"cases": cases,
}
def build_doc(receipt: dict[str, Any]) -> str:
lines = [
"# Whitespace-Zero Grammar Probe",
"",
"**Date:** 2026-05-09",
"",
receipt["claim_boundary"],
"",
"## Rule",
"",
"- Store symbol payloads.",
"- Store zero ordinary whitespace codes.",
"- Reconstruct one canonical display space between adjacent symbols.",
"- HOLD any non-canonical whitespace unless a residual is declared.",
"",
"## Status",
"",
f"- Lean module: `{receipt['lean_module']}`",
f"- Lean build: `{receipt['lean_build']['status']}`",
f"- Probe status: `{receipt['summary']['status']}`",
f"- Admitted canonical fixtures: `{receipt['summary']['admit_count']}`",
f"- HOLD fixtures needing residual: `{receipt['summary']['hold_count']}`",
f"- Stored whitespace codes total: `{receipt['summary']['stored_whitespace_codes_total']}`",
"",
"## Cases",
"",
]
for case in receipt["cases"]:
lines.append(
f"- `{case['case_id']}`: `{case['status']}`, symbols `{case['symbol_count']}`, "
f"payload `{case['payload_bytes']}` bytes, raw `{case['raw_bytes']}` bytes, "
f"derived boundaries `{case['derived_boundary_count']}`, exact replay `{case['exact_replay']}`"
)
lines.extend(
[
"",
"## Machine Receipt",
"",
f"- `{rel(OUT)}`",
]
)
return "\n".join(lines) + "\n"
def main() -> int:
OUT_DIR.mkdir(parents=True, exist_ok=True)
receipt = build_receipt()
OUT.write_text(json.dumps(receipt, indent=2, sort_keys=True), encoding="utf-8")
DOC.write_text(build_doc(receipt), encoding="utf-8")
print(json.dumps({"receipt": rel(OUT), "doc": rel(DOC), "status": receipt["summary"]["status"]}, indent=2))
return 0 if receipt["summary"]["status"] == "PASS_ZERO_WHITESPACE_CANONICAL" else 1
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,406 @@
//! Rust OISC decompressor target.
//!
//! This module is a reference backend for the Lean surface
//! `Semantics.RustOISCDecompressor`. It is not a compression benchmark and does
//! not claim silicon-target readiness.
use crate::{CompressionError, Compressor};
pub const MAGIC: &[u8; 4] = b"OISC";
pub const VERSION: u8 = 1;
pub const HEADER_LEN: usize = 5;
pub const INSTRUCTION_LEN: usize = 3;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum OiscDecision {
Done,
Nan0,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct OiscReceipt {
pub output: Vec<u8>,
pub instruction_count: usize,
pub accumulator: u8,
pub decision: OiscDecision,
pub input_hash: u64,
pub output_hash: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct OiscReplayMeta {
pub instruction_count: usize,
pub accumulator: u8,
pub decision: OiscDecision,
pub input_hash: u64,
pub output_hash: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct OiscState {
pc: usize,
acc: u8,
halted: bool,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct Instruction {
symbol: u8,
residual: u8,
final_flag: bool,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Transition {
Continue,
Done,
Nan0,
}
pub struct OiscCompressor;
fn stable_hash(bytes: &[u8]) -> u64 {
let mut hash = 0xcbf2_9ce4_8422_2325u64;
for &byte in bytes {
hash ^= byte as u64;
hash = hash.wrapping_mul(0x0000_0100_0000_01b3);
}
hash
}
fn step(
state: &mut OiscState,
output: &mut Vec<u8>,
capacity: usize,
instr: Instruction,
) -> Transition {
if state.halted {
return Transition::Done;
}
if output.len() >= capacity {
state.halted = true;
return Transition::Nan0;
}
output.push(instr.symbol);
state.pc += 1;
state.acc = state
.acc
.wrapping_add(instr.symbol)
.wrapping_add(instr.residual);
state.halted = instr.final_flag;
if instr.final_flag {
Transition::Done
} else {
Transition::Continue
}
}
fn parse_instruction(input: &[u8], pos: usize) -> Result<Instruction, CompressionError> {
if pos + INSTRUCTION_LEN > input.len() {
return Err(CompressionError::CorruptStream);
}
let final_flag = match input[pos + 2] {
0 => false,
1 => true,
_ => return Err(CompressionError::CorruptStream),
};
Ok(Instruction {
symbol: input[pos],
residual: input[pos + 1],
final_flag,
})
}
fn validate_instruction_stream(input: &[u8]) -> Result<(), CompressionError> {
let mut pos = HEADER_LEN;
while pos < input.len() {
let instr = parse_instruction(input, pos)?;
if instr.final_flag && pos + INSTRUCTION_LEN < input.len() {
return Err(CompressionError::CorruptStream);
}
pos += INSTRUCTION_LEN;
}
Ok(())
}
pub fn decompress_oisc(
input: &[u8],
output_capacity: usize,
) -> Result<OiscReceipt, CompressionError> {
let mut output = Vec::with_capacity(output_capacity);
let meta = decompress_oisc_into(input, output_capacity, &mut output)?;
Ok(OiscReceipt {
output,
instruction_count: meta.instruction_count,
accumulator: meta.accumulator,
decision: meta.decision,
input_hash: meta.input_hash,
output_hash: meta.output_hash,
})
}
pub fn decompress_oisc_into(
input: &[u8],
output_capacity: usize,
output: &mut Vec<u8>,
) -> Result<OiscReplayMeta, CompressionError> {
output.clear();
if output.capacity() < output_capacity {
return Err(CompressionError::CapacityExceeded);
}
if input.len() < HEADER_LEN {
return Err(CompressionError::InvalidData);
}
if &input[0..4] != MAGIC {
return Err(CompressionError::InvalidData);
}
if input[4] != VERSION {
return Err(CompressionError::UnsupportedVersion);
}
if input.len() == HEADER_LEN {
return Ok(OiscReplayMeta {
instruction_count: 0,
accumulator: 0,
decision: OiscDecision::Done,
input_hash: stable_hash(input),
output_hash: stable_hash(output),
});
}
let body_len = input.len() - HEADER_LEN;
if body_len % INSTRUCTION_LEN != 0 {
return Err(CompressionError::CorruptStream);
}
validate_instruction_stream(input)?;
let mut state = OiscState {
pc: 0,
acc: 0,
halted: false,
};
let mut pos = HEADER_LEN;
let mut instruction_count = 0usize;
while pos < input.len() {
let instr = parse_instruction(input, pos)?;
let decision = step(&mut state, output, output_capacity, instr);
instruction_count += 1;
match decision {
Transition::Done => {
return Ok(OiscReplayMeta {
instruction_count,
accumulator: state.acc,
decision: OiscDecision::Done,
input_hash: stable_hash(input),
output_hash: stable_hash(output),
});
}
Transition::Nan0 => {
return Err(CompressionError::CapacityExceeded);
}
Transition::Continue => {}
}
pos += INSTRUCTION_LEN;
}
Ok(OiscReplayMeta {
instruction_count,
accumulator: state.acc,
decision: OiscDecision::Done,
input_hash: stable_hash(input),
output_hash: stable_hash(output),
})
}
impl Compressor for OiscCompressor {
fn compress(&self, input: &[u8]) -> Vec<u8> {
let mut out = Vec::with_capacity(HEADER_LEN + input.len() * INSTRUCTION_LEN);
out.extend_from_slice(MAGIC);
out.push(VERSION);
for (idx, &symbol) in input.iter().enumerate() {
out.push(symbol);
out.push(0);
out.push((idx + 1 == input.len()) as u8);
}
out
}
fn decompress(&self, input: &[u8]) -> Result<Vec<u8>, CompressionError> {
let instruction_capacity = if input.len() >= HEADER_LEN {
(input.len() - HEADER_LEN) / INSTRUCTION_LEN
} else {
0
};
decompress_oisc(input, instruction_capacity).map(|receipt| receipt.output)
}
fn name(&self) -> &'static str {
"oisc-replay-v0.1"
}
fn ratio(&self) -> f64 {
1.0
}
}
#[cfg(test)]
mod tests {
use super::*;
const ABC_WIRE: &[u8] = b"OISC\x01A\x01\x00B\x01\x00C\x01\x01";
const RESIDUAL_WIRE: &[u8] = b"OISC\x01A\xff\x00B\x01\x00C\x02\x01";
const TRANSFER_FIXTURE: &[u8] = br#"Hutter transfer readiness fixture.
This small corpus window is intentionally local and inspectable. It carries
repeated phrases, shifted byte contexts, punctuation, digits 0123456789, and
line breaks so replay code must preserve more than a three-symbol toy stream.
Window alpha binds local recurrence: stack stack stack, receipt receipt receipt,
offset offset offset. Window beta changes cadence with JSON-ish tokens:
{"decision":"HOLD","window":128,"route":"oisc-replay"}.
Window gamma adds mixed case and separators: Alpha/BETA/gamma; phase_00,
phase_01, phase_02. The fixture stops here because the readiness gate is about
manifest discipline, not corpus scale.
"#;
fn expected_accumulator(instructions: &[(u8, u8)]) -> u8 {
instructions.iter().fold(0, |acc, &(symbol, residual)| {
acc.wrapping_add(symbol).wrapping_add(residual)
})
}
#[test]
fn abc_matches_lean_fixture() {
let receipt = decompress_oisc(ABC_WIRE, 8).expect("valid ABC fixture");
assert_eq!(receipt.output, b"ABC");
assert_eq!(receipt.instruction_count, 3);
assert_eq!(receipt.accumulator, 201);
assert_eq!(receipt.decision, OiscDecision::Done);
}
#[test]
fn empty_stream_closes() {
let receipt = decompress_oisc(b"OISC\x01", 8).expect("header-only stream");
assert_eq!(receipt.output, b"");
assert_eq!(receipt.instruction_count, 0);
assert_eq!(receipt.accumulator, 0);
assert_eq!(receipt.decision, OiscDecision::Done);
}
#[test]
fn capacity_overflow_fails_closed() {
let err = decompress_oisc(ABC_WIRE, 2).expect_err("capacity should overflow");
assert!(matches!(err, CompressionError::CapacityExceeded));
}
#[test]
fn invalid_magic_rejected() {
let err = decompress_oisc(b"XISC\x01", 8).expect_err("bad magic");
assert!(matches!(err, CompressionError::InvalidData));
}
#[test]
fn invalid_version_rejected() {
let err = decompress_oisc(b"OISC\x02", 8).expect_err("bad version");
assert!(matches!(err, CompressionError::UnsupportedVersion));
}
#[test]
fn truncated_instruction_rejected() {
let err = decompress_oisc(b"OISC\x01A\x01", 8).expect_err("truncated instruction");
assert!(matches!(err, CompressionError::CorruptStream));
}
#[test]
fn trailing_after_final_rejected() {
let err = decompress_oisc(b"OISC\x01A\x01\x01B\x01\x00", 8)
.expect_err("trailing bytes after final");
assert!(matches!(err, CompressionError::CorruptStream));
}
#[test]
fn compressor_round_trips_without_claiming_ratio() {
let compressor = OiscCompressor;
let wire = compressor.compress(b"ABC");
let output = compressor.decompress(&wire).expect("round trip");
assert_eq!(output, b"ABC");
assert_eq!(compressor.name(), "oisc-replay-v0.1");
assert_eq!(compressor.ratio(), 1.0);
}
#[test]
fn residual_accumulator_is_deterministic_receipt_field() {
let first = decompress_oisc(RESIDUAL_WIRE, 8).expect("valid residual fixture");
let second = decompress_oisc(RESIDUAL_WIRE, 8).expect("valid residual fixture");
assert_eq!(first, second);
assert_eq!(first.output, b"ABC");
assert_eq!(
first.accumulator,
expected_accumulator(&[(b'A', 0xff), (b'B', 1), (b'C', 2)])
);
assert_eq!(first.instruction_count, 3);
assert_eq!(first.decision, OiscDecision::Done);
}
#[test]
fn final_instruction_rejects_trailing_without_executing_it() {
let mut output = Vec::with_capacity(8);
let err = decompress_oisc_into(b"OISC\x01A\x01\x01Z\xff\x00", 8, &mut output)
.expect_err("trailing instruction after final must be rejected");
assert!(matches!(err, CompressionError::CorruptStream));
assert!(output.is_empty());
assert!(!output.contains(&b'Z'));
}
#[test]
fn decompress_into_reuses_caller_buffer() {
let mut output = Vec::with_capacity(8);
output.extend_from_slice(b"stale");
let original_capacity = output.capacity();
let meta = decompress_oisc_into(ABC_WIRE, 8, &mut output).expect("valid ABC fixture");
assert_eq!(output, b"ABC");
assert_eq!(output.capacity(), original_capacity);
assert_eq!(meta.instruction_count, 3);
assert_eq!(meta.accumulator, 201);
assert_eq!(meta.output_hash, stable_hash(b"ABC"));
}
#[test]
fn decompress_into_rejects_undersized_caller_buffer_before_writing() {
let mut output = Vec::with_capacity(2);
output.extend_from_slice(b"xy");
let err = decompress_oisc_into(ABC_WIRE, 3, &mut output)
.expect_err("caller buffer capacity is part of the receipt contract");
assert!(matches!(err, CompressionError::CapacityExceeded));
assert!(output.is_empty());
assert_eq!(output.capacity(), 2);
}
#[test]
fn non_toy_transfer_fixture_replays_byte_exact() {
let compressor = OiscCompressor;
let wire = compressor.compress(TRANSFER_FIXTURE);
let receipt =
decompress_oisc(&wire, TRANSFER_FIXTURE.len()).expect("transfer fixture replay");
assert_eq!(receipt.output, TRANSFER_FIXTURE);
assert_eq!(receipt.instruction_count, TRANSFER_FIXTURE.len());
assert_eq!(receipt.decision, OiscDecision::Done);
assert_eq!(receipt.input_hash, stable_hash(&wire));
assert_eq!(receipt.output_hash, stable_hash(TRANSFER_FIXTURE));
assert!(receipt.instruction_count > 128);
assert!(wire.len() > TRANSFER_FIXTURE.len());
}
}

View file

@ -0,0 +1,37 @@
# Beaver Mask Freshness Negative Controls
**Date:** 2026-05-09
This closes one narrow shaky part: adaptive/topology-derived coefficients are not admitted as privacy-equivalent Beaver masks in the local Lean gate.
## Claim Boundary
Finite freshness/admission gate only. This rejects unsafe mask sources in the local model; it does not prove full MPC privacy, entropy quality, or adaptive Beaver security.
## Gate
- Lean module: `Semantics.BeaverMaskFreshness`
- Lean build: `PASS`
- Case status: `PASS_NEGATIVE_CONTROLS`
- Positive controls: `2`
- Negative controls: `4`
- Promotion effect: `PARTIAL_EVIDENCE_ONLY_SECURITY_HOLD_REMAINS`
## Cases
- `fresh_unused_admits`: expected `ADMIT`, observed `ADMIT`, Lean witness `freshUnusedAdmits`
- `distinct_fresh_sequence_admits`: expected `ADMIT`, observed `ADMIT`, Lean witness `distinctFreshSequenceAdmits`
- `reused_source_rejected`: expected `REJECT`, observed `REJECT`, Lean witness `reusedSourceRejected`
- `reused_mask_id_mislabeled_fresh_rejected`: expected `REJECT`, observed `REJECT`, Lean witness `reusedMaskIdRejected`
- `topology_derived_rejected`: expected `REJECT`, observed `REJECT`, Lean witness `topologyDerivedRejected`
- `adversarial_chosen_rejected`: expected `REJECT`, observed `REJECT`, Lean witness `adversarialChosenRejected`
## Remaining Security Debt
- This does not prove true randomness or independence of a deployed entropy source.
- This does not prove a full Beaver Triples privacy theorem.
- This does not admit topology-derived coefficients as masks; they remain useful routing coefficients only after separate receipt gates.
## Machine Receipt
- `shared-data/data/stack_solidification/beaver_mask_freshness_negative_controls.json`

View file

@ -0,0 +1,84 @@
# DESI Epoviz to MaNGA Population Cell Join
Status: `COARSE_CELL_PRIOR`
Decision: `ADMIT_COARSE_CELL_PRIOR_HOLD_OBJECT_CROSSMATCH`
This note uses the DESI EDR epoviz visualization/outreach VAC as a lightweight
population prior and compares it with the local MaNGA stellar-gas grouping study
by shared sky/redshift cells.
Claim boundary: this is not an object-level crossmatch, not a direct gas-density
map, and not a cosmology fit. It is a population-shape prior for deciding where
the stellar-gas model has local support and where it should stay in `HOLD`.
## Source
- DESI EDR epoviz CSV: `shared-data/artifacts/stellar_gas_observation/desi_epoviz/EDR-Viz-Outreach-VAC.csv.gz`
- DESI source hash: `83a4c4c4c4e8eb309e6a71459b3e6087113b82ef03edeeb09752ce368443b6d6`
- DESI documentation: https://data.desi.lbl.gov/doc/releases/edr/vac/epoviz/
- MaNGA grouping study: `shared-data/data/stellar_gas_observation/stellar_gas_population_grouping_study.json`
## DESI Population
Rows read: `669377`
Tracer counts:
- `BGS`: 228630
- `ELG`: 261489
- `LRG`: 125174
- `QSO`: 54084
Local redshift bins used for MaNGA overlap:
- `z_000_002`: 580
- `z_002_004`: 4313
- `z_004_006`: 3729
- `z_006_008`: 7017
- `z_008_plus`: 653738
Redshift summary:
```json
{
"count": 669377,
"min": 0.010233,
"max": 3.499648,
"mean": 0.765852
}
```
## Coarse Join
Join key:
```text
raXX_north_or_south__z_000_002/z_002_004/z_004_006/z_006_008/z_008_plus
```
Joined cells: `25` out of `56` MaNGA cells.
Top joined cells:
- `ra03_north__z_008_plus`: DESI 315331, MaNGA 305, shock proxy 0.918033
- `ra04_north__z_008_plus`: DESI 181621, MaNGA 219, shock proxy 0.922374
- `ra02_north__z_008_plus`: DESI 44035, MaNGA 364, shock proxy 0.881868
- `ra03_north__z_002_004`: DESI 2439, MaNGA 1622, shock proxy 0.676326
- `ra03_north__z_006_008`: DESI 3456, MaNGA 374, shock proxy 0.754011
- `ra04_north__z_002_004`: DESI 1177, MaNGA 960, shock proxy 0.60625
- `ra03_south__z_008_plus`: DESI 100231, MaNGA 8, shock proxy 1.0
- `ra03_north__z_004_006`: DESI 1401, MaNGA 531, shock proxy 0.572505
## Holds
- `HOLD_OBJECT_LEVEL_CROSSMATCH`
- `HOLD_DIRECT_GAS_DENSITY_INFERENCE`
- `HOLD_SELECTION_FUNCTION_FIT`
- `HOLD_COSMOLOGY_FIT`
## Receipt Backlinks
- Receipt: `shared-data/data/stellar_gas_observation/desi_epoviz_manga_population_cell_join_receipt.json`
- Data: `shared-data/data/stellar_gas_observation/desi_epoviz_manga_population_cell_join.json`
- Tiddler: `6-Documentation/tiddlywiki-local/wiki/tiddlers/DESI Epoviz MaNGA Population Cell Join.tid`

View file

@ -0,0 +1,76 @@
# DESI Epoviz Row Eigenmass Probe
Status: `DESI_ROW_EIGENMASS`
Decision: `ADMIT_DESI_ROW_EIGENMASS_HOLD_PHYSICAL_MASS`
This probe streams the DESI EDR epoviz CSV row-by-row and computes the dominant
correlation eigenvector over geometry, redshift, rosette phase, and tracer
identity. It is a literal-data stress pass over the DESI epoviz surface.
Claim boundary: this is SMN/evidence-load mass, not physical mass, not stellar
mass, not gas-density inference, and not a cosmology fit.
## Result
Rows read: `669377`
Dominant eigenvalue:
```text
3.276998814
```
Dominant explained mass share:
```text
0.327699881
```
## Dominant Eigenvector
- `x_glyr`: -0.339632035
- `y_glyr`: -0.350843233
- `z_glyr`: 0.284495161
- `redshift`: 0.51941062
- `rosette_sin`: -0.005671442
- `rosette_cos`: -0.058708375
- `tracer_QSO`: 0.178308871
- `tracer_ELG`: 0.373691964
- `tracer_LRG`: -0.001480357
- `tracer_BGS`: -0.485709225
## Eigensolver Diagnostics
```text
method: jacobi_symmetric
converged: True
iterations: 149
final max off-diagonal: 1.5584514983054888e-13
dominant residual L2: 0.0
```
## Population Counts
Tracer counts:
- `BGS`: 228630
- `ELG`: 261489
- `LRG`: 125174
- `QSO`: 54084
Cosmic redshift bins:
- `z_0_0p1`: 24267
- `z_0p1_0p5`: 224101
- `z_0p5_1`: 213259
- `z_1_2`: 196842
- `z_2_plus`: 10908
## Holds
- `HOLD_PHYSICAL_MASS_INTERPRETATION`
- `HOLD_DIRECT_STELLAR_GAS_INFERENCE`
- `HOLD_OBJECT_LEVEL_MANGA_CROSSMATCH`
- `HOLD_SELECTION_FUNCTION_FIT`
- `HOLD_COSMOLOGY_FIT`

View file

@ -0,0 +1,78 @@
# DESI NOIRLab2610 Calibration Note
**Date:** 2026-05-09
**Status:** `EXTERNAL_CALIBRATION_PRIOR`
**Source:** https://noirlab.edu/public/news/noirlab2610/
**Claim boundary:** this is an external observation-scale calibration note. It
is not imported data, not a cosmology fit, not a proof of the universe model,
and not a stellar-gas emission-line calibration by itself.
## Source Facts
The NOIRLab release is titled:
```text
DESI Completes Planned 3D Map of the Universe and Continues Exploring
```
The page metadata states that the Dark Energy Spectroscopic Instrument has
mapped more than 47 million galaxies and quasars, producing the largest
high-resolution 3D map of the Universe to date. It also states that DESI is
mounted on the NSF Nicholas U. Mayall 4-meter telescope and will continue
observations into 2028 because of instrument performance and hints that dark
energy might evolve.
## Why This Calibrates The Stack
This belongs in the stack as a large-scale-structure calibration prior:
```text
redshift sample count
+ sky footprint / selection function
+ galaxy/quasar tracer class
+ BAO / expansion-distance surface
+ survey extension window
```
For the project model, this is useful because it supplies an external anchor for
how information physics propagates through large countable tracers rather than
through a single local channel. In SMN terms, the high semantic mass comes from
population size, tracer diversity, selection-function burden, and cosmological
parameter pressure.
## Calibration Variables To Track
```text
N_tracers > 47 million galaxies and quasars
instrument DESI
mount Mayall 4-meter telescope
map_kind high-resolution 3D large-scale-structure map
continuation observations extended into 2028
calibration_use expansion-distance / dark-energy evolution prior
```
## Minimal Equation Hooks
These are hooks, not fitted results:
```text
z -> comoving_distance(z; H0, Omega_m, Omega_DE, w)
BAO scale -> D_M(z) / r_d and H(z) * r_d
semantic_load_DESI ~ log(N_tracers) + tracer_diversity + selection_function_burden
```
The next useful tool step is to keep DESI as a calibration-prior source for
large-scale topology and expansion models, while keeping MaNGA line-ratio data
as the local stellar-gas/shock-proxy lane.
## HOLD Conditions
- HOLD any numerical cosmology claim until a DESI data table, covariance, and
model-fit receipt are imported.
- HOLD any stellar-gas transfer until a bridge explicitly maps large-scale
structure statistics to local gas diagnostics.
- HOLD any dark-energy-evolution claim until source papers/data products are
cited and fit locally.

View file

@ -0,0 +1,86 @@
# DESI Stellar Gas Distribution Prior
**Date:** 2026-05-09
**Status:** `PRIOR_BRIDGE_HOLD_FIT`
**Decision:** `ADMIT_PRIOR_BRIDGE_HOLD_FIT`
**Claim boundary:** DESI large-scale structure can supply an environment prior
for stellar-gas distribution. A coarse DESI/MaNGA cell join exists; object-level
crossmatch remains `HOLD`. This is not a direct gas map, not a cosmology fit,
and not a local shock model.
## Core Idea
DESI does not directly measure stellar gas. It measures a huge galaxy/quasar
redshift tracer field. That is still useful because gas is not randomly placed:
it follows halos, filaments, nodes, voids, star-formation history, feedback, and
selection effects.
So the safe inference is:
```text
DESI tracer density
-> matter / environment prior
-> probable stellar-gas distribution
-> local update from MaNGA line ratios
-> residual map where the prior fails
```
## Inference Ladder
```text
L0: delta_g(x,z) = (n_g(x,z) - mean(n_g(z))) / mean(n_g(z))
L1: delta_m(x,z) ~= delta_g(x,z) / b_g(type,z)
L2: P(env | x,z) = softmax([void, sheet, filament, node]; delta_m, grad(delta_m), Hessian(Phi))
L3: rho_gas_prior(x,z) proportional_to f_b * rho_m(x,z) * G(env,z) * S(selection)
L4: P(gas_state | DESI_env, MaNGA_lines) proportional_to P(MaNGA_lines | gas_state) * P(gas_state | DESI_env)
```
The important step is L4. DESI supplies the wide-field environment prior; MaNGA
supplies local emission-line evidence.
## What This Gives Us Now
- A principled way to turn large-scale topology into a gas-distribution prior.
- A join target for MaNGA line-ratio diagnostics.
- A residual surface: places where cosmic-web environment predicts gas state
poorly become high-value FAMM/scar targets.
- A clean SMN interpretation: DESI environment classes and MaNGA diagnostic
classes are high semantic-load objects, but still not proof.
## What Remains HOLD
```text
direct stellar gas map HOLD
cosmology fit HOLD
dark-energy evolution claim HOLD
shock transfer claim HOLD
object-level DESI/MaNGA crossmatch HOLD
coarse DESI/MaNGA cell join EXISTS
```
## Next Tool Step
Use the existing coarse DESI/MaNGA cell join as the population prior, then keep
the object-level crossmatch path explicitly held until a cone and redshift
window receipt exists:
```text
match if angular_sep < theta_max and |z_DESI - z_MaNGA| < dz_max
```
After that, fit environment bins against MaNGA line-ratio classes and keep the
negative controls visible.
## Receipt
```text
shared-data/data/stack_solidification/desi_stellar_gas_distribution_prior_receipt.json
```
## Receipt Backlinks
- Coarse cell join receipt: `shared-data/data/stellar_gas_observation/desi_epoviz_manga_population_cell_join_receipt.json`
- Tiddler: `6-Documentation/tiddlywiki-local/wiki/tiddlers/DESI Stellar Gas Distribution Prior.tid`

View file

@ -0,0 +1,79 @@
# External sem Entity-Diff Probe
**Date:** 2026-05-09
## Decision
- Status: `OPTIONAL_AUDIT_AID_READY`
- No implementation code was imported.
- No repository dependency was added.
- Use only as an optional external audit aid unless explicitly vendored later.
## Why It Helps
- Primary fit: `FAIL-WORKTREE-SCOPE-007`, because entity-level diffs reduce broad dirty-tree risk.
- Secondary fit: `FAIL-RECEIPT-GATE-006`, because entity IDs can be attached to receipt and rollback checklists.
- It does not close security, coefficient, topology prediction, or FPGA transport gates by itself.
## Tool Boundary
- Source: `https://github.com/Ataraxy-Labs/sem`
- License: `MIT OR Apache-2.0`
- Requested binary: `/tmp/sem_probe/sem/crates/target/release/sem`
- Version result: `sem 0.5.3`
- `/usr/bin/sem` collision: `True`
## Entity Probe
### `4-Infrastructure/shim/stack_fail_closure_register.py`
- Status: `PASS`
- `function` `load_json` lines 20-21
- `function` `ticket` lines 24-43
- `function` `build_register` lines 46-215
- `function` `build_doc` lines 218-252
- `function` `main` lines 255-261
### `4-Infrastructure/shim/tang9k_uart_beacon_probe.py`
- Status: `PASS`
- `function` `read_port` lines 18-50
- `function` `main` lines 53-83
### `4-Infrastructure/shim/stack_solidification_audit.py`
- Status: `PASS`
- `class` `CmdResult` lines 43-49
- `function` `rel` lines 52-53
- `function` `run_cmd` lines 56-72
- `function` `load_json` lines 75-76
- `function` `file_sha256` lines 79-86
- `function` `json_gate` lines 89-105
- `function` `py_compile_gate` lines 108-111
- `function` `compiler_gate` lines 114-127
- `function` `tri_cycle_gate` lines 130-150
- `function` `lean_gate` lines 153-157
- `function` `hardware_bitstream_gate` lines 160-168
- `function` `worktree_gate` lines 171-194
- `function` `build_doc` lines 197-259
- `function` `main` lines 262-290
### `4-Infrastructure/shim/rrc_tri_cycle_audit.py`
- Status: `PASS`
- `class` `CmdResult` lines 106-112
- `function` `run_cmd` lines 115-131
- `function` `load_json` lines 134-135
- `function` `load_uart_beacon_diagnostics` lines 138-194
- `function` `scan_less_solid_surfaces` lines 197-216
- `function` `bucket_for_line` lines 219-231
- `function` `run_q16_witnesses` lines 234-308
- `function` `summarize_rrc` lines 311-331
- `function` `bucket_counts` lines 334-338
- `function` `active_gate_register` lines 341-347
- `function` `build_doc` lines 350-419
- `function` `main` lines 422-491
## Machine Receipt
- `shared-data/data/stack_solidification/external_sem_entity_diff_probe_receipt.json`

View file

@ -0,0 +1,206 @@
# FPGA Direct Probe Report
**Date:** 2026-05-09
**FPGA Model:** Gowin GW1NR-9 (Tang Nano 9K)
**Interface:** FTDI FT2232C (ID 0403:6010)
**Status:** SUPERSEDED / DO NOT USE AS FABRIC-UART PROOF
> **Supersession note, 2026-05-09:** Later SRAM-loaded beacon tests in
> `fpga_rrc_q16_accel_setup_2026-05-09.md` show no fabric UART bytes on
> `/dev/ttyUSB0` or `/dev/ttyUSB1` for both standard and swapped pins. Treat the
> `faXX` / `ffXX` patterns below as FTDI/JTAG/MPSSE or board-bridge behavior
> until proven otherwise. They are not valid evidence that the user fabric
> bitstream is communicating over UART.
---
## Direct FPGA Communication Results
### Interface A (/dev/ttyUSB0) - Primary UART
**Communication Pattern:**
- **Sent:** `00`**Received:** `fa00`
- **Sent:** `01`**Received:** `fa01`
- **Sent:** `02`**Received:** `fa02`
- **Sent:** `03`**Received:** `fa03`
- **Sent:** `04`**Received:** `fa04`
- **Sent:** `FF`**Received:** `faff`
- **Sent:** `FE`**Received:** `fafe`
**Analysis:**
- FPGA echoes data with `fa` prefix
- Pattern: `fa` + input byte
- This is a simple echo/transformation function
- `fa` likely indicates protocol identifier or framing
**Continuous Data Stream:**
```
fa00fa01fa02fa03fa04fab0fac0fad0fae0faf0
```
- FPGA appears to be streaming sequential data
- Pattern suggests counter or state machine
- Data increments from 00 to F0 with `fa` prefix
### Interface B (/dev/ttyUSB1) - Secondary UART
**Communication Pattern:**
- **Sent:** `02`**Received:** `ff00`
- **Sent:** `0102030405060708`**Received:** `ff031000ff0c`
- **Sent:** `A0A0A0A0`**Received:** `ff40` (decoded as `@`)
- **Sent:** `FFFFFFFF`**Received:** `ffff`
**Analysis:**
- Different protocol than Interface A
- Uses `ff` prefix
- Responds with variable-length data
- Appears to be a control/status interface
---
## Current FPGA Bitstream Analysis
### What the FPGA Says Directly
**Interface A says:** "I am in echo mode. I prefix all data with `fa` and echo it back."
**Interface B says:** "I respond to commands with `ff` prefix and variable-length responses."
### Current Bitstream Identity
**NOT the Meta-Manifold Prover:**
- The Meta-Manifold Prover design (metamanifold_prover_gowin.v) implements:
- Mass Number gates
- Torus topology distance
- Menger sponge hash
- Fold energy calculation
- Surface check
- Current bitstream does NOT implement these operations
- Operation select commands (00-04) returned no response
**Current Bitstream is:**
- Simple echo/transformation program
- Likely a bootloader, test program, or default factory bitstream
- Implements UART communication with FTDI interface
- No complex mathematical operations detected
### Hardware Identification
**Confirmed FPGA:**
- **Model:** Gowin GW1NR-9 (Tang Nano 9K)
- **LUTs:** 8640
- **FFs:** 6480
- **BRAM:** 468Kb (26 × 18Kb blocks)
- **DSP:** 20 multipliers (16×16)
- **PLLs:** 2
- **IO:** 68 user I/O
**Connection:**
- **USB Device:** FTDI FT2232C (ID 0403:6010)
- **Bus:** 005 Device 002
- **Interfaces:** A (ttyUSB0), B (ttyUSB1)
- **Baud Rate:** 115200
---
## Comparison: Expected vs Actual
### Expected (Meta-Manifold Prover Design)
```
Operations:
- op_select 000: Mass Number Gate
- op_select 001: Torus Distance
- op_select 010: Menger Hash
- op_select 011: Fold Energy
- op_select 100: Surface Check
Inputs:
- admissible, residual, epsilon, threshold (Q16_16)
- coord1, coord2 (torus coordinates)
- menger_x, menger_y, menger_z (Menger sponge)
- torus_energy, menger_energy, horn_energy (fold energy)
- alpha, beta, gamma (weights)
- surface_height, surface_ridge (surface check)
Outputs:
- mass_le_result (1 bit)
- torus_distance (12 bits)
- menger_address (16 bits)
- fold_energy_total (16 bits)
- surface_admissible (1 bit)
```
### Actual (Current Bitstream)
```
Operations:
- Echo with fa prefix (Interface A)
- Command response with ff prefix (Interface B)
Inputs:
- Single byte commands
- Multi-byte commands
Outputs:
- fa + input byte (Interface A)
- ff + variable response (Interface B)
```
---
## Interpretation
### What the FPGA is Currently Saying
**Direct Quote from FPGA:**
- Interface A: "fa00, fa01, fa02, fa03, fa04..."
- Interface B: "ff00, ff031000ff0c, ff40, ffff..."
**Translation:**
- "I am a simple UART echo/transformation program"
- "I am NOT the Meta-Manifold Prover design"
- "I am likely a bootloader or test program"
- "I am ready to receive a new bitstream"
### Why the Meta-Manifold Prover is Not Loaded
**Possible Reasons:**
1. **Bitstream not programmed:** The Meta-Manifold Prover design exists in Verilog but has not been synthesized and programmed to the FPGA
2. **Different bitstream loaded:** A different bitstream (bootloader/test) is currently loaded
3. **Configuration lost:** FPGA configuration was lost (power cycle or reconfiguration)
4. **Development in progress:** Meta-Manifold Prover is still being developed and not yet deployed
---
## Next Steps
### Option 1: Program Meta-Manifold Prover
1. Synthesize metamanifold_prover_gowin.v with Gowin toolchain
2. Generate bitstream file (.fs)
3. Program bitstream to FPGA via FTDI interface
4. Test Meta-Manifold Prover operations
### Option 2: Analyze Current Bitstream
1. Extract current bitstream from FPGA
2. Reverse-engineer bitstream to understand design
3. Document current bitstream functionality
### Option 3: Develop New Bitstream
1. Use current simple bitstream as starting point
2. Add Meta-Manifold Prover functionality
3. Test incremental additions
---
## Conclusion
**FPGA Direct Probe Results:**
- FPGA is alive and responding via USB
- Current bitstream is a simple echo/transformation program
- Meta-Manifold Prover design is NOT currently loaded
- FPGA is ready to receive a new bitstream
**What the FPGA Says Directly:**
- Interface A: "fa" + echoed data (simple echo mode)
- Interface B: "ff" + variable responses (command mode)
- Overall: "I am a simple UART communication program, not the Meta-Manifold Prover"
**Recommendation:**
The FPGA hardware is confirmed to be a Gowin GW1NR-9 (Tang Nano 9K) with FTDI FT2232C interface. To use the Meta-Manifold Prover design, the bitstream needs to be synthesized and programmed to the FPGA. The current simple bitstream suggests the FPGA is in a development or test state.

View file

@ -0,0 +1,171 @@
# FPGA Rainbow Raccoon Q16 Accelerator Setup
**Date:** 2026-05-09
**Board:** Tang Nano 9K / Gowin GW1N(R)-9C
**Status:** Bitstream builds and loads to SRAM; USB-UART path is not currently receiving fabric UART responses.
## Added Surface
Created a narrow FPGA witness lane for Rainbow Raccoon Compiler fixed-point lowering:
- `4-Infrastructure/hardware/tangnano9k_rrc_q16_accel.v`
- `4-Infrastructure/hardware/build_rrc_q16_accel.sh`
- `4-Infrastructure/shim/tang9k_rrc_q16_accel.py`
This is custom hardware work, not an off-the-shelf Gowin example. The RTL defines a purpose-built Q16.16 arithmetic witness surface for the Rainbow Raccoon pipeline, the build script targets that surface through the open-source Gowin flow, and the host shim emits machine receipts for each deterministic witness operation.
The hardware surface accepts framed UART requests:
```text
A5 02 seq opcode len payload crc8
```
and returns:
```text
A6 02 seq status len payload crc8
```
## Opcodes
| Opcode | Meaning | Lean receipt lane |
| --- | --- | --- |
| `0x20` | `x >>> 16` Q16.16 divide-by-65536 witness | `shiftRightEqDiv` |
| `0x21` | `(E * alpha) >>> 16 <= E` bounded weighted term witness | `weightedTermBounded` |
| `0x22` | monotone arithmetic shift witness | `shiftRightMonotone` |
The FPGA surface is intentionally not a proof authority. It accelerates deterministic arithmetic witnesses only; Lean and the host admission path remain authoritative.
## Custom Work Boundary
Accurate claim:
> This is a custom hardware witness surface for Lean-linked Q16.16 arithmetic lowering, built for the Rainbow Raccoon pipeline. The bitstream builds and SRAM-loads, and the software witness lane passes. Live FPGA receipts remain blocked by the UART/fabric route, not by the Q16 arithmetic design.
This boundary matters: build success, SRAM load, and software witness receipts are real evidence, but hardware acceleration is not promoted until the UART route returns matching hardware receipts.
## Build Evidence
Command:
```bash
cd 4-Infrastructure/hardware
./build_rrc_q16_accel.sh
```
Result:
- Yosys synthesis: pass, 0 check problems
- nextpnr-himbaechel: pass at 27 MHz
- Maximum reported clock frequency after route: `102.13 MHz`
- Device usage: `730/8640 LUT4`, `269/6480 DFF`, `1/5 MULT36X36`
- Bitstream SHA-256: `cdd8ae8f089c0b6dc5b638b112a3f4662497f2cee1b240cd5fcdb939b67166ff`
## Programming Evidence
SRAM load succeeds:
```bash
openFPGALoader -b tangnano9k 4-Infrastructure/hardware/tangnano9k_rrc_q16_accel.fs
```
Result:
```text
Load SRAM: 100.00%
CRC check: Success
```
Flash programming attempt was not accepted as durable:
```text
write Flash: 100.00%
CRC check : FAIL
Read: 0x00000000 checksum: 0x5c15
```
Use SRAM load for now. Do not claim persistent flash install until the flash readback failure is resolved.
## Host Receipts
Software receipt checks pass:
- `4-Infrastructure/shim/tang9k_rrc_q16_accel_shift_software_receipt.json`
- `4-Infrastructure/shim/tang9k_rrc_q16_accel_weighted_software_receipt.json`
- `4-Infrastructure/shim/tang9k_rrc_q16_accel_monotone_software_receipt.json`
USB-UART hardware receipt checks currently fail with no response on `/dev/ttyUSB1`:
- `4-Infrastructure/shim/tang9k_rrc_q16_accel_shift_hw_receipt.json`
- `4-Infrastructure/shim/tang9k_rrc_q16_accel_weighted_hw_receipt.json`
- `4-Infrastructure/shim/tang9k_rrc_q16_accel_monotone_hw_receipt.json`
A control test with the preexisting UART loopback bitstream also produced no `/dev/ttyUSB1` fabric response, while `/dev/ttyUSB0` shows FTDI MPSSE/JTAG command echoes such as `faXX`. Therefore the present blocker is the USB-UART-to-fabric pin path, not Q16 arithmetic synthesis.
A forced JTAG reset and SRAM reload was also tested. The JTAG path detects the Gowin device and reloads bitstreams successfully, but the beacon and Q16 hardware receipts remain silent on the host-visible serial endpoints. This makes the next useful test an external USB-UART adapter on the fabric pins, not another JTAG clear loop.
A PTY-backed virtual serial responder was added as a host-side control. It passes the same shift, weighted, and monotone Q16 operations through the existing serial harness, which confirms the host framing and receipt parser are working independently of the blocked physical UART route.
The UART path is now modeled through a transport route table. The active non-hardware route is `virtual://q16-pty`, while `/dev/ttyUSB0` and `/dev/ttyUSB1` remain blocked physical route entries. This lets the Rainbow Raccoon pipeline continue exercising serial receipts without confusing virtual-route success for live FPGA fabric success.
## UART Beacon Isolation
To remove the Q16 protocol parser from the failure surface, a TX-only fabric beacon was added:
- `4-Infrastructure/hardware/tangnano9k_uart_beacon.v`
- `4-Infrastructure/hardware/build_uart_beacon.sh`
The beacon repeatedly emits:
```text
A6 42 51 31 36 0A
```
which is the byte `0xA6`, beacon version byte `0x42`, followed by ASCII `Q16\n`.
Two beacon builds were tested:
| Constraint set | TX pin | RX pin | Bitstream SHA-256 | Probe receipt | Result |
| --- | ---: | ---: | --- | --- | --- |
| `tangnano9k_uart_loopback.cst` | 17 | 18 | `45feef98a527b4b1643dcbee983e1b36f9505a0170e0bd03abee65ccca30db17` | `4-Infrastructure/shim/tang9k_uart_beacon_probe_receipt.json` | no beacon bytes on `/dev/ttyUSB0` or `/dev/ttyUSB1` |
| `tangnano9k_uart_swapped.cst` | 18 | 17 | `46d68bd12371f203839c289e7bd7667ebdc33dafd0d6d2f33f9cc7caacc7b86d` | `4-Infrastructure/shim/tang9k_uart_beacon_swapped_probe_receipt.json` | no beacon bytes on `/dev/ttyUSB0` or `/dev/ttyUSB1` |
The second beacon uses an internal reset counter rather than relying on the external reset pin. This means the current blocker is no longer attributable to the Q16 parser, the host framing code, an external reset hold, or a simple TX/RX swap. Treat the live-hardware claim as blocked until the USB-UART bridge mode or physical UART route is verified with board documentation, BL702 bridge configuration, or an external USB-UART adapter on the fabric pins.
See the dedicated route analysis for the current hardware conclusion and external-adapter path:
- `6-Documentation/docs/fpga_uart_route_analysis_2026-05-09.md`
## Current Claim Boundary
What is ready:
- Open-source Gowin build pipeline for this accelerator
- SRAM-loaded FPGA bitstream
- Host-side framed Q16 receipt harness
- PTY-backed virtual serial control for host protocol validation
- Lean-linked opcode contract
What is not ready:
- Durable flash programming
- Live USB-UART fabric receipts
- Integration into automatic Rainbow Raccoon Compiler routing
## Next Fix
Resolve the UART physical route:
1. Confirm Tang Nano 9K USB-UART bridge mode and pins for this board revision.
2. If the onboard USB bridge is not routing fabric UART to `/dev/ttyUSB1`, attach an external USB-UART adapter to the actual fabric pins and re-run the beacon first.
3. Re-run the simple loopback or TX-only beacon bitstream before testing the Q16 accelerator.
4. Once the beacon responds, re-run:
```bash
python3 4-Infrastructure/shim/tang9k_uart_beacon_probe.py --port /dev/ttyUSB2
python3 4-Infrastructure/shim/tang9k_rrc_q16_accel.py --op shift --x 0x00038000 --port /dev/ttyUSB2
python3 4-Infrastructure/shim/tang9k_rrc_q16_accel.py --op weighted --energy 0x000a0000 --alpha 0x00008000 --port /dev/ttyUSB2
python3 4-Infrastructure/shim/tang9k_rrc_q16_accel.py --op monotone --a 0x00010000 --b 0x00030000 --port /dev/ttyUSB2
```
Use the actual external adapter path if it enumerates as a different device, for example `/dev/ttyACM0`.

View file

@ -0,0 +1,173 @@
# FPGA UART Route Analysis
**Date:** 2026-05-09
**Board:** Tang Nano 9K / Gowin GW1N(R)-9C
**Status:** USB-UART bridge path blocked; external adapter solution required
## Current Situation
### Problem Summary
The Tang Nano 9K USB-UART bridge is not routing fabric UART to `/dev/ttyUSB0` or `/dev/ttyUSB1`. Both standard and swapped pin configurations have been tested with TX-only beacon bitstreams, and no fabric UART bytes were observed on either device.
This route failure does not erase the custom FPGA work already completed. The Q16.16 accelerator RTL, open-source build script, Lean-linked opcodes, host receipt harness, and TX-only beacon are custom surfaces for this stack. The current blocker is specifically the live serial route from host to fabric.
### Evidence of Blockage
**Beacon Test Results:**
- Standard pin config (TX=17, RX=18): No beacon bytes on `/dev/ttyUSB0` or `/dev/ttyUSB1`
- Swapped pin config (TX=18, RX=17): No beacon bytes on `/dev/ttyUSB0` or `/dev/ttyUSB1`
- Beacon pattern: `A6 42 51 31 36 0A` (0xA6 + beacon version 0x42 + ASCII "Q16\n")
**Control Test Results:**
- Preexisting UART loopback bitstream also produced no `/dev/ttyUSB1` fabric response
- `/dev/ttyUSB0` shows FTDI MPSSE/JTAG command echoes (`faXX` patterns)
- This confirms the blocker is the USB-UART-to-fabric pin path, not Q16 arithmetic synthesis
**Forced JTAG Clear Results:**
- `openFPGALoader --detect` sees the Gowin `GW1N(R)-9C` over the FT2232 JTAG path
- JTAG reset and SRAM reload of the TX-only beacon complete with `CRC check: Success`
- Beacon probe after forced JTAG reset/reload still reads zero bytes on `/dev/ttyUSB0` and `/dev/ttyUSB1`
- Loading the loopback bitstream after the JTAG clear produced `faXX`-style bytes on `/dev/ttyUSB0`, matching FTDI/MPSSE bridge behavior rather than the expected fabric beacon payload
- Reloading the Q16 accelerator to SRAM succeeds, but hardware receipts still return empty/short frames
**JTAG Clear Conclusion:**
The forced JTAG clear confirms that the USB-UART bridge issue is not a stale state or configuration problem that can be resolved by JTAG reset. The `faXX` patterns are definitively FTDI/MPSSE bridge behavior, not fabric UART responses. The BL702 bridge appears to be operating in JTAG/MPSSE mode only, not transparent UART bridge mode for fabric communication.
**Virtual Serial Control:**
- A PTY-backed virtual serial responder was added to exercise the same Q16 host framing and receipt parser without live FPGA fabric
- The virtual serial probe passes shift, weighted, and monotone Q16 operations
- This narrows the remaining failure to the physical/JTAG-bridge-to-fabric route, not the host protocol encoder, parser, or opcode semantics
- Machine receipt: `shared-data/data/stack_solidification/tang9k_rrc_q16_virtual_serial_probe.json`
**Transport Router Control:**
- The UART routes are now represented as a route table rather than loose device paths
- The active non-hardware route is `virtual://q16-pty`
- Onboard FTDI routes remain visible as blocked physical entries, and the external USB-UART route remains the required live-hardware closure path
- Machine receipt: `shared-data/data/stack_solidification/tang9k_uart_transport_routes.json`
## Current Pin Assignments
### Standard Configuration (tangnano9k_uart_loopback.cst)
```
IO_LOC "uart_tx_pin" 17;
IO_LOC "uart_rx_pin" 18;
```
### Swapped Configuration (tangnano9k_uart_swapped.cst)
```
IO_LOC "uart_tx_pin" 18;
IO_LOC "uart_rx_pin" 17;
```
### Sparkle Project Reference
```
// USB-UART via onboard BL702.
IO_LOC "uart_tx" 17;
```
## Root Cause Analysis
### Likely Issue: BL702 Bridge Configuration
The Tang Nano 9K uses an onboard BL702 microcontroller as a USB-UART bridge. The BL702 may be:
1. Configured for JTAG/MPSSE mode only (not UART bridge mode)
2. Not routing the expected fabric pins to the USB UART endpoints
3. Requiring specific configuration commands to enable UART bridging
### Alternative Issue: Pin Multiplexing
The physical pins 17/18 may be multiplexed with other functions:
- Could be configured as GPIO instead of UART
- May require BL702 firmware configuration
- Bridge may not be in transparent UART mode
## Proposed Solution: External USB-UART Adapter
### Rationale
Since the onboard BL702 bridge is not functioning as expected, the most reliable path forward is to bypass it entirely and use an external USB-UART adapter connected directly to the fabric pins.
### Implementation Plan
#### Hardware Required
- External USB-UART adapter (FTDI or CP210x based)
- Connection wires (female-to-female dupont cables)
- Access to Tang Nano 9K GPIO header pins 17 and 18
#### Connection Diagram
```
External USB-UART Adapter Tang Nano 9K
------------------------ --------------
TX (adapter) --------> RX (pin 18)
RX (adapter) <-------- TX (pin 17)
GND --------> GND
```
#### Software Steps
1. Identify external USB-UART device (e.g., `/dev/ttyUSB2` or `/dev/ttyACM0`)
2. Modify probe scripts to use external device path
3. Rebuild beacon bitstream with standard pin assignment (TX=17, RX=18)
4. Load beacon bitstream to FPGA via SRAM
5. Test beacon reception on external UART device
6. If beacon succeeds, proceed to Q16 accelerator testing
#### Modified Probe Commands
```bash
# Test beacon with external adapter
python3 4-Infrastructure/shim/tang9k_uart_beacon_probe.py --port /dev/ttyUSB2
# Test Q16 operations with external adapter
python3 4-Infrastructure/shim/tang9k_rrc_q16_accel.py --op shift --x 0x00038000 --port /dev/ttyUSB2
python3 4-Infrastructure/shim/tang9k_rrc_q16_accel.py --op weighted --energy 0x000a0000 --alpha 0x00008000 --port /dev/ttyUSB2
python3 4-Infrastructure/shim/tang9k_rrc_q16_accel.py --op monotone --a 0x00010000 --b 0x00030000 --port /dev/ttyUSB2
```
## Fallback: BL702 Bridge Investigation
If external adapter is not available, investigate BL702 bridge configuration:
### Investigation Steps
1. Obtain Tang Nano 9K board schematic and BL702 documentation
2. Check if BL702 requires firmware update or configuration
3. Test with Gowin official toolchain to see if they use different pin assignments
4. Examine BL702 USB descriptors to determine current mode
5. Research if BL702 can be reconfigured or if alternative bridge mode exists
### Risk Assessment
- BL702 configuration may be board-specific and not publicly documented
- Firmware updates may brick the bridge if not done correctly
- Time investment may be high with uncertain success probability
## Recommendation
**Primary Path:** Use external USB-UART adapter
- Higher probability of success
- Bypasses undocumented BL702 behavior
- Provides direct fabric access for debugging
- Minimal configuration required
**Secondary Path:** BL702 bridge investigation
- Only pursue if external adapter unavailable
- Requires board documentation research
- May need Gowin vendor support
- Higher risk and time investment
## Next Actions
1. **Immediate:** Acquire or locate external USB-UART adapter
2. **Hardware:** Connect adapter to Tang Nano 9K pins 17 (TX) and 18 (RX)
3. **Software:** Identify adapter device path and modify probe scripts
4. **Test:** Run beacon probe with external adapter
5. **Validate:** If beacon succeeds, proceed to Q16 accelerator testing
6. **Document:** Update FPGA setup documentation with external adapter procedure
## Success Criteria
- Beacon pattern `A6 42 51 31 36 0A` received on external UART device
- Q16 shift operation returns correct hardware receipt
- Q16 weighted term operation returns correct hardware receipt
- Q16 monotone operation returns correct hardware receipt
- All operations match software receipt expectations
## References
- Original FPGA setup: `6-Documentation/docs/fpga_rrc_q16_accel_setup_2026-05-09.md`
- Superseded probe report: `6-Documentation/docs/fpga_direct_probe_report_2026-05-09.md`
- Hardware constraint files: `4-Infrastructure/hardware/tangnano9k_uart_*.cst`

View file

@ -0,0 +1,207 @@
# Hopfion Topological Soliton Lane
**Date:** 2026-05-09
**Status:** `TOPOLOGICAL_SOLITON_DESIGN_PRIOR`
**Claim boundary:** this note folds laser-created isolated magnetic hopfions
into the Research Stack as a topology/field-configuration primitive. Hopfions
are particle-like topological magnetic solitons, not elementary particles. This
does not claim new Standard Model particle physics, device readiness, or
spintronic engineering success.
## Source
Phys.org reported the first direct observation of laser-created isolated
hopfions, based on the Nature Physics paper:
```text
Laser-induced nucleation of magnetic hopfions
Nature Physics (2026)
DOI: 10.1038/s41567-026-03236-0
```
Useful source facts:
- The observed objects are isolated magnetic hopfions in cubic chiral FeGe.
- They were nucleated by femtosecond laser pulses and observed by transmission
electron microscopy.
- Quantitative agreement between experiment and micromagnetic simulation was
used as evidence.
- Algebraic topology was used to classify the observed magnetic
configurations.
- The observed isolated hopfion can be characterized by an integer Hopf charge,
with examples including `H = -1`.
Sources:
- `https://phys.org/news/2026-05-laser-isolated-hopfions.html`
- `https://doi.org/10.1038/s41567-026-03236-0`
## Why This Matters For The Stack
This is a nearly perfect physical analogue for the stack's receipt discipline:
```text
local field texture
-> projection through an instrument
-> simulation replay
-> topological invariant
-> admitted particle-like state
```
That is exactly the stack pattern:
```text
structure -> projection -> receipt -> replay -> invariant gate
```
The important upgrade is that this is not just a 2D braid metaphor. A hopfion is
a 3D field texture whose nontrivial topology can survive deformation unless a
singular/unwinding event occurs. That makes it a strong model for:
- braided rope states;
- torsional memory-bearing trajectories;
- logogram folds with nontrivial closure;
- AMMR leaves that carry topological charge;
- FAMM scars that are local minima in an energy landscape.
## Core Equations And Invariants
The broader reusable equation pack is:
```text
6-Documentation/docs/topological_soliton_equation_pack_2026-05-09.md
```
The Nature Physics paper frames the topology as maps of pairs of spaces:
```text
f : (I^3, partial I^3) -> (A, B)
```
Where:
```text
I^3 = localization domain
partial I^3 = boundary of the localization domain
A = S^2, the order-parameter sphere
B = constrained boundary subspace
```
The key softened-boundary invariant is:
```text
pi_3(S^2, S^2 \ union_i X_i) = Z, n >= 1
```
This matters because it keeps integer Hopf charge available under realistic
boundary constraints, not only idealized one-point boundary conditions.
For the stack:
```text
H in Z
H = 0 trivial / unwindable class
H != 0 nontrivial topological receipt
|H| = 1 generator / anti-generator class
```
The micromagnetic energy surface includes exchange, DMI, Zeeman, and
demagnetizing terms:
```text
E = int_Vm dr [
A * sum_i |grad m_i|^2
+ D * m . (grad x m)
- M_s * m . B
]
+ (1 / (2 mu_0)) * int_R3 dr sum_i |grad A_d,i|^2
```
Where:
- `m(r) = M(r) / M_s` is the normalized magnetization field.
- `A` is the Heisenberg exchange constant.
- `D` is the DMI constant.
- `B` is the external plus demagnetizing magnetic field.
- `A_d` is the demagnetizing vector potential.
## Receipt Gate
Minimum admission gate:
```text
if projected image is missing:
HOLD_MISSING_PROJECTION
elif simulation replay is missing:
HOLD_MISSING_MICROMAGNETIC_REPLAY
elif topological invariant H is missing:
HOLD_MISSING_HOPF_CHARGE
elif H == 0:
HOLD_TRIVIAL_TOPOLOGY
elif projection and simulation disagree above tolerance:
HOLD_PROJECTION_REPLAY_MISMATCH
else:
ADMIT_TOPOLOGICAL_SOLITON_PRIOR
```
This is deliberately a design-prior gate. It does not assert that the stack can
create or control hopfions. It says the stack can borrow the logical shape:
```text
particle-like state = localized field + replay projection + integer topology
```
## Mapping To Existing Stack Surfaces
| Hopfion paper concept | Stack surface |
|---|---|
| Femtosecond laser perturbation | controlled energy kick / topology crossing gate |
| Complex energy landscape | FAMM basin / frustration surface |
| Local minimum | stable receipt-bearing state |
| TEM projection | projection receipt / rendered view |
| Micromagnetic simulation | replay witness |
| Hopf charge `H` | integer topological invariant |
| Boundary subspace `B` | residual / admissibility boundary |
| Punctured sphere | allowed field state with excluded singular regions |
| `H = -1` | anti-generator / oriented rope charge |
## Fit With The Eigen/Topology Work
This should sharpen the topology lane more than the shock lane. The strongest
local bridge is:
```text
topological chain reduction
+ torsional rope memory
+ energy-landscape FAMM scars
+ projection/replay receipts
+ integer invariant gates
```
The likely future Lean shape is not continuous micromagnetics first. The first
Lean shape should be finite and receipt-friendly:
```text
structure HopfionReceipt where
projection_present : Bool
replay_present : Bool
hopf_charge : Int
projection_residual_q0_16 : UInt16
residual_bound_q0_16 : UInt16
```
Then prove the gate rejects missing projection, missing replay, zero charge,
and over-bound residual before it admits a nonzero topological class.
## Next Work
1. Add `TopologicalSolitonReceipt` as the general finite Lean gate surface.
2. Add `HopfionTopologicalSoliton` as a fixture family over that gate.
3. Add fixtures for missing projection, missing replay, `H = 0`, `H = -1`, and
projection/replay mismatch.
4. Re-run the topology/eigen remapper and check whether the soliton/topology
lane gains a cleaner support signature.
5. Keep device, memory, spintronic, and elementary-particle claims HOLD until
direct receipts exist.

View file

@ -0,0 +1,127 @@
# Hutter Eigenmass Transfer Plan
Status: `TRANSFER_PROTOCOL_HOLD_HUTTER_CLAIM`
Decision: `ADMIT_TRANSFER_PROTOCOL_HOLD_HUTTER_CLAIM`
This document ports the DESI/MaNGA multiscale eigenmass pattern into the Hutter
compression lane as a tuning protocol. It is a method-transfer receipt, not a
compression benchmark.
Claim boundary: no enwik9 run, no compression-gain claim, no Hutter Prize claim,
and no FPGA/ASIC promotion is made here.
## Imported Signal
```text
DESI/MaNGA tracer cosine: 0.702922832
DESI/MaNGA sharpening factor: 1.784206501
```
Only the evidence-sharpening pattern transfers. Astronomy values do not become
compression scores.
## Hutter Feature Basis After Tuning
- `byte_offset_phase`
- `symbol_class`
- `local_context_hash_class`
- `long_range_recurrence_distance`
- `prediction_cache_hit`
- `ram_trace_reuse`
- `residual_entropy_proxy`
- `oisc_instruction_density`
- `ammr_receipt_depth`
- `replay_delta_cost`
## Transfer Ladder
- DESI `literal row surface` -> Hutter `raw corpus windows`; gate `WINDOW_FIXTURE_ONLY_UNTIL_CANONICAL_ENWIK9`
- DESI `joined gas/shock constrained cells` -> Hutter `byte-exact replay fixtures constrained by Rust OISC closure`; gate `LEAN_RUST_REPLAY_REQUIRED`
- DESI `constraint sharpening factor` -> Hutter `compression candidate must sharpen prediction/replay axes without losing byte-exact closure`; gate `SHARPENING_WITH_EXACT_REPLAY_ONLY`
- DESI `multiscale cosine alignment` -> Hutter `candidate feature direction must align across raw windows, token/logogram windows, and OISC replay receipts`; gate `MULTISCALE_ALIGNMENT_BEFORE_PROMOTION`
## JPEG XL Starfield First Sweep
Add a non-claim visual sidecar:
```text
Hutter text fixture
-> byte/window projection
-> JPEG XL sidecar image
-> pixel-density pattern collections
-> eigenprobe over density groups
-> "grouped stars" observation receipt
```
This is not a classifier and not a compression claim. It only asks whether a
projected pixel-density field shows grouped starfield-like patches worth
returning to the byte fixture for replay checks.
Allowed decisions:
```text
OBSERVE_GROUPING
OBSERVE_NO_GROUPING
HOLD_PROJECTION_NOISE
QUARANTINE_MISSING_PROVENANCE
```
Receipt: `shared-data/data/stack_solidification/hutter_jxl_starfield_eigenprobe_first_sweep_receipt.json`
## Minimum Receipt Shape
```json
{
"corpus_id": "canonical hash or fixture label",
"window_id": "byte offset + length + hash",
"baseline_sizes": "raw, zlib/lzma, current candidate if available",
"candidate_features": [
"byte_offset_phase",
"symbol_class",
"local_context_hash_class",
"long_range_recurrence_distance",
"prediction_cache_hit",
"ram_trace_reuse",
"residual_entropy_proxy",
"oisc_instruction_density",
"ammr_receipt_depth",
"replay_delta_cost"
],
"eigenmass": "dominant eigenvalue + explained share + feature vector",
"replay": "input hash + output hash + instruction count + decision",
"decision": "ADMIT_FIXTURE / HOLD / QUARANTINE"
}
```
## Promotion Gates
- `canonical_enwik9_sha256_or_fixture_label_required`
- `raw_baseline_required`
- `candidate_wire_format_required`
- `byte_exact_decompressor_receipt_required`
- `negative_controls_required`
- `eigenmass_sharpening_required`
- `no_competitive_hutter_claim_without_full_prize_envelope`
## Holds
- `HOLD_CANONICAL_ENWIK9`
- `HOLD_FULL_CORPUS_RUN`
- `HOLD_COMPETITIVE_COMPRESSION_CLAIM`
- `HOLD_PRODUCTION_DECOMPRESSOR`
- `HOLD_FPGA_ASIC_PROMOTION`
## Receipt
```text
plan hash: 85f1daf7bb2a7ae5236904937d77f10bcdda2d5b97b2acf25ee3ae763dfbe22e
```
## Receipt Backlinks
- Receipt: `shared-data/data/stack_solidification/hutter_eigenmass_transfer_plan_receipt.json`
- Source multiscale receipt: `shared-data/data/stellar_gas_observation/stellar_gas_multiscale_eigenmass_alignment_receipt.json`
- Data: `shared-data/data/stack_solidification/hutter_eigenmass_transfer_plan.json`
- Tiddler: `6-Documentation/tiddlywiki-local/wiki/tiddlers/Hutter Eigenmass Transfer Plan.tid`
- JPEG XL first-sweep tiddler: `6-Documentation/tiddlywiki-local/wiki/tiddlers/Hutter JPEG XL Starfield Eigenprobe First Sweep.tid`

View file

@ -0,0 +1,32 @@
# Hutter Transfer Readiness Fixture Manifest
Decision: `ADMIT_FIXTURE`
This is a small fixture-readiness manifest. It records byte provenance,
window receipts, baseline sizes, negative controls, and OISC replay
requirements before any later spectral analysis can be considered.
## Fixture
- Fixture id: `htrf_local_text_window_2026_05_09_v0`
- Source path: `shared-data/data/stack_solidification/fixtures/hutter_transfer_readiness_fixture.txt`
- Byte length: `669`
- SHA-256: `eacd7a17d3485ff0e9fd3ec19c3c2dfcc7419e906eafe18817b07596a99ad6e6`
## Baseline Matrix
| route | bytes | note |
| --- | ---: | --- |
| raw | 669 | source bytes |
| zlib | 423 | stdlib level 9 |
| lzma | 520 | stdlib preset 6 |
| current-wire | 2012 | OISC replay wire |
## Controls
Negative controls are registered as `HOLD` before any later result gate.
## Receipt
- Manifest: `shared-data/data/stack_solidification/hutter_transfer_readiness_fixture_manifest.json`
- OISC Rust test: `non_toy_transfer_fixture_replays_byte_exact`

View file

@ -0,0 +1,164 @@
# Matched-Reference Genomics For Logogram Replay
**Date:** 2026-05-09
**Status:** `DESIGN_PRIOR_RECEIPT`
**Claim boundary:** this note uses matched-reference genomics as a design prior
for logogram replay and static decompressor surfaces. It does not claim a
biological codec implementation, clinical utility, or compression-ratio win.
## Source
Nature Communications published:
```text
Cell line-matched reference enables high-precision functional genomics
DOI: 10.1038/s41467-025-66155-3
Published: 2025-11-20
```
The paper's core result is that sequencing and functional-genomics analysis
improve when reads from a laboratory cell line are mapped against a reference
assembly from the same cell line, especially in divergent, repetitive,
haplotype-specific regions such as centromeres.
Useful source facts:
- A generic reference genome can introduce reference bias.
- The RPE-1 matched diploid reference improves DNA/RNA read mapping.
- The gain is strongest in divergent centromeric and pericentromeric loci.
- Haplotype-specific centromere organization matters for guide design,
epigenetic peak calling, and kinetochore localization.
Source URL:
```text
https://www.nature.com/articles/s41467-025-66155-3
```
## Stack Translation
The useful abstraction is:
```text
payload should be decoded against the matching reference basis
```
For the Research Stack:
```text
generic reference -> generic grammar / generic dictionary / generic route prior
cell-line reference -> matched logogram basis / matched dictionary / matched route prior
reference bias -> replay bias / residual inflation / false mismatch
haplotype-specificity -> branch-specific receipt basis
centromere divergence -> high-repetition, high-ambiguity logogram region
```
This gives a clean rule for the static decompressor:
```text
do not unspool against a universal reference if the payload declares
a more specific matched basis
```
## Isogenomic Replay Primitive
Define an `isogenomic replay basis` as a reference surface whose identity is
matched to the payload's declared substrate family.
Minimum fields:
```yaml
matched_reference_basis:
basis_id:
substrate_family:
alphabet:
haplotype_or_branch:
source_hash:
payload_hash:
reference_hash:
mismatch_policy:
residual_policy:
```
Replay gate:
```text
if payload declares matched_reference_basis and basis is missing:
HOLD_MISSING_MATCHED_REFERENCE
elif reference_hash mismatch:
QUARANTINE_REFERENCE_BIAS
elif replay residual <= declared bound:
ADMIT_MATCHED_REPLAY
else:
HOLD_RESIDUAL_INFLATION
```
## Why This Helps The Decompressor
A static decompressor is intentionally dumb: it should not improvise a better
world model at decode time. The compressor must therefore emit enough reference
identity for the decompressor to replay the intended path.
Matched-reference genomics supports that design:
```text
the same observed symbols can map differently under a generic versus matched
reference, and the mismatch is most dangerous in repetitive divergent regions
```
For logogram / Hachimoji / OISC surfaces, those risky regions are:
- repeated symbols
- dense alphabets
- folded or mirrored branches
- branch-local dictionaries
- whitespace-zero grammar boundaries
- highly reused AMMR leaves
So the decompressor should prefer:
```text
payload + matched reference hash + residual
```
over:
```text
payload + generic global dictionary
```
## Relation To Existing Surfaces
This design prior attaches to:
- `6-Documentation/docs/specs/OMINDIRECTION_LOGOGRAM_DESIGN_AND_COMPILER.md`
- `4-Infrastructure/infra/embedded_surface/omni_lut/sequence_surface_lut.py`
- `5-Applications/compression-core/src/oisc.rs`
- `shared-data/data/stack_solidification/rust_oisc_decompressor_target_receipt.json`
It also sharpens the earlier hachimoji / sequence-surface split:
```text
Q4-ROISC -> smaller carrier, simpler static unspooling
Hachimoji/BF -> larger alphabet, richer command surface
matched basis -> required whenever richer alphabets create replay ambiguity
```
## Decision
```text
ADMIT_AS_DESIGN_PRIOR
HOLD_FOR_IMPLEMENTATION
```
## Next Work
1. Add a `reference_hash` and `basis_id` field to the Rust OISC receipt shape.
2. Add a negative fixture where the payload decodes under a generic basis but
fails under the declared matched basis.
3. Add a hachimoji/logogram replay fixture where branch-specific basis metadata
lowers residual size without changing byte-exact output.
4. Keep any compression-efficiency claim HOLD until fixture and corpus receipts
exist.

View file

@ -0,0 +1,51 @@
# Network Topology HOLD Manifests
**Date:** 2026-05-09
These manifests separate hypothesis weights and predictions from calibrated or validated claims.
## Coefficient Calibration
- Rows: `9`
- Status: `HOLD_CALIBRATION`
- Receipt: `shared-data/data/stack_solidification/network_topology_coefficient_calibration_manifest.json`
| Methodology | Raw Weight | Receipt Weight | Decision |
| --- | ---: | ---: | --- |
| `backhaul_providers` | 0.07 | 0.06254 | `HOLD_TOPOLOGY_PREDICTION_VALIDATION` |
| `civic_design_mathematics` | 0.1 | 0.108488 | `HOLD_COEFFICIENT_RECEIPT_DEBT` |
| `hft_infrastructure` | 0.21 | 0.147415 | `HOLD_COEFFICIENT_RECEIPT_DEBT` |
| `major_consumer_nodes` | 0.08 | 0.071474 | `HOLD_TOPOLOGY_PREDICTION_VALIDATION` |
| `public_internet_map` | 0.12 | 0.183791 | `KEEP_OBSERVED_PRIOR_HIGH` |
| `regional_infrastructure` | 0.09 | 0.103382 | `HOLD_PROVENANCE` |
| `slime_mold_physics` | 0.1 | 0.102106 | `HOLD_ANALOGY_ADAPTER` |
| `soliton_wave_analysis` | 0.15 | 0.134014 | `HOLD_ANALOGY_ADAPTER` |
| `subway_underground` | 0.08 | 0.08679 | `HOLD_ANALOGY_ADAPTER` |
## Prediction HOLD Registry
- Rows: `15`
- Status: `HOLD_PREDICTION_VALIDATION`
- Receipt: `shared-data/data/stack_solidification/network_topology_prediction_hold_registry.json`
| Prediction | Kind | Status |
| --- | --- | --- |
| `node::disney::ashburn_va` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `node::disney::new_york_ny` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `node::disney::los_angeles_ca` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `node::comcast::new_york_ny` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `node::comcast::los_angeles_ca` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `node::warner_discovery::new_york_ny` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `node::warner_discovery::los_angeles_ca` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `node::paramount::new_york_ny` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `node::netflix::los_angeles_ca` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `node::disney::san_francisco_ca` | `predicted_network_node` | `HOLD_PREDICTION_VALIDATION` |
| `path::ashburn_va::dallas_tx` | `novel_network_path` | `HOLD_PREDICTION_VALIDATION` |
| `path::chicago_il::seattle_wa` | `novel_network_path` | `HOLD_PREDICTION_VALIDATION` |
| `path::new_york_ny::miami_fl` | `novel_network_path` | `HOLD_PREDICTION_VALIDATION` |
| `path::los_angeles_ca::denver_co` | `novel_network_path` | `HOLD_PREDICTION_VALIDATION` |
| `path::london_uk::frankfurt_germany` | `novel_network_path` | `HOLD_PREDICTION_VALIDATION` |
## Claim Boundary
These rows are accounting and validation queues. They do not calibrate coefficients or validate topology predictions.

View file

@ -0,0 +1,151 @@
# Palindrome Hex Symmetry Markers
Status: `DETERMINISTIC_MARKER_DESIGN_PRIOR`
Source prompt: Scientific American Proof Positive puzzle context, user supplied
palindrome probability counts, 2026-05-09.
Source page:
```text
https://www.scientificamerican.com/article/why-some-mathematicians-think-we-should-abandon-pi/
```
## Claim Boundary
Palindrome hex values are useful as deterministic symmetry markers, frame
sentinels, mirror-route tags, and replay witnesses.
They are not:
- entropy sources
- cryptographic identifiers
- collision-resistant hashes
- uniqueness guarantees
- proof of payload correctness by themselves
The correct role is:
```text
palindrome marker + marker namespace + payload hash + receipt hash
```
not:
```text
palindrome marker = secure identity
```
## Counting Rule
The decimal puzzle gives the intuition: if a number must read the same forward
and backward, the first symbol controls both ends. That makes leading-zero
constraints matter.
For a four-hex-digit marker:
```text
ABBA
```
with no leading zero:
```text
A in {1..F} -> 15 choices
B in {0..F} -> 16 choices
palindromes -> 15 * 16 = 240
4-hex-digit values from 0x1000..0xFFFF -> 61440
probability -> 240 / 61440 = 1 / 256
```
If leading zeros are allowed inside a fixed 16-bit lane:
```text
palindromes -> 16 * 16 = 256
all 16-bit values -> 65536
probability -> 1 / 256
```
So the marker is deterministic and sparse enough to be visible, but nowhere
near sparse enough to act as a security token.
## Marker Examples
These are reserved as examples until a Lean/Rust marker registry exists:
| Marker | Meaning |
|---|---|
| `0xA55A` | mirror-route synchronization marker |
| `0x5AA5` | reverse replay checkpoint marker |
| `0xC33C` | closure witness marker |
| `0xE11E` | admitted edge marker |
| `0xF00F` | frame boundary marker |
## Static Decompressor Use
For a static decompressor, palindrome hex markers can provide cheap boundaries
inside a zero-whitespace or compact base stream:
```text
packet stream
-> palindrome frame marker
-> marker namespace
-> payload hash
-> residual policy
-> replay receipt
```
The decompressor may use the marker to regain synchronization, choose the mirror
replay lane, or reject a malformed frame early. It still must verify hashes and
receipts before admitting payload bytes.
## Remote Test Use
For local/remote reproducibility tests, palindrome markers are useful because
they make packet boundaries deterministic across environments:
```text
local packet marker == remote packet marker
local payload hash == remote payload hash
local receipt hash == remote receipt hash
```
If marker equality holds but payload or receipt hashes diverge, the marker has
done its job: it localized the mismatch without pretending the result is valid.
## Gates
```text
ADMIT_MARKER
if marker is palindromic
and namespace is declared
and payload hash exists
and receipt hash exists
and mirror replay agrees
HOLD_NOT_SYMMETRIC
if a marker in the palindrome lane is not palindromic
HOLD_NAMESPACE_COLLISION
if the same marker is reused without a namespace or packet kind
HOLD_MIRROR_REPLAY_MISMATCH
if forward and reverse replay disagree
QUARANTINE_ENTROPY_MISUSE
if a palindrome marker is used as a secret, random nonce, or security proof
```
## Integration Rule
Palindrome hex values are structural punctuation for the receipt stream:
```text
symbol count gives spacing
palindrome gives mirror boundary
hash gives payload identity
receipt gives replay authority
```
This pairs cleanly with whitespace-zero grammar, omindirection, static OISC
decompression, and remote reproducibility tests.

View file

@ -0,0 +1,181 @@
# Predictive Harmony Social Synchrony Prior
Status: `EXTERNAL_NEUROACOUSTIC_ROUTE_PRIOR`
Source:
```text
PsyPost summary:
https://www.psypost.org/scientists-show-how-common-chord-progressions-unlock-social-bonding-in-the-brain/
Primary study:
Watts, Allsop, Compton, Zhang, Noah, Hirsch.
"Listening to a Consonant Chord Progression during Live Face-to-Face Gaze
Enhances Neural Activity in Social Systems."
Journal of Neuroscience, 2026.
DOI: 10.1523/JNEUROSCI.1116-25.2026
```
## Claim Boundary
This source is admitted as a neuroacoustic route prior:
```text
predictable harmonic structure + live mutual gaze
-> increased social-system activity and cross-brain synchrony in dyads
```
It is not admitted as:
- proof that any chord progression causes bonding in all settings
- a clinical therapy claim
- a social-control mechanism
- a payload authority for the logogram layer
- evidence that music notation alone certifies reconstruction
## Useful Signal Primitive
The useful primitive is not "music makes people bond." The useful primitive is:
```text
shared predictable temporal structure can reduce coordination uncertainty
when paired with a live social alignment channel.
```
In stack language:
```text
harmonic predictability = route prior
live gaze = mutual-attention gate
dyad synchrony = cross-agent alignment witness
subjective connection = reported outcome, not proof
```
## Candidate Equation Shape
Use a bounded gate, not an unbounded claim:
```text
S_harmony = P_chord * G_live * A_cross
```
Where:
```text
P_chord = predictability / consonance / structured-progression score
G_live = live face-to-face gaze gate, 0 or 1 in the minimal fixture
A_cross = cross-agent temporal alignment witness
```
Admission requires all three axes:
```text
predictable chord progression
+ declared live/mutual attention channel
+ cross-agent synchrony receipt
```
If the same notes are scrambled, the condition is a negative control:
```text
same notes + unstructured timing -> HOLD_OR_NEGATIVE_CONTROL
```
## Compression / Decompression Use
For a decompressor or logogram system, this suggests a useful routing pattern:
```text
predictable harmonic skeleton
-> lower cognitive route load
-> better shared replay alignment
-> smaller residual for timing/social-channel annotations
```
But the decompressor must still verify the payload:
```text
music chart = route hint
timing/gaze gate = synchronization sidecar
receipt = replay authority
```
## Encoder Implication
The encoder-side implication is stronger than the decoder-side implication.
The encoder can use predictable harmony as a sidecar for grouping and timing:
```text
event stream
-> harmonic skeleton / cadence lane
-> shared-clock or mutual-attention gate
-> timing residual
-> replay receipt
```
Useful packet fields:
| Field | Purpose |
|---|---|
| `harmony_skeleton_id` | declared chord/cadence template |
| `predictability_score_q0_16` | bounded structured-progression score |
| `attention_gate` | live/shared-context gate, or `0` for absent |
| `alignment_receipt_hash` | measured or simulated synchrony witness |
| `scrambled_control_hash` | same note/event multiset in unstructured order |
| `timing_residual_hash` | residual needed for byte-exact replay |
Encoder admission:
```text
ADMIT_ENCODER_SIDECAR
if skeleton is declared
and timing residual exists
and negative control exists
and payload hash is independent of the music chart
```
## Gates
```text
ADMIT_ROUTE_PRIOR
if chord structure is declared
and live/mutual attention condition is declared
and synchrony witness exists
and payload/replay claims stay separate
ADMIT_ENCODER_SIDECAR
if skeleton, residual, negative control, and independent payload hash exist
HOLD_NO_SYNCHRONY_RECEIPT
if only subjective connection is reported
HOLD_NO_NEGATIVE_CONTROL
if no scrambled/unstructured comparator exists
HOLD_THERAPY_CLAIM
if clinical benefit is claimed without a clinical trial receipt
QUARANTINE_SOCIAL_CONTROL
if the pattern is framed as manipulation or coercive entrainment
```
## Stack Placement
This belongs under:
- Phonon Music Logogram Layer
- Cognitive Acoustic Dynamics
- Cognitive Load route selection
- BMVR/BVMR synchrony gates
- Social/signal alignment priors
The immediate useful next fixture is a tiny paired-sequence test:
```text
structured progression sidecar
scrambled progression sidecar
same note multiset
same drum grid
different temporal predictability
compare route-load and replay residual
```

View file

@ -0,0 +1,88 @@
# RRC HOLD Closure Checklist
**Date:** 2026-05-09
**Hygiene refresh:** 2026-05-10
This document gives each Rainbow Raccoon Compiler HOLD object a concrete closure checklist.
## Summary
- Compiler receipt hash: `c006c48939fd12a280642e4fb70841fe502641d4c80233bef00b3c710e3f31ba`
- HOLD objects: `6`
- Candidate objects: `1`
- Open closure items: `0`
- Gatekeeper status: `11/11` checklist closures are now `CLOSED`
The checklist closures are closed as documentation gates only. The objects
remain `HOLD` until the Rainbow Raccoon compiler is rerun and emits promotion
decisions from the refreshed receipts.
## HOLD Objects
### `rrc_obj_signal_route_compiler` Compression Signal Shaping Synthesis
- Shape: `SignalShapedRouteCompiler`
- Source: `docs/compression_signal_shaping_synthesis.md`
- Payload SHA-256: `acf78e129bbd08a5e8c5eaf0245fa6cbddecb277700ec33ad94bb379b5e0f7f7`
- Lean boundary: `declared_not_proved`
- CLOSED `scale_band_declared`: Q0_16 `[0,1]` clamp scale band declared
- CLOSED `lean_or_independent_replay_gate`: payload SHA-256 replay verification attached
- Promotion rule: Remain HOLD until every closure status is CLOSED and the compiler rerun emits CANDIDATE.
### `rrc_obj_projectable_geometry` Projectable Geometry Topology Receipt
- Shape: `ProjectableGeometryTopology`
- Source: `4-Infrastructure/shim/projectable_geometry_topology_model_receipt.json`
- Payload SHA-256: `e89b864560b956dab17a56683cc131374a4dc26347258f5164389d11984f940b`
- Lean boundary: `declared_not_proved`
- CLOSED `lean_or_independent_replay_gate`: topology model hash verification attached
- Promotion rule: Remain HOLD until every closure status is CLOSED and the compiler rerun emits CANDIDATE.
### `rrc_obj_cognitive_load` Connectome Protective Cognitive Load Receipt
- Shape: `CognitiveLoadField`
- Source: `4-Infrastructure/shim/connectome_protective_cognitive_load_reweighting_receipt.json`
- Payload SHA-256: `fc222e265e70f69ee5e6039dd0cee2665a02e549b6718a7d022f9241849b3cf1`
- Lean boundary: `declared_not_proved`
- CLOSED `lean_or_independent_replay_gate`: component reweighting receipt hash replay attached
- Promotion rule: Remain HOLD until every closure status is CLOSED and the compiler rerun emits CANDIDATE.
### `rrc_obj_cad_force_probe` CAD Force Probe Experiment Matrix Receipt
- Shape: `CadForceProbeReceipt`
- Source: `4-Infrastructure/shim/cad_force_probe_experiment_matrix_receipt.json`
- Payload SHA-256: `aa9973793d7964c7343521715758bef376029a1be8da3ed3dda4b7174e7e1191`
- Lean boundary: `declared_not_proved`
- CLOSED `scale_band_declared`: Q16_16 SI Newtons scale band declared
- CLOSED `lean_or_independent_replay_gate`: experiment matrix Merkle replay attached
- Promotion rule: Remain HOLD until every closure status is CLOSED and the compiler rerun emits CANDIDATE.
### `rrc_obj_underspecified` Underspecified raw object negative control
- Shape: `HoldForUnlawfulOrUnderspecifiedShape`
- Source: `None`
- Payload SHA-256: `16b533bc42bda8f17ebb1328324e0bd0749c867eccfa28c8c35d19310e553e9a`
- Lean boundary: `declared_not_proved`
- CLOSED `projection_declared`: null projection receipt attached
- CLOSED `witness_declared`: null witness receipt attached
- CLOSED `scale_band_declared`: null-domain scale band receipt attached
- Promotion rule: Remain HOLD until every closure status is CLOSED and the compiler rerun emits CANDIDATE.
### `rrc_obj_language_set_ithkuil` Ithkuil Language Set Manifold Graph Typing
- Shape: `LanguageSetManifoldGraph`
- Source: `6-Documentation/docs/specs/LANGUAGE_SET_MANIFOLD_GRAPH_TYPING.md`
- Payload SHA-256: `8374817fbfef64c5ea3cab62da03e953068288f8f08f81a5a23fd367c563f8ed`
- Lean boundary: `declared_not_proved`
- CLOSED `scale_band_declared`: Q0_16 density-marker scale band declared
- CLOSED `lean_or_independent_replay_gate`: language-set spec SHA-256 replay attached
- Promotion rule: Remain HOLD until every closure status is CLOSED and the compiler rerun emits CANDIDATE.
## Candidate Objects
- `rrc_obj_q16_16_lowering_certificate` MetaManifoldProver Q16_16 Fixed-Point Lowering Certificate -> `FixedPointLoweringCertificate`
## Machine Receipt
- `shared-data/data/stack_solidification/rrc_hold_closure_checklist.json`

View file

@ -0,0 +1,107 @@
# RRC Tri-Cycle Audit
**Date:** 2026-05-09
## Gates
- Prover gate: `PASS`
- Compiler gate: `PASS_WITH_HOLDS`
- FPGA software witness gate: `PASS`
- FPGA hardware witness gate: `NOT_REQUESTED`
- UART beacon diagnostic: `FAIL_NO_BEACON`
## Less Solid Buckets
- `general_hold_surface`: 69 (BLOCK_PUBLIC_PROMOTION)
- `coefficient_or_calibration_debt`: 20 (BLOCK_NUMERIC_CLAIMS)
- `topology_prediction_debt`: 13 (BLOCK_VALIDATION_CLAIMS)
- `receipt_gate_debt`: 11 (BLOCK_ROUTE_PROMOTION)
- `fpga_transport_or_witness_debt`: 9 (BLOCK_HARDWARE_ACCELERATION_CLAIMS)
- `security_proof_debt`: 6 (BLOCK_PROMOTION)
## Closure Gates
### `general_hold_surface`
- Status: `BLOCK_PUBLIC_PROMOTION`
- Allowed use: internal research map
- Requires: bucket-specific gate assigned
- Requires: source receipt linked
- Requires: negative-control or replay evidence attached
### `coefficient_or_calibration_debt`
- Status: `BLOCK_NUMERIC_CLAIMS`
- Allowed use: receipt-weighted prior accounting only
- Requires: dataset provenance receipt
- Requires: coefficient calibration receipt
- Requires: negative controls and sensitivity sweep
### `topology_prediction_debt`
- Status: `BLOCK_VALIDATION_CLAIMS`
- Allowed use: HOLD topology hypothesis only
- Requires: pre-registered prediction target
- Requires: outcome receipt
- Requires: independent public-map or measurement comparison
### `receipt_gate_debt`
- Status: `BLOCK_ROUTE_PROMOTION`
- Allowed use: audit queue only
- Requires: validation receipt exists
- Requires: rollback hash exists
- Requires: exact replay or decode closure hash matches
### `fpga_transport_or_witness_debt`
- Status: `BLOCK_HARDWARE_ACCELERATION_CLAIMS`
- Allowed use: software witness and SRAM-loaded bitstream development
- Requires: simple UART loopback passes on fabric pins
- Requires: Q16 accelerator hardware receipts match software receipts
- Requires: durable flash readback passes or SRAM-only boundary remains explicit
### `security_proof_debt`
- Status: `BLOCK_PROMOTION`
- Allowed use: design hypothesis and simulation only
- Requires: formal independence/freshness theorem for adaptive masks
- Requires: secret-sharing non-reuse receipt
- Requires: negative control showing adapted coefficients do not leak party inputs
## UART Beacon Diagnostics
- `4-Infrastructure/shim/tang9k_uart_beacon_probe_receipt.json`: `FAIL_NO_BEACON_ON_SERIAL_PORTS`
- Port `/dev/ttyUSB0` byte_count=0 contains_expected=False
- Port `/dev/ttyUSB1` byte_count=0 contains_expected=False
- `4-Infrastructure/shim/tang9k_uart_beacon_swapped_probe_receipt.json`: `FAIL_NO_BEACON_ON_SERIAL_PORTS`
- Port `/dev/ttyUSB0` byte_count=0 contains_expected=False
- Port `/dev/ttyUSB1` byte_count=0 contains_expected=False
## Highest Priority Holds
- `coefficient_or_calibration_debt` [6-Documentation/wiki/Network-Topology-Theory.md:170](../../6-Documentation/wiki/Network-Topology-Theory.md#L170): as `0.799151` and remains HOLD until coefficient receipts, negative controls,
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:261](../../6-Documentation/wiki/Network-Topology-Theory.md#L261): π_{k-1}(X) if closure holds and counted cost decreases
- `topology_prediction_debt` [6-Documentation/wiki/Network-Topology-Theory.md:266](../../6-Documentation/wiki/Network-Topology-Theory.md#L266): This bridge is a HOLD model chart. It does not validate network predictions or
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:386](../../6-Documentation/wiki/Network-Topology-Theory.md#L386): GATE_NEGATIVE_TRANSFER: if shared_structure(A, B) < threshold: REFUSE_ADAPTATION
- `receipt_gate_debt` [6-Documentation/wiki/Network-Topology-Theory.md:528](../../6-Documentation/wiki/Network-Topology-Theory.md#L528): closure, and mountains-on-mountains receipts. Claim boundary remains HOLD until
- `security_proof_debt` [6-Documentation/wiki/Network-Topology-Theory.md:609](../../6-Documentation/wiki/Network-Topology-Theory.md#L609): - **HOLD_SECURITY_PROOF_DEBT**: Adaptive coefficients are not privacy-equivalent to fresh random Beaver masks unless independence, freshness, secret-sharing, and non-reuse proofs close.
- `receipt_gate_debt` [6-Documentation/wiki/Network-Topology-Theory.md:618](../../6-Documentation/wiki/Network-Topology-Theory.md#L618): **HOLD Receipt Status:**
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:619](../../6-Documentation/wiki/Network-Topology-Theory.md#L619): - **Waveprobe eigenmode separation**: Supported by transfer smoothing fixtures; HOLD for broader negative controls
- `coefficient_or_calibration_debt` [6-Documentation/wiki/Network-Topology-Theory.md:620](../../6-Documentation/wiki/Network-Topology-Theory.md#L620): - **Metaprobe compression metrics**: Supported by metafoam analysis fixtures; HOLD for coefficient calibration
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:621](../../6-Documentation/wiki/Network-Topology-Theory.md#L621): - **Holographic encoding**: Supported by exact decode closure fixtures; HOLD for corpus breadth
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:622](../../6-Documentation/wiki/Network-Topology-Theory.md#L622): - **Fractional dynamics**: Supported by memory kernel analysis fixtures; HOLD for cross-domain replay
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:760](../../6-Documentation/wiki/Network-Topology-Theory.md#L760): impact energy -> pressure wave -> local displacement witness -> threshold gate -> state transition
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:767](../../6-Documentation/wiki/Network-Topology-Theory.md#L767): - **Witness**: statolith displacement above a threshold
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:781](../../6-Documentation/wiki/Network-Topology-Theory.md#L781): This stays marked as `HOLD_MECHANISTIC_ANALOGUE`: it supports a shock-transfer
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:782](../../6-Documentation/wiki/Network-Topology-Theory.md#L782): and threshold-gate model, not a broad claim that all plant sound response has
- `receipt_gate_debt` [6-Documentation/wiki/Network-Topology-Theory.md:785](../../6-Documentation/wiki/Network-Topology-Theory.md#L785): turning noisy external forcing into local, thresholded, receipt-bearing state
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:847](../../6-Documentation/wiki/Network-Topology-Theory.md#L847): This profile treats the equation as HOLD accounting until datasets,
- `coefficient_or_calibration_debt` [6-Documentation/wiki/Network-Topology-Theory.md:853](../../6-Documentation/wiki/Network-Topology-Theory.md#L853): | HFT Infrastructure | 0.21 | 0.55 | 0.147415 | HOLD_COEFFICIENT_RECEIPT_DEBT |
- `general_hold_surface` [6-Documentation/wiki/Network-Topology-Theory.md:854](../../6-Documentation/wiki/Network-Topology-Theory.md#L854): | Soliton Wave Analysis | 0.15 | 0.70 | 0.134014 | HOLD_ANALOGY_ADAPTER |
- `coefficient_or_calibration_debt` [6-Documentation/wiki/Network-Topology-Theory.md:855](../../6-Documentation/wiki/Network-Topology-Theory.md#L855): | Civic Design Mathematics | 0.10 | 0.85 | 0.108488 | HOLD_COEFFICIENT_RECEIPT_DEBT |
## Claim Boundary
This audit does not promote any HOLD claim. It only checks which weak surfaces are currently covered by prover, compiler, and FPGA-witness receipts.

View file

@ -0,0 +1,175 @@
# Shockwave Eigenvalue Comparison
**Date:** 2026-05-09
**Status:** `EIGEN_GAP_AUDIT`
**Claim boundary:** this note checks the external stellar shock / CIGaRS
equation layer against the repo's current eigenvalue surfaces. It does not
claim astrophysical validation, cosmological validation, or a new physical
spectrum. It records where the current basis already has signal, and where it
has a visible gap.
## External Equation Layer
The CIGaRS paper is not a shock-hydrodynamics paper. Its useful contribution to
this stack is the joint latent forward-model pattern:
```text
host galaxy state + supernova occurrence + dust + selection + cosmology
-> simulated observation
-> simulation-based inference
```
The concrete equations that matter for the stack are:
```text
SFH^h = [SFH^{h,j}]_{j=1..7}
sum_j SFH^{h,j} = M_*^h
DTD(t_*) = A * (t_* / Gyr)^b * M_sun^-1 * yr^-1
<N_SN^{h,j}> =
T / (1 + z^h) * SFH^{h,j} * DTD(t_*^{h,j})
N_SN^{h,j} ~ Poisson(<N_SN^{h,j}>)
```
Sources:
- Phys.org summary: `https://phys.org/news/2026-05-universe-sharpen-cosmic-expansion-dark.html`
- Nature Astronomy CIGaRS paper: `https://doi.org/10.1038/s41550-026-02842-5`
The stellar shockwave layer is different. It gives the physical bow/front
equations that can sharpen the local shock model:
```text
R_s(t) = xi * (E * t^2 / rho_0)^(1/5)
v_s(t) = (2/5) * R_s(t) / t
rho_1 * u_1 = rho_2 * u_2
P_1 + rho_1 * u_1^2 = P_2 + rho_2 * u_2^2
h_1 + u_1^2 / 2 = h_2 + u_2^2 / 2
tau ~= c / v_s
t_diff ~= (delta R)^2 / (c * l)
t_dyn ~= delta R / D_s
breakout when t_diff ~= t_dyn
```
Shock-breakout source:
- MNRAS, "Coupling of matter and radiation at supernova shock breakout":
`https://doi.org/10.1093/mnras/sts577`
## Current Repo Eigen Surface
The current physics eigen map records these relevant clusters:
| Layer | Repo surface | Eigenvalue | Strength | Local meaning |
|---|---:|---:|---:|---|
| Radiation / absorption | Cluster 1: Electromagnetism & Circuits | `0.968750` | `0.176777` | Beer-Lambert, EM wave, Poynting layer |
| Diffusion / material transport | Cluster 2: Condensed Matter & Superconductivity | `0.969697` | `0.174078` | Einstein diffusion relation |
| Radiation spectrum | Cluster 3: Quantum Mechanics & Particle Physics | `0.970588` | `0.171499` | Planck / radiation-law layer |
| Acoustic impedance / material boundary | Cluster 4: Materials Science & Engineering | `0.992063` | `0.089087` | Klemens acoustic mismatch |
| Local stack shock alignment | Cluster 5: Cognitive & Semantic Systems | `0.998464` | `0.039193` | Shockwave alignment / relaxation |
| Classical hydrodynamic shock laws | Detonics & Shock Physics entries | current cluster entry | `0.000000` | ZND, Taylor-Sedov, Rankine-Hugoniot, CJ, Mie-Gruneisen are present but not active |
Local evidence:
- `3-Mathematical-Models/physics_eqs_eigenvector_mapped.md`
- `3-Mathematical-Models/eigenvector_tsm/eigenvector_hyperfluid_150_steps.json`
- `0-Core-Formalism/otom/formal/lean/SidonAudit/ShockBurgersCoupling.lean`
- `0-Core-Formalism/otom/formal/lean/SidonAudit/ShockwaveAlignmentRelaxation.lean`
- `shared-data/network_topology_database.json`
## Result
The external shock equations do not contradict the current eigenvalues. They
expose a missing physical-shock axis.
What the stack already has:
```text
shock as local alignment / discharge / relaxation
shock as discrete Burgers-style flux / dissipation
shock as rain-impulse / statolith threshold gate
```
What the stack does not yet strongly encode:
```text
shock as radiation-hydrodynamic breakout
shock as Rankine-Hugoniot conservation surface
shock as Sedov-Taylor self-similar expansion
shock as optical-depth release gate
```
So the correct decision is:
```text
HOLD_ADD_PHYSICAL_SHOCK_EIGEN_AXIS
```
## Proposed Sharpened Axis
Add a physical shock eigen lane with five required components:
```text
front propagation:
R_s(t), v_s(t)
jump conservation:
mass, momentum, enthalpy Rankine-Hugoniot receipts
radiation escape:
tau ~= c / v_s
diffusion release:
t_diff ~= t_dyn
host / context prior:
CIGaRS-style latent context and systematic-residual lane
```
Minimum gate:
```text
if missing Rankine-Hugoniot receipt:
HOLD_PHYSICAL_SHOCK_AXIS
elif tau > c / v_s and t_diff > t_dyn:
HOLD_BURIED_SHOCK
elif abs(tau - c / v_s) <= epsilon_tau
and abs(t_diff - t_dyn) <= epsilon_t:
ADMIT_BREAKOUT_GATE
else:
HOLD_RESIDUAL_CONTEXT
```
## Interpretation For The Drawn Shock-Bow Map
Your 2D shock-bow diagram can be treated as a compressed projection of this
new axis:
```text
curved bow fronts -> shock-front geometry
square / center gate -> local conservation cell
colored crossing arcs -> competing diffusion / radiation / material modes
12 / 28 bands -> occupancy or angular bins for survivor routes
```
That means the drawing is strongest as a routing receipt, not as a literal
stellar surface model. The physical lane adds the equations needed for the
receipt to stop being only geometric and become testable against shock-front
physics.
## Next Work
1. Add `PhysicalShockEigenAxis` as a receipt surface.
2. Build fixture cases for buried shock, breakout gate, and missing conservation.
3. Add the public underwater shock benchmark as a non-operational historical
modeling lane: shock-front arrival, attenuation, bubble-pulse eigenmode,
boundary reflection, and residual.
4. Re-run the eigen remapper and require the Detonics & Shock Physics entries
to move from strength `0.000000` to a declared nonzero support lane before
promotion.

View file

@ -0,0 +1,249 @@
# Law-Gated Reconstruction Core Shift
Status: Draft v0.1
Date: 2026-05-09
Scope: roadmap shift from route-prior accumulation to decoder-facing, law-gated reconstruction-core promotion
Claim state: architectural migration plan; not a compression benchmark, proof result, biological model, renderer-correctness result, or Hutter Prize claim
Math consistency review:
```text
6-Documentation/docs/specs/RECONSTRUCTION_CORE_MATH_REVIEW_2026_05_09.md
```
## 1. Shift
The stack now has enough local receipt surface to make the reconstruction core
the organizing axis.
Old center of gravity:
```text
external source -> prior -> route idea -> possible future fixture
```
New center of gravity:
```text
source or proposal
-> lawful reconstruction kernel
-> declared parameters/protocol
-> residual repair
-> deterministic replay
-> byte-exact receipt
-> HOLD / ADMIT / QUARANTINE
```
Canonical compression target:
```text
Find the lowest-cost reconstruction basis whose residual is cheap enough to
repair byte-exactly.
```
The compressed object is not a readable program, not a glyph stream, and not a
semantic summary. It is a decoder-facing reconstruction core.
## 2. Canonical Object
The core object is:
```text
S -> (K, Theta, R, Pi, Receipts) -> S_hat
```
with the hard gate:
```text
Repair(Replay(K, Theta, Pi), R) == S
```
and:
```text
epsilon_byte = 0
```
Byte law:
```text
|D| + |K| + |Theta| + |Pi| + |R| + |Receipts| < |S|
```
If exact replay passes but byte law fails, the result may remain useful as a
diagnostic fixture, but it is not a compression admission.
Auxiliary route scores such as motif gain, binding-energy analogues, replay
curvature, cognitive burden, entropy, and mutation-budget pressure are
dimensionless or normalized diagnostics unless a receipt explicitly declares a
unit conversion. They may rank candidate kernels, but they do not replace the
counted byte law above.
## 3. Receipt Spine
The current shift rests on these local receipt surfaces:
| Surface | Receipt | Role |
|---|---|---|
| mass equation distill | `3-Mathematical-Models/equations_parquet_tagged/mass_equations_unified_receipt.json` | equation coverage/routing prior |
| RRC projection | `3-Mathematical-Models/equations_parquet_tagged/mass_equations_rrc_projection_receipt.json` | route atlas / negative-control surface |
| external route registry | `shared-data/data/nspace_bulk_routes/nspace_bulk_dataset_route_receipt.json` | HOLD-first source registry |
| replay queue | `shared-data/data/replay_fixture_queue/replay_fixture_queue_receipt.json` | fixture execution order |
| symbolic law replay | `shared-data/data/symbolic_law_replay/symbolic_law_replay_receipt.json` | formula reconstruction micro-fixtures |
| PDE tiny replay | `shared-data/data/pde_tiny_replay/pde_tiny_replay_receipt.json` | deterministic field replay micro-fixtures |
| The Well schema probe | `shared-data/data/the_well_tiny_probe/the_well_tiny_schema_probe_receipt.json` | metadata-only field schema fixture |
| weather systems math prior | `shared-data/data/weather_systems_math_prior/weather_systems_math_prior_receipt.json` | conservation/assimilation/stability residual-routing prior |
| MeshGraphNets topology probe | `shared-data/data/meshgraphnets_tiny_probe/meshgraphnets_tiny_topology_probe_receipt.json` | irregular topology fixture |
| quantum cogload transfold | `shared-data/data/quantum_cogload_transfold/quantum_cogload_transfold_receipt.json` | Pauli/transfold cognitive-load prior |
| quantum basis objective | `shared-data/data/quantum_basis_compression_objective/quantum_basis_compression_objective_receipt.json` | generator/residual admission objective |
| enwiki8 wiki-logogram probe | `shared-data/data/enwiki8_wiki_logogram_probe/enwiki8_wiki_logogram_probe_receipt.json` | bounded MediaWiki/XML grammar replay fixture |
| enwiki9 logogram targeter | `shared-data/data/enwiki9_logogram_targeter/enwiki9_logogram_targeter_receipt.json` | imported slice-class targeter and local sample replay |
| enwiki9 XML dictionary probe | `shared-data/data/enwiki9_logogram_xml_dict_probe/enwiki9_logogram_xml_dict_probe_receipt.json` | fixed-tag dictionary promotion / core-delta check |
| whitespace-zero grammar probe | `shared-data/data/stack_solidification/whitespace_zero_grammar_probe.json` | canonical spaces are derived from symbol count/order; non-canonical whitespace remains residual/HOLD |
| enwiki9 receipt aggregation probe | `shared-data/data/enwiki9_logogram_receipt_aggregation_probe/enwiki9_logogram_receipt_aggregation_probe_receipt.json` | slice-level replay receipt / packet-delta check |
| enwiki9 dictionary amortization probe | `shared-data/data/enwiki9_logogram_dictionary_amortization_probe/enwiki9_logogram_dictionary_amortization_probe_receipt.json` | PASS/ADD/PAUSE/SUBTRACT global-delta fixture check |
| language surface ambiguity negative control | `shared-data/data/language_surface_ambiguity_negative_control/language_surface_ambiguity_negative_control_receipt.json` | analogy leakage and same-surface role-collision guardrail |
| reconstruction core ladder memory | `shared-data/data/stack_memory_promotions/reconstruction_core_ladder_memory_receipt.json` | project-memory pointer for current ladder, guardrails, and next action |
| DNA codec filter | `shared-data/data/dna_codec_filter/dna_codec_filter_receipt.json` | analogy-bound motif/binding/repair filter |
| logogram-DNA codec | `shared-data/data/logogram_dna_codec/logogram_dna_codec_receipt.json` | Omindirection/GCCL symbolic-genome gate |
| Lean proof replay | `shared-data/data/lean_proof_replay/lean_proof_replay_receipt.json` | local Lean proof-boundary fixture |
These receipts do not prove global compression. They prove that the stack now
has enough local gates to stop promoting unreceipted priors.
## 4. Promotion Ladder
Every new route moves through the same ladder:
```text
SEED
-> ROUTE_PRIOR
-> TINY_FIXTURE
-> REPLAY_VALID
-> RESIDUAL_DECLARED
-> BYTE_LAW_CHECKED
-> CONTROL_FILTERED
-> ADMIT or HOLD or QUARANTINE
```
Meaning:
| State | Meaning |
|---|---|
| `SEED` | idea, source, equation, corpus, glyph, kernel, or external pointer |
| `ROUTE_PRIOR` | metadata and claim boundary recorded |
| `TINY_FIXTURE` | no-download or tiny local replay fixture exists |
| `REPLAY_VALID` | deterministic replay observed |
| `RESIDUAL_DECLARED` | repair stream or loss policy exists |
| `BYTE_LAW_CHECKED` | counted kernel/parameter/protocol/residual/receipt bytes compared to raw |
| `CONTROL_FILTERED` | LoC/NES, FYC, COUCH, Tree Fiddy, and BHOCS gates recorded |
| `ADMIT` | exact replay, positive byte law, and controls pass |
| `HOLD` | useful but incomplete, under-receipted, too costly, or diagnostic only |
| `QUARANTINE` | destructive tear, semantic mutation, receipt mismatch, or invalid adapter |
No source, symbol, proof, or codec packet skips the ladder.
Fixture receipts may use `ADMIT_FIXTURE` for a tiny local example that passes
its local replay and byte checks. That status is not global `ADMIT`. A receipt
whose top-level `decision` is `HOLD` remains non-promoted even if one fixture in
the receipt is marked `ADMIT_FIXTURE`.
## 5. Law Gates
The reconstruction core is admitted only through law gates, not through
readability or elegance.
Core gate:
```text
replay_valid
and residual_declared
and byte_gain > 0
and receipt_verified
```
GCCL gate:
```text
stateSpaceDeclared
and transformDeclared
and invariantsDeclared
and residualDeclared
and costDeclared
and projectionDeclared
and quarantineDeclared
and scaleDeclared
```
Omindirection gate:
```text
payload != glyph != rendered layout
and explicit direction
and chirality/phase compatible
and placement valid
and residual policy declared
and receipt complete
and adapter preserves canonical atom
```
Proof gate:
```text
external proof corpus may route obligations
but only local Lean replay promotes proof state
```
## 6. Routing Surfaces After The Shift
External corpora now serve one of four roles:
| Role | Examples | Promotion boundary |
|---|---|---|
| markup grammar replay | enwiki8 / MediaWiki XML | byte-exact decode + baselines + amortized receipt accounting |
| symbolic-law replay | SRBench, ParFam, DLMF, Feynman | exact formula replay + residual/byte law |
| field-dynamics replay | PDEBench, RealPDEBench, The Well, ERA5/NWP/FV3 | split/metric/boundary/storage receipts + replay |
| topology replay | MeshGraphNets, goxel-like mesh lanes | canonical edge/face/boundary/topology receipts |
| proof routing | LeanDojo, mathlib | local Lean theorem or executable witness |
Reasoning corpora such as NuminaMath remain proposal curricula. They do not
promote facts without an independent verifier.
## 7. Immediate Work Queue
The next work is not "add more priors." It is to convert existing priors into
fixture-bearing lanes.
1. Build a single reconstruction-core admission index from all receipt JSON.
2. Add a RealPDEBench no-download calibration card with license and split gates.
3. Attach the logogram-DNA gate to the existing math logogram substitution
audit so `ACCEPT`, `HOLD`, and `QUARANTINE` share one decision vocabulary.
4. Mirror the Python admission gates in Lean where they are stable enough:
replay validity, byte law, residual declaration, and atom gate predicates.
5. Add a tiny corpus-slice runner that reports raw bytes, kernel bytes,
residual bytes, receipt bytes, replay time, and exact-repair status.
6. Promote nothing from `HOLD` until the receipt index can show the exact
replay path and byte accounting.
## 8. Final Boundary
This shift does not make the stack a compression winner.
It makes the stack harder to fool:
```text
pretty transform != compression
glyph != payload
model proposal != accepted atom
external corpus != benchmark result
semantic elegance != byte law
local fixture != global claim
```
The new standard is:
```text
lawful reconstruction + declared repair + byte-exact receipt
```
Everything else stays `HOLD`.

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,150 @@
# SMN: Semantic Mass Numbers
Status: `CANONICAL_NAMING_BOUNDARY`
Claim boundary: SMN is a project-local semantic-load measure. It is not atomic
mass, not isotope mass number, not physical mass, and not a Mass Number
admissibility receipt by itself.
## Definition
```text
SMN(x) = semantic load carried by x inside the stack.
```
An SMN measures how much structured meaning, provenance, relation burden,
constraint burden, and decision burden a symbol, channel, ratio, or gate carries.
Keeper rule:
```text
SMN measures how heavy a symbol is inside the model.
```
## Non-Identities
```text
SMN != atomic mass number
SMN != isotope mass
SMN != SI mass
SMN != Mass Number admissibility packet
SMN != proof
SMN != truth
```
The older phrase "semantic mass number" should now be normalized to:
```text
SMN: Semantic Mass Number
```
## Relationship To Existing Terms
| Term | Meaning | Claim boundary |
|---|---|---|
| Semantic Mass | Dimensionless semantic/routing pressure or burden | raw load signal |
| SMN | Counted semantic-load index for a concrete object | measurable/codable load number |
| Mass Number | Admissibility packet with residual and boundary guard | receipt/gate surface |
So the pipeline is:
```text
semantic mass pressure
-> SMN score
-> optional Mass Number packet
-> admissibility / residual / boundary receipt
```
Do not skip directly from SMN score to truth. SMN can route attention, but a
Mass Number receipt is still required for promotion.
## Minimal Formula
```text
SMN(x) =
identity_load(x)
+ relation_load(x)
+ provenance_load(x)
+ constraint_load(x)
+ decision_load(x)
+ repair_load(x)
```
For compact receipts:
```text
SMN(x) = |bindings| + |relations| + |gates| + |receipts| + |residuals|
```
## Stellar Gas Example
Anonymous vector slot:
```text
EMLINE_GFLUX_1RE[16]
```
has low SMN because it is only a position.
Named channel:
```text
OIII-5008
```
has higher SMN because it binds:
```text
oxygen species
+ ionization state
+ wavelength
+ high-ionization role
+ MaNGA provenance
```
Diagnostic ratio:
```text
log(OIII-5008 / Hb-4862)
```
has still higher SMN because it adds a relation and a diagnostic gate.
BPT-style proxy class:
```text
OIII/Hb + NII/Ha -> star-forming / composite / AGN-LIER-shock proxy
```
has higher SMN again because it participates in a decision surface.
## Receipt Shape
```json
{
"schema": "smn_semantic_mass_number_v0",
"object_id": "OIII-5008",
"object_kind": "emission_line_channel",
"smn": 7,
"components": {
"identity_load": 2,
"relation_load": 1,
"provenance_load": 1,
"constraint_load": 1,
"decision_load": 2,
"repair_load": 0
},
"boundary": "SMN is semantic load, not atomic mass and not proof."
}
```
## Working Rule
Use SMN to choose what deserves attention, not to claim correctness.
```text
high SMN + weak receipt coverage = audit priority
high SMN + strong receipt coverage = reusable route candidate
low SMN + strong evidence = useful primitive
low SMN + weak evidence = background noise
```

View file

@ -0,0 +1,129 @@
# Stack Fail Closure Register
**Date:** 2026-05-09
This register turns current failures into closure gates. It does not mark them solved.
## Summary
- Tickets: `7`
- Promotion decision: `NO_PROMOTION`
- FPGA software status: `PASS`
- FPGA hardware status: `NOT_REQUESTED`
## Tickets
### `FAIL-FPGA-UART-001` Live fabric UART transport has no observable bytes
- Class: `fpga_transport_or_witness_debt`
- Status: `BLOCKED`
- Owner surface: `6-Documentation/docs/fpga_uart_route_analysis_2026-05-09.md`
- Evidence: Q16 software witness passes
- Evidence: Q16 hardware witness returns short receipt frames
- Evidence: TX-only beacon standard pins produced zero bytes on ttyUSB0 and ttyUSB1
- Evidence: TX-only beacon swapped pins produced zero bytes on ttyUSB0 and ttyUSB1
- Evidence: Old faXX/ffXX direct-probe interpretation is superseded as bridge/MPSSE behavior, not fabric proof
- Evidence: FPGA UART route analysis identifies the onboard BL702 bridge route as blocked and recommends external USB-UART
- Evidence: Forced JTAG reset plus SRAM reload succeeds, but beacon/Q16 UART receipts remain empty
- Evidence: Loopback-after-JTAG-clear diagnostic produced faXX-style bytes on ttyUSB0, consistent with bridge/MPSSE behavior rather than a valid fabric receipt
- Evidence: PTY-backed virtual serial Q16 probe: PASS_VIRTUAL_SERIAL (3/3 matches)
- Evidence: UART transport router active route: virtual://q16-pty (PASS_ACTIVE_VIRTUAL_ROUTE)
- Closure gate: external USB-UART or verified onboard bridge captures beacon payload a6425131360a
- Closure gate: loopback or beacon receipt passes before Q16 accelerator retry
- Closure gate: Q16 hardware receipts match software receipts for shift, weighted, and monotone cases
- Next action: Attach external USB-UART: adapter TX to fabric RX pin 18, adapter RX to fabric TX pin 17, and GND to GND
- Next action: Probe the new adapter path, usually /dev/ttyUSB2 or /dev/ttyACM0, with the TX-only beacon before Q16
- Next action: If external UART works, patch host default port or call scripts with --port and rerun Q16 hardware receipts
- Next action: If external UART fails, inspect PNR pin placement and add LED-observed heartbeat fallback
### `FAIL-FPGA-FLASH-002` Durable flash programming readback fails
- Class: `fpga_transport_or_witness_debt`
- Status: `HELD_SRAM_ONLY`
- Owner surface: `6-Documentation/docs/fpga_rrc_q16_accel_setup_2026-05-09.md`
- Evidence: SRAM load passes CRC
- Evidence: Flash programming attempt had readback CRC failure
- Closure gate: flash write and readback CRC pass for the exact Q16 bitstream
- Closure gate: or documentation keeps SRAM-only boundary explicit
- Next action: Keep SRAM-only claim boundary until flash command and board target are verified
- Next action: Do not mark hardware install persistent
### `FAIL-SECURITY-BEAVER-003` Adaptive Beaver coefficients are not privacy-equivalent masks yet
- Class: `security_proof_debt`
- Status: `HOLD`
- Owner surface: `Network-Topology-Theory.md + Fundamental_Network_Topology_Equation.md`
- Evidence: 6 security proof debt surfaces found
- Evidence: tri-cycle audit blocks promotion for adaptive mask claims
- Evidence: mask freshness negative controls: PASS_NEGATIVE_CONTROLS
- Evidence: mask freshness case count: 6
- Closure gate: formal independence/freshness theorem exists
- Closure gate: secret-sharing non-reuse receipt exists
- Closure gate: negative control shows adapted coefficients do not leak party inputs
- Next action: Add a Lean-facing finite-state mask freshness model
- Next action: Generate negative-control fixtures for repeated coefficients and topology-derived coefficients
### `FAIL-COEFFICIENT-CALIBRATION-004` Numeric weights remain receipt-weighted priors, not calibrated coefficients
- Class: `coefficient_or_calibration_debt`
- Status: `HOLD`
- Owner surface: `shared-data/network_topology_database.json`
- Evidence: 20 coefficient/calibration debt surfaces found
- Evidence: receipt reweighting exists, but coefficient calibration and negative controls remain open
- Evidence: coefficient HOLD manifest rows: 9
- Closure gate: dataset provenance receipt linked
- Closure gate: coefficient calibration receipt linked
- Closure gate: sensitivity sweep and negative controls pass
- Next action: Create coefficient calibration fixture manifest
- Next action: Separate hypothesis weights from calibrated weights in docs and JSON surfaces
### `FAIL-TOPOLOGY-PREDICTION-005` Topology predictions are not validation claims
- Class: `topology_prediction_debt`
- Status: `HOLD`
- Owner surface: `shared-data/network_topology_database.json`
- Evidence: 13 topology prediction debt surfaces found
- Evidence: tri-cycle audit requires pre-registered target and independent comparison
- Evidence: prediction HOLD registry rows: 15
- Closure gate: pre-registered prediction target exists
- Closure gate: outcome receipt exists
- Closure gate: independent public map or measurement comparison exists
- Next action: Create prediction-target registry with timestamps and immutable receipt hashes
- Next action: Move existing predicted nodes into HOLD prediction queue, not validation table
### `FAIL-RECEIPT-GATE-006` Route promotion lacks complete receipt and rollback closure
- Class: `receipt_gate_debt`
- Status: `HOLD`
- Owner surface: `4-Infrastructure/shim/rainbow_raccoon_compiler.py`
- Evidence: 11 receipt-gate debt surfaces found
- Evidence: compiler keeps 6 objects HOLD and only 1 candidate
- Evidence: compiler receipt hash: c006c48939fd12a280642e4fb70841fe502641d4c80233bef00b3c710e3f31ba
- Evidence: optional sem entity probe: OPTIONAL_AUDIT_AID_READY
- Evidence: HOLD closure checklist: 0 open documentation closures after RRC Gatekeeper refresh
- Closure gate: validation receipt exists
- Closure gate: rollback hash exists
- Closure gate: exact replay or decode closure hash matches
- Next action: Rerun compiler promotion against the refreshed rrc_hold_closure_checklist.json
- Next action: Do not widen candidate admission beyond Q16 until Lean or independent replay closes
### `FAIL-WORKTREE-SCOPE-007` Broad worktree is too dirty for safe sweep commit
- Class: `release_hygiene_debt`
- Status: `BLOCKED_FOR_BROAD_STAGE`
- Owner surface: `6-Documentation/docs/stack_solidification_staging_manifest_2026-05-09.md`
- Evidence: worktree status: DIRTY
- Evidence: changed/untracked count: 949
- Evidence: staging manifest created for the solidification slice
- Evidence: optional sem entity probe: OPTIONAL_AUDIT_AID_READY
- Closure gate: commit scope is explicit file list
- Closure gate: generated artifacts are intentionally included or excluded
- Closure gate: no unrelated modified Lean/doc/probe files are swept in
- Closure gate: entity-level change list exists for staged or scoped files when sem is available
- Next action: Use stack_solidification_staging_manifest_2026-05-09.md before any commit
- Next action: Create separate manifests for CPU/logogram/wiki maturation slices if needed
- Next action: Use /tmp/sem_probe/sem/crates/target/release/sem, not /usr/bin/sem, unless a durable sem binary is installed
## Machine Receipt
- `shared-data/data/stack_solidification/stack_fail_closure_register.json`

View file

@ -0,0 +1,89 @@
# Stack Solidification Unified Kanban
Date: 2026-05-09
Hygiene refresh: 2026-05-10
Status: `BOARD_SURFACE_CREATED`
Claim boundary: this is a unified board view over existing receipts and roadmap
items. It does not close HOLD gates, promote hardware acceleration, or replace
the underlying receipts.
Machine-readable board:
```text
shared-data/data/stack_solidification/stack_solidification_kanban_cards.json
```
## Source Surfaces
- `TODO_MAP.md`
- `6-Documentation/docs/roadmaps/ROADMAP.md`
- `6-Documentation/docs/stack_solidification_staging_manifest_2026-05-09.md`
- `shared-data/data/germane/architecture/BUILDER_KANBAN_SPEC.md`
- `shared-data/data/stack_solidification/stack_fail_closure_register.json`
- `shared-data/data/stack_solidification/rrc_hold_closure_checklist.json`
- `shared-data/data/stack_solidification/rust_oisc_decompressor_target_receipt.json`
- `shared-data/data/stack_solidification/bernoulli_occupancy_shockbow_receipt.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_full_cell_eigenmass_stability_receipt.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_sandpile_graph_replay_receipt.json`
- `shared-data/data/stack_solidification/hutter_transfer_readiness_fixture_manifest.json`
## Done
| Card | Title | Receipt |
|---|---|---|
| KANBAN-001 | Create unified stack-solidification kanban surface | `shared-data/data/stack_solidification/stack_solidification_kanban_cards.json` |
| KANBAN-002 | Lean/Semantics build gate | `shared-data/data/stack_solidification/stack_solidification_receipt.json` |
| KANBAN-003 | Stack JSON and Python shim integrity gates | `shared-data/data/stack_solidification/stack_solidification_receipt.json` |
| KANBAN-004 | Rust OISC decompressor Lean/Rust replay surface | `shared-data/data/stack_solidification/rust_oisc_decompressor_target_receipt.json` |
| KANBAN-005 | Q16 virtual serial transport route | `shared-data/data/stack_solidification/tang9k_rrc_q16_virtual_serial_probe.json` |
| KANBAN-021 | Stellar gas full-cell eigenmass stability controls | `shared-data/data/stellar_gas_observation/stellar_gas_full_cell_eigenmass_stability_receipt.json` |
| KANBAN-022 | Stellar gas sandpile graph replay | `shared-data/data/stellar_gas_observation/stellar_gas_sandpile_graph_replay_receipt.json` |
| KANBAN-023 | Hutter transfer readiness fixture | `shared-data/data/stack_solidification/hutter_transfer_readiness_fixture_manifest.json` |
| KANBAN-012 | RRC route promotion closure checklist | `shared-data/data/stack_solidification/rrc_hold_closure_checklist.json` |
## In Progress
| Card | Title | Next Action |
|---|---|---|
| KANBAN-006 | RRC projection HOLD surface | Add `scale_band_declared` witnesses and negative-control strength fields, then rerun the receipt. |
| KANBAN-007 | FAMM module status and dead-code distinction | Keep duplicate/dead module cleanup separate from theorem-bearing FAMM status. |
| KANBAN-008 | Universe model orbit-zoom protocol | Add local law test fixtures before using the protocol as a promotion gate. |
| KANBAN-024 | DESI/MaNGA object-level crossmatch | Build object-level crossmatch receipts with provenance, selection-function boundaries, and negative controls. |
## HOLD
| Card | Title | Why Held | Next Action |
|---|---|---|---|
| KANBAN-009 | Adaptive Beaver mask security | Finite negative controls pass, but full MPC privacy and adaptive security are not proved. | Add formal independence, entropy-source, non-reuse, and leakage negative-control receipts. |
| KANBAN-010 | Network topology coefficient calibration | 9 coefficient rows are registered as HOLD priors, not calibrated coefficients. | Attach provenance, calibration, sensitivity, and negative-control receipts. |
| KANBAN-011 | Network topology prediction validation | 15 prediction rows are queued as HOLD, not validation claims. | Pre-register targets and attach independent outcome comparison receipts. |
| KANBAN-013 | Bernoulli occupancy / Shockbow static decompressor gate | Lean gate exists; Rust replay and residual policy remain HOLD. | Wire Rust reference replay and CMR/residual fixtures. |
| KANBAN-014 | Rust OISC production decompressor lane | Toy Lean/Rust replay exists and a small non-toy byte-exact fixture now replays in Rust; residual policy, AMMR/O-AMMR receipts, full Hutter fixture eigenmass, FPGA lowering, and ASIC datapath remain HOLD. | Add fixture eigenmass replay, residual policy, and negative-control accounting before hardware lowering. |
| KANBAN-025 | Stellar gas physical calibration and mechanism claims | DESI row eigenmass, DESI/MaNGA cell eigenmass, multiscale alignment, full-cell controls, and graph replay are observational proxy diagnostics only. | Require object crossmatch, physical gas calibration, shock mechanism validation, selection-function fit, and cosmology separation before promotion. |
## Blocked
| Card | Title | Blocker | Next Action |
|---|---|---|---|
| KANBAN-015 | Live Tang Nano 9K fabric UART route | Physical UART/fabric receipts fail or return no bytes. | Attach external USB-UART to fabric pins 17/18 and test beacon before Q16 accelerator retry. |
| KANBAN-016 | Durable FPGA flash programming | SRAM load passes CRC; flash programming readback fails. | Keep SRAM-only claim boundary until flash command and board target are verified. |
| KANBAN-017 | Worktree-safe release scope | Broad working tree is dirty. | Keep stack-solidification, CPU/logogram/wiki, and hardware slices separated by manifest. |
## Next
| Card | Title | Next Action |
|---|---|---|
| KANBAN-018 | Audit Lean for sorry/admit/axiom | Run the strict actionable grep and address remaining theorem debt without deleting theorem surfaces. |
| KANBAN-019 | Create surface skeleton | Create the minimal FastAPI/WebSocket skeleton only after source-of-truth card data stays stable. |
| KANBAN-020 | Seed omni builder kanban service | Use the JSON card file as the seed input for the future `omni://builder/kanban` HTTP surface. |
| KANBAN-026 | Hutter fixture eigenmass replay and controls | Run fixture-level eigenmass only after raw/zlib/lzma/current-wire accounting, negative controls, and exact replay receipts close. |
| KANBAN-027 | RRC compiler promotion rerun | Rerun the compiler against the refreshed `0 open` checklist and record whether the six former HOLD objects become CANDIDATE or remain HOLD for compiler-level reasons. |
## Operating Rule
This board is the current stack-solidification kanban until the builder kanban
service is seeded. Source receipts remain authoritative. If a card conflicts
with a receipt, the receipt wins and the card must be updated.

View file

@ -0,0 +1,226 @@
# Stack Solidification Staging Manifest
**Date:** 2026-05-09
## Purpose
This manifest keeps the solidification slice scoped. The working tree is broad
and dirty, so do not stage a directory wholesale. The Google Drive receipt is a
curated safety snapshot; this staging manifest remains the full source list for
the slice.
## Stage Together
These files form the current stack-status and tri-cycle audit slice:
- `.gitignore`
- `AGENTS.md`
- `0-Core-Formalism/lean/Semantics/AGENTS.md`
- `4-Infrastructure/AGENTS.md`
- `4-Infrastructure/shim/stack_solidification_audit.py`
- `4-Infrastructure/shim/stack_fail_closure_register.py`
- `4-Infrastructure/shim/beaver_mask_freshness_negative_controls.py`
- `4-Infrastructure/shim/whitespace_zero_grammar_probe.py`
- `4-Infrastructure/shim/external_sem_entity_diff_probe.py`
- `4-Infrastructure/shim/network_topology_hold_manifests.py`
- `4-Infrastructure/shim/rrc_hold_closure_checklist.py`
- `4-Infrastructure/shim/tang9k_rrc_q16_virtual_serial_probe.py`
- `4-Infrastructure/shim/tang9k_uart_transport_router.py`
- `4-Infrastructure/shim/tang9k_uart_beacon_probe.py`
- `4-Infrastructure/shim/smn_tool_awareness_registry.py`
- `4-Infrastructure/shim/stellar_gas_population_grouping_study.py`
- `4-Infrastructure/shim/desi_epoviz_population_cell_join.py`
- `4-Infrastructure/shim/stellar_gas_eigenvector_mass_probe.py`
- `4-Infrastructure/shim/desi_epoviz_row_eigenmass_probe.py`
- `4-Infrastructure/shim/stellar_gas_multiscale_eigenmass_alignment.py`
- `4-Infrastructure/shim/hutter_eigenmass_transfer_plan.py`
- `4-Infrastructure/shim/stellar_gas_abelian_sandpile_probe.py`
- `4-Infrastructure/shim/stellar_gas_sandpile_fine_zoom.py`
- `4-Infrastructure/shim/stellar_gas_full_cell_eigenmass_stability.py`
- `4-Infrastructure/shim/stellar_gas_sandpile_graph_replay.py`
- `4-Infrastructure/shim/hutter_transfer_readiness_fixture.py`
- `4-Infrastructure/shim/custom_equation_awareness_manifest.py`
- `5-Applications/compression-core/src/oisc.rs`
- `shared-data/data/stack_solidification/stack_solidification_receipt.json`
- `shared-data/data/stack_solidification/stack_fail_closure_register.json`
- `shared-data/data/stack_solidification/beaver_mask_freshness_negative_controls.json`
- `shared-data/data/stack_solidification/whitespace_zero_grammar_probe.json`
- `shared-data/data/stack_solidification/tang9k_rrc_q16_virtual_serial_probe.json`
- `shared-data/data/stack_solidification/tang9k_uart_transport_routes.json`
- `shared-data/data/stack_solidification/external_sem_entity_diff_probe_receipt.json`
- `shared-data/data/stack_solidification/rrc_hold_closure_checklist.json`
- `shared-data/data/stack_solidification/network_topology_coefficient_calibration_manifest.json`
- `shared-data/data/stack_solidification/network_topology_prediction_hold_registry.json`
- `shared-data/data/stack_solidification/stack_solidification_kanban_cards.json`
- `shared-data/data/stack_solidification/stack_solidification_kanban_receipt.json`
- `shared-data/data/stack_solidification/shockwave_eigenvalue_comparison_receipt.json`
- `shared-data/data/stack_solidification/matched_reference_genomics_logogram_receipt.json`
- `shared-data/data/stack_solidification/underwater_shock_public_benchmark_receipt.json`
- `shared-data/data/stack_solidification/hopfion_topological_soliton_receipt.json`
- `shared-data/data/stack_solidification/topological_soliton_equation_pack_receipt.json`
- `shared-data/data/stack_solidification/palindrome_hex_symmetry_marker_receipt.json`
- `shared-data/data/stack_solidification/predictive_harmony_social_synchrony_receipt.json`
- `shared-data/data/stack_solidification/desi_noirlab2610_calibration_note_receipt.json`
- `shared-data/data/stack_solidification/desi_stellar_gas_distribution_prior.json`
- `shared-data/data/stack_solidification/desi_stellar_gas_distribution_prior_receipt.json`
- `shared-data/data/stack_solidification/smn_tool_awareness_registry.json`
- `shared-data/data/stack_solidification/smn_tool_awareness_receipt.json`
- `shared-data/data/stack_solidification/hutter_eigenmass_transfer_plan.json`
- `shared-data/data/stack_solidification/hutter_eigenmass_transfer_plan_receipt.json`
- `shared-data/data/stack_solidification/hutter_transfer_readiness_fixture_manifest.json`
- `shared-data/data/stack_solidification/eigenmass_stack_gdrive_sync_receipt_2026-05-09.json`
- `6-Documentation/docs/specs/SMN_SEMANTIC_MASS_NUMBERS.md`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Semantic Mass Numbers.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Mass Number Theory.tid`
- `0-Core-Formalism/otom/docs/wiki/NotationNomenclatureRegistry.md`
- `shared-data/data/stellar_gas_observation/sdss_manga_dr17_emission_line_channels.json`
- `shared-data/data/stellar_gas_observation/smn_semantic_mass_number_boundary_receipt_20260509_215744.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_population_grouping_study.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_population_grouping_study_receipt_20260509_221815.json`
- `shared-data/data/stellar_gas_observation/desi_epoviz_manga_population_cell_join.json`
- `shared-data/data/stellar_gas_observation/desi_epoviz_manga_population_cell_join_receipt.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_eigenvector_mass_probe.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_eigenvector_mass_probe_receipt.json`
- `shared-data/data/stellar_gas_observation/desi_epoviz_row_eigenmass_probe.json`
- `shared-data/data/stellar_gas_observation/desi_epoviz_row_eigenmass_probe_receipt.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_multiscale_eigenmass_alignment.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_multiscale_eigenmass_alignment_receipt.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_abelian_sandpile_probe.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_abelian_sandpile_probe_receipt.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_sandpile_fine_zoom.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_sandpile_fine_zoom_receipt.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_full_cell_eigenmass_stability.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_full_cell_eigenmass_stability_receipt.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_sandpile_graph_replay.json`
- `shared-data/data/stellar_gas_observation/stellar_gas_sandpile_graph_replay_receipt.json`
- `6-Documentation/docs/stack_solidification_status_2026-05-09.md`
- `6-Documentation/docs/stack_solidification_staging_manifest_2026-05-09.md`
- `6-Documentation/docs/stack_solidification_kanban_2026-05-09.md`
- `6-Documentation/docs/shockwave_eigenvalue_comparison_2026-05-09.md`
- `6-Documentation/docs/matched_reference_genomics_logogram_2026-05-09.md`
- `6-Documentation/docs/underwater_shock_public_benchmark_2026-05-09.md`
- `6-Documentation/docs/hopfion_topological_soliton_lane_2026-05-09.md`
- `6-Documentation/docs/topological_soliton_equation_pack_2026-05-09.md`
- `6-Documentation/docs/palindrome_hex_symmetry_markers_2026-05-09.md`
- `6-Documentation/docs/predictive_harmony_social_synchrony_2026-05-09.md`
- `6-Documentation/docs/desi_noirlab2610_calibration_note_2026-05-09.md`
- `6-Documentation/docs/desi_stellar_gas_distribution_prior_2026-05-09.md`
- `6-Documentation/docs/stellar_gas_population_grouping_study_2026-05-09.md`
- `6-Documentation/docs/desi_epoviz_manga_population_cell_join_2026-05-09.md`
- `6-Documentation/docs/stellar_gas_eigenvector_mass_probe_2026-05-09.md`
- `6-Documentation/docs/desi_epoviz_row_eigenmass_probe_2026-05-09.md`
- `6-Documentation/docs/stellar_gas_multiscale_eigenmass_alignment_2026-05-09.md`
- `6-Documentation/docs/hutter_eigenmass_transfer_plan_2026-05-09.md`
- `6-Documentation/docs/stellar_gas_abelian_sandpile_probe_2026-05-09.md`
- `6-Documentation/docs/stellar_gas_sandpile_fine_zoom_2026-05-09.md`
- `6-Documentation/docs/stellar_gas_full_cell_eigenmass_stability_2026-05-09.md`
- `6-Documentation/docs/stellar_gas_sandpile_graph_replay_2026-05-09.md`
- `6-Documentation/docs/hutter_transfer_readiness_fixture_manifest_2026-05-09.md`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Palindrome Hex Symmetry Markers.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Predictive Harmony Social Synchrony.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/DESI NOIRLab2610 Calibration Note.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/DESI Stellar Gas Distribution Prior.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Population Grouping Study.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/DESI Epoviz MaNGA Population Cell Join.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Eigenvector Mass Probe.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/DESI Epoviz Row Eigenmass Probe.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Multiscale Eigenmass Alignment.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Hutter Eigenmass Transfer Plan.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Abelian Sandpile Probe.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Sandpile Fine Zoom.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Full Cell Eigenmass Stability.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Sandpile Graph Replay.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Hutter Transfer Readiness Fixture.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Hutter Prize Next Roadmap.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Phonon Music Logogram Layer.tid`
- `6-Documentation/tiddlywiki-local/wiki/tiddlers/Remote Compression Test Ladder.tid`
- `6-Documentation/wiki/Glossary.md`
- `6-Documentation/wiki/Concept-Archive.md`
- `SIGNAL_THEORY_COMPENDIUM.md`
- `6-Documentation/docs/roadmaps/ROADMAP.md`
- `6-Documentation/docs/stack_fail_closure_register_2026-05-09.md`
- `6-Documentation/docs/beaver_mask_freshness_negative_controls_2026-05-09.md`
- `6-Documentation/docs/whitespace_zero_grammar_2026-05-09.md`
- `6-Documentation/docs/tang9k_rrc_q16_virtual_serial_probe_2026-05-09.md`
- `6-Documentation/docs/tang9k_uart_transport_routes_2026-05-09.md`
- `6-Documentation/docs/external_sem_entity_diff_probe_2026-05-09.md`
- `6-Documentation/docs/rrc_hold_closure_checklist_2026-05-09.md`
- `6-Documentation/docs/network_topology_hold_manifests_2026-05-09.md`
- `6-Documentation/docs/specs/OMINDIRECTION_LOGOGRAM_DESIGN_AND_COMPILER.md`
- `6-Documentation/docs/specs/LAW_GATED_RECONSTRUCTION_CORE_SHIFT.md`
- `4-Infrastructure/shim/rrc_tri_cycle_audit.py`
- `shared-data/data/rrc_tri_cycle_audit/rrc_tri_cycle_audit_receipt.json`
- `6-Documentation/docs/rrc_tri_cycle_audit_2026-05-09.md`
- `4-Infrastructure/hardware/tangnano9k_uart_beacon.v`
- `4-Infrastructure/hardware/build_uart_beacon.sh`
- `4-Infrastructure/hardware/tangnano9k_uart_swapped.cst`
- `4-Infrastructure/shim/tang9k_uart_beacon_probe_receipt.json`
- `4-Infrastructure/shim/tang9k_uart_beacon_swapped_probe_receipt.json`
- `4-Infrastructure/shim/tang9k_uart_loopback_after_jtag_clear_probe_receipt.json`
- `6-Documentation/docs/fpga_rrc_q16_accel_setup_2026-05-09.md`
- `6-Documentation/docs/fpga_uart_route_analysis_2026-05-09.md`
- `6-Documentation/docs/fpga_direct_probe_report_2026-05-09.md`
- `0-Core-Formalism/lean/Semantics/Semantics.lean`
- `0-Core-Formalism/lean/Semantics/Semantics/BeaverMaskFreshness.lean`
- `0-Core-Formalism/lean/Semantics/Semantics/WhitespaceFreeGrammar.lean`
## Optional Same-Slice Hardware Artifacts
These are generated bitstream/build products. Stage only if this repository wants
to preserve generated FPGA artifacts:
- `4-Infrastructure/hardware/tangnano9k_uart_beacon.fs`
- `4-Infrastructure/hardware/tangnano9k_uart_beacon.json`
- `4-Infrastructure/hardware/tangnano9k_uart_beacon_pnr.json`
- `4-Infrastructure/hardware/tangnano9k_rrc_q16_accel.fs`
- `4-Infrastructure/hardware/tangnano9k_rrc_q16_accel.json`
- `4-Infrastructure/hardware/tangnano9k_rrc_q16_accel_pnr.json`
## Do Not Sweep
Avoid broad commands such as:
```bash
git add 0-Core-Formalism 4-Infrastructure 6-Documentation shared-data
```
The current tree contains many unrelated generated probes, modified Lean
experiments, and documentation surfaces. Stage by explicit file path only.
## Current Evidence
- Full Lean/Semantics build: PASS
- JSON integrity gate: PASS
- Python shim compile gate: PASS
- Support receipt refresh gate: PASS
- Rainbow Raccoon compiler: PASS_WITH_HOLDS, 1 candidate and 6 HOLD
- Tri-cycle audit: PASS_WITH_BLOCKED_HARDWARE
- FPGA software witness: PASS
- FPGA hardware witness: FAIL
- UART beacon: no bytes observed on `/dev/ttyUSB0` or `/dev/ttyUSB1`
- Q16 virtual serial probe: PASS, proving host framing/parser and opcode semantics over a PTY-backed serial device
- UART transport routes: active non-hardware route is `virtual://q16-pty`; physical routes remain blocked or pending
- Forced JTAG clear: JTAG detect/reset and SRAM reload work; UART/fabric receipts still fail, with loopback-after-clear showing FTDI/MPSSE-style `faXX` bytes on `/dev/ttyUSB0`
- FPGA UART route analysis: onboard BL702 bridge route remains blocked; next live test uses external USB-UART on fabric pins 17/18
- Optional external `sem` entity-diff aid: ready from isolated `/tmp` binary; `/usr/bin/sem` is GNU Parallel
- Stack fail closure register: PASS_TICKETS_DECLARED, 7 tickets
- RRC HOLD closure checklist: 0 open documentation closures across 6 HOLD objects; compiler promotion rerun still pending
- Network topology HOLD manifests: 9 coefficient rows and 15 prediction rows separated from validation claims
- Beaver mask freshness controls: Lean-backed finite negative controls added; security HOLD remains partial
- Whitespace-zero grammar: canonical spaces derive from symbol count/order with zero stored whitespace codes; non-canonical spacing remains HOLD/residual
- Palindrome hex markers: deterministic mirror/frame markers added for zero-whitespace decompressor and remote/local replay packets; marker use is ADMIT only with namespace, payload hash, receipt hash, and replay agreement
- Predictive harmony social synchrony: structured chord progression plus live mutual-attention gate added as a neuroacoustic route prior; therapy and social-control claims remain HOLD/QUARANTINE
- DESI epoviz to MaNGA population-cell join: 669,377 DESI EDR epoviz rows joined to 25 MaNGA sky/redshift cells as a coarse population prior; object-level crossmatch and direct gas-density inference remain HOLD
- Stellar gas eigenvector mass: dominant SMN/evidence-load eigenvalue 4.872368819 over 25 DESI/MaNGA cells, explaining 58.4684258% of joined-cell variance; physical mass, gas density, and cosmology interpretations remain HOLD
- DESI row eigenmass: 669,377 DESI epoviz rows streamed into a row-level SMN/evidence-load eigenprobe; dominant eigenvalue 3.276998814 explains 32.7699881% of row-level geometry/tracer variance
- Multiscale eigenmass alignment: row-level DESI tracer eigenmass and DESI/MaNGA joined-cell tracer eigenmass align with cosine 0.702922832; MaNGA gas/shock overlap sharpens the dominant explained share by 1.784206501x
- Hutter eigenmass transfer: DESI multiscale eigenmass method ported into a Hutter tuning protocol with seven promotion gates; no Hutter compression-gain claim is made
- Stellar gas Abelian sandpile probe: eigenmass treated as grains and gas/shock channels as toppling pressure; 5 avalanche-candidate cells found, with eigenmass most correlated to DESI density, stellar sigma, and shock proxies
- Stellar gas sandpile fine zoom: 911 MaNGA objects found inside 5 avalanche-candidate cells; 40 object-level proxy examples emitted for follow-up while mechanism proof remains HOLD
- Stellar gas full-cell stability: all 25 joined cells checked against full-cell and null/ablation controls; stored 25-cell comparison cosine is 1.0 and leave-one-cell-out minimum cosine is 0.996495428
- Stellar gas sandpile graph replay: graph diagnostic emits 25 nodes, 45 edges, 5 seeded avalanche candidates, graph hash `fdfc686a022b726f550f73ad663cd9e222122a1048193dd22dafee0d891c62af`, and replay hash `95e938b7f6ed3a3c64af0fdaec45e9296316ae9e77b6b937aaa9ee16f7665393`
- Hutter transfer readiness fixture: 669-byte fixture `htrf_local_text_window_2026_05_09_v0`, SHA-256 `eacd7a17d3485ff0e9fd3ec19c3c2dfcc7419e906eafe18817b07596a99ad6e6`, raw/zlib/lzma/current-wire baseline matrix, negative controls, and Rust OISC byte-exact replay; no Hutter compression, full-corpus, FPGA, or ASIC claim
- Drive sync receipt: curated eigenmass-stack safety snapshot copied to `Gdrive:topological_storage/research-stack/eigenmass-stack-2026-05-09` with 36 remote entries observed after final sync; this manifest remains the full source list
- Agent routing files: repo root, Lean/Semantics, and Infrastructure contracts added
- Receipt visibility: stack solidification receipts and `shared-data/network_topology_database.json` are no longer hidden by broad `shared-data/` ignore rules
- Promotion decision: NO_PROMOTION

View file

@ -0,0 +1,81 @@
# Stack Solidification Status
**Date:** 2026-05-09
**Hygiene refresh:** 2026-05-10
## Bottom Line
The stack is buildable and internally gateable, but not promotable as a live hardware-accelerated system yet.
## Gates
- Full Lean/Semantics build: `SKIPPED`
- JSON integrity: `PASS`
- Python shim compile: `PASS`
- Support receipt refresh: `PASS`
- Rainbow Raccoon compiler: `PASS_WITH_HOLDS` (1 candidate, 6 HOLD)
- Tri-cycle audit: `PASS`; promotion decision `NO_PROMOTION`
- FPGA software witness: `PASS`
- FPGA hardware witness: `NOT_REQUESTED`
- UART beacon seen: `False`
- Hardware bitstreams present: `PASS`
- Optional sem entity-diff aid: `OPTIONAL_AUDIT_AID_READY`
- RRC HOLD closure checklist: `PASS_ALL_ITEMS_CLOSED` (0 open; compiler rerun still required before promotion)
- Network HOLD manifests: `PASS_HOLD_QUEUES_DECLARED` (9 coefficient rows, 15 prediction rows)
- Beaver mask freshness controls: `PASS_NEGATIVE_CONTROLS` (6 cases)
- Whitespace-zero grammar: `PASS_ZERO_WHITESPACE_CANONICAL` (3 admitted, 3 HOLD)
- Q16 virtual serial probe: `PASS_VIRTUAL_SERIAL` (3/3 matches)
- UART transport routes: `PASS_ACTIVE_VIRTUAL_ROUTE` (active `virtual://q16-pty`)
- Stack fail closure register: `PASS_TICKETS_DECLARED` (7 tickets)
- SMN tool awareness: `PASS` (ADMIT_SMN_TOOL_AWARENESS)
- Worktree: `DIRTY`
- Staging manifest: `6-Documentation/docs/stack_solidification_staging_manifest_2026-05-09.md`
## Current Solid Core
- Lean/Semantics builds end to end.
- Core JSON receipts and network topology database parse.
- Compiler gate admits only the Q16 fixed-point lowering certificate as candidate.
- Q16 software witness lane passes.
- Q16 host UART framing and parser pass over a PTY-backed virtual serial device.
- UART route table now selects the PTY-backed Q16 route while keeping blocked physical routes visible.
- HOLD buckets are explicit rather than silently promoted.
- Optional sem entity extraction is available for scoped Python audit files.
- SMN is tool-visible as Semantic Mass Number and explicitly separated from Mass Number admissibility packets.
- Every compiler HOLD object now has an explicit closure checklist, and the RRC Gatekeeper has closed `11/11` documentation closures.
- Current failures and broad HOLD buckets have closure tickets in a stack fail register.
- Network topology coefficients and predictions are split into HOLD queues.
- Beaver mask freshness has Lean-backed finite negative controls; full MPC security remains HOLD.
- Canonical logogram grammar can derive ordinary spaces from symbol count/order with zero stored whitespace codes.
- Agent routing now has repo-root, Lean/Semantics, and Infrastructure contracts.
- Stack receipts and the network topology database are visible through narrow `.gitignore` exceptions instead of broad `shared-data/` exposure.
## Current Blockers
- Live FPGA UART transport remains blocked: beacon receipts show no bytes on `/dev/ttyUSB0` or `/dev/ttyUSB1`.
- Hardware acceleration claims remain blocked until the UART route or external adapter path produces matching receipts.
- Security, coefficient, topology-prediction, and receipt-gate debts remain HOLD surfaces.
- The worktree is broad and dirty; do not stage by directory sweep.
- `/usr/bin/sem` is GNU Parallel on this machine; use the isolated sem binary path if sem is needed.
## Less Solid Surface Counts
- `coefficient_or_calibration_debt`: 20
- `fpga_transport_or_witness_debt`: 9
- `general_hold_surface`: 69
- `receipt_gate_debt`: 11
- `security_proof_debt`: 6
- `topology_prediction_debt`: 13
## Next Stabilization Moves
1. Resolve fabric UART transport with board bridge docs or an external USB-UART adapter.
2. Work the closure register tickets in order: UART transport, flash persistence, adaptive-mask security, coefficient calibration, topology predictions, receipt gates, then worktree scope.
3. Rerun the Rainbow Raccoon compiler against the refreshed closure receipts before changing any HOLD/CANDIDATE status.
4. Produce a scoped staging manifest before any commit because the working tree contains many unrelated/generated surfaces.
## Receipt
- Machine receipt: `shared-data/data/stack_solidification/stack_solidification_receipt.json`
- Staging manifest: `6-Documentation/docs/stack_solidification_staging_manifest_2026-05-09.md`

View file

@ -0,0 +1,69 @@
# Stellar Gas Abelian Sandpile Probe
Status: `SANDPILE_DIAGNOSTIC`
Decision: `ADMIT_SANDPILE_DIAGNOSTIC_HOLD_PHYSICAL_SANDPILE`
This probe treats the stellar-gas eigenmass surface as an Abelian-sandpile-style
diagnostic. Cells carry normalized eigenmass as "grains"; gas/shock observables
act as toppling pressure; high grain/high-pressure cells become avalanche
candidates for fine-grained follow-up.
Claim boundary: this is a metaphor-backed diagnostic over observational proxies.
It is not a physical sandpile simulation, not stellar mass, not direct gas
density inference, and not a cosmology fit.
## Channel Correlations With Eigenmass
- `log_desi_count`: 0.881673842
- `stellar_sigma_mean`: 0.878922036
- `partial_full_shock_fraction`: 0.847109384
- `shock_score_mean`: 0.839508306
- `shock_lier_fraction`: 0.811272006
- `star_forming_fraction`: -0.718656679
- `agn_liner_or_shock_fraction`: 0.708425115
- `gas_sigma_mean`: 0.654818267
- `gas_sigma_p90`: 0.605869064
- `snr_mean`: -0.435451409
- `log_manga_count`: 0.097331989
## Toppling Candidates
- `ra03_north__z_008_plus`: state `AVALANCHE_CANDIDATE`, grains `0.11642`, pressure `1.157125106`, index `1.596155368`
- `ra03_south__z_008_plus`: state `AVALANCHE_CANDIDATE`, grains `0.112959`, pressure `1.198262192`, index `1.570637955`
- `ra04_north__z_008_plus`: state `AVALANCHE_CANDIDATE`, grains `0.11274`, pressure `1.17842835`, index `1.557804876`
- `ra02_south__z_008_plus`: state `AVALANCHE_CANDIDATE`, grains `0.107057`, pressure `1.208401092`, index `1.497117602`
- `ra02_north__z_008_plus`: state `AVALANCHE_CANDIDATE`, grains `0.110828`, pressure `1.09454727`, index `1.490404542`
- `ra02_south__z_006_008`: state `LOADED`, grains `0.047288`, pressure `1.540624514`, index `0.867357748`
- `ra03_north__z_006_008`: state `LOADED`, grains `0.039897`, pressure `0.645551377`, index `0.321404162`
- `ra02_north__z_006_008`: state `LOADED`, grains `0.03392`, pressure `0.577987139`, index `0.208033555`
- `ra04_north__z_006_008`: state `LOADED`, grains `0.034318`, pressure `0.362305657`, index `0.105492499`
- `ra03_south__z_006_008`: state `STABLE`, grains `0.024638`, pressure `0.337625686`, index `-0.035744352`
## Pressure Components
```json
[
"partial_full_shock_fraction",
"shock_lier_fraction",
"shock_score_mean",
"gas_sigma_mean",
"gas_sigma_p90",
"agn_liner_or_shock_fraction"
]
```
## Holds
- `HOLD_PHYSICAL_SANDPILE_SIMULATION`
- `HOLD_DIRECT_STELLAR_MASS`
- `HOLD_DIRECT_GAS_DENSITY_INFERENCE`
- `HOLD_OBJECT_LEVEL_CROSSMATCH`
- `HOLD_COSMOLOGY_FIT`
## Receipt Backlinks
- Receipt: `shared-data/data/stellar_gas_observation/stellar_gas_abelian_sandpile_probe_receipt.json`
- Source cell eigenmass receipt: `shared-data/data/stellar_gas_observation/stellar_gas_eigenvector_mass_probe_receipt.json`
- Data: `shared-data/data/stellar_gas_observation/stellar_gas_abelian_sandpile_probe.json`
- Tiddler: `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Abelian Sandpile Probe.tid`

View file

@ -0,0 +1,69 @@
# Stellar Gas Eigenvector Mass Probe
Status: `SMN_EIGENVECTOR_MASS`
Decision: `ADMIT_SMN_EIGENVECTOR_MASS_HOLD_PHYSICAL_MASS`
This probe computes the dominant covariance eigenvector over the coarse DESI
epoviz to MaNGA population-cell join. The output is an SMN/evidence-load mass
direction: it ranks the current coarse joined cells by this diagnostic score so
later zoom work can choose explicit follow-up targets.
Claim boundary: this is not physical mass, not stellar mass, not a direct gas
density map, and not a cosmology fit.
## Result
Dominant eigenvalue:
```text
4.872368819
```
Dominant explained mass share:
```text
0.584684258
```
## Dominant Eigenvector
- `log_desi_count`: 0.407663945
- `log_manga_count`: 0.045003218
- `partial_full_shock_fraction`: 0.391682145
- `shock_lier_fraction`: 0.375112377
- `BGS_share`: -0.370321192
- `ELG_share`: 0.443075496
- `LRG_share`: 0.443094949
- `QSO_share`: -0.088734468
## Eigensolver Diagnostics
```text
method: jacobi_symmetric
converged: True
iterations: 87
final max off-diagonal: 6.353076600207743e-13
dominant residual L2: 0.0
```
## Top Cell Masses
- `ra03_north__z_008_plus`: mass `0.11642`, score `4.401578`, DESI `315331`, MaNGA `305`
- `ra03_south__z_008_plus`: mass `0.112959`, score `4.202232`, DESI `100231`, MaNGA `8`
- `ra04_north__z_008_plus`: mass `0.11274`, score `4.18963`, DESI `181621`, MaNGA `219`
- `ra02_north__z_008_plus`: mass `0.110828`, score `4.079512`, DESI `44035`, MaNGA `364`
- `ra02_south__z_008_plus`: mass `0.107057`, score `3.862327`, DESI `12520`, MaNGA `15`
- `ra02_south__z_006_008`: mass `0.047288`, score `0.419776`, DESI `91`, MaNGA `3`
- `ra03_north__z_006_008`: mass `0.039897`, score `-0.005909`, DESI `3456`, MaNGA `374`
- `ra04_north__z_006_008`: mass `0.034318`, score `-0.327257`, DESI `2102`, MaNGA `143`
- `ra02_north__z_006_008`: mass `0.03392`, score `-0.350201`, DESI `608`, MaNGA `259`
- `ra03_north__z_002_004`: mass `0.032102`, score `-0.454923`, DESI `2439`, MaNGA `1622`
## Holds
- `HOLD_PHYSICAL_MASS_INTERPRETATION`
- `HOLD_OBJECT_LEVEL_CROSSMATCH`
- `HOLD_DIRECT_GAS_DENSITY_INFERENCE`
- `HOLD_SELECTION_FUNCTION_FIT`
- `HOLD_COSMOLOGY_FIT`

View file

@ -0,0 +1,85 @@
# Stellar Gas Full Cell Eigenmass Stability
Status: `FULL_CELL_EIGENMASS_STABILITY`
Decision: `REPORT_FULL_CELL_EIGENMASS_STABILITY_WITH_NULL_CONTROLS_HOLD_PHYSICAL_CLAIMS`
This probe checks the 25-cell DESI/MaNGA joined-cell eigenmass against all
available joined cells, leave-one-cell-out slices, deterministic null shuffles,
and feature ablations.
Claim boundary: this is evidence-geometry quality control only. It does not
promote physical mass, gas density, shock proof, object-level crossmatch, or
cosmology.
## Full-Cell Baseline
Cell count: `25`
Dominant eigenvalue:
```text
4.872368819
```
Dominant explained mass share:
```text
0.584684258
```
Dominant eigenvector:
- `log_desi_count`: 0.407663945
- `log_manga_count`: 0.045003218
- `partial_full_shock_fraction`: 0.391682145
- `shock_lier_fraction`: 0.375112377
- `BGS_share`: -0.370321192
- `ELG_share`: 0.443075496
- `LRG_share`: 0.443094949
- `QSO_share`: -0.088734468
## Stored 25-Cell Comparison
```text
common-basis cosine to stored probe: 1.0
top-cell overlap at 5: 5
max abs component delta: 0.0
```
## Eigensolver Diagnostics
```text
method: jacobi_symmetric
converged: True
iterations: 87
final max off-diagonal: 6.353076600207743e-13
dominant residual L2: 0.0
```
## Leave-One-Cell-Out Stability
```text
loo count: 25
min cosine to original: 0.996495428
median cosine to original: 0.99984468
mean cosine to original: 0.999501223
mean top5 overlap: 0.96
```
## Null And Ablation Controls
- `shuffled_feature_columns`: cosine `0.32053877`, explained share `0.251750194`, top5 overlap `0.0`
- `shuffled_shock_channels`: cosine `0.738854689`, explained share `0.460461221`, top5 overlap `1.0`
- `desi_count_removed`: cosine `0.998817144`, explained share `0.56433804`, top5 overlap `1.0`
- `shock_proxy_removed`: cosine `0.993658602`, explained share `0.589466219`, top5 overlap `1.0`
- `tracer_mix_removed`: cosine `0.992371346`, explained share `0.629616192`, top5 overlap `1.0`
## Holds
- `HOLD_PHYSICAL_MASS_INTERPRETATION`
- `HOLD_DIRECT_GAS_DENSITY_INFERENCE`
- `HOLD_SHOCK_PROOF`
- `HOLD_OBJECT_LEVEL_CROSSMATCH`
- `HOLD_SELECTION_FUNCTION_FIT`
- `HOLD_COSMOLOGY_FIT`

View file

@ -0,0 +1,62 @@
# Stellar Gas Multiscale Eigenmass Alignment
Status: `MULTISCALE_EIGENMASS_ALIGNMENT`
Decision: `ADMIT_MULTISCALE_EIGENMASS_ALIGNMENT_HOLD_PHYSICAL_MASS`
This probe compares the row-level DESI epoviz eigenmass with the DESI/MaNGA
joined-cell eigenmass. It reports a tracer-subspace cosine and explained-share
ratio between the literal row data and the coarse joined-cell overlap surface.
Claim boundary: this is not physical mass, not stellar mass, not gas-density
inference, and not a cosmology fit.
## Alignment Result
Tracer-subspace cosine:
```text
0.702922832
```
Alignment class:
```text
MODERATE_ALIGNMENT
```
Constraint sharpening factor:
```text
1.784206501
```
Dominant eigenvalue ratio, cell over row:
```text
1.48683875
```
## Tracer Subvectors
- `QSO`: row `0.178308871`, cell `-0.088734468`
- `ELG`: row `0.373691964`, cell `0.443075496`
- `LRG`: row `-0.001480357`, cell `0.443094949`
- `BGS`: row `-0.485709225`, cell `-0.370321192`
## Scale Comparison
```text
row level rows: 669377
row explained share: 0.327699881
cell level cells: 25
cell explained share: 0.584684258
```
## Holds
- `HOLD_PHYSICAL_MASS_INTERPRETATION`
- `HOLD_DIRECT_GAS_DENSITY_INFERENCE`
- `HOLD_OBJECT_LEVEL_CROSSMATCH`
- `HOLD_SELECTION_FUNCTION_FIT`
- `HOLD_COSMOLOGY_FIT`

View file

@ -0,0 +1,100 @@
# Stellar Gas Population Grouping Study
**Date:** 2026-05-09
**Decision:** `ADMIT_POPULATION_GROUPING_SURFACE`
**Claim boundary:** MaNGA stellar-gas population grouping only. A coarse
DESI/MaNGA cell join now exists; object-level crossmatch remains `HOLD`.
Proxy classes do not prove physical shock, AGN, or gas mechanism.
## Population Surface
```text
rows seen, all DAP types: 43128
unique Plate-IFU count: 10782
selected population: 10782
preferred DAPTYPE: HYB10-MILESHC-MASTARSSP
```
## Aggregate
```json
{
"bpt_proxy_classes": {
"agn_liner_or_shock_proxy": 2740,
"star_forming_proxy": 5029,
"composite_proxy": 2883,
"unclassified": 130
},
"shock_lier_proxy_classes": {
"shock_lier_proxy": 4589,
"partial_shock_proxy": 2458,
"no_shock_proxy": 3735
},
"partial_or_full_shock_fraction": 0.653589,
"gas_sigma_summary": {
"count": 10609,
"min": 0.384927,
"max": 929.010071,
"mean": 157.969176,
"median": 77.881104,
"p90": 420.896637
},
"stellar_sigma_summary": {
"count": 10297,
"min": 3.115719,
"max": 765.25354,
"mean": 118.838681,
"median": 94.604462,
"p90": 233.339722
}
}
```
## Redshift Bins
| Bin | Count | Shock+Partial Fraction | Gas Sigma Median | Stellar Sigma Median |
|---|---:|---:|---:|---:|
| `z_002_004` | 4838 | 0.619678 | 55.327721 | 72.146851 |
| `z_004_006` | 2415 | 0.57764 | 78.61245 | 101.009289 |
| `z_008_plus` | 1250 | 0.896 | 284.684052 | 230.705811 |
| `z_000_002` | 1241 | 0.635778 | 30.074999 | 52.191267 |
| `z_006_008` | 1038 | 0.717726 | 157.058464 | 172.332016 |
## DESI-Ready Sky/Redshift Cells
These are population cells for the existing coarse DESI/MaNGA cell join. They
are not object-level DESI/MaNGA crossmatches and not DESI environment classes
yet.
| Cell | Count | Shock+Partial Fraction | Main BPT Counts |
|---|---:|---:|---|
| `ra03_north__z_002_004` | 1622 | 0.676326 | `{'star_forming_proxy': 811, 'agn_liner_or_shock_proxy': 342, 'composite_proxy': 437, 'unclassified': 32}` |
| `ra02_north__z_002_004` | 1289 | 0.584174 | `{'agn_liner_or_shock_proxy': 222, 'composite_proxy': 286, 'star_forming_proxy': 781}` |
| `ra04_north__z_002_004` | 960 | 0.60625 | `{'agn_liner_or_shock_proxy': 219, 'composite_proxy': 257, 'star_forming_proxy': 483, 'unclassified': 1}` |
| `ra02_north__z_004_006` | 684 | 0.554094 | `{'composite_proxy': 188, 'star_forming_proxy': 319, 'agn_liner_or_shock_proxy': 174, 'unclassified': 3}` |
| `ra03_north__z_004_006` | 531 | 0.572505 | `{'star_forming_proxy': 246, 'composite_proxy': 152, 'agn_liner_or_shock_proxy': 131, 'unclassified': 2}` |
| `ra00_north__z_000_002` | 450 | 0.72 | `{'star_forming_proxy': 214, 'agn_liner_or_shock_proxy': 79, 'composite_proxy': 117, 'unclassified': 40}` |
| `ra04_north__z_004_006` | 420 | 0.6 | `{'agn_liner_or_shock_proxy': 128, 'star_forming_proxy': 159, 'composite_proxy': 133}` |
| `ra03_north__z_006_008` | 374 | 0.754011 | `{'star_forming_proxy': 83, 'agn_liner_or_shock_proxy': 164, 'composite_proxy': 127}` |
| `ra02_north__z_008_plus` | 364 | 0.881868 | `{'agn_liner_or_shock_proxy': 163, 'composite_proxy': 143, 'star_forming_proxy': 58}` |
| `ra03_north__z_008_plus` | 305 | 0.918033 | `{'composite_proxy': 130, 'star_forming_proxy': 43, 'agn_liner_or_shock_proxy': 129, 'unclassified': 3}` |
| `ra03_north__z_000_002` | 279 | 0.670251 | `{'agn_liner_or_shock_proxy': 40, 'star_forming_proxy': 188, 'unclassified': 14, 'composite_proxy': 37}` |
| `ra02_north__z_000_002` | 270 | 0.581481 | `{'star_forming_proxy': 210, 'composite_proxy': 30, 'agn_liner_or_shock_proxy': 28, 'unclassified': 2}` |
## What This Gives Us
- A population baseline over unique MaNGA Plate-IFU rows.
- Grouped shock/LIER and BPT proxy counts by redshift and sky cell.
- DESI-ready cells now used by the coarse DESI/MaNGA cell join.
- Residual target: cells whose local gas state diverges from later DESI environment priors.
## Receipt
`shared-data/data/stellar_gas_observation/stellar_gas_population_grouping_study_receipt_*.json`
## Receipt Backlinks
- Coarse cell join receipt: `shared-data/data/stellar_gas_observation/desi_epoviz_manga_population_cell_join_receipt.json`
- Tiddler: `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Population Grouping Study.tid`

View file

@ -0,0 +1,43 @@
# Stellar Gas Sandpile Fine Zoom
Status: `FINE_ZOOM_OBJECT_EXAMPLES`
Decision: `ADMIT_FINE_ZOOM_OBJECT_EXAMPLES_HOLD_MECHANISM_PROOF`
This fine-zoom pass drills from sandpile avalanche-candidate cells down to MaNGA
Plate-IFU examples. It is meant to show which concrete objects carry the
strongest local proxy pressure under the cell-level eigenmass surface.
Claim boundary: proxy classes and object-pressure scores route follow-up; they
do not prove physical shock, AGN, stellar mass, gas density, or cosmology.
## Summary
```text
candidate cells: 5
candidate-cell objects: 911
examples per cell: 8
```
## Top Object Per Candidate Cell
- `ra03_north__z_008_plus`: candidates `305`, top `8333-6101`, pressure `4.587696`, class `shock_lier_proxy`
- `ra03_south__z_008_plus`: candidates `8`, top `11024-12705`, pressure `4.423534`, class `shock_lier_proxy`
- `ra04_north__z_008_plus`: candidates `219`, top `9883-3702`, pressure `4.700404`, class `shock_lier_proxy`
- `ra02_south__z_008_plus`: candidates `15`, top `12518-6104`, pressure `4.145211`, class `shock_lier_proxy`
- `ra02_north__z_008_plus`: candidates `364`, top `8943-3704`, pressure `4.656886`, class `shock_lier_proxy`
## Holds
- `HOLD_PHYSICAL_SHOCK_PROOF`
- `HOLD_DIRECT_STELLAR_MASS`
- `HOLD_DIRECT_GAS_DENSITY_INFERENCE`
- `HOLD_OBJECT_LEVEL_DESI_CROSSMATCH`
- `HOLD_COSMOLOGY_FIT`
## Receipt Backlinks
- Receipt: `shared-data/data/stellar_gas_observation/stellar_gas_sandpile_fine_zoom_receipt.json`
- Source sandpile receipt: `shared-data/data/stellar_gas_observation/stellar_gas_abelian_sandpile_probe_receipt.json`
- Data: `shared-data/data/stellar_gas_observation/stellar_gas_sandpile_fine_zoom.json`
- Tiddler: `6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Sandpile Fine Zoom.tid`

View file

@ -0,0 +1,157 @@
# Stellar Gas Sandpile Graph Replay
Status: `GRAPH_TOPPLING_PROXY`
Decision: `ADMIT_GRAPH_TOPPLING_PROXY_HOLD_PHYSICAL_SANDPILE_SIMULATION`
This hardening pass turns the sandpile metaphor into a reproducible graph
diagnostic. Nodes are sky/redshift cells. Edges are defined by sky-sector
neighbors plus adjacent redshift bins. Grain and toppling values are diagnostic
proxies derived from the existing cell-level toppling index.
Claim boundary: this is not a physical sandpile simulation, not a stellar-gas
mechanism proof, not direct stellar mass, not gas density inference, and not a
cosmology fit.
## Replay Receipt
```json
{
"avalanche_sizes": [
{
"avalanche_size_cells": 10,
"seed_cell": "ra02_north__z_008_plus",
"terminated": true,
"topple_events": 21
},
{
"avalanche_size_cells": 10,
"seed_cell": "ra02_south__z_008_plus",
"terminated": true,
"topple_events": 21
},
{
"avalanche_size_cells": 10,
"seed_cell": "ra03_north__z_008_plus",
"terminated": true,
"topple_events": 21
},
{
"avalanche_size_cells": 10,
"seed_cell": "ra03_south__z_008_plus",
"terminated": true,
"topple_events": 21
},
{
"avalanche_size_cells": 10,
"seed_cell": "ra04_north__z_008_plus",
"terminated": true,
"topple_events": 21
}
],
"created": "2026-05-10T04:07:14+00:00",
"decision": "ADMIT_GRAPH_TOPPLING_PROXY_HOLD_PHYSICAL_SANDPILE_SIMULATION",
"edge_count": 45,
"graph_hash": "fdfc686a022b726f550f73ad663cd9e222122a1048193dd22dafee0d891c62af",
"node_count": 25,
"receipt_type": "stellar_gas_sandpile_graph_replay_receipt",
"replay_hash": "95e938b7f6ed3a3c64af0fdaec45e9296316ae9e77b6b937aaa9ee16f7665393",
"seed_avalanche_cell_count": 5,
"seed_candidate_object_count": 911,
"validated_outputs": [
"shared-data/data/stellar_gas_observation/stellar_gas_sandpile_graph_replay.json",
"shared-data/data/stellar_gas_observation/stellar_gas_sandpile_graph_replay_receipt.json",
"6-Documentation/docs/stellar_gas_sandpile_graph_replay_2026-05-09.md",
"6-Documentation/tiddlywiki-local/wiki/tiddlers/Stellar Gas Sandpile Graph Replay.tid"
]
}
```
## Seed Evidence
```text
avalanche cells: 5
candidate objects: 911
graph nodes: 25
graph edges: 45
graph hash: fdfc686a022b726f550f73ad663cd9e222122a1048193dd22dafee0d891c62af
replay hash: 95e938b7f6ed3a3c64af0fdaec45e9296316ae9e77b6b937aaa9ee16f7665393
```
## Toppling Threshold Table
| cell | degree | threshold |
| --- | ---: | ---: |
| `ra02_north__z_000_002` | 3 | 4 |
| `ra02_north__z_002_004` | 4 | 5 |
| `ra02_north__z_004_006` | 4 | 5 |
| `ra02_north__z_006_008` | 4 | 5 |
| `ra02_north__z_008_plus` | 3 | 4 |
| `ra02_south__z_000_002` | 3 | 4 |
| `ra02_south__z_002_004` | 4 | 5 |
| `ra02_south__z_004_006` | 4 | 5 |
| `ra02_south__z_006_008` | 4 | 5 |
| `ra02_south__z_008_plus` | 3 | 4 |
| `ra03_north__z_000_002` | 4 | 5 |
| `ra03_north__z_002_004` | 5 | 6 |
| `ra03_north__z_004_006` | 5 | 6 |
| `ra03_north__z_006_008` | 5 | 6 |
| `ra03_north__z_008_plus` | 4 | 5 |
| `ra03_south__z_000_002` | 3 | 4 |
| `ra03_south__z_002_004` | 4 | 5 |
| `ra03_south__z_004_006` | 4 | 5 |
| `ra03_south__z_006_008` | 4 | 5 |
| `ra03_south__z_008_plus` | 3 | 4 |
| `ra04_north__z_000_002` | 2 | 3 |
| `ra04_north__z_002_004` | 3 | 4 |
| `ra04_north__z_004_006` | 3 | 4 |
| `ra04_north__z_006_008` | 3 | 4 |
| `ra04_north__z_008_plus` | 2 | 3 |
## Initial Grain Table
| cell | initial grains | index z proxy | seed objects |
| --- | ---: | ---: | ---: |
| `ra02_north__z_000_002` | 0 | -0.949257801 | 0 |
| `ra02_north__z_002_004` | 0 | -0.553842296 | 0 |
| `ra02_north__z_004_006` | 0 | -0.41882282 | 0 |
| `ra02_north__z_006_008` | 2 | 0.228692003 | 0 |
| `ra02_north__z_008_plus` | 7 | 1.638406841 | 364 |
| `ra02_south__z_000_002` | 0 | -1.463289664 | 0 |
| `ra02_south__z_002_004` | 0 | -0.737986126 | 0 |
| `ra02_south__z_004_006` | 0 | -0.194645543 | 0 |
| `ra02_south__z_006_008` | 5 | 0.953489356 | 0 |
| `ra02_south__z_008_plus` | 7 | 1.64578653 | 15 |
| `ra03_north__z_000_002` | 0 | -0.368783497 | 0 |
| `ra03_north__z_002_004` | 0 | -0.100706818 | 0 |
| `ra03_north__z_004_006` | 0 | -0.372968233 | 0 |
| `ra03_north__z_006_008` | 3 | 0.353320701 | 0 |
| `ra03_north__z_008_plus` | 9 | 1.754659087 | 305 |
| `ra03_south__z_000_002` | 0 | -1.161555303 | 0 |
| `ra03_south__z_002_004` | 0 | -0.731256364 | 0 |
| `ra03_south__z_004_006` | 0 | -1.186176214 | 0 |
| `ra03_south__z_006_008` | 0 | -0.039293889 | 0 |
| `ra03_south__z_008_plus` | 7 | 1.726607707 | 8 |
| `ra04_north__z_000_002` | 0 | -1.291360152 | 0 |
| `ra04_north__z_002_004` | 0 | -0.333483573 | 0 |
| `ra04_north__z_004_006` | 0 | -0.226002461 | 0 |
| `ra04_north__z_006_008` | 1 | 0.115968267 | 0 |
| `ra04_north__z_008_plus` | 6 | 1.71250026 | 219 |
## Avalanche Sizes
| seed cell | size cells | topple events | terminated |
| --- | ---: | ---: | --- |
| `ra02_north__z_008_plus` | 10 | 21 | `True` |
| `ra02_south__z_008_plus` | 10 | 21 | `True` |
| `ra03_north__z_008_plus` | 10 | 21 | `True` |
| `ra03_south__z_008_plus` | 10 | 21 | `True` |
| `ra04_north__z_008_plus` | 10 | 21 | `True` |
## Holds
- `HOLD_PHYSICAL_SANDPILE_SIMULATION`
- `HOLD_STELLAR_GAS_MECHANISM_PROOF`
- `HOLD_DIRECT_STELLAR_MASS`
- `HOLD_DIRECT_GAS_DENSITY_INFERENCE`
- `HOLD_COSMOLOGY_FIT`

View file

@ -0,0 +1,21 @@
# Tang Nano 9K Q16 Virtual Serial Probe
**Date:** 2026-05-09
Virtual serial probe only. This validates the host Q16 UART framing, receipt parser, and opcode semantics over a PTY-backed serial device; it does not validate live FPGA fabric or the Tang Nano UART route.
## Status
- Status: `PASS_VIRTUAL_SERIAL`
- Cases: `3`
- Matches: `3`
## Cases
- `shift`: match `True`, receipt `4-Infrastructure/shim/tang9k_rrc_q16_virtual_shift_receipt.json`
- `weighted`: match `True`, receipt `4-Infrastructure/shim/tang9k_rrc_q16_virtual_weighted_receipt.json`
- `monotone`: match `True`, receipt `4-Infrastructure/shim/tang9k_rrc_q16_virtual_monotone_receipt.json`
## Machine Receipt
- `shared-data/data/stack_solidification/tang9k_rrc_q16_virtual_serial_probe.json`

View file

@ -0,0 +1,31 @@
# Tang Nano 9K UART Transport Routes
**Date:** 2026-05-09
Transport route table only. The active virtual route validates host Q16 serial framing and parser behavior. It does not validate live FPGA fabric or the Tang Nano onboard UART bridge.
## Active Route
- Active route: `virtual://q16-pty`
- Status: `PASS_ACTIVE_VIRTUAL_ROUTE`
## Route Table
| Route | Device | Kind | Status |
| --- | --- | --- | --- |
| `onboard-ftdi-a` | `/dev/ttyUSB0` | `physical_ftdi_mpsse_or_bridge` | `BLOCKED_FOR_FABRIC_UART` |
| `onboard-ftdi-b` | `/dev/ttyUSB1` | `physical_ftdi_secondary_endpoint` | `BLOCKED_FOR_FABRIC_UART` |
| `external-usb-uart` | `/dev/ttyUSB2_or_/dev/ttyACM0` | `physical_external_adapter` | `PENDING_HARDWARE` |
| `virtual-q16-pty` | `virtual://q16-pty` | `pty_backed_virtual_serial` | `PASS_VIRTUAL_SERIAL` |
## Routing Policy
- Default non-hardware route: `virtual://q16-pty`
- Live hardware promotion requires:
- external or verified onboard route captures beacon payload a6425131360a
- Q16 shift, weighted, and monotone hardware receipts match software expectations
- stack audit reports FPGA hardware witness PASS
## Machine Receipt
- `shared-data/data/stack_solidification/tang9k_uart_transport_routes.json`

View file

@ -0,0 +1,371 @@
# Topological Soliton Equation Pack
**Date:** 2026-05-09
**Status:** `EQUATION_PACK_DESIGN_PRIOR`
**Claim boundary:** this is an equation and receipt pack for topological
solitons as stable field configurations. It does not claim new elementary
particles, device readiness, or physical control of solitons. It gives the
Research Stack a reusable mathematical basis for knots, braids, hopfions,
skyrmions, kinks, FAMM scars, and receipt-bearing topology.
## Why Topological Solitons Matter Here
Topological solitons are directly applicable to the stack because they are:
```text
localized structure
+ preserved invariant
+ deformation resistance
+ energy barrier
+ projection/replay evidence
```
That is the same shape as:
```text
braid -> rope -> trajectory -> AMMR leaf -> replay receipt
```
The practical stack rule is:
```text
do not promote a soliton-like state because it looks stable;
promote it only when the invariant, energy/residual, and replay receipt close.
```
## Equation 1: Generic Topological Charge
For any field `phi` with boundary values in distinct vacuum classes:
```text
Q = boundary_class(phi(+infinity)) - boundary_class(phi(-infinity))
```
For the sine-Gordon field:
```text
Q_sg = [phi(+infinity) - phi(-infinity)] / (2*pi)
```
Stack use:
```text
Q = 0 trivial route / no preserved topology
Q != 0 nontrivial route / receipt required
```
## Equation 2: Sine-Gordon Kink
The sine-Gordon equation:
```text
partial_t^2 phi - partial_x^2 phi + sin(phi) = 0
```
One kink solution:
```text
phi(x,t) = 4 * arctan(exp(gamma * (x - v*t - x0)))
gamma = 1 / sqrt(1 - v^2)
```
Boundary behavior:
```text
phi(-infinity) = 0
phi(+infinity) = 2*pi
Q_sg = 1
```
Stack use:
```text
kink = smallest one-dimensional receipt-bearing transition
antikink = same structure with opposite orientation
```
## Equation 3: 2D Skyrmion Number
For a normalized magnetization field:
```text
m : R^2 -> S^2
|m| = 1
```
The skyrmion number is:
```text
Q_sk = (1 / 4*pi) * integral m . (partial_x m x partial_y m) dx dy
```
Stack use:
```text
Q_sk measures whether a 2D projected spin/field texture wraps the sphere.
```
This is the 2D cousin of the hopfion lane. It is useful for projection
receipts: a 3D state may cast a 2D image, but the 2D charge alone is not the
whole 3D invariant.
## Equation 4: Hopf Invariant
For a field:
```text
n : R^3 compactified to S^3 -> S^2
```
Define the emergent two-form / field:
```text
B_i = (1/2) * epsilon_ijk * n . (partial_j n x partial_k n)
```
If:
```text
curl A = B
```
then the Hopf invariant can be written as a helicity integral:
```text
H = (1 / (4*pi)^2) * integral A . B d^3x
```
Stack use:
```text
H counts linking / knotting of preimage loops.
H = 0 no hopfion receipt
H != 0 nontrivial 3D topological receipt
```
## Equation 5: Relative Homotopy For Realistic Hopfions
The hopfion paper uses maps of pairs:
```text
f : (I^3, partial I^3) -> (A, B)
```
with:
```text
A = S^2
B = S^2 \ union_i X_i
```
The softened-boundary invariant is:
```text
pi_3(S^2, S^2 \ union_i X_i) = Z, n >= 1
```
Stack use:
```text
realistic boundaries can still preserve integer topological charge.
```
This is important because the stack rarely has perfect boundary conditions.
Most real data arrives through partial projections, residuals, and excluded
regions.
## Equation 6: Skyrme-Faddeev / Hopfion Energy
For a unit vector field:
```text
n : R^3 -> S^2
|n| = 1
```
A common Hopf-soliton energy shape is:
```text
E_FS = integral [
alpha * sum_i |partial_i n|^2
+ beta * sum_{i<j} |partial_i n x partial_j n|^2
+ V(n)
] d^3x
```
The first term penalizes gradients. The second term prevents simple collapse
under scaling and helps stabilize knotted configurations. `V(n)` is an optional
potential or boundary preference.
Stack use:
```text
gradient term -> smoothness / local cost
quartic term -> anti-collapse / topology preservation cost
potential term -> boundary or substrate preference
```
## Equation 7: Micromagnetic Hopfion Energy
For chiral magnetic hopfions, the Nature Physics paper uses a micromagnetic
energy functional containing exchange, Dzyaloshinskii-Moriya interaction,
Zeeman, and demagnetizing terms:
```text
E = integral_Vm dr [
A * sum_i |grad m_i|^2
+ D * m . (grad x m)
- M_s * m . B
]
+ (1 / (2*mu_0)) * integral_R3 dr sum_i |grad A_d,i|^2
```
Where:
```text
m(r) = M(r) / M_s
B = B_ext + curl A_d
```
Stack use:
```text
exchange -> local alignment pressure
DMI -> chirality / torsion preference
Zeeman -> external field bias
demagnetizing -> long-range residual field
```
This is the best direct bridge from hopfion physics into your torsion/rope
model.
## Equation 8: Landau-Lifshitz-Gilbert Dynamics
The dynamical evolution of magnetization is commonly modeled by:
```text
partial_t m = -gamma * m x H_eff + alpha * m x partial_t m
```
with:
```text
H_eff = - delta E / delta m
```
Stack use:
```text
precession term -> reversible rotation / phase flow
damping term -> energy descent / basin settling
effective field -> gradient of the declared energy receipt
```
The stack analogue is:
```text
torsion update = reversible phase flow + dissipative FAMM settling
```
## Equation 9: Energy Descent Gate
For a damped soliton system, the usable receipt is not only that an invariant
exists. It also needs an energy condition:
```text
DeltaE = E(next_state) - E(current_state)
```
Gate:
```text
if Q changes unexpectedly:
QUARANTINE_TOPOLOGY_BREAK
elif DeltaE <= 0 and residual <= bound:
ADMIT_STABLE_DESCENT
elif DeltaE > 0 but external_kick_receipt exists:
HOLD_EXCITED_TRANSITION
else:
HOLD_UNEXPLAINED_ENERGY_GROWTH
```
This maps directly to FAMM: unexplained energy growth is a frustration scar.
## Equation 10: Projection / Replay Closure
A topological soliton often cannot be observed directly. The hopfion result
uses projected microscopy images plus simulation replay.
Stack closure:
```text
P_observed = projection(field_state)
P_simulated = projection(replay(field_state, parameters))
R_projection = norm(P_observed - P_simulated)
```
Gate:
```text
R_projection <= epsilon_projection
```
This is the same rule as logogram projection:
```text
projected view is not proof unless replay closes.
```
## Direct Stack Mapping
| Soliton concept | Stack primitive |
|---|---|
| Topological charge `Q` / `H` | invariant receipt |
| Kink / antikink | oriented one-dimensional route transition |
| Skyrmion number | 2D projection/wrapping receipt |
| Hopf invariant | 3D linking/knotting receipt |
| Energy barrier | FAMM scar / promotion cost |
| DMI chirality | torsional rope handedness |
| LLG precession | reversible phase flow |
| Gilbert damping | dissipative settling |
| Projection residual | replay mismatch bound |
| Boundary punctures | excluded / quarantined state regions |
## Minimal Finite Receipt Shape
The first Lean surface should be finite. Continuous equations become source
authority and later extraction targets.
```text
TopologicalSolitonReceipt:
projection_present : Bool
replay_present : Bool
invariant_kind : {kink, skyrmion, hopfion}
invariant_charge : Int
energy_delta_q0_16 : UInt16
energy_direction : {descent, excited, unexplained_growth}
projection_residual_q0_16 : UInt16
residual_bound_q0_16 : UInt16
```
Admission:
```text
ADMIT iff
projection_present
replay_present
invariant_charge != 0
projection_residual <= residual_bound
and energy_direction != unexplained_growth
```
## Next Work
1. Add `Semantics.TopologicalSolitonReceipt` as the general gate.
2. Keep `Semantics.HopfionTopologicalSoliton` as a specific fixture family.
3. Add negative controls for zero charge, missing replay, residual overflow,
and unexplained energy growth.
4. Re-run the topology/eigen remapper after the finite gate exists.

View file

@ -0,0 +1,233 @@
# Underwater Shock Public Benchmark
**Date:** 2026-05-09
**Status:** `PUBLIC_HISTORY_MODELING_PRIOR`
**Claim boundary:** this note uses public historical underwater detonation
records as free modeling data for shock-front, acoustic, bubble-pulse,
reflection, and attenuation behavior. It is not a weapon-design document, not a
charge-sizing guide, not target-vulnerability analysis, and not an operational
placement model.
## Why Sea-Based Records Are Useful
Underwater detonations are over-documented historical events. Humans made an
enormous number of public visual, acoustic, radiological, naval, and historical
records around them. That makes them useful as a low-cost validation source for
general shock physics:
```text
impulsive source
-> compressive water shock
-> pressure-release surface interaction
-> gas / vapor bubble expansion
-> bubble collapse and pulse train
-> acoustic propagation and attenuation
-> sediment / boundary reflection
```
For this stack, the value is not the weapon. The value is the medium response:
water is dense, nearly incompressible, acoustically conductive, and creates a
clean separation between the first shock front and the slower bubble-pulse
sequence.
There is also a practical economic reason. A single serious underwater shock
test chamber campaign would be expensive enough to erase the available research
budget before the model had a chance to mature. Public historical records are
therefore not just convenient; they are the only sane first validation lane.
They let the stack fit waveform shape, timing, attenuation, and residuals
without pretending that a private chamber test is feasible.
The rule is:
```text
use public history to learn the medium response;
do not use the model to optimize destructive operation.
```
## Public Historical Source Class
Useful public source classes:
- official history pages and fact sheets for underwater tests such as
Operation Crossroads BAKER;
- medical / environmental / historical reviews that describe the test context;
- public technical reports that summarize shock-wave and bubble-pulse signal
characteristics;
- open acoustic literature on underwater explosion sound and bubble-pulse
timing;
- generic bubble-dynamics literature using Rayleigh-Plesset-type equations.
Examples:
- Atomic Heritage Foundation / National Museum of Nuclear Science & History,
Operation Crossroads overview:
`https://ahf.nuclearmuseum.org/ahf/history/operation-crossroads`
- NCBI Bookshelf, "Mortality of Veteran Participants in the Crossroads Nuclear
Test", historical description:
`https://www.ncbi.nlm.nih.gov/books/NBK233207/`
- OSTI technical report, "Signal characteristics of an underwater explosive
acoustic telemetry system":
`https://www.osti.gov/biblio/6625697`
- Acoustics Today, "The Sound from Underwater Explosions":
`https://acousticstoday.org/wp-content/uploads/2023/02/The-Sound-from-Underwater-Explosions-David-R.-DallOsto-Peter-H.-Dahl-and-N.-Ross-Chapman.pdf`
## Safe Modeling Variables
The benchmark lane should use observable signal variables:
```text
t_arrival acoustic arrival time
p_peak_proxy observed or normalized peak pressure proxy
tau_decay shock decay time constant
t_bubble_1 first bubble pulse arrival
t_bubble_k later bubble pulse arrivals
A_k relative pulse amplitudes
alpha_water fitted propagation attenuation
Gamma_surface pressure-release reflection coefficient
Gamma_bottom fitted seabed / boundary reflection coefficient
```
The benchmark lane must not optimize:
```text
charge mass
device design
placement depth
standoff distance
target damage
ship / hull response
casualty or infrastructure effects
```
Those fields are explicitly outside the modeling target.
## Equations For The Benchmark Lane
The first useful abstraction is a normalized waveform model:
```text
p_obs(t, r) =
A_s(r) * exp(-(t - t_a) / tau_s) * H(t - t_a)
+ sum_k A_k(r) * B_k(t - t_b,k)
+ epsilon(t)
```
Where:
- `t_a = r / c_w` is acoustic arrival time in water.
- `A_s(r)` is a fitted initial shock-front amplitude proxy.
- `tau_s` is a fitted decay constant.
- `B_k` are bubble-pulse basis functions.
- `epsilon(t)` is residual sensor / environment error.
Attenuation can be tracked as:
```text
A_s(r) = A_0 * G(r) * exp(-alpha_water * r)
```
Where `G(r)` is a declared geometry-spreading term, not a weapon calibration.
The bubble-motion receipt can use the Rayleigh-Plesset shape as a qualitative
dynamics gate:
```text
rho * (R * R_ddot + 3/2 * R_dot^2)
= p_b(t) - p_infty(t) - 2*sigma/R - 4*mu*R_dot/R
```
For stack use, this equation says:
```text
bubble pulse timing is a medium-response eigenmode,
not a second independent source event
```
Surface reflection can be modeled as a receipt gate:
```text
p_reflected = Gamma_boundary * p_incident
```
For a pressure-release surface, `Gamma_boundary` is expected to be negative in
the simplified acoustic model. The exact value remains a fitted receipt field.
## Eigenvalue Connection
This public benchmark should sharpen the physical-shock eigen gap found in:
```text
6-Documentation/docs/shockwave_eigenvalue_comparison_2026-05-09.md
```
Current repo state:
```text
shock alignment / relaxation exists as a local stack mode
classical hydrodynamic shock equations exist but have zero-strength support
```
The underwater benchmark can add a measured public-data bridge:
```text
Rankine-Hugoniot conservation
+ water acoustic attenuation
+ bubble-pulse eigenmode
+ boundary reflection
+ residual receipt
```
## Gate
Minimum gate:
```text
if source class is not public / archival:
HOLD_SOURCE_PROVENANCE
elif requested variable is operational weapon design:
QUARANTINE_OPERATIONAL_OPTIMIZATION
elif waveform lacks arrival/pulse/residual receipt:
HOLD_SIGNAL_RECEIPT
elif fitted residual <= declared bound:
ADMIT_PUBLIC_SHOCK_BENCHMARK
else:
HOLD_RESIDUAL_TOO_LARGE
```
## Stack Interpretation
This is the clean bridge:
```text
stellar shock breakout:
radiation escape through optical depth
underwater public shock:
acoustic escape through dense medium + bubble pulse
rain/statolith shock:
local displacement threshold in biological medium
```
All three share the same receipt grammar:
```text
impulse -> medium transfer -> boundary condition -> local witness -> residual
```
That gives the stack a free, public, non-operational benchmark for the physical
shock eigen lane.
## Next Work
1. Add a `PublicUnderwaterShockBenchmark` receipt surface.
2. Add an economic feasibility field that records why public data is the
primary lane before any lab/chamber validation.
3. Use only normalized waveform fixtures at first: arrival, relative pulse
intervals, attenuation fit, and residual.
4. Add negative controls for missing source provenance, operational-variable
requests, missing residuals, and overfit waveforms.
5. Re-run the physics eigen remapper after the benchmark exists and check
whether Detonics & Shock Physics gains a nonzero support lane.

View file

@ -0,0 +1,34 @@
# Whitespace-Zero Grammar Probe
**Date:** 2026-05-09
Zero whitespace-code grammar for canonical single-space token streams. Whitespace is reconstructed from symbol count/order. Non-canonical spacing requires an explicit residual and is not admitted by this gate.
## Rule
- Store symbol payloads.
- Store zero ordinary whitespace codes.
- Reconstruct one canonical display space between adjacent symbols.
- HOLD any non-canonical whitespace unless a residual is declared.
## Status
- Lean module: `Semantics.WhitespaceFreeGrammar`
- Lean build: `PASS`
- Probe status: `PASS_ZERO_WHITESPACE_CANONICAL`
- Admitted canonical fixtures: `3`
- HOLD fixtures needing residual: `3`
- Stored whitespace codes total: `0`
## Cases
- `canonical_1`: `ADMIT_FIXTURE`, symbols `5`, payload `42` bytes, raw `46` bytes, derived boundaries `4`, exact replay `True`
- `canonical_2`: `ADMIT_FIXTURE`, symbols `5`, payload `21` bytes, raw `25` bytes, derived boundaries `4`, exact replay `True`
- `canonical_3`: `ADMIT_FIXTURE`, symbols `6`, payload `37` bytes, raw `42` bytes, derived boundaries `5`, exact replay `True`
- `hold_1`: `HOLD_NEEDS_WHITESPACE_RESIDUAL`, symbols `4`, payload `21` bytes, raw `25` bytes, derived boundaries `3`, exact replay `False`
- `hold_2`: `HOLD_NEEDS_WHITESPACE_RESIDUAL`, symbols `4`, payload `25` bytes, raw `29` bytes, derived boundaries `3`, exact replay `False`
- `hold_3`: `HOLD_NEEDS_WHITESPACE_RESIDUAL`, symbols `3`, payload `16` bytes, raw `18` bytes, derived boundaries `2`, exact replay `False`
## Machine Receipt
- `shared-data/data/stack_solidification/whitespace_zero_grammar_probe.json`