Research-Stack/6-Documentation/docs/plans/SilverSight_completion_pipeline.md

SilverSight Completion Pipeline

Status: Design draft — to be approved before execution

Version: 0.1

Date: 2026-06-21 Ground truth: Lean 4 (/tmp/SilverSight) + Research Stack documentation (/home/allaun/Research Stack)

---

1. Current State

The SilverSight core rebas has reached the end of its Phase 0/1 boundary:

• ✅ YaFF-inspired Semantics core exists and compiles:
  • Schema.lean, Layout.lean, WireFormat.lean, View.lean, LayoutBridge.lean, CanalLayout.lean
• ✅ lake build green: 2987 jobs, 0 errors
• ✅ Q16_16 roundtrip proven across Python, C, and Lean
• ✅ Repo hygiene scripts active: glossary lint, doc sync, project map generator
• ✅ Specification drafted in 6-Documentation/docs/specs/SilverSight_Spec.md
• ⬜ Product-type encoders not yet implemented
• ⬜ DynamicCanal physics not linked to layout selection
• ⬜ Research Stack theorems not triaged or ported
• ⬜ No claim-state manifest exists yet
• ⬜ Extraction targets (Python/Rust/Verilog) not generated from Lean

This pipeline describes the shortest safe path from the current state to a completed SilverSight 1.0: a fully-Lean-grounded system with searchable concepts, verified compression receipts, substrate-extractable hardware specifications, and promotion-ready documentation.

---

2. Design Principles

1. Lean is the source of truth. Every decision, cost, gate, and receipt invariant is defined and proved in Lean. Python, Rust, and Verilog are extraction targets only.
2. No sorry in the main build. Proof holes live in quarantined modules with a TODO(lean-port) ticket and human sign-off.
3. No Float in compute paths. Q0_16 is the default scalar. Q16_16 is allowed only with a theorem-backed justification.
4. Receipts are the compressed state. A receipt is not metadata; it is the encoding. Invertibility of a receipt is the definition of lossless transformation.
5. Promotion is gated. Claims advance only through the claim-state ladder with reproducible evidence and reviewer provenance.
6. Library method. Core/ defines contracts; libraries implement them. Libraries never import other libraries.

---

3. Claim-State Ladder

Every claim, module, and receipt advances through exactly these states:

State	Meaning	Exit gate
BEAUTIFUL_PROVISIONAL	Intuition captured in a spec or stub.	Spec reviewed; module skeleton compiles.
CALIBRATED_ENGINEERING_DELTA	Implementation exists with #eval witnesses or unit tests.	lake build / py_compile / roundtrip tests pass.
REVIEWED	Independent review (LLM or human) confirms no logic leakage and no float.	Review receipt emitted and signed.
VERIFIED	A Lean theorem or hardware receipt proves the claim.	lake build green; theorem or instrument receipt present.

No claim may skip a state. Promotion out of BEAUTIFUL_PROVISIONAL requires a Lean module or spec section; out of CALIBRATED_ENGINEERING_DELTA requires a checkable artifact; out of REVIEWED requires a reviewer provenance receipt; out of VERIFIED requires a theorem or live-hardware receipt.

---

4. Phase Overview

Phase	Goal	Deliverable	Build gate
1	Finish Core Semantics	Product encoders, DynamicCanal link, Receipt/Bind	lake build SilverSightCore
2	Triage & Port Research Stack	Inventory, keep/delete/transform list, ported theorems	lake build SilverSightFormal
3	Build Search Space Corpus	Corpus250, PIST matrix builder, concept index	lake build SilverSightRRC + fixture emit
4	Port Mathematical Models	Braid/eigensolid compression, physics manifolds	lake build SilverSightFormal
5	Rewrite Shims	Pure-I/O Python scripts, no decision logic	py_compile + pytest
6	Hardware Extraction	Verilog from Lean, FPGA receipts, 1-Wire mapping	Hardware receipt + bitstream hash
7	Applications	CAD harness, LLM review emitter, dashboard wiring	End-to-end smoke tests
8	Documentation & Promotion	Specs, claim manifests, evidence DAGs	Review + signed manifest

---

5. Phase 1 — Finish Core Semantics

Goal: Make the schema/layout/wireformat stack useful for real structured objects and link it to substrate physics.

5.1 Product-Type Encodings

Implement certified WireFormat and LayoutBridge instances for fixed-size product types so that braid states, fixture rows, and receipts can be encoded without marshalling through JSON.

Core/SilverSight/Semantics/
  ProductSchema.lean      -- Schema instance for tuples / structures
  ProductWireFormat.lean  -- Row-major, columnar, compact encoders
  ProductLayoutBridge.lean-- rowMajor <-> columnar <-> compact bridges

Required theorems:

• product_encode_size — encoded product size equals sum of field sizes.
• product_roundTrip — decode after encode is identity.
• bridge_product_correct — layout bridge preserves product encoding.

Example target: BraidState (8 strands × fixed fields) encoded to a 64-byte row-major ByteArray.

5.2 DynamicCanal Integration

Move CanalRegime from a free-standing enum to a physics-backed decision:

formal/CoreFormalism/DynamicCanal/
  Regime.lean       -- CanalRegime + classifyRegime from pressure
  Lane.lean         -- Lane geometry and λ_eff(P)
  Timing.lean       -- Timing parameters derived from canal state

Link: CanalLayout.chooseLayout calls DynamicCanal.classifyRegime to override the cost-based choice when the substrate is in stressed or throat.

Required theorem: regime_override_preserves_cost_bound — a regime override never selects a layout more than a documented ε factor worse than the pure cost optimum.

5.3 Receipt and Bind Core

Implement the receipt type and the bind primitive that composes gates:

Core/SilverSight/
  Receipt.lean      -- Receipt schema, wellFormed, validation
  Bind.lean         -- bind : Gate a b -> Gate b c -> Gate a c
Core/SilverSightCore.lean  -- integrate Receipt and bind into the invariant center

Required theorems:

• Receipt.wellFormed_decidable
• bind_associative
• bind_identity_left / bind_identity_right

5.4 Fixed-Point Completion

Ensure Q0_16 is the default dimensionless scalar and Q16_16 is available with a documented reason. Remove any remaining Float from core compute paths.

Verification:

lake build SilverSightCore
lake build SilverSightFormal
python3 tests/test_q16_roundtrip.py
.lake/build/bin/q16-roundtrip

---

6. Phase 2 — Triage and Port Research Stack Theorems

Goal: Decide what survives from Research Stack and port it cleanly.

6.1 Inventory

Run the existing inventory scripts to produce a machine-readable map:

cd /home/allaun/Research Stack
python3 scripts/inventory_everything.py              # Postgres + Gremlin + repo
python3 extraction/generate_porting_candidates.py    # if available
cd /tmp/SilverSight
python3 docs/generate_research_stack_usage_map.py    # cross-reference usage
python3 docs/generate_porting_candidates.py          # produce keep/delete/transform list

Output:

• extraction/all_concepts_merged.json (Research Stack)
• docs/research_stack_triage_report.md (SilverSight)

6.2 Triage Rules

Apply the spec rules:

Action	Criterion
Keep	Compiles under lake build; has a theorem or #eval witness; no Float in compute path.
Delete	Duplicate Q16_16, demo scripts, SaaS integrations, unproven Python decision logic.
Transform	Python shims → pure I/O; markdown specs → structured sections; Rust/Verilog → extraction targets.
Quarantine	Contains sorry or unsafe and is not on the critical path; mark TODO(lean-port).

6.3 Port Order

1. Fixed-point and numeric lemmas (Semantics.FixedPoint family).
2. Sidon sets and interaction graphs (SidonSets.lean, InteractionGraphSidon.lean).
3. Braid dynamics (BraidStrand.lean, BraidCross.lean, BraidEigensolid.lean).
4. Sieve lemmas and weak-axis CRT (SieveLemmas.lean).
5. Remaining physics modules only if needed by Phase 4 compression claims.

6.4 Module Destination Map

Research Stack module	SilverSight destination	Notes
Semantics.FixedPoint.lean	Core/SilverSight/FixedPoint.lean	Already canonical.
Semantics.SidonSets.lean	formal/CoreFormalism/SidonSets.lean	Keep.
Semantics.InteractionGraphSidon.lean	formal/CoreFormalism/InteractionGraphSidon.lean	Keep.
Semantics.SieveLemmas.lean	formal/CoreFormalism/SieveLemmas.lean	Keep.
Semantics.Braid*.lean	formal/CoreFormalism/Braid*.lean	Keep.
Semantics.RRC.*.lean	formal/SilverSight/RRC/*.lean	Port selectively.
Semantics.AVMIsa.*.lean	formal/SilverSight/AVMIsa/*.lean	Already present; reconcile diffs.
4-Infrastructure/shim/*.py	python/ or delete	Strip decision logic.

6.5 Verification

lake build SilverSightFormal   # must stay green after each ported module
lake build SilverSightRRC      # must stay green after RRC ports

---

7. Phase 3 — Build Search Space Corpus

Goal: Produce the 250-equation corpus and the PIST matrix pipeline so that RRC alignment has real inputs.

7.1 Corpus250 Builder

Create or port the 250-equation fixture corpus as a pure data module:

formal/SilverSight/RRC/Corpus250.lean   -- 250 FixtureRow values
python/pist_matrix_builder.py           -- generate 8x8 strand matrices
python/build_corpus250.py               -- merge PIST labels into FixtureRows

Rules:

• Equations are raw text; no admissibility logic in Python.
• pistProxyLabel / pistExactLabel are populated only when PIST predictions exist.
• Alignment status remains missingPrediction until labels are present.

7.2 Concept Index

Generate a searchable concept index:

python/inventory_concepts.py
  → docs/concepts.json
  → docs/concepts.md

Each concept gets a stable ID:

silversight_<type>_<source_hash>_<name_hash>

7.3 RRC Emit Fixture Executable

Ensure exe/RrcEmitFixture.lean emits the full corpus as AVM-stamped JSON:

lake build rrc-emit-fixture
.lake/build/bin/rrc-emit-fixture > /tmp/rrc_fixture_emitted.json
python3 python/validate_rrc_predictions.py /tmp/rrc_fixture_emitted.json

Verification:

lake build SilverSightRRC
lake build rrc-emit-fixture
python3 -m py_compile python/*.py
python3 python/validate_rrc_predictions.py /tmp/rrc_fixture_emitted.json

---

8. Phase 4 — Port Mathematical Models

Goal: Prove the compression and physics claims that justify SilverSight.

8.1 Braid / Eigensolid Compression

Port the braid dynamics and prove the two compressor theorems:

formal/CoreFormalism/
  BraidEigensolid.lean      -- eigensolid_convergence
  BraidReceipt.lean         -- receipt_invertible

Required theorems:

• eigensolid_convergence — crossStep reaches a fixed point.
• receipt_invertible — given the receipt, the original state is reconstructible within bounded error, including gaps/timing/absence.

8.2 Physics Models (Optional)

Only port physics manifolds if they feed a receipt claim:

• ManifoldStructures.lean / CausalGeometry.lean / ExoticSpacetime.lean
• Migrate to canonical Semantics.FixedPoint.Q16_16 using the PhysicsScalarBridge pattern.

Verification:

lake build SilverSightFormal

---

9. Phase 5 — Rewrite Shims

Goal: Convert all Python scripts to pure I/O wrappers.

9.1 Decision-Logic Removal

For every Python file in python/ and qubo/:

1. Identify functions that branch on admissibility, cost, or classification.
2. Move the decision to Lean.
3. Replace Python logic with:
   • JSON read/write
   • subprocess call to lake exe <gate>
   • result formatting

9.2 New/Updated Shims

python/
  q16_canonical.py              # reference implementation (already pure)
  sidon_address.py              # raw Sidon label generation
  pist_matrix_builder.py        # raw 8x8 counts
  build_corpus250.py            # merge only; no alignment logic
  validate_rrc_predictions.py   # schema check only
  emit_review_receipt.py        # call Lean emitter, store JSON
  inventory_concepts.py         # generate concept index
qubo/
  qubo_builder.py               # format QUBO matrix; solver call
  finsler_metric.py             # format metric; no routing decision
  qaoa_adapter.py               # format adapter inputs

9.3 Verification

python3 -m py_compile python/*.py qubo/*.py tests/*.py
python3 -m pytest tests/          # if pytest available

---

10. Phase 6 — Hardware Extraction

Goal: Generate substrate implementations from Lean and collect hardware receipts.

10.1 1-Wire Mapping

Formalize the 1-Wire trit VM in Lean:

formal/CoreFormalism/DynamicCanal/
  LanePayload.lean      -- trit VM state
  OneWire.lean          -- pulse ↔ trit ↔ canal regime mapping

10.2 Verilog Extraction

For combinational circuits derived from Lean defs, generate Verilog:

hardware/
  verilog/
    braid_cross_step.v    -- generated from BraidCross.crossStep
    q16_add_sat.v         -- generated from FixedPoint.add

Rule: Verilog is an extraction target. The authoritative definition remains in Lean.

10.3 FPGA Receipts

Burn a bitstream and collect a hardware receipt:

{
  "schema": "hardware_fpga_receipt_v1",
  "bitstream_hash": "sha256:...",
  "uart_beacon": "...",
  "continuous_state": "...",
  "gate": "q16_add_sat",
  "result": "passed"
}

Verification:

• Bitstream hash matches generated Verilog source hash.
• UART beacon observed.
• Continuous state verification (not just exit code).

---

11. Phase 7 — Applications

Goal: Wire user-facing tools to the verified core without leaking logic.

11.1 CAD Harness

• Text-to-CAD pipeline calls Lean geometry gate.
• Python only formats inputs and renders outputs.

11.2 LLM Review Emitter

• python/emit_review_receipt.py calls the canonical review emitter.
• Review receipts include answer_sha256 and are verifiable.

11.3 Dashboard

• Hermes dashboard displays receipt state, build status, and concept index.
• No decision logic in the dashboard.

---

12. Phase 8 — Documentation and Promotion

Goal: Produce the final evidence package and promote claims to VERIFIED.

12.1 Spec Completion

Finalize:

• 6-Documentation/docs/specs/SilverSight_Spec.md
• docs/ARCHITECTURE.md
• docs/TESTING.md
• docs/GLOSSARY.md

12.2 Claim-State Manifest

Create a machine-readable manifest:

docs/claims/
  manifest_v1.json       -- every claim, current state, evidence URLs
  manifest_v1.md         -- human-readable summary

Each entry:

{
  "claim_id": "silversight_claim_eigensolid_convergence",
  "state": "VERIFIED",
  "evidence": ["formal/CoreFormalism/BraidEigensolid.lean:theorem eigensolid_convergence"],
  "reviewer": "...",
  "promoted_at": "..."
}

12.3 Evidence DAG

Build a directed acyclic graph of evidence:

docs/claims/evidence_dag.dot

Each node is a theorem, receipt, or review; edges are dependency arrows.

12.4 Final Review

• Independent review of all REVIEWED claims.
• Signed review receipts.
• Promotion to VERIFIED only after review.

---

13. Verification Gates by Phase

Phase	Lean build	Python	Receipt / hardware	Review
1	lake build green	py_compile	N/A	Spec review
2	lake build SilverSightFormal green	N/A	N/A	Triage report
3	lake build SilverSightRRC + fixture emit	validate_rrc_predictions.py	AVM-stamped JSON	Corpus review
4	lake build SilverSightFormal	N/A	Compression receipts	Theorem review
5	N/A	py_compile + pytest	N/A	Shim audit
6	Verilog generated	N/A	FPGA receipt	Hardware review
7	N/A	smoke tests	Review receipts	UX review
8	N/A	N/A	Claim manifest	Final sign-off

---

14. Continuous Integration

The existing GitHub workflows must cover the full pipeline:

.github/workflows/
  lean-check.yml          -- lake build on push
  python-check.yml        -- py_compile + pytest
  doc-sync.yml            -- glossary lint + check_doc_sync
  q16-roundtrip.yml       -- Python/C/Lean roundtrip
  rrc-emit-check.yml      -- emit fixture + validate
  hardware-receipt.yml    -- check FPGA receipts (manual trigger)

---

15. Risks and Mitigations

Risk	Mitigation
Research Stack modules have sorry	Quarantine; do not block main build.
Duplicate Q16_16 definitions	Unify to Semantics.FixedPoint.Q16_16; bridge old code.
Python decision logic leaks back	Shim audit checklist; enforce via python/check_for_decision_logic.py.
Hardware receipts cannot be collected	Document software witness vs. live-hardware distinction; keep claims bounded.
Mathlib version drift	Pin toolchain in lean-toolchain; vendor .lake if necessary.
Claim promotion by hand	Manifest is the only promotion authority; manual edits require reviewer signature.

---

16. Open Questions to Resolve Before Execution

1. Should the 250-equation corpus be regenerated from Research Stack data or rewritten for SilverSight?
2. Is AVM rebuilt in Phase 1, or does the existing AVM ISA remain canonical?
3. Which physics modules are required for 1.0 vs. deferred to 1.1?
4. Which FPGA board is the hardware extraction target?
5. Who are the human reviewers for the REVIEWED → VERIFIED transitions?

---

17. Immediate Next Steps

If this pipeline is approved, the first executable batch is:

1. Phase 1.1 — Implement ProductSchema.lean, ProductWireFormat.lean, and ProductLayoutBridge.lean with a BraidState instance.
2. Phase 1.2 — Link CanalLayout.chooseLayout to CoreFormalism.DynamicCanal.classifyRegime.
3. Phase 1.3 — Implement Core/SilverSight/Receipt.lean and Core/SilverSight/Bind.lean.
4. Phase 2.1 — Run Research Stack inventory and produce docs/research_stack_triage_report.md.

Each step is a separate commit with its own build baseline and claim-state update.

---

18. Acceptance Criteria for SilverSight 1.0

• lake build passes with zero errors across SilverSightCore, SilverSightFormal, and SilverSightRRC.
• No sorry in the main import path.
• No Float in core compute paths.
• All Python shims pass py_compile and contain no admissibility logic.
• The 250-equation RRC corpus emits AVM-stamped JSON that validates.
• Braid compression has eigensolid_convergence and receipt_invertible theorems.
• At least one hardware receipt (FPGA or 1-Wire) is present with bitstream hash and continuous-state evidence.
• A claim-state manifest exists with every in-scope claim at VERIFIED.
• All documentation is in sync: glossary_lint.py and check_doc_sync.py pass.