Research-Stack/4-Infrastructure/AGENTS.md
allaun 0149a5c93d feat(infra): add MCP-only Drive GCCL compression shim
Adds 4-Infrastructure/shim/mcp_drive_gccl_compress.py:
- downloads a Google Drive folder via @piotr-agier/google-drive-mcp
- GCCL-delta-compresses contents into .tar.zst + JSON manifest
- uploads archive + manifest back to Drive
- optionally trashes the original folder
- handles MCP listFolder pagination

Updates 4-Infrastructure/AGENTS.md with the new shim anchor.

No rclone or egress-heavy operations; all Drive I/O goes through MCP.
2026-06-22 01:11:17 -05:00

28 KiB
Raw Blame History

AGENTS.md - Infrastructure And Hardware

Scope: 4-Infrastructure/

Repository Extraction Notice (2026-06-02)

The following infrastructure components have been extracted to distributed-compute-fabric repository:

  • kubernetes/ - Kubernetes manifests and configurations
  • k3s-flake/ - K3s flake configurations
  • kube/ - Additional Kubernetes configurations
  • shim/ray-actors/ - Ray actor implementations
  • shim/vcn_*.py - VCN compute substrate shims
  • shim/hermes/ - Hermes orchestration layer
  • shim/vectorless_spatial_hash_backend*.py - Spatial hash backends
  • docs/vcn-lupine-setup.md - VCN setup documentation
  • docs/mesh-networking-over-ray-plan.md - Ray networking documentation

This AGENTS.md now covers only the remaining infrastructure components: hardware bring-up, storage stack, cloudflare edge, and non-VCN shims.

Rules

  • Keep infrastructure scripts receipt-bearing: every probe should have a machine-readable output or update an existing receipt.
  • Separate software witnesses from live hardware witnesses.
  • Do not claim FPGA acceleration from bitstream generation alone.
  • Do not claim UART/fabric success without observed bytes or a matching hardware receipt.
  • Treat /usr/bin/sem as GNU Parallel on this machine unless proven otherwise; use the isolated sem binary documented in stack solidification receipts when needed.
  • Remote model/API probes must be secret-clean. Read provider credentials from environment variables only (OLLAMA_API_KEY, DEEPSEEK_API_KEY, etc.); never embed literal keys in scripts, receipts, prompts, or docs.
  • LLM/model outputs are reviewer receipts, not validation. If a model review is promoted, store the answer and a machine-readable receipt with prompt/answer hashes under shared-data/artifacts/, and state which files formed the context.

Preferred Checks

python3 -m py_compile 4-Infrastructure/shim/<script>.py
python3 -m json.tool <receipt>.json >/dev/null

For API-facing or receipt-writing scripts, also run a touched-file secret scan before staging. Treat the repository credential hook as a backstop, not the first detector.

For Tang Nano 9K work, keep the boundaries explicit:

  • bitstream present: Compiled tangnano9k_uart_beacon.fs and tangnano9k_uart_loopback.fs via build_uart_beacon.sh and build_loopback.sh.
  • SRAM load: Loaded tangnano9k_uart_beacon.fs to SRAM using sudo openFPGALoader -b tangnano9k tangnano9k_uart_beacon.fs (CRC check: Success).
  • flash persistence: Pending.
  • UART beacon: Tested onboard BL702 bridge; physical UART route is blocked due to bridge firmware limitations (documented in 6-Documentation/docs/fpga_uart_route_analysis_2026-05-09.md). Custom virtual serial transport virtual://q16-pty acts as active verification path.
  • Q16/software witness: Verified.
  • Q16/live hardware witness: Requires external USB-UART adapter connected to pins 17/18.

Storage Stack: restic + Garage + rclone

Three tools, three distinct jobs — no overlap:

Tool Job Does NOT do
restic Deduplicated, encrypted, content-addressed snapshots. Point-in-time restore. Verifiable integrity. Raw sync, remotes management
Garage Self-hosted S3-compatible object store across Tailscale nodes. restic's primary backend. Dedup, encryption, scheduling
rclone Moves raw objects between remotes (Garage↔gdrive, gdrive↔Garage). Cold-copy of restic chunks to gdrive. Dedup, encryption, snapshots

Data flow

git commit
    └─(post-commit hook, async)─▶ restic snap ──────────────────▶ Garage:research-stack
                                   restic snap-db (SQLite)         (deduplicated, encrypted)
                                   restic snap-rds (pg_dump|zstd)  │
                                                                    │
Daily 03:00 (systemd timer)                                        │
    ├─ rclone copy ─────────────────────────────────────────────── ▶ gdrive:restic-mirror/
    │  (cold copy of restic chunks — survive Garage loss)
    └─ rclone sync ─── gdrive:research-stack ──▶ Garage:gdrive-mirror
                        (S3-native access to gdrive data)

Restore path A (Garage up):
    restic restore <id> -r s3:http://localhost:3900/research-stack

Restore path B (Garage down, gdrive available):
    restic restore <id> -r rclone:gdrive:restic-mirror

Garage S3 — node topology (Tailscale mesh)

All object storage for this stack uses Garage v2.3.0 — a single-binary, Dynamo-style S3-compatible store written in Rust. Replaced rclone serve s3.

Node topology (Tailscale mesh)

Node Tailscale IP k3s Garage Zone Disk SSH
qfox-1 (this machine) 100.88.57.96 worker 780 GiB local 1.8 TB NVMe local
cupfox 100.72.130.76 control-plane 69 GiB fra 125 GB key OK (361395)
nixos-laptop 100.102.173.61 worker 347 GiB ord 459 GB NVMe key OK
racknerd 100.80.39.40 worker 954 MiB vps 9.1 GB VPS key OK
neon-64gb 100.92.88.64 rebuilt (standalone k3s) netcup-arm 2 TB key OK (allaun)
steamdeck 100.85.244.73 worker 373 GiB gpu 476 GB NVMe key OK
rs-vps (netcup) 2 TB SSH via password
dracocomp 100.100.140.27 unreachable
  • RPC port: 3901 (Tailscale-only, not exposed to internet)
  • S3 API port: 3900 (qfox-1 only; other nodes bind loopback)
  • Admin API port: 3903 (loopback only on all nodes)

Garage S3 buckets

Bucket Purpose
research-stack Primary project objects
db-scratch Active SQLite scratch databases
rds-overflow pg_dump / COPY TO exports from Aurora RDS
snap-zone ZFS send/receive snapshots
gdrive-mirror Mirror of gdrive:research-stack

Credentials

Credentials live in /etc/garage/garage.env (mode 600, never committed). Sourced automatically by all storage scripts. Inside devcontainer, set:

source /etc/garage/garage.env
export AWS_ACCESS_KEY_ID=$GARAGE_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$GARAGE_SECRET_ACCESS_KEY
export AWS_ENDPOINT_URL=http://host.containers.internal:3900
export AWS_DEFAULT_REGION=garage

Restic scripts (4-Infrastructure/storage/restic/)

Script Purpose
restic.env Source this to load all credentials + repo paths
backup.sh snap [tag] Snapshot repo tree → Garage
backup.sh snap-db [dir] Snapshot SQLite scratch DBs → Garage
backup.sh snap-rds <table> Stream pg_dump | zstd → restic stdin → Garage
backup.sh cold-copy rclone copy Garage:research-stack → gdrive:restic-mirror
backup.sh sync-gdrive rclone sync gdrive:research-stack → Garage:gdrive-mirror
backup.sh forget Apply retention (7 daily / 4 weekly / 6 monthly) + prune
backup.sh verify restic check --read-data-subset=5%
backup.sh snapshots List all snapshots
backup.sh restore <id> <dst> Restore a snapshot
backup.sh full snap + cold-copy + sync-gdrive + forget

Restic repo: s3:http://localhost:3900/research-stack (Garage primary) Cold copy: rclone:gdrive:restic-mirror (survives Garage loss) Password: /etc/garage/restic-password (chmod 644, not committed)

Daily timer: restic-backup.timer fires at 03:00 ±30 min, runs backup.sh full.

Garage scripts (4-Infrastructure/storage/garage/)

Script Purpose
zfs-pool-setup.sh Create ZFS pool on local NVMe (run after reboot into 7.0.9 kernel)
garage-node-bootstrap.sh <ip> Install Garage on a new node, register in node-registry.json
garage-cluster-init.sh Connect nodes, assign layout, bump replication_factor to 3
db-consolidate.sh offload [dir] Push SQLite DBs → s3://db-scratch/
db-consolidate.sh rds-dump <table> Dump RDS table → s3://rds-overflow/
db-consolidate.sh consolidate Restore static s3://rds-overflow/ objects → RDS
db-consolidate.sh sync-gdrive Mirror gdrive:research-stack → s3://gdrive-mirror/
db-consolidate.sh status Show cluster and bucket state

Replication status

replication_factor = 3, zone redundancy enforced, 6 nodes across 6 zones. ~1.3 TiB total capacity, ~440 GiB effective (RF3).

Add a new Garage node:

bash 4-Infrastructure/storage/garage/garage-node-bootstrap.sh <tailscale_ip>

Git post-commit hook

.git/hooks/post-commit automatically runs db-consolidate.sh offload + db-consolidate.sh consolidate in the background after every commit. Non-blocking. Skipped silently if Garage isn't running. Log at ~/.cache/garage-post-commit.log.

gdrive integration

gdrive is still mounted via rclone at /home/allaun/gdrive for direct file access. The db-consolidate.sh sync-gdrive command mirrors it into the gdrive-mirror Garage bucket for S3-native access without hitting Drive API quotas on every read.

Drive API safe-use rules still apply to any direct rclone → gdrive operations:

  • --drive-pacer-min-sleep 200ms — ≤5 TPS sustained
  • --drive-pacer-burst 10 — limits burst before pacer beats
  • --dir-cache-time 10m — warm cache = zero API calls

Storage Agent (4-Infrastructure/storage/storage_agent.py)

Full-loop observer, optimizer, and actor for the restic + Garage + rclone stack.

Design contract

Shim boundary (per AGENTS.md §7.1):

  • ALLOWED: subprocess calls to existing CLI tools (backup.sh, db-consolidate.sh, restic, aws s3), JSON I/O, JSONL log, receipt assembly, threshold comparisons on Q16_16-encoded integers.
  • FORBIDDEN: reimplementing restic/Garage/rclone logic, Float arithmetic, cost functions (those belong in Lean), new external dependencies.

Observe → Decide → Act → Emit loop

Observation          Decision              ActionResult
─────────────────    ──────────────────    ──────────────────
garage_up            trigger_snap          actions_attempted
garage_buckets       trigger_cold_copy     actions_succeeded
restic_snapshot_count trigger_verify       actions_failed
dedup_ratio_q16      trigger_forget        details
backup_log_last_ok   trigger_offload
cold_copy_needed     trigger_garage_restart
errors               alerts / rationale

All numeric thresholds are Q16_16 (UInt32, one = 0x00010000 = 65536). No Float arithmetic. Threshold constants defined at the top of the file.

Receipt schema: storage_agent_receipt_v1

Every cycle emits one receipt:

{
  "schema":           "storage_agent_receipt_v1",
  "version":          "1.0.0",
  "generated_at_utc": "<ISO-8601>",
  "tick":             <int>,
  "parent_hash":      "<sha256 of previous receipt>",
  "observation":      { ... },
  "decision":         { ... },
  "action_result":    { ... },
  "claim_boundary":   "storage-agent-observe-decide-act-only",
  "receipt_hash":     "<sha256 of canonical preimage>"
}

Two sinks:

  1. Local JSONL hash-chain: ~/.cache/storage-agent.jsonl — fast local access, survives Garage loss.
  2. Garage S3: s3://research-stack/agent-receipts/<date>/<hash16>.json — durable, indexed by date.

Trigger model

Trigger Condition Action
trigger_snap No snapshots, or backup log shows no recent success backup.sh snap agent-triggered
trigger_cold_copy Newest snapshot > 26 h old (daily timer may have missed) backup.sh cold-copy
trigger_verify dedup_ratio_q16 < 0.3 AND snapshot_count > 5 backup.sh verify
trigger_forget snapshot_count > 30 backup.sh forget (prune)
trigger_offload Garage is up (idempotent) db-consolidate.sh offload
trigger_garage_restart Garage unreachable systemctl restart garage.service

Systemd units

File Purpose
storage-agent.service One-shot service (Type=oneshot, User=allaun)
storage-agent.timer Fires every 15 min (OnCalendar=*:0/15, RandomizedDelaySec=60)

Installation:

sudo cp 4-Infrastructure/storage/storage-agent.{service,timer} /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now storage-agent.timer

Post-commit integration

.git/hooks/post-commit runs the agent (--once) in the background after each backup.sh snap completes. The agent observes the post-snap state and takes any follow-on corrective actions (offload, cold-copy if stale, etc.). Log at ~/.cache/storage-agent.log.

Usage

# One-shot probe (observe only, no actions)
python3 4-Infrastructure/storage/storage_agent.py --probe-only

# One-shot, full loop
python3 4-Infrastructure/storage/storage_agent.py --once

# Show what would be done
python3 4-Infrastructure/storage/storage_agent.py --dry-run

# Skip S3 receipt upload (local JSONL only)
python3 4-Infrastructure/storage/storage_agent.py --no-s3

# Daemon mode (rarely needed; systemd timer is preferred)
python3 4-Infrastructure/storage/storage_agent.py --loop --interval 900

Log paths

Path Contents
~/.cache/storage-agent.jsonl Hash-chained JSONL receipt log
~/.cache/storage-agent.log Human-readable stdout/stderr from systemd and hook runs
s3://research-stack/agent-receipts/ Durable S3 receipts (Garage)

Current Stack-Solidification Anchors

  • 4-Infrastructure/shim/mcp_drive_gccl_compress.py — MCP-only Google Drive folder compressor: downloads a Drive folder via @piotr-agier/google-drive-mcp, GCCL-delta-compresses contents into a .tar.zst archive with JSON manifest, uploads archive + manifest back to Drive, optionally trashes the original. No rclone.
  • 4-Infrastructure/shim/rrc_arxiv_kernel_refine.py — RRC arXiv kernel refinement: title+abstract keyword search against arxiv_papers for unmatched equations
  • 4-Infrastructure/infra/lean_lsp_mcp_wrapper.py — Python wrapper for lean-lsp-mcp to fix the schema of lean_diagnostic_messages for compatibility with strict validators like Moonshot/Kimi API.
  • 4-Infrastructure/shim/stack_solidification_audit.py
  • 4-Infrastructure/shim/stack_fail_closure_register.py
  • 4-Infrastructure/shim/beaver_mask_freshness_negative_controls.py
  • 4-Infrastructure/shim/tang9k_uart_beacon_probe.py
  • 4-Infrastructure/shim/hutter_jxl_starfield_eigenprobe.py
  • 4-Infrastructure/shim/hutter_jxl_starfield_replay_verify.py
  • 4-Infrastructure/shim/braid_diat_codec.py — Python extraction of BraidDiatCodec (ChiralityDIAT + MountainPacked + BraidResidual + BraidDiatFrame); benchmark artifact at shared-data/artifacts/braid_diat_codec_benchmark.json
  • 4-Infrastructure/shim/spirv_copy_if_optimizer.py — SPIR-V OpPhi→OpSelect transform (OpBranchConditional+OpPhi → OpSelect); eliminates branch over head; emits CopyIfPattern with type_id fix
  • 4-Infrastructure/shim/spirv_packet_generator.py — OpPhi-driven packet descriptor generator: SPIR-V asm → copy-if optimizer → JSON packet descriptors (5 OpPhi fields: type_id, cond_id, true_val_id, false_val_id, result_id)
  • 4-Infrastructure/shim/virtio_net_transform.py — Virtio-net ring as computation pipeline: three Class-1 primitives (HASH_REPORT RSS Toeplitz, TSO gso_size split, MRG_RXBUF merge) via virtio_net_hdr_v1_hash; zero backend changes needed
  • 4-Infrastructure/shim/vcn_compute_substrate.py — AMD VCN / NVIDIA NVENC H.264/H.265 hardware video encoder as compute device via MKV trick; dynamically detects GPU vendor (NVIDIA/AMD/Intel) to load math-optimized lossless parameters (YUV444p10le, full-range PC spacing, CABAC offload, VAAPI/NVENC/AMF wrappers); carries BraidStrand/BraidBracket payloads; vectorized packing
  • 4-Infrastructure/shim/qemu_framebuffer_packer.py — QEMU graphics framebuffer packer mapping Q16_16 scalars to ARGB8888/RGB24 pixels for mmap-based zero-copy display DMA loopback
  • 4-Infrastructure/shim/rrc_ray_tagger.py — RRC Ray Layer Tagger; classifies math payloads into RRC shapes and matches them to swappable compute slots and transports
  • 4-Infrastructure/shim/gccl_transfer_pipeline.py — GCCL-gated parallel transfer pipeline wrapper; integrates WaveProbe sampling, MetaProbe validation, and Delta+RLE encoding for secure transport
  • 4-Infrastructure/shim/braid_mutation_optimizer.py — Multi-core parallel genetic algorithm optimizer for State8 braid crossing structures; outputs optimization receipts
  • 4-Infrastructure/shim/burgers_2d_simplification.py — 2D Burgers equation multi-core solver with Helmholtz solenoidal/dilatational energy decay analysis
  • 4-Infrastructure/shim/erdos_discrepancy_probe.py — Multi-core homogeneous arithmetic progression discrepancy scanner
  • 4-Infrastructure/shim/entropic_collision_prober.py — Entropic collision deficit and sumset prober for Sidon sets
  • 4-Infrastructure/shim/quandela_erdos_search.py — Photonic quantum simulation optimizer using Perceval SDK to shave combinatorial complexity
  • 4-Infrastructure/shim/openai_unit_distance_verifier.py — Euclidean unit-distance density verifier for planar configurations
  • 4-Infrastructure/shim/galois_orbit_trimmer.py — Galois Orbit Trimming (DST) conjugation symmetry-based search trimmer
  • 4-Infrastructure/shim/braid_shock_16d.py — 16D BraidShock simulation prototype with Dimensional Shock Trim (DST) and underverse bleed ledger tracking
  • 4-Infrastructure/shim/verify_all_shims.py — Master verification pipeline running all 7 simulation shims sequentially on EPYC
  • 4-Infrastructure/shim/verify_ene_schema.py — Verifies the presence and structure of Braid Eigensolid Compressor tables in ENE schema and writes validation receipt
  • 4-Infrastructure/shim/wolfram_verify.py — Queries Wolfram Alpha API to verify algebraic/physical equations
  • 4-Infrastructure/shim/ingest_eigensolid_data.py — Database integration shim for eigensolid crossing weights, snapshots, and braid strands in pure Q16_16 fixed-point format
  • 4-Infrastructure/shim/eigensolid_lean_bridge.py — Lean-to-Postgres bridge for BraidEigensolid: executes Lean evaluations, extracts Q16_16 coordinates/crossing weights from #eval witnesses, seeds Sidon labels (powers of 2), binds verifier identities to ene.prover_instances for audit trails
  • 4-Infrastructure/shim/geometric_entropy_explorer.py — Entropy exploration candidate generator for RRC: places 8 braid strands on torus/sphere/cube, maximizes Shannon entropy of pairwise-distance distribution via gradient descent, exports candidate BraidReceipt JSON. Exploration phase only — no gating decisions.
  • 4-Infrastructure/shim/candidate_certification_bridge.py — Bridge from entropy exploration → Lean certification pipeline: reads candidate JSON files, generates Candidates.lean with BraidState fixtures (Fin 8 → BraidStrand lambdas), allCandidates list, and verifyAllCandidates function that runs crossStep + IsEigensolid check.
  • 0-Core-Formalism/lean/Semantics/Semantics/RRC/EntropyCandidates/Candidates.lean — Auto-generated Lean candidate file from entropy exploration runs. Built by candidate_certification_bridge.py. Contains ranked BraidState definitions sorted by final entropy. Certified by crossStepeigensolid_convergencereceipt_invertible theorem chain.
  • shared-data/data/stack_solidification/candidates/ — Generated candidate JSON files + batch manifests from geometric entropy explorer. Per-batch directories with manifest.json ranking by entropy.
  • 6-Documentation/docs/specs/DP_RRC_RECEIPT_ENCODING_SPEC.md — Depth-prefix receipt encoding spec for RRC. Maps dot-prefixed depth markers (dp-expr) to Sidon labels, structural tokens to scar absence (∅), and defines DP-RRC ↔ JSON translation. Design proposal.
  • 4-Infrastructure/cloudflare/src/lib.rs — Cloudflare Workers edge WASM trinary VM core implementing the Q0_16 scalar compute floor
  • 4-Infrastructure/cloudflare/src/index.js — Cloudflare Workers entry point, POST-only, JSON + binary protocol
  • 4-Infrastructure/cloudflare/wrangler.toml — Wrangler config, deployed at https://wasm-compute-edge.researchstack.workers.dev
  • 4-Infrastructure/hardware/emergency_boot/emergency_boot_shim.py — Python I/O shim for Geometry Emergency Boot Witness (6502 calculator-efficiency FPGA controller) Specification: 6-Documentation/docs/specs/GEOMETRY_EMERGENCY_BOOT_WITNESS_2026-04-08.md
  • 4-Infrastructure/surface/main.py — Topological FastAPI surface server with WebSocket telemetry hooks and /api/nuvmap projection endpoint
  • 4-Infrastructure/surface/static/index.html — Sovereign Surface UI displaying the dynamic NUVMAP projection grid and metrics dashboard
  • 4-Infrastructure/shim/burgers_chaos_game.py — Continuous-time quantum walk over the Burgers representation graph (22 representations, adjacency from proven codebase isomorphisms). Confirms the 0D DualQuat Braid as the universal hub by eigenvector centrality, quantum walk probability, and degree (all rank #1). Receipt: burgers_chaos_game_receipt.json.
  • 4-Infrastructure/shim/rrc_domain_manifold_graph.py — Build expanding manifold graph across all gathered math domains. Produces 6 edge types (shared_paper, manifold route, regime, shared_topic, domain adapter, dimension chain) plus 4 intrinsic fallback types (kernel_internal, kernel_hub, topic_overlap, unverified shared_paper) that ensure dense output (~0.61 density) even when live arxiv DB access is unavailable.
  • 4-Infrastructure/shim/rrc_refactor_oracle.py — Chaos-game-driven RRC self-refactoring oracle. Reads the domain manifold graph, runs BurgersChaosGame quantum walk eigenvector centrality, generates PRUNE/MERGE/PROMOTE/SPLIT commands from centrality deltas. When graph density < 0.005, attempts live rebuild via rrc_domain_manifold_graph.py, then falls back to intrinsic edge inference from node metadata. --no-live flag forces fallback for containerized deployments. Receipt: rrc_refactor_oracle_receipt.json. Canonical configuration from SLO sweep: --max-merges 10 --threshold-prune 0.005 --threshold-merge 0.01 --max-iterations 5.
  • 4-Infrastructure/shim/rrc_slo_analyzer.py — SLO analyzer for the refactoring oracle. Measures structural SLOs (spectral_gap, modularity, conductance, isolation_ratio, centrality_spread, edge_efficiency, community_count) and performance SLOs (adj_build_ms, eigenvector_ms, evolution_ms, total_ms). Compares baseline vs target graph. Receipt: rrc_slo_receipt.json.
  • 4-Infrastructure/shim/pist_trace_classify_offline.py — Pure-I/O shim for offline proof-trace classification. Reads trace JSON, forwards it to the pist-classify-trace Lean executable for spectral feature extraction and shape/tactic decisions, and optionally invokes rrc-watchdog for alignment. Contains no classification logic, no spectral arithmetic, and no floating-point compute path.
  • 4-Infrastructure/shim/rrc_slo_sweep.py — Parameter sweep over merge aggressiveness (max_merges ∈ {20,10,5,2,1}). Composite score: speedup × community_retention × node_retention × (1 + mod_gain) × (1 - cent_spread_loss). Found optimal at max_merges=10: 3.1× speedup, 71% community retention, 100% isolation elimination. Receipt: rrc_slo_sweep_receipt.json.
  • 4-Infrastructure/shim/rrc_bosonic_tensor_gpu.py — GPU-accelerated Bosonic Tensor Network Centrality: computes exp(-iAtheta) using adaptive RK4 on GPU with wgpu (Vulkan) to scale N to 10000+ without CPU eigendecomposition bottleneck. Receipt: rrc_bosonic_tensor_gpu_receipt.json.
  • 4-Infrastructure/shim/rrc_bosonic_db_buffer.py — Asynchronous database ingestion buffer: queues, batches, and flushes PostgreSQL inserts for bosonic tensor network receipts and metrics in thread-safe worker pools.
  • 4-Infrastructure/shim/pist_trace_classify_offline.py — Offline, token-free trace classifier implementing Lean's Semantics.PIST.Classify color-space model and executing the local rrc-watchdog binary in the container.

Compute Dispatch (WGSL → any substrate)

All compute shaders live as WGSL source. Dispatch follows a single pattern:

RDS SELECT (input strands, weights) → wgpu SSBO → WGSL compute → readback → RDS INSERT

The wgpu Rust dispatch (pattern: 5-Applications/parquet_compressor/src/gpu.rs) probes the adapter and chooses the best available backend transparently:

Adapter probe: └── Vulkan → GPU (discrete or integrated) └── Vulkan (lavapipe/SwiftShader) → CPU blitter (L1 cache, ~112 ops/step) └── WebGPU (WASM) → Browser GPU or WASM CPU fallback

The algorithm is always WGSL. The dispatch is always wgpu. The backend is transparent. No path specialization is needed because Q16_16 integer arithmetic is deterministic across all substrates.

Known dispatch entry points:

  • 5-Applications/parquet_compressor/src/gpu.rs — Rust wgpu compute + XOR/S-box
  • 5-Applications/scripts/rgflow_gpu_pipeline.py — Python wgpu with Vulkan backend
  • 4-Infrastructure/gpu/wasmgpu/ — TypeScript WebGPU engine with 47 WGSL shaders
  • 4-Infrastructure/shim/erdos_surface_orchestrator/src/main.rs — WGSL generator

For the braid eigensolid compressor, the dispatch is planned at: 4-Infrastructure/shim/braid_blitter/ (Rust, following parquet_compressor/src/gpu.rs)

ENE schema additions for DSP volunteer computing (PipeWire/FLAC)

Any Linux node with PipeWire can act as a DSP compute worker regardless of physical audio hardware. A virtual sound card is created via PipeWire, exposing FLAC audio chunks as compute workloads.

  • ene.dsp_nodes — PipeWire/FLAC DSP node capabilities: node_id, dsp_available, pipewire_available, virtual_soundcard_supported, physical_soundcard, max_sample_rate, spectral_bands, latency_target_us, fft_size, overlap_factor, last_seen_at, receipt_hash
  • Dispatch: FLAC chunks in MKV audio track → routed to DSP-capable nodes → results returned via separate reply channel
  • Shim: 4-Infrastructure/shim/flac_dsp_node.py — node registration, PipeWire probe, FLAC chunk spectral analysis (FFT peaks, spectral centroid, RMS level)
  • Receipt: every DSP operation writes to ~/.cache/flac_dsp_receipts.jsonl

ENE schema additions for braid eigensolid compressor (created)

These tables extend ene_substrate_schema.sql to support the Braid Eigensolid Compressor:

  • ene.prover_state — Lean theorem registry: theorem_id, name, statement, formalization_status (raw/forming/proven/verified), signature_hash, dependencies
  • ene.prover_instances — Concrete theorem evaluations/fixtures: instance_id, theorem_id, input_fixture, evaluation_witness, verifier_identity, verified_at, receipt_hash
  • ene.sidon_labels — Power-of-2 strand address maps: label_set_name, strand_index, label_value, sidon_slack
  • ene.crossing_weights — Braid crossing weights (fixed-point integer Q16_16) with row sum/value range checks
  • ene.eigensolid_snapshots — Converged stable state snapshots: package_id, receipt_id, step_count, convergence_metric, entropy, is_stable
  • ene.braid_strands — Strand coordinates (x/y phase vectors in Q16_16 format): snapshot_id, strand_index, phase_x, phase_y
  • ene.receipts ADD theorem_id — FK to prover_state
  • ene.receipts ADD dispatch_path — tracks backend dispatch channel (e.g. vulkan_gpu, cpu_blitter)

Verification is driven by 4-Infrastructure/shim/verify_ene_schema.py, which validates the schema layout and emits shared-data/data/stack_solidification/ene_schema_validation_receipt.json.

Cross-References

See root AGENTS.md for:

  • Post-Interaction Workflow (mandatory 5-step session-end procedure)
  • Programming Choice Flow (Lean owns decisions; Python owns I/O — shims must not contain decision/gating/scoring logic)
  • Do Not Sweep rules (no broad git add .)
  • Git Remote Hygiene