Commit graph

34 commits

Author SHA1 Message Date
762c88057a docs(infra): document build-log automation in AGENTS.md
Adds a Build-log automation section noting the post-commit hook runs lake_build_ingest.py Compiler in the background, inserting into ENE on neon-64gb. Includes log path and reinstall note.
2026-06-22 02:37:00 -05:00
f3d11359c6 docs(infra): document canonical database locations and lake build ingest shim
Adds canonical database location notes to 4-Infrastructure/AGENTS.md:
- Postgres canonical instance is the arxiv-pg container on neon-64gb.
- Gremlin canonical instance is the Azure Cosmos DB endpoint.
- Explicitly warns against spinning up local Postgres on qfox for
  production data.

Also adds `lake_build_ingest.py` to the stack-solidification anchors.
2026-06-22 02:22:34 -05:00
040b0ef61f docs(infra): document hybrid arxiv search infrastructure
Added embed_arxiv.py, hybrid_search.sql, and jaccard_hybrid.py to the
Stack-Solidification Anchors section. Replaced slow Jaccard matcher with
397x faster static-retrieval-mrl-en-v1 model + pgvector HNSW index.

Build: 3314 jobs, 0 errors (lake build Compiler)
2026-06-22 01:11:17 -05:00
d28e9de6ca fix(infra): optimize drive compression memory usage and download skip
Check file sizes before loading bytes into memory and stream large files
directly into the tarball to prevent OOM errors on Steam Deck. Skip downloading
files that already exist locally.

Build: 0 errors (py_compile)
2026-06-22 01:11:17 -05:00
0149a5c93d feat(infra): add MCP-only Drive GCCL compression shim
Adds 4-Infrastructure/shim/mcp_drive_gccl_compress.py:
- downloads a Google Drive folder via @piotr-agier/google-drive-mcp
- GCCL-delta-compresses contents into .tar.zst + JSON manifest
- uploads archive + manifest back to Drive
- optionally trashes the original folder
- handles MCP listFolder pagination

Updates 4-Infrastructure/AGENTS.md with the new shim anchor.

No rclone or egress-heavy operations; all Drive I/O goes through MCP.
2026-06-22 01:11:17 -05:00
67e77d8f24 feat(lean,infra): SilverSight-first offline PIST trace classifier
Moves all classification authority from Python into Lean:

- Adds `0-Core-Formalism/lean/Semantics/PistClassifyTrace.lean` executable.
  Reads a raw trace JSON and emits spectral radius, RRC shape, and tactic
  family using `Semantics.PIST.Spectral` and `Semantics.PIST.Classify`.
- Registers `pist-classify-trace` in `lakefile.toml`.
- Fixes `Semantics.PIST.Spectral` power iteration to handle directed
  transition matrices:
  - `symmetrize` now preserves half-integer weights as Q16_16 raw values.
  - `matVecMul` uses saturated Q16_16 arithmetic to prevent overflow.
- Rewrites `4-Infrastructure/shim/pist_trace_classify_offline.py` as a pure
  I/O wrapper: reads JSON, calls the Lean classifier, optionally calls
  `rrc-watchdog`, and emits the combined JSON. Removes Python-side spectral
  computation, shape thresholds, tactic heuristic, and KNN.
- Updates `AGENTS.md` and `4-Infrastructure/AGENTS.md` with the new build
  baseline and shim contract.

Verification:
- `lake build` → 8604 jobs, 0 errors
- Canary trace outputs match previous Python outputs to within Q16_16 rounding
  (e.g., apply_chain λ_q16 = 59044 vs 59045).
- `python3 -m py_compile` on the rewritten shim passes.
2026-06-22 01:11:17 -05:00
7b731f87c0 feat(rrc): add offline token-free trace classifier
Created a local, offline Python shim `pist_trace_classify_offline.py` that computes trace transition matrix spectra, maps the max eigenvalue to the Q16.16 spectral radius, evaluates the color-space shape classification logic from `Semantics.PIST.Classify`, and invokes the local `rrc-watchdog` Lean binary inside the podman container to verify alignment. This avoids querying the dead AWS RDS instance and saves LLM API tokens.

Build: 3314 jobs, 0 errors (lake build Compiler)
2026-06-22 01:11:17 -05:00
5bea52a09a fix(infra): wrap lean-lsp-mcp to correct severity schema for Moonshot API
Added a python-based wrapper script for lean-lsp-mcp that intercepts and cleans the 'severity' schema field from the tools/list output, resolving the 400 error from Moonshot API which strictly forbids defining 'type' in parent schemas alongside 'anyOf'.

Build: 3314 jobs, 0 errors (lake build)
2026-06-22 01:11:12 -05:00
f00ead35c2 feat(infra): add async database buffering program for RRC receipts
Implemented rrc_bosonic_db_buffer.py containing AsyncDatabaseBuffer
which queues, batches, and flushes PostgreSQL inserts for bosonic tensor
network receipts and metrics. Documented both rrc_bosonic_tensor_gpu.py
and rrc_bosonic_db_buffer.py in 4-Infrastructure/AGENTS.md.

Build: 0 jobs, 0 errors (lake build)
2026-06-18 23:10:46 -05:00
558431fc45 feat(lean,infra): Corpus278→250 rename + SLOS-calibrated braid defaults
- Renamed Corpus278/Matrices278 to Corpus250/Matrices250 across all
  Lean modules, Python shims, JSON schemas, and documentation
- Applied SLOS-calibrated defaults to braid_search.py:
  bracket_cost base=8x/16x, crossing_penalty gap_reward=0.01
- Synced same defaults to research-compute-fabric submodule
- Updated Burgers 0D defaults: nu=0.9995, advection=0.075
- Fixed 278→250 refs in pist_predictions JSON (claim_boundary, total)
- Fixed stale schema refs in .devin/skills/lean-proof/SKILL.md
- Updated AGENTS.md build baselines to post-clean counts (3314 jobs)
- Updated lake-manifest.json sparkle rev to match lakefile.toml
- Fixed Q16_16 signed conversion bug in qaoa_adapter tunable costs
- NBody.lean: pre-existing dirty changes (introN failures, NOT our work)

Build: 3314 jobs, 0 errors (lake build Compiler)
2026-06-18 22:16:52 -05:00
c0ec099e6d feat(infra): Burgers chaos game — quantum walk over 22-representation graph
Encodes each Burgers representation as a node connected by proven
isomorphisms from the codebase. Continuous-time quantum walk e^{-iAt}
on the adjacency matrix converges to eigenvector centrality; the
0D DualQuat Braid is confirmed as the universal hub (#1 in all three
rankings: centrality, quantum walk probability, and degree).

Build: N/A (Python shim, no Lean files touched)
2026-06-18 21:39:18 -05:00
00e9eed399 fix(lean): complete projectionOrdering proof in GeometricCompressionWorkspace
Replace the TODO(lean-port) sorry with a complete proof of the
projectionOrdering theorem: for positive SourceValue pairs s1 < s2
with s2 ≤ maxExpected, projectToCoding preserves strict ordering
of the Q0_64 values.

The proof uses Nat-only arithmetic (no Float) and handles two cases:
  - a2 < d: both values fit in Q0_64 range, ordering follows from
    monotonicity of integer division
  - a2 = d: a2*s/d = s clamped to q0_64MaxRaw; a1*s/d < q0_64MaxRaw
    via the key inequality (d-1)*s < (s-1)*d

Build: 8598 jobs, 0 errors (lake build)
2026-06-18 15:06:50 -05:00
475f6319ea chore(repo): push local 768-commit branch state onto clean remote baseline
This squashes all local history (768 commits) onto the scrubbed PR #90
baseline. Individual commits were lost during filter-repo corruption;
the working tree content is preserved intact.

Build: N/A (working tree state only)
2026-06-15 22:46:50 -05:00
Brandon Schneider
7c1f690e0f feat(infra): Garage 6-node RF3 cluster, Cloudflare Workers deploy, k3s node additions
- Garage scale-out complete: 6 nodes across 6 zones with replication_factor=3
- neon-64gb: added to Garage cluster (zone netcup-arm, 93 GiB)
- steamdeck: installed Garage v2.3.0, added to cluster (zone gpu, 373 GiB)
- nixos-laptop: converted from k3s server to agent in cupfox cluster
- qfox-1: k3s agent rejoined cupfox cluster
- Cloudflare Workers: WASM trinary VM deployed at wasm-compute-edge.researchstack.workers.dev
- node-registry.json: updated with all 6 Garage nodes
- Documentation: AGENTS.md, WIKI.md, ROADMAP.md, LLM-Context.md updated
- Lake build: 3313 jobs, 0 errors

Build: 3313 jobs, 0 errors (lake build Compiler)
2026-05-31 03:39:11 -05:00
Brandon Schneider
d9a647cb4d chore(infra): bootstrap racknerd and configure multi-node garage cluster
- Bootstrapped microvm-racknerd node on its live Tailscale IP 100.80.39.40.
- Connected racknerd to the central QFox cluster.
- Assigned racknerd to its own vps zone with 1G capacity.
- Made garage-cluster-init.sh robust and idempotent.
- Updated comments and topology mappings.

Build: 3313 jobs, 0 errors (lake build)
2026-05-31 02:04:32 -05:00
Brandon Schneider
5320a08105 feat(infra): integrate edge WASM and GitHub batch compute tiers
- Update device_capability_probe.py to add BATCH and WASM tiers and fix a NameError bug on has_virtio_net.
- Build Cloudflare Workers WASM compilation and JS fetch handler in 4-Infrastructure/cloudflare/ executing trinary VM steps.
- Create GitHub Actions batch_compute.yml workflow to harvest runner minutes.
- Keep 4-Infrastructure/AGENTS.md updated with the WASM core library anchor.

Build: 3313 jobs, 0 errors (lake build)
2026-05-30 20:08:31 -05:00
Brandon Schneider
cf907e2835 feat(infra): add QEMU graphics framebuffer packing shim and spec
- Add qemu_framebuffer_packer.py supporting ARGB8888/RGB24 raw matrix mapping
- Implement zero-copy mmap write/read interface to /dev/fb0 with signature headers
- Document the QEMU graphics framebuffer backplane in Section 11 of spec
- Update 4-Infrastructure/AGENTS.md and walkthrough.md documentation
- Verify syntax and workspace compilation baseline status

Build: 3313 jobs, 0 errors (lake build)
2026-05-30 19:49:25 -05:00
Brandon Schneider
6ea34fbae6 feat(infra): add GPU-specific math optimization loader for H.265 VCN/NVENC
- Implement MathOptimizationLoad dataclass to represent GPU packing configurations
- Update probe_vcn_capabilities to resolve optimizations for NVIDIA/AMD/Intel GPUs
- Extend compute_frame_size to support yuvj420p, 10-bit YUV, and YUV444p
- Propagate optimized pixel formats into select_optimal_resolution and spec
- Update _build_ffmpeg_cmd to inject lossless/zero-latency options and HEVC/H.265 metadata SEI NAL parameters
- Update 4-Infrastructure/AGENTS.md documentation

Build: 3313 jobs, 0 errors (lake build)
2026-05-30 19:45:09 -05:00
Brandon Schneider
61b38ca697 refactor(infra): optimize YUV420 frame packing using numpy
- Vectorized create_yuv420_frame when numpy is available to eliminate the 500k-iteration scalar Python loop.
- Pre-filled memoryview slice buffers in the fallback path.
- Updated 4-Infrastructure/AGENTS.md to document the optimization.

Build: 3313 jobs, 0 errors (lake build)
2026-05-30 19:32:11 -05:00
Brandon Schneider
30d8c56158 feat(infra): improve RRC Ray Layer Tagger and align registry
Improved rrc_ray_tagger.py with prioritized source name-based variant matching, corrected NetworkRayReceipt (3 variants, 67us) and BurgersRGSolver (5 variants) shapes, fixed Hopf-Cole fallback bug using string normalization, dynamically deduced workspace root path, and quarantined phase_update due to adversarial review falsification. Registered anchor in 4-Infrastructure/AGENTS.md.

Build: 3313 jobs, 0 errors (lake build Compiler)
2026-05-30 19:18:02 -05:00
Brandon Schneider
f63e4b5179 feat(infra): virtio-net ring as compute pipeline
Add virtio_net_transform.py: three Class-1 computation primitives via
virtio-net TX/RX rings — zero backend code changes needed.

  1. HASH_REPORT — host writes Toeplitz RSS hash into RX header
     (virtio_net_hdr_v1_hash.hash_value return channel)
  2. TSO gso_size — host splits large buffer via TCP segmentation offload
     (spatial partition into N × gso_size chunks)
  3. MRG_RXBUF — host merges multiple RX buffers (aggregation primitive)

Structs: VirtioNetHdr (12B), VirtioNetHdrHash (20B), VringDesc (16B).
Receipt schema: virtio_transform_receipt_v1 with CRC32 witness_hash.

The copy-if filter (skip zero deltas, process non-zeros) maps directly
onto the HASH_REPORT return channel: delta=0 → hash skip, delta≠0 →
hash_as_function_of_payload. This is the ambient compute model:
any QEMU/firecracker microVM is already a computation device without
knowing it.

Build: 3313 jobs, 0 errors (lake build)
Tools: glslang, spirv-as, spirv-dis (native); tint from nixpkgs for WGSL
2026-05-30 16:54:09 -05:00
Brandon Schneider
c64e4f15eb docs: update AGENTS.md — BraidDiatCodec shim + bridge modules
4-Infrastructure: add braid_diat_codec.py to stack-solidification anchors
Semantics: document BraidSpherionBridge admits (IntNodeToPhaseVec linearity, receipt_correspondence)
2026-05-30 13:30:55 -05:00
Brandon Schneider
e5fb0a5f4d chore: commit accumulated working tree changes
Lean: update Semantics modules, add new numerics/physics data files
Hardware: update FPGA bitstreams (tangnano9k_uart_loopback)
Infra: k3s-flake tests, netcup-vps configuration, VCN compute substrate
Docs: ARCHITECTURE, specs, citation updates
2026-05-30 00:10:02 -05:00
Brandon Schneider
066a96f03f docs(infra): update node topology with rs-vps (netcup ARM64) 2026-05-29 14:26:46 -05:00
Brandon Schneider
00e88b53a3 feat(infra): add DSP node schema and flac_dsp_node.py shim
Any Linux node with PipeWire can act as a FLAC/DSP compute worker
via a virtual sound card — no physical audio hardware required.

- ene.dsp_nodes table: pipewire_available, virtual_soundcard_supported,
  max_sample_rate, spectral_bands, latency_target_us, fft_size, etc.
- flac_dsp_node.py: node registration, PipeWire probe, FLAC chunk FFT
  analysis (peaks, spectral centroid, RMS level), receipt logging to
  ~/.cache/flac_dsp_receipts.jsonl
- AGENTS.md: document DSP volunteer computing schema addition

Build: 0 errors (py_compile)
2026-05-29 01:31:13 -05:00
Brandon Schneider
31b2f1f66f feat(infra): compile and SRAM-flash UART beacon with reset bypass
Synthesized and placed-and-routed the UART beacon design on Tang Nano 9K with physical reset bypassed (rst_n_internal = 1'b1). Programmed the SRAM using openFPGALoader (CRC check: Success). Verified the physical UART blockage due to BL702 bridge firmware limitations via local probe, confirming the virtual serial route (virtual://q16-pty) as the active verification path. Updated scoped AGENTS.md files with the latest hardware status and Lean build baselines.

Build: 3313 jobs, 0 errors (lake build)
2026-05-28 14:10:46 -05:00
Brandon Schneider
e203a5215e feat(hardware): Emergency Boot Witness reference implementation
Add compiling Lean modules and Python shim for the Geometry Emergency
Boot Witness specification (graphene supercapacitor geometric encoding).

Lean modules (Semantics.Hardware.EmergencyBoot.*):
- EmergencyBootTypes: HexCoord, CapClass, OpticalPath, VoltageDifferential,
  GrapheneMemristor, HybridOpticalPath, material properties
- EmergencyBootState: PowerState, SolarPowerState, ScanState, seed assembly,
  emergency boot state machine with 6502 calculator efficiency targets
- EmergencyBootShell: Command opcodes, status byte encoding, process
  definitions, executeCommand dispatch

All use Q16_16 fixed-point arithmetic (no Float in compute paths).
Verified theorems: utilizationWithinBounds, powerFailureMonotonic,
commandOpcode_roundTrip.

Python shim (4-Infrastructure/hardware/emergency_boot/):
- EmergencyBootEngine simulating FPGA geometric scan and seed extraction
- Demo CLI showing power failure → self-powered calculator mode →
  geometric scan → seed assembly flow

Build: 3302 jobs, 0 errors (narrow target), 3313 jobs, 0 errors (Compiler)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-27 23:00:39 -05:00
Brandon Schneider
a8a163650a fix(infra): resolve k3s services startup issues and update ingress configuration
- Revert builder image in embedded_surface Dockerfile to rust:bookworm for glibc compatibility with bookworm-slim runner.
- Replace racknerd-surface.json Git LFS pointer with the actual retrieved JSON config to resolve parse errors.
- Separated and imported regenerated docker images for stubs and credential-server on all cluster nodes.
- Fixed Ingress resources and middleware to target the correct authentik-server service name.
- Rewrote verification script checks to handle iptables-based ports and relative redirects robustly.
- Updated nixos-laptop IP in node topology documentation.

Build: 0 jobs, 0 errors
2026-05-27 17:39:00 -05:00
Brandon Schneider
3044f36df7 docs(agents): project-wide AGENTS.md audit — cross-refs, baseline, contracts
Actions taken from 5-agent audit sweep (audit date 2026-05-26):

AGENTS.md / docs sync:
- root AGENTS.md: add scripts/qc-flag and lean_expert_agent to Nested Contracts
- All 6 nested AGENTS.md files: append Cross-References section pointing to
  root for Post-Interaction Workflow, Programming Choice Flow, Do Not Sweep,
  Git Remote Hygiene (Lean, Infra, text-to-cad, docs, qc-flag, lean_expert_agent)
- 6-Documentation/docs/AGENTS.md: cross-ref also lists AVMIsa.Emit sole output
  boundary and Compiler surface blessing

Lean build baseline:
- 0-Core-Formalism/lean/Semantics/AGENTS.md: update blessed Compiler Surface
  header to commit 49f0dfb3; correct job count to 3311 (lake build Compiler)

ARCHITECTURE.md:
- §7 repo table: add RRC.Emit, AVMIsa.Emit, RRC.Corpus278 to Lean/Semantics entry
- New §7.1 Compiler Surface: documents 3-root pipeline and sole output boundary
- §4 Data Flow: annotate output with AVMIsa.Emit sole-boundary note

TODO_MAP.md:
- Phase A6: add 3 new Lean deliverables (Corpus278, RRC.Emit, AVMIsa.Emit);
  update status/result with Compiler build baseline; refine next action

Build: Compiler 3311 jobs, 0 errors (no Lean changes).

Generated with Devin (https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-26 22:34:46 -05:00
Brandon Schneider
073a70eb86 WIP: accumulated changes 2026-05-25 16:24:21 -05:00
Brandon Schneider
fd863af6fd Expand devcontainer with full Python stack, add MCP servers (Notion/AWS), strengthen Lean theorems
- .devcontainer/Dockerfile: add PostgreSQL client libs, OpenSSL/libffi headers, gfortran/BLAS for scipy, rclone; install full Python dependency set (boto3, psycopg2-binary, fastapi, uvicorn, notion-client, httpx, pytest, numpy, scipy, etc.) in uv-managed venv; add rclone S3 gateway init script as ENTRYPOINT
- .devcontainer/devcontainer.json: switch from build to pre-built image (localhost/research
2026-05-19 01:52:14 -05:00
Brandon Schneider
99fc5d750b Add storage observer/optimizer agent (storage_agent.py)
Full Observe→Decide→Act→Emit loop for the restic + Garage + rclone stack:

- storage_agent.py: probes Garage health, restic snapshot count and dedup
  ratio (Q16_16), backup log staleness, cold-copy drift; triggers corrective
  actions (snap, cold-copy, verify, forget, offload, garage restart) based
  on Q16_16 threshold comparisons. Emits hash-chained receipts to both a
  local JSONL log (~/.cache/storage-agent.jsonl) and
  s3://research-stack/agent-receipts/ in Garage.

- storage-agent.service / storage-agent.timer: systemd units installed
  system-wide; timer fires every 15 min (RandomizedDelaySec=60). Includes
  PATH=/home/allaun/.local/bin:... so aws cli and restic are resolved.

- .git/hooks/post-commit updated: agent runs (--once) in the background
  after each restic snap, so post-commit state is always observed.

- 4-Infrastructure/AGENTS.md: full documentation of agent contract,
  receipt schema, trigger table, log paths, and usage examples.

Shim boundary: zero logic — pure subprocess calls to existing CLI tools.
No Float, no new dependencies. py_compile and live dry-run verified.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-18 22:32:52 -05:00
Brandon Schneider
cb96c6bed2 Update repository agent operating contracts 2026-05-11 22:48:54 -05:00
Brandon Schneider
29f9b78b6d Stage stack solidification source slice 2026-05-11 22:08:10 -05:00