feat(infra): Garage 6-node RF3 cluster, Cloudflare Workers deploy, k3s node additions

- Garage scale-out complete: 6 nodes across 6 zones with replication_factor=3
- neon-64gb: added to Garage cluster (zone netcup-arm, 93 GiB)
- steamdeck: installed Garage v2.3.0, added to cluster (zone gpu, 373 GiB)
- nixos-laptop: converted from k3s server to agent in cupfox cluster
- qfox-1: k3s agent rejoined cupfox cluster
- Cloudflare Workers: WASM trinary VM deployed at wasm-compute-edge.researchstack.workers.dev
- node-registry.json: updated with all 6 Garage nodes
- Documentation: AGENTS.md, WIKI.md, ROADMAP.md, LLM-Context.md updated
- Lake build: 3313 jobs, 0 errors

Build: 3313 jobs, 0 errors (lake build Compiler)
This commit is contained in:
Brandon Schneider 2026-05-31 03:39:11 -05:00
parent 008cf8fefe
commit 7c1f690e0f
9 changed files with 281 additions and 42 deletions

View file

@ -80,15 +80,16 @@ Dynamo-style S3-compatible store written in Rust. Replaced rclone serve s3.
### Node topology (Tailscale mesh)
| Node | Tailscale IP | Role | Disk | SSH Status |
|------|-------------|------|------|----------|
| qfox-1 (this machine) | 100.88.57.96 | primary, S3 endpoint, GPU compute | 1.8 TB NVMe | local |
| 361395-1 (old cupfox) | 100.110.163.82 | Netcup VPS, 2 vCPU EPYC-Genoa | 125 GB | key OK (recovered) |
| rs-vps (netcup) | — | Lean LSP, Python LSP, Ollama, Jellyfin, k3s (ARM64) | 2 TB | SSH via password (creds in 1Password) |
| nixos-laptop | 100.102.173.61 | Authentik SSO, Uptime Kuma, storage node, AMD GPU compute | 459 GB NVMe | key OK |
| microvm-racknerd | 100.80.39.40 | Caddy reverse proxy, Homer dashboard, chat placeholder, storage node (RackNerd VPS) | 9.1 GB | key OK |
| nixos-steamdeck-1 | 100.85.244.73 | GPU compute, planned edge LLM (3B-7B), RDNA 2 | NixOS | just onboarded |
| dracocomp | 100.100.140.27 | offline | — | unreachable (3+ days)
| Node | Tailscale IP | k3s | Garage | Zone | Disk | SSH |
|------|-------------|-----|--------|------|------|-----|
| **qfox-1** (this machine) | 100.88.57.96 | ✅ worker | ✅ 780 GiB | local | 1.8 TB NVMe | local |
| **cupfox** | 100.110.163.82 | ✅ control-plane | ✅ 69 GiB | fra | 125 GB | key OK (361395) |
| **nixos-laptop** | 100.102.173.61 | ✅ worker | ✅ 347 GiB | ord | 459 GB NVMe | key OK |
| **racknerd** | 100.80.39.40 | ✅ worker | ✅ 954 MiB | vps | 9.1 GB VPS | key OK |
| **neon-64gb** | 100.64.19.78 | ✅ worker (ARM64) | ✅ 93 GiB | netcup-arm | 2 TB | root key OK |
| **steamdeck** | 100.85.244.73 | ✅ worker | ✅ 373 GiB | gpu | 476 GB NVMe | key OK |
| rs-vps (netcup) | — | ❌ | ❌ | — | 2 TB | SSH via password |
| dracocomp | 100.100.140.27 | ❌ | ❌ | — | — | unreachable
- RPC port: **3901** (Tailscale-only, not exposed to internet)
- S3 API port: **3900** (qfox-1 only; other nodes bind loopback)
@ -153,12 +154,12 @@ Daily timer: `restic-backup.timer` fires at 03:00 ±30 min, runs `backup.sh full
### Replication status
Currently `replication_factor = 1` (single node, qfox-1 only).
Bump to 3 after bootstrapping 3 nodes:
`replication_factor = 3`, zone redundancy enforced, 6 nodes across 6 zones.
~1.3 TiB total capacity, ~440 GiB effective (RF3).
Add a new Garage node:
```bash
bash 4-Infrastructure/storage/garage/garage-node-bootstrap.sh 100.110.163.82
bash 4-Infrastructure/storage/garage/garage-node-bootstrap.sh 100.119.165.120
bash 4-Infrastructure/storage/garage/garage-cluster-init.sh
bash 4-Infrastructure/storage/garage/garage-node-bootstrap.sh <tailscale_ip>
```
### Git post-commit hook
@ -311,6 +312,8 @@ python3 4-Infrastructure/storage/storage_agent.py --loop --interval 900
- `4-Infrastructure/shim/qemu_framebuffer_packer.py` — QEMU graphics framebuffer packer mapping Q16_16 scalars to ARGB8888/RGB24 pixels for mmap-based zero-copy display DMA loopback
- `4-Infrastructure/shim/rrc_ray_tagger.py` — RRC Ray Layer Tagger; classifies math payloads into RRC shapes and matches them to swappable compute slots and transports
- `4-Infrastructure/cloudflare/src/lib.rs` — Cloudflare Workers edge WASM trinary VM core implementing the Q0_16 scalar compute floor
- `4-Infrastructure/cloudflare/src/index.js` — Cloudflare Workers entry point, POST-only, JSON + binary protocol
- `4-Infrastructure/cloudflare/wrangler.toml` — Wrangler config, deployed at `https://wasm-compute-edge.researchstack.workers.dev`
- `4-Infrastructure/hardware/emergency_boot/emergency_boot_shim.py` — Python I/O shim
for Geometry Emergency Boot Witness (6502 calculator-efficiency FPGA controller)
Specification: `6-Documentation/docs/specs/GEOMETRY_EMERGENCY_BOOT_WITNESS_2026-04-08.md`

View file

@ -1,4 +1,7 @@
import { VmState } from "./wasm_compute.wasm";
import wasmModule from "./wasm_compute_bg.wasm";
import { initSync, VmState } from "./wasm_compute.js";
initSync(wasmModule);
export default {
async fetch(request, env, ctx) {
@ -26,7 +29,6 @@ export default {
headers: { "content-type": "application/json" }
});
} else {
// Default to binary stream: 3 bytes per step (op, idx, val)
const buffer = await request.arrayBuffer();
const view = new DataView(buffer);
const len = buffer.byteLength;
@ -34,16 +36,14 @@ export default {
for (let offset = 0; offset + 2 < len; offset += 3) {
const op = view.getUint8(offset);
const idx = view.getUint8(offset + 1);
const val = view.getInt8(offset + 2); // signed 8-bit integer
const val = view.getInt8(offset + 2);
vm.step(op, idx, val);
}
const scalar = vm.derive_scalar();
// Return 2-byte binary scalar response
const responseBuf = new ArrayBuffer(2);
const responseView = new DataView(responseBuf);
responseView.setUint16(0, scalar, false); // big-endian
responseView.setUint16(0, scalar, false);
return new Response(responseBuf, {
headers: { "content-type": "application/octet-stream" }

View file

@ -0,0 +1,183 @@
/* @ts-self-types="./wasm_compute.d.ts" */
export class VmState {
__destroy_into_raw() {
const ptr = this.__wbg_ptr;
this.__wbg_ptr = 0;
VmStateFinalization.unregister(this);
return ptr;
}
free() {
const ptr = this.__destroy_into_raw();
wasm.__wbg_vmstate_free(ptr, 0);
}
/**
* @returns {number}
*/
derive_scalar() {
const ret = wasm.vmstate_derive_scalar(this.__wbg_ptr);
return ret;
}
constructor() {
const ret = wasm.vmstate_new();
this.__wbg_ptr = ret;
VmStateFinalization.register(this, this.__wbg_ptr, this);
return this;
}
reset() {
wasm.vmstate_reset(this.__wbg_ptr);
}
/**
* @param {number} op
* @param {number} idx
* @param {number} val
*/
step(op, idx, val) {
wasm.vmstate_step(this.__wbg_ptr, op, idx, val);
}
}
if (Symbol.dispose) VmState.prototype[Symbol.dispose] = VmState.prototype.free;
function __wbg_get_imports() {
const import0 = {
__proto__: null,
__wbg___wbindgen_throw_1506f2235d1bdba0: function(arg0, arg1) {
throw new Error(getStringFromWasm0(arg0, arg1));
},
__wbindgen_init_externref_table: function() {
const table = wasm.__wbindgen_externrefs;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
},
};
return {
__proto__: null,
"./wasm_compute_bg.js": import0,
};
}
const VmStateFinalization = (typeof FinalizationRegistry === 'undefined')
? { register: () => {}, unregister: () => {} }
: new FinalizationRegistry(ptr => wasm.__wbg_vmstate_free(ptr, 1));
function getStringFromWasm0(ptr, len) {
return decodeText(ptr >>> 0, len);
}
let cachedUint8ArrayMemory0 = null;
function getUint8ArrayMemory0() {
if (cachedUint8ArrayMemory0 === null || cachedUint8ArrayMemory0.byteLength === 0) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
let cachedTextDecoder = new TextDecoder('utf-8', { ignoreBOM: true, fatal: true });
cachedTextDecoder.decode();
const MAX_SAFARI_DECODE_BYTES = 2146435072;
let numBytesDecoded = 0;
function decodeText(ptr, len) {
numBytesDecoded += len;
if (numBytesDecoded >= MAX_SAFARI_DECODE_BYTES) {
cachedTextDecoder = new TextDecoder('utf-8', { ignoreBOM: true, fatal: true });
cachedTextDecoder.decode();
numBytesDecoded = len;
}
return cachedTextDecoder.decode(getUint8ArrayMemory0().subarray(ptr, ptr + len));
}
let wasmModule, wasmInstance, wasm;
function __wbg_finalize_init(instance, module) {
wasmInstance = instance;
wasm = instance.exports;
wasmModule = module;
cachedUint8ArrayMemory0 = null;
wasm.__wbindgen_start();
return wasm;
}
async function __wbg_load(module, imports) {
if (typeof Response === 'function' && module instanceof Response) {
if (typeof WebAssembly.instantiateStreaming === 'function') {
try {
return await WebAssembly.instantiateStreaming(module, imports);
} catch (e) {
const validResponse = module.ok && expectedResponseType(module.type);
if (validResponse && module.headers.get('Content-Type') !== 'application/wasm') {
console.warn("`WebAssembly.instantiateStreaming` failed because your server does not serve Wasm with `application/wasm` MIME type. Falling back to `WebAssembly.instantiate` which is slower. Original error:\n", e);
} else { throw e; }
}
}
const bytes = await module.arrayBuffer();
return await WebAssembly.instantiate(bytes, imports);
} else {
const instance = await WebAssembly.instantiate(module, imports);
if (instance instanceof WebAssembly.Instance) {
return { instance, module };
} else {
return instance;
}
}
function expectedResponseType(type) {
switch (type) {
case 'basic': case 'cors': case 'default': return true;
}
return false;
}
}
function initSync(module) {
if (wasm !== undefined) return wasm;
if (module !== undefined) {
if (Object.getPrototypeOf(module) === Object.prototype) {
({module} = module)
} else {
console.warn('using deprecated parameters for `initSync()`; pass a single object instead')
}
}
const imports = __wbg_get_imports();
if (!(module instanceof WebAssembly.Module)) {
module = new WebAssembly.Module(module);
}
const instance = new WebAssembly.Instance(module, imports);
return __wbg_finalize_init(instance, module);
}
async function __wbg_init(module_or_path) {
if (wasm !== undefined) return wasm;
if (module_or_path !== undefined) {
if (Object.getPrototypeOf(module_or_path) === Object.prototype) {
({module_or_path} = module_or_path)
} else {
console.warn('using deprecated parameters for the initialization function; pass a single object instead')
}
}
if (module_or_path === undefined) {
module_or_path = new URL('wasm_compute_bg.wasm', import.meta.url);
}
const imports = __wbg_get_imports();
if (typeof module_or_path === 'string' || (typeof Request === 'function' && module_or_path instanceof Request) || (typeof URL === 'function' && module_or_path instanceof URL)) {
module_or_path = fetch(module_or_path);
}
const { instance, module } = await __wbg_load(await module_or_path, imports);
return __wbg_finalize_init(instance, module);
}
export { initSync, __wbg_init as default };

Binary file not shown.

View file

@ -1,7 +1,7 @@
name = "wasm-compute-edge"
main = "src/index.js"
compatibility_date = "2026-05-30"
compatibility_date = "2026-05-31"
account_id = "47b5018a9ddab552d535c8e046a6bc7e"
workers_dev = true
[[rules]]
globs = ["**/*.wasm"]
type = "CompiledWasm"

View file

@ -1,11 +1,56 @@
[
{
"hostname": "100.80.39.40",
"hostname": "qfox-1",
"tailscale_ip": "100.88.57.96",
"ssh_user": "allaun",
"ssh_port": 22,
"garage_node_id": "3e08a71b73fa2b1099301844d1f199caab50f7a9209b9929d9bfb2bfeeb937f4@100.88.57.96:3901",
"garage_data_dir": "/var/lib/garage/data",
"added_at": "2026-05-31T08:00:00.000000+00:00"
},
{
"hostname": "cupfox",
"tailscale_ip": "100.110.163.82",
"ssh_user": "root",
"ssh_port": 22,
"garage_node_id": "75fac43bc53eb201315c09f3e590fff15f27454b7e9f974f64522204612821db@100.110.163.82:3901",
"garage_data_dir": "/var/lib/garage/data",
"added_at": "2026-05-31T07:03:13.628137+00:00"
},
{
"hostname": "nixos-laptop",
"tailscale_ip": "100.102.173.61",
"ssh_user": "allaun",
"ssh_port": 22,
"garage_node_id": "a7e6c283056a4d77b4a5bb820fd25dc336c5443a1ad17110dfadc2b150ebfb49@100.102.173.61:3901",
"garage_data_dir": "/var/lib/garage/data",
"added_at": "2026-05-31T08:15:00.000000+00:00"
},
{
"hostname": "racknerd",
"tailscale_ip": "100.80.39.40",
"ssh_user": "root",
"ssh_port": 22,
"garage_node_id": "1951197c0ce140a7724155d0d68db490682fa3b63f8cc98acdceb8c7b1d41766@100.80.39.40:3901",
"garage_data_dir": "/var/lib/garage/data",
"added_at": "2026-05-31T07:03:13.628137+00:00"
},
{
"hostname": "neon-64gb",
"tailscale_ip": "100.64.19.78",
"ssh_user": "root",
"ssh_port": 22,
"garage_node_id": "036bdd161f089d2302b95453078ab1738d571f90d904752da1a9506dacb4f452@100.64.19.78:3901",
"garage_data_dir": "/var/lib/garage/data",
"added_at": "2026-05-31T10:00:00.000000+00:00"
},
{
"hostname": "steamdeck",
"tailscale_ip": "100.85.244.73",
"ssh_user": "allaun",
"ssh_port": 22,
"garage_node_id": "3536017f8688fc3cae7b7408274809a8acefeff634004c43de99f615e53e0efb@100.85.244.73:3901",
"garage_data_dir": "/var/lib/garage/data",
"added_at": "2026-05-31T10:30:00.000000+00:00"
}
]

View file

@ -488,8 +488,10 @@ The Lean 4 formalization lives in `0-Core-Formalism/lean/Semantics/`. As of May
| File | Description |
|------|-------------|
| `src/lib.rs` | Rust WASM entry point for Cloudflare Workers — 512 B payload, 8 ms CPU budget |
| `wrangler.toml` | Worker configuration, route bindings, WASM build settings |
| `src/lib.rs` | Rust WASM trinary VM (7 ops, 32 trits) — SET/ADD/SUB/SHIFT/MERGE/PROJECT/WEIGHT |
| `src/index.js` | Worker entry point — POST only, JSON + binary protocols, `initSync` WASM load |
| `wrangler.toml` | Worker configuration, account `47b5018a9ddab552d535c8e046a6bc7e` |
| **Deployed** | `https://wasm-compute-edge.researchstack.workers.dev` — Q0_16 scalar compute floor |
### GitHub Actions (`.github/workflows/`)

View file

@ -298,7 +298,7 @@ No promotion without domain-appropriate evidence. Compression claims require SI
2. Prove `receipt_invertible` for braid eigensolid compressor (second required theorem)
3. Add RRC scale-band witness schema for equation records; rerun `4-Infrastructure/shim/rrc_equation_classifier.py`
4. Add negative-control strength witnesses for HOLD rows before promotion
5. Bootstrap Garage replication (nixos-laptop + 361395-1 → replication_factor=3); 361395-1 SSH recovered
5. Garage replication_factor=3 with 6 nodes across 6 zones (qfox-1, cupfox, nixos-laptop, racknerd, neon-64gb, steamdeck) — ✅ done
6. UART packet format design (start byte + 3-byte payload + checksum)
7. Create `4-Infrastructure/surface/` FastAPI skeleton
8. Flash Tang Nano 9K with generated bitstream; verify LED behavior *(blocked: requires physical Tang Nano 9K board + USB programmer attached to host)*
@ -348,7 +348,7 @@ Three tools, non-overlapping roles:
| Tool | Role |
|------|------|
| **restic** | Deduplicated, encrypted, content-addressed snapshots. Primary backend: Garage S3. |
| **Garage v2.3.0** | Self-hosted S3-compatible object store over Tailscale mesh. 5 buckets. `replication_factor=1` (scale-out to 3 planned). |
| **Garage v2.3.0** | Self-hosted S3-compatible object store over Tailscale mesh. 6 nodes across 6 zones, `replication_factor=3`, zone redundancy enforced. ~1.6 TiB total capacity, ~440 GiB effective. |
| **rclone** | Raw sync between remotes (Garage↔gdrive). Cold copy of restic chunks to gdrive. |
Data flow: `git commit → post-commit hook → restic snap → Garage:research-stack`; daily 03:00 timer → `rclone copy → gdrive:restic-mirror`.

View file

@ -74,14 +74,15 @@ Research-Stack/
### Tailscale Mesh
| Node | IP | OS | Role | Reachable? |
|------|-----|-----|------|------------|
| qfox-1 | 100.88.57.96 | Arch Linux (CachyOS) | Primary, Garage S3, GPU compute, 1.8 TB NVMe | local (this machine) |
| 361395-1 | 100.110.163.82 | Debian 13 | Netcup VPS, 2 vCPU EPYC-Genoa, 125 GB | SSH key auth OK (recovered) |
| nixos-laptop | 100.119.165.120 | NixOS 25.11 | Authentik SSO, storage node, AMD GPU compute, 459 GB | Key auth OK |
| microvm-racknerd | 100.101.247.127 | Debian 13 | Caddy reverse proxy, credential server, 1 vCPU | Root password OK |
| steam-deck | TBD | SteamOS (Arch) | **Planned**: Edge LLM inference (3B-7B), RDNA 2 GPU compute | Control stick drift — repurposing |
| dracocomp | 100.100.140.27 | — | Offline | Last seen 3+ days ago |
| Node | IP | OS | k3s | Garage Zone | Role | Reachable? |
|------|-----|-----|-----|-------------|------|------------|
| qfox-1 | 100.88.57.96 | CachyOS | ✅ worker | local (780 GiB) | Primary, GPU compute, 1.8 TB NVMe | local |
| cupfox | 100.110.163.82 | Debian 13 | ✅ control-plane | fra (69 GiB) | k3s server, Netcup VPS | key OK |
| nixos-laptop | 100.102.173.61 | NixOS 26.05 | ✅ worker | ord (347 GiB) | GPU compute, 459 GB NVMe | key OK |
| neon-64gb | 100.64.19.78 | Debian 13 | ✅ worker (ARM64) | netcup-arm (93 GiB) | Netcup ARM64 VPS, 2 TB | root key OK |
| racknerd | 100.80.39.40 | Debian 13 | ✅ worker | vps (954 MiB) | RackNerd VPS, reverse proxy | key OK |
| steamdeck | 100.85.244.73 | NixOS 25.11 | ✅ worker | gpu (373 GiB) | RDNA 2 GPU compute, LLM inference | key OK |
| dracocomp | 100.100.140.27 | — | ❌ | — | Offline | unreachable |
### Distributed Storage Surface
@ -98,16 +99,21 @@ qfox-1 NVMe (1.8 TB) Google Drive (5 TB)
└── snap-zone
```
| Layer | Capacity | Used | Free | Protocol |
|-------|----------|------|------|----------|
| qfox-1 NVMe (Garage primary) | 1.8 TB | ~291 GB | ~1.6 TB | local NVMe |
| nixos-laptop NVMe (future Garage node) | 459 GB | ~36 GB | ~423 GB | local NVMe |
| Layer | Capacity | Zone | Protocol |
|-------|----------|------|----------|
| qfox-1 NVMe | 780 GiB (allocated) | local | Tailscale :3901 |
| nixos-laptop NVMe | 347 GiB (allocated) | ord | Tailscale :3901 |
| cupfox VPS | 69 GiB (allocated) | fra | Tailscale :3901 |
| neon-64gb VPS | 93 GiB (allocated) | netcup-arm | Tailscale :3901 |
| steamdeck NVMe | 373 GiB (allocated) | gpu | Tailscale :3901 |
| racknerd VPS | 954 MiB (allocated) | vps | Tailscale :3901 |
| **Total** | **~1.6 TiB** (6 zones, RF3) | — | ~440 GiB effective |
| 361395-1 disk (future Garage node) | 125 GB | ~48 GB | ~77 GB | local SSD |
| **Google Drive** | **5.0 TB** | **1.9 TB** | **3.1 TB** | **rclone FUSE mount** |
| **Total addressable** | **~7.4 TB** | **~2.3 TB** | **~5.2 TB** | **Garage + rclone + Drive** |
**Garage S3 Cluster**
- **Status**: Single-node (`replication_factor = 1`) on qfox-1
- **Status**: 6-node cluster (`replication_factor = 3`) across 6 zones — qfox-1 (local), cupfox (fra), nixos-laptop (ord), racknerd (vps), neon-64gb (netcup-arm), steamdeck (gpu)
- **Primary endpoint**: `s3:http://100.88.57.96:3900`
- **Buckets**: `research-stack`, `db-scratch`, `rds-overflow`, `snap-zone`, `gdrive-mirror`
- **Restic snapshots**: 2 (last: 2026-05-18, 5.4 GiB)