allaun/Research-Stack
1
0
Fork 0
mirror of https://github.com/allaunthefox/Research-Stack.git synced 2026-07-31 03:05:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
Research-Stack/4-Infrastructure
History
Brandon Schneider 02f1c928d7 refactor(rds): consolidate 14 psycopg2 connect patterns into shared rds_connect module 
Creates 4-Infrastructure/shim/rds_connect.py with a single connect_rds()
function that resolves connection parameters in priority order:
  1. explicit kwargs
  2. DATABASE_URL env var (postgres://user:pass@host:port/dbname?sslmode=...)
  3. individual RDS_* env vars (RDS_HOST, RDS_PORT, RDS_USER, etc.)
  4. built-in defaults

Auth resolution (when password is empty or RDS_IAM=1):
  1. RDS_IAM_TOKEN env var (pre-computed)
  2. boto3 SDK generate_db_auth_token (preferred)
  3. subprocess aws rds generate-db-auth-token (fallback)
  4. RDS_PASSWORD env var (non-IAM)

Replaces 8 connection pattern variants across 14 active shims:
  - subprocess + RDS_IAM_TOKEN fallback: pist_trace_classify_mcp, joint_classifier,
    pist_prove_and_classify, ingest_57_flexures
  - boto3 SDK: ene_wiki_body_reingest, ene_migrate_and_tag, dataset_ingest_rds
  - subprocess + RDS_PASSWORD: batch_embed_artifacts, sync_wiki_to_rds, seed_flexure_dataset
  - RDS_IAM_AUTH: pist_classify
  - bashrc parsed: credential_loader

v1.4a benchmark confirmed at 100% after refactor.
2026-05-26 15:09:34 -05:00
..
auto archive: remove dated receipts, experimental probes, and uncompiled prototypes 2026-05-25 16:51:58 -05:00
bin initial: sovereign research stack (consolidated, weightless, and lfs-optimized) 2026-05-04 18:11:36 -05:00
config initial: sovereign research stack (consolidated, weightless, and lfs-optimized) 2026-05-04 18:11:36 -05:00
docs WIP: accumulated changes 2026-05-25 16:24:21 -05:00
drivers initial: sovereign research stack (consolidated, weightless, and lfs-optimized) 2026-05-04 18:11:36 -05:00
exploit-infra/CATEGORY/TSM initial: sovereign research stack (consolidated, weightless, and lfs-optimized) 2026-05-04 18:11:36 -05:00
hardware Track remaining source and documentation inventory 2026-05-11 22:18:31 -05:00
infra Add proof worker pool routing 2026-05-25 22:27:18 -05:00
k3s-flake refactor(rds): consolidate 14 psycopg2 connect patterns into shared rds_connect module 2026-05-26 15:09:34 -05:00
kernel chore: preserve working tree before secure wipe 2026-05-13 17:36:02 -05:00
nano-kernel Track remaining source and documentation inventory 2026-05-11 22:18:31 -05:00
NoDupeLabs Scrub tracked API key material 2026-05-11 22:25:43 -05:00
packaging collapse: prover orchestration layers, FAMM verilator harness, swarm topological prober, spec sheets, virtual FPGA system tests, merge conflict resolution 2026-05-06 23:42:01 -05:00
rds_probe Fix Dependabot vulns: update sqlx 0.7->0.8.6, add ene-rds Cargo.lock 2026-05-19 08:09:34 +00:00
scripts feat(pist): Route-Repair v1.1 — 60 failure flexures ingested, obstruction-type voting 2026-05-26 12:17:42 -05:00
servo-fetch fix(servo-fetch): align dom query with smoothie 2026-05-20 23:07:32 -05:00
shim refactor(rds): consolidate 14 psycopg2 connect patterns into shared rds_connect module 2026-05-26 15:09:34 -05:00
shims initial: sovereign research stack (consolidated, weightless, and lfs-optimized) 2026-05-04 18:11:36 -05:00
storage WIP: accumulated changes 2026-05-25 16:24:21 -05:00
surface initial: sovereign research stack (consolidated, weightless, and lfs-optimized) 2026-05-04 18:11:36 -05:00
witness Refactor provenance sources for open witness backends 2026-05-12 05:57:04 -05:00
AGENTS.md WIP: accumulated changes 2026-05-25 16:24:21 -05:00
README.md docs: full documentation + roadmap refresh (2026-05-19) 2026-05-19 09:12:16 +00:00

README.md

4-Infrastructure

Purpose: Rust/Python shims, GPU duty assignment, cloud storage, web interaction, hardware designs, drivers, and the ENE RDS Rust workspace.

Depends on: 0 through 3

Directory Layout

Path Purpose
infra/ Python shims, ENE distributed node, credential server
infra/ene-rds/ ENE RDS Rust workspace (8 crates — Rust replacement for Python RDS stack)
hardware/ FPGA Verilog, FAMM verilator, KiCad templates
drivers/ Hardware drivers
gpu/ WGSL shaders, wgpu compute dispatch
storage/ restic + Garage S3 + rclone storage stack
storage/storage_agent.py Storage observer/optimizer agent
shim/ Stack solidification probes, adversarial harnesses, gate library
surface/ FastAPI/WebSocket surface (spec'd, pending implementation)

ENE RDS Rust Workspace (infra/ene-rds/)

8-crate Rust workspace replacing the Python RDS stack. Build: cd 4-Infrastructure/infra/ene-rds && cargo build --release.

Crate Purpose
ene-rds-core Shared PostgreSQL client, DSN builder, receipts
ene-rds-wiki Wiki CRUD + full-text search + revision tracking
ene-rds-ephemeral EphemeralNode thermal zones, tasks, receipts, scars, metrics
ene-rds-chat Chat session ingestion, keyword/semantic search
ene-api Axum HTTP server on :3000
ene-node Node identity and gossip primitives
ene-storage S3/Garage object storage client
ene-sync Polls opencode.db SQLite → upserts into RDS chat tables

sqlx 0.8.6 (Dependabot vuln from 0.7 resolved 2026-05-19).

Storage Stack (storage/)

Three tools, non-overlapping roles. Full contract: 4-Infrastructure/AGENTS.md §Storage Stack.

Tool Job
restic Deduplicated, encrypted, content-addressed snapshots
Garage v2.3.0 Self-hosted S3-compatible object store (Tailscale mesh, 5 buckets)
rclone Raw sync between remotes (Garage↔gdrive cold copy)

Storage agent (storage/storage_agent.py): observe→decide→act loop, systemd timer (every 15 min), Q16_16 thresholds, JSONL hash-chain receipts.

GPU Status

  • Device: NVIDIA GeForce RTX 4070
  • Memory: 12.3GB total, ~10GB available
  • CUDA: 13.0
  • Compute dispatch: WGSL → wgpu (Vulkan GPU / lavapipe CPU / WebGPU WASM). Dispatch entry points: 5-Applications/parquet_compressor/src/gpu.rs, gpu/wasmgpu/ (47 WGSL shaders)

Components

  • ENE RDS — Rust workspace replacing Python RDS (see above)
  • Storage Stack — restic + Garage S3 + rclone (see above)
  • Lean Shim — Lean ↔ Python bidirectional interface
  • ENE Shim — ENE node management from Python
  • GPU Duty — WGSL/wgpu compute dispatch (Q16_16 deterministic across all substrates)
  • Credential Gateway — infra/credential_server.py, apiProvider service kind, cupfox routing
  • Adversarial Harnesses — shim/adversarial_duals/: Anti-FAMM, Anti-BraidStorm, 20+ gate library entries