allaun/Research-Stack
1
0
Fork 0
mirror of https://github.com/allaunthefox/Research-Stack.git synced 2026-07-31 03:05:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
610 commits 18 branches 0 tags 425 MiB
Commit graph

34 commits

Author SHA1 Message Date
Brandon Schneider
aba1fba7ad fix(adversarial-review): resolve 35 critical coding bugs across 8 subsystems 
Security & correctness fixes from full adversarial review:

Lean (7 fixes):
- FixedPoint.lean: guard false theorem with n > 0 precondition
- QFactor.lean: remove double-scaling error in energy decrease
- AVMIsa/Step.lean: implement addSatQ16/subSatQ16 primitives
- BraidEigensolid.lean: fix crossStep second output argument swap
- SSMS.lean: complete ACI preservation proof (with rounding caveat)
- HouseholderQR.lean: add n > 0 precondition to spectral theorem

Verilog (7 fixes):
- q16_lut_core.v: fix multiply shift (16 → 32 bits)
- q16_lut_top.v: fix valid bit (0 → 1)
- cff_accelerator.v: fix SHA-256 padding (len < 448 check)
- research_stack_top.v: fix trigger aliasing (unique counters)
- Blitter6502OISC_small.v: fix address width (15 → 16 bits)
- spatial_hash_bram.v: add OOB write guard
- tmr_oepi_safety_fsm.v: fix double-increment race

WGSL (6 fixes):
- shaders.wgsl: atomicAdd for concurrent writes
- frustration_qubo.wgsl: double-buffer + CAS loop
- braid_fft.wgsl: workgroupBarrier synchronization
- burgers_scar_filter.wgsl: atomic E_bins array

Rust (9 fixes):
- thermodynamic.rs: Arc::from_raw → Arc::clone (double-free)
- thermodynamic.rs: Box::into_raw → Box (leak)
- tools/src/lib.rs: shell injection → shlex.quote
- ene-node/src/lib.rs: LRU caps, constant-time HMAC, peer caps

Python (6 fixes):
- similarity/__init__.py: pickle.load → RestrictedUnpickler
- AI-Feynman: torch.load → weights_only=True (14 calls)
- fetch_arxiv.py, fetch_s2.py: eval → ast.literal_eval
- topology.py: os.system → shutil.copy2
- SSH pipe: os.system → base64 pipe

Build: lake build 3572 jobs, 0 errors
 2026-05-31 23:38:03 -05:00
Brandon Schneider
e5fb0a5f4d chore: commit accumulated working tree changes 
Lean: update Semantics modules, add new numerics/physics data files
Hardware: update FPGA bitstreams (tangnano9k_uart_loopback)
Infra: k3s-flake tests, netcup-vps configuration, VCN compute substrate
Docs: ARCHITECTURE, specs, citation updates
 2026-05-30 00:10:02 -05:00
Brandon Schneider
a8a163650a fix(infra): resolve k3s services startup issues and update ingress configuration 
- Revert builder image in embedded_surface Dockerfile to rust:bookworm for glibc compatibility with bookworm-slim runner.
- Replace racknerd-surface.json Git LFS pointer with the actual retrieved JSON config to resolve parse errors.
- Separated and imported regenerated docker images for stubs and credential-server on all cluster nodes.
- Fixed Ingress resources and middleware to target the correct authentik-server service name.
- Rewrote verification script checks to handle iptables-based ports and relative redirects robustly.
- Updated nixos-laptop IP in node topology documentation.

Build: 0 jobs, 0 errors
 2026-05-27 17:39:00 -05:00
Allaun Silverfox
3dcb5a2d0a chore(pending): quarantine python ENE ContextStream MCP surface (Lean-first unification) 2026-05-26 17:59:30 -05:00
Allaun Silverfox
d200887e42 cleanup(ene): make provenance node/lake_seed/tailscale_ip configurable via env vars 2026-05-26 16:39:43 -05:00
Allaun Silverfox
c4c50bb78f cleanup(ene): read canonical SQL schema file for chat tables; remove embedded DDL string 2026-05-26 16:34:23 -05:00
Allaun Silverfox
b8cc57943e cleanup(ene): add canonical SQL schema for chat/session sync tables 2026-05-26 16:23:51 -05:00
Brandon Schneider
f73d49b342 Add proof worker pool routing 2026-05-25 22:27:18 -05:00
Brandon Schneider
e18a284082 Wire ENE context into remote proof checks 2026-05-25 22:07:58 -05:00
Brandon Schneider
b57e1c2740 Stabilize ENE API and context shim 2026-05-25 21:06:46 -05:00
Brandon Schneider
51408bb590 Stabilize remote proof endpoint and RDS shims 2026-05-25 20:48:25 -05:00
Brandon Schneider
d4180194d7 archive: remove experimental tools-scripts, scripts, and shim probes 
- Move 38 experimental tools-scripts directories to archive/ (famm, ptos, crypto, market, geoweird, cognitive, carrier, tsm, semi_jack, hachimoji, chemistry, bt20, optimization, gpgpu, hardware, infrastructure, defense, security, connectome, encoding, formula_optimization, manifold, metafoam, model, verifier, substrate, audio, ingestion, literature, domain, crossbreed, external, physics, pipeline, design, classification, database, dashboard, monitor, braid, compression, waveprobe, data, ingested, demo, publish, blockchain, regret, simulation, build)
- Move 386 one-shot scripts to archive/ (ask_swarm*, execute*, swarm_* probes, test_* scripts, computational controllers, topology experiments, shell scripts)
- Move 2124 experimental shim probe files to archive/ (research probes, prior*, metaprobe*, erdos*, blockchain*, hutter*, tang9k*, stellar_gas*, enwiki*, quandela* probes, experimental shell scripts, ffmpeg-plugins, erdos_surface_orchestrator, codebase-memory, receipts, data files, MCP bus probes)
 2026-05-25 18:14:31 -05:00
Brandon Schneider
073a70eb86 WIP: accumulated changes 2026-05-25 16:24:21 -05:00
Brandon Schneider
e6f770324d fix(rds): restore ENE storage observation probe 2026-05-20 18:53:55 -05:00
Brandon Schneider
51a1898e11 ene-session-sync: complete Python→Rust port; fix all 6 pre-existing test failures 
**New Rust modules (batch 2 — 9 files)**
- src/deepseek_adapter.rs    — DeepSeek/Ollama chat + DeepSeekProver
- src/ene_cloud_credential_manager.rs — ENE cloud credential + node balancer (SQLite)
- src/enhanced_swarm.rs      — enhanced swarm stub
- src/gemma_integration.rs   — SQLite task queue for Gemma 4 model tasks
- src/hyperbolic_encoding.rs — Poincaré disk math, HyperbolicManifoldEncoder
- src/knowledge_ingestion.rs — WolframAlpha, OpenMath, nLab wiki adapters
- src/manifold_perception.rs — filesystem manifest scanner / topological report
- src/s3c_lean_review.rs     — CLI adapter submitting S3C.lean to Gemma4Integration
- src/search_adapter.rs      — Google (stub) + Brave search providers

All 9 wired into main.rs as mod declarations.

**Test fixes (6 pre-existing failures → 0)**
- s3c.rs: fix shell decomp width formula (a+b not a+b+1); correct test
  expectations for n=9 (b=7, not b=1); invariant a+b=2k+1 not 2k
- math.rs: fix test_avg_chain expected avg to 10/6 (all-pairs average,
  not just A→* paths)
- ene_core.rs: fix AES-GCM decrypt AAD mismatch in retrieve_sensitive_data —
  SELECT now fetches pkg column and passes it as AAD (matches store path)
- hyperbolic_encoding.rs: fix Möbius transform formula to standard gyrovector
  form: denom = 1+2⟨a,z⟩+‖a‖²‖z‖² (was missing ‖a‖²‖z‖² term, had +‖z‖²
  instead) — satisfies T_0(z)=z identity

**cargo test: 145 passed, 0 failed**

**Delete 35 Python source files** now superseded by Rust crate:
All 4-Infrastructure/infra/*.py and embedded_surface/server.py removed.

**Deploy scripts updated** to use rs-surface binary instead of Python:
- gcl_edge_in_place_upgrade.sh: CURRENT_SERVER → rs-surface binary; validate
  with test -x; smoke-test exec binary directly; rollback saves rs-surface
- xen_alpine/install_rs_surface_openrc.sh: SERVER_SRC → musl release binary;
  drop python3 from apk; install as rs-surface (not server.py)
- xen_alpine/run_qemu_alpine_surface.sh: default SURFACE_IMPL=rust; RUST_BIN
  var for musl binary; else-branch copies rs-surface; boot script exec binary
- recover_credential_server.sh: upload rs-surface binary; ExecStart → binary
  with RS_SURFACE_PORT=8444 (credential endpoint built into rs-surface /credentials)
- nixos-setup-cred-server.sh: same — ExecStart uses /opt/rs-surface/rs-surface

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-05-19 14:44:19 +00:00
Brandon Schneider
ec4166edb2 flake.nix: add rs-surface binary + OCI image outputs; fix Dockerfile base 
flake.nix:
- rsSurface: pkgs.rustPlatform.buildRustPackage for
  4-Infrastructure/infra/embedded_surface/rs-surface — uses cargoLock.lockFile
  (no cargoHash/cargoSha256 for Cargo.lock-based fetching).
- rsSurfaceImage: pkgs.dockerTools.buildLayeredImage — pure Nix OCI tarball,
  no Alpine, no musl. Closure: rs-surface binary + cacert + coreutils.
  Healthcheck, ExposedPorts, Env, Labels all set.
- packages.${system}.rs-surface and .rs-surface-image registered.

Build:
  nix build .#rs-surface          # binary only
  nix build .#rs-surface-image    # OCI tarball → docker load < result

Dockerfile:
- Switch fallback runtime from alpine:3.20 → debian:bookworm-slim (glibc,
  no musl compat issues). Builder stage debian:bookworm-slim with libssl-dev.
- Added comment directing users to `nix build .#rs-surface-image` as the
  canonical build path on NixOS.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-05-19 14:14:17 +00:00
Brandon Schneider
a9f3d3d487 ene-session-sync: add meta_autotype.rs; embedded_surface: Rust rs-surface daemon 
- src/meta_autotype.rs (396L): port ene_meta_autotype.py — deterministic
  ContingentField classifier with scalar_type, bind_class, surface_hint,
  autotype_payload, handle_request, and 3 unit tests (all green).
- Cargo.toml: wire meta_autotype mod, add tempfile dev-dep for existing tests.
- embedded_surface/rs-surface/ (1025L): full Rust port of server.py.
  - axum 0.7 HTTP server + WebSocket binary surface-frame protocol.
  - All 16 op-codes: HEALTH, STATUS, METRICS, ATTEST, COMPRESS, RGFLOW,
    ROUTE, MOUNT_STATUS, SNAPSHOT, ENTER_RECOVERY, PRIMITIVES, PLAN_ROUTE,
    WIKI, FRACTAL_FOLD, META_AUTOTYPE, CREDENTIALS.
  - Inline omni_lut choose_route (choose_route fn, shannon_entropy,
    detect_sequence_surface, s3c_shear) — no Python deps.
  - Inline meta_autotype_payload — mirrors handle_request from ene_meta_autotype.py.
  - Same wire framing: [v][flags][codec][op][req_id][len][crc32][payload].
- Dockerfile: multi-stage Rust builder (alpine) + minimal runtime image;
  Python server.py kept as fallback alongside binary.
- cargo build: 0 errors on both crates; meta_autotype tests: 3/3 pass.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-05-19 14:11:47 +00:00
Brandon Schneider
2d2b71eb93 ene-session-sync: fix compile errors, add Cargo.lock, delete lean_unified_shim 
- Add rusqlite::OptionalExtension import to ene_core.rs (fixes 3 .optional() errors)
- Fix borrow conflict in compression.rs DeltaGclService::compress_manifest
  (split previous.is_some() out before mutable insert)
- Fix serde_json Map::get type error in compression.rs (use .as_str())
- Fix tokio::process::Child moved-after-use in misc.rs (capture child.id() before
  wait_with_output consumes child; use system kill for timeout case)
- Fix misc.rs S3CIterativeImprover field names (emit/j_score.total/handles.handle_a)
- Fix wiki.rs stmt lifetime error (bind collect result to named variable)
- Add Cargo.lock (new dep resolution for sha2, base64, hex, aes-gcm)
- Delete lean_unified_shim.py (minimal SwarmAPISystem shim — replaced by Rust)

Build: cargo build passes with 0 errors (8 dead-code warnings only).

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-05-19 13:58:44 +00:00
Brandon Schneider
03100f0a1c Rust ene-session-sync: complete rewrite with Python adaptation layer 
Major expansion of the ene-session-sync Rust service with 9 new modules
(~9.8k lines) that mirror the existing Python infrastructure:

New Rust modules:
- compression.rs: Delta GCL compression service (ports delta_gcl_compression_service.py)
- credential.rs: Credential provider + minimal HTTP credential server
- ene_core.rs: Core ENE data structures and operations
- fractal_fold.rs: Fractal folding algorithms and data structures
- math.rs: Mathematical utilities and Q16_16 fixed-point arithmetic
- misc.rs: Miscellaneous utilities and helpers
- s3c.rs: S3-compatible storage client abstraction
- swarm.rs: Swarm API client and orchestration
- topology.rs: Topology management and 5D torus operations
- wiki.rs: Wiki/TiddlyWiki ingestion and processing

Cargo.toml:
- Add crypto deps: aes-gcm, sha2, hex, base64
- Keep existing deps: anyhow, chrono, clap, reqwest, rusqlite, serde, tokio, etc.

Lean proof quarantine:
- Replace detailed proofs with sorry + TODO(lean-port) placeholders in:
  * FiveDTorusTopology.lean (torusDistanceSymmetric, torusDiameterFormula)
  * PISTMachine.lean (mirror_preserves_mass, zero_mass_iff_square)
  * TorsionalPIST.lean (torsionPreservesMass)
  * SubagentOrchestrator.lean (subagentInvariantPreservation)
- Simplify UnifiedCompression.lean proof structure
- Add TODO comments for future Lean proof completion

This creates a complete Rust foundation that can operate independently
while the Lean proofs are completed incrementally.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-05-19 13:53:19 +00:00
Brandon Schneider
2cc9fc8dc7 Rust ene-session-sync: complete rewrite with Python adaptation layer 
- source.rs: Add ClawSource adapter that reads .claw/sessions/*.jsonl (skips
  LFS stubs), parses both full-JSON and JSONL-stream formats, normalises into
  ChatSession/ChatMessage identical to the OpenCode path.

- models.rs: Add title/agent/model fields to ChatSession; remove unused
  chrono import.

- normalize.rs: Populate title/agent/model from OpenCodeSession.

- sink.rs: Full rewrite:
  - DDL adds title/agent/model columns + ALTER TABLE IF NOT EXISTS migration
    path for existing clusters.
  - upsert_session and upsert_messages parameter counts match placeholders
    ($19 and $13 respectively).
  - FTS index uses COALESCE to handle NULL text_content.
  - sslmode= token stripped before parse so NoTls connect doesn't reject
    it; README documents the SSL upgrade path (tokio-postgres-rustls).
  - with-serde_json-1 feature added to tokio-postgres dep for JSONB params.

- main.rs:
  - Replace DefaultHasher sha256_text with FNV-1a 64-bit (correct hex digest).
  - Add ClawSync, List, Get subcommands.
  - Remove unused Context import.

- bridge.rs: Rewrite to use bridge_wrapper.py subprocess rather than inline
  -c script; resolves wrapper path relative to infra_dir or CARGO_MANIFEST_DIR;
  unwraps {"ok":true,"data":{}} envelope; health_check() tests python3 only.

- embed.rs: Remove unused OllamaEmbedRequest import.

- systemd/: Add ene-session-sync.service + .timer (5-min cadence, 2-min
  OnBootSec, Persistent=true for catch-up after suspend).

- README.md: Architecture diagram, CLI reference, schema, build/install,
  systemd install, Python bridge setup.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-05-19 09:16:19 +00:00
Brandon Schneider
8e5efbe11f Fix Dependabot vulns: update sqlx 0.7->0.8.6, add ene-rds Cargo.lock 
Security fixes:
- rds_probe: bump sqlx 0.7.4 -> 0.8.6 (GHSA-xmrp-424f-vfpx MEDIUM)
  Also removes transitive rustls-webpki 0.101.7 dependency chain,
  fixing GHSA-82j2-j2ch-gfr8 (HIGH), GHSA-xgp8-3hg3-c2mh,
  GHSA-965h-392x-2mh5 (LOW) in this workspace.
- ene-rds already has rustls-webpki 0.103.13 (patched); commit its
  Cargo.lock for reproducible builds.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-05-19 08:09:34 +00:00
Brandon Schneider
9d450756c1 Expand ene-rds Rust workspace and refine devcontainer 
- Add ene-node, ene-rds-chat, ene-rds-ephemeral, ene-rds-wiki,
  ene-storage, ene-sync crates to the ene-rds workspace
- Extend ene-rds-core with shared types and cluster messaging
- Add review doc and wiki page for the RDS Rust workspace
- Fix devcontainer: custom /etc/passwd with researcher user,
  add gcc/glibc/binutils for VS Code server compatibility,
  include util-linux, gnutar, gzip; add LD_LIBRARY_PATH
- Ignore Nix build output (`result`) and Rust artifacts

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-05-19 07:52:39 +00:00
Brandon Schneider
fd863af6fd Expand devcontainer with full Python stack, add MCP servers (Notion/AWS), strengthen Lean theorems 
- .devcontainer/Dockerfile: add PostgreSQL client libs, OpenSSL/libffi headers, gfortran/BLAS for scipy, rclone; install full Python dependency set (boto3, psycopg2-binary, fastapi, uvicorn, notion-client, httpx, pytest, numpy, scipy, etc.) in uv-managed venv; add rclone S3 gateway init script as ENTRYPOINT
- .devcontainer/devcontainer.json: switch from build to pre-built image (localhost/research
 2026-05-19 01:52:14 -05:00
Brandon Schneider
edab29b0d5 Fix Porkbun DNS, automate postgres backups, fix rds_probe vulns 
- Porkbun API keys verified working (SUCCESS ping)
- Automated daily AppFlowy postgres backups via systemd timer
- rds_probe: switch from rustls to native-tls (fixes 3 Dependabot alerts)
- ec2-config.nix: add appflowy-backup service and timer
 2026-05-18 11:52:01 -05:00
Brandon Schneider
9a97983a64 Add apiProvider service kind, credential gateway, and cupfox routing 
- TopologyNode.lean: apiProvider service kind with network-only requirement
- server.py: OP_CREDENTIALS handler, RDS-backed wiki/fractal fold layers,
  /credentials HTTP endpoint via SurfaceHandler
- cupfox-config.nix: Caddy routes for /api/credentials/ -> EC2 (100.69.1.43)
  and /git/ -> local Forgejo
 2026-05-18 10:58:35 -05:00
Brandon Schneider
c957fac96e Add EC2 recovery backup: NixOS config, AppFlowy compose/env template, credential server bootstrap, recovery guide 
- ec2-configuration.nix: full NixOS config for aws-nixos-node-1
- docker-compose.minimal.yml: AppFlowy Cloud compose with search_path fix
- .env.example: sanitized AppFlowy env template
- nixos-setup-cred-server.sh: credential server bootstrap
- RECOVERY.md: step-by-step rebuild instructions
- .gitignore: secrets dir excluded
- credential_provider.py reverted to repo HEAD (EC2 had hardcoded AWS creds)
- racknerd_root.txt removed from working tree
 2026-05-18 10:44:23 -05:00
Brandon Schneider
c5b6a31a8b feat: add RDS probe tool, credential server, and Notion/Linear ingestion pipeline 
New infrastructure components:
- rds_probe: Rust database inspection tool with IAM auth
- credential_server.py: REST credential provider server
- credential_provider.py: credential resolution chain
- ene_rds_fractal_fold.py / ene_rds_wiki_layer.py: RDS-backed ENE layers
- import_dumps_to_rds.py / export_linear_from_rds.py: ingestion pipeline
- recover_credential_server.sh: deployment script (sanitized)

Sanitize hardcoded secrets across codebase:
- Strip API keys from recover_credential_server.sh → env var lookups
- Replace hardcoded Wolfram appid (HYJE3R3R63) → env var in 5 scripts
- Strip fallback key values from config/index.js
- Add .claude/ and optimized_basis_v3.bin to .gitignore

Ingested 2,685 records into RDS: 2,421 Linear issues + 264 wiki pages
 2026-05-18 00:31:44 -05:00
Brandon Schneider
a6311ed940 chore: preserve working tree before secure wipe 
- Update .gitignore with **/target/ for Rust build artifacts
- Add eval receipts to UniversalBridge.lean (compile-time verification comments)
- Add PCIe Idle-Cycle Compute Harvester to ROADMAP.md
- Clean up deprecated scripts, generated Verilog, and old tools (23 deletions)
- Stage new infrastructure: Xen/Alpine embedded surface, QFOX topology manager
- Stage new probes: boundary activation field, holographic carving
- Stage new applications: finance manager, script roots
- Stage new research spec: PCIe idle-cycle substrate
 2026-05-13 17:36:02 -05:00
Brandon Schneider
a99e839bab Track remaining source and documentation inventory 2026-05-11 22:18:31 -05:00
Brandon Schneider
1225404b72 Point ENE scripts at shared data 2026-05-11 22:10:55 -05:00
Brandon Schneider
eff316ff3f wip: refined investigation script for Erdős–Gyárfás conjecture 
Created refined investigation script for Erdős–Gyárfás conjecture
where previous test found no power-of-two cycles (conjecture holds: False).

Refinements:
- Regular graph construction (all vertices same degree)
- Exhaustive DFS cycle detection
- More samples per n (5 instead of 3)
- Extended n values [8, 10, 12, 14, 16]

Goal: Determine if previous negative result was due to random graph construction
or if regular graphs with exhaustive cycle detection find power-of-two cycles.

Script created: investigate_erdos_gyarfas_refined.py
Execution canceled by user - awaiting further instructions.
 2026-05-08 14:50:03 -05:00
Brandon Schneider
0cf775c80e collapse: prover orchestration layers, FAMM verilator harness, swarm topological prober, spec sheets, virtual FPGA system tests, merge conflict resolution 
- Prover-Integrated Orchestration Layers (L0-L3): Goedel-Prover-V2 watchdog, BFS-Prover-V2 swarm consensus, bf4prover topology adaptation
- FAMM Verilator benchmark: uniform vs preshaped delay comparison (4.4x speedup)
- Swarm topological device prober: 11 agents probing traces, caps, delays, errors, vias, PDN
- Spec sheet puller: 10 components with key params and topological relevance
- Virtual FPGA system tests: 6/6 passed, 134K ops/s throughput
- Fixed merge conflicts in AI-Newton test_experiment.ipynb
 2026-05-06 23:42:01 -05:00
Brandon Schneider
0709b298b3 Consolidate research stack updates 2026-05-05 21:09:48 -05:00
Brandon Schneider
5f88abf618 initial: sovereign research stack (consolidated, weightless, and lfs-optimized) 2026-05-04 18:11:36 -05:00