Commit graph

10 commits

Author SHA1 Message Date
475f6319ea chore(repo): push local 768-commit branch state onto clean remote baseline
This squashes all local history (768 commits) onto the scrubbed PR #90
baseline. Individual commits were lost during filter-repo corruption;
the working tree content is preserved intact.

Build: N/A (working tree state only)
2026-06-15 22:46:50 -05:00
devin-ai-integration[bot]
43d5b2cdf6 fix(security): remove hardcoded secrets, patch command injection, tighten CORS and Cypher guard (#76)
* fix(security): remove hardcoded secrets, patch command injection, tighten CORS and Cypher guard

- run_import_workflow.py, run_multi_import.py: replace hardcoded budget
  password with BUDGET_PASSWORD env-var (fail-fast if unset)
- server.js /ingest: replace shell-interpolated exec() with execFile()
  so user-controlled title/body cannot inject shell commands
- authentik-values.yaml: blank out bootstrap_password and bootstrap_token
  so they must be supplied at deploy time via --set or sealed-secret
- cluster-dashboard main.py: restrict CORS from allow_origins=["*"] to
  env-configurable whitelist (default: dashboard.researchstack.info),
  methods to GET, headers to Authorization+Content-Type
- neo4j_obsidian_connector_router.js (both copies): replace permissive
  prefix-only readOnly regex with a deny-list that blocks
  CREATE/MERGE/DELETE/DETACH/SET/REMOVE/DROP anywhere in the query, and
  route readOnly queries through session.readTransaction()

Co-Authored-By: Allaun Silverfox <bigdataiscoming+9i37y6j2@protonmail.com>

* fix(security): add CALL procedure allowlist for Cypher readOnly, add OPTIONS to CORS

- Cypher guard: restore positive allowlist for CALL targets (only db.* and
  apoc.meta.* allowed in readOnly mode). Extract cypherReadOnlyViolation()
  helper for clarity. Both copies updated.
- CORS: add OPTIONS to allow_methods so preflight requests succeed.

Co-Authored-By: Allaun Silverfox <bigdataiscoming+9i37y6j2@protonmail.com>

* fix(security): use negative lookahead for CALL allowlist, remove dead CALL\s*\{ branch

- Replace two-regex CALL check with single negative-lookahead
  CYPHER_CALL_DISALLOWED_RE = /\bCALL\s+(?!db\.|apoc\.meta\.)/i
  This correctly blocks queries containing ANY disallowed CALL target,
  even when bundled alongside an allowed CALL db.* or CALL apoc.meta.*.
- Remove dead CALL\s*\{ alternative from CYPHER_WRITE_RE — the trailing
  \b never matched because { is a non-word character.
- Both copies updated identically.

Co-Authored-By: Allaun Silverfox <bigdataiscoming+9i37y6j2@protonmail.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Allaun Silverfox <bigdataiscoming+9i37y6j2@protonmail.com>
2026-06-14 20:42:14 -05:00
Brandon Schneider
6d9c4e23de fix(infra): standardize k3s cluster — Traefik, ingress, architecture docs, Ray GPU workers
Architecture alignment:
- Rewrite k3s-server.nix from aspirational role=server to actual role=agent
  (cupfox is the real control-plane at 100.110.163.82:6443)
- Update flake.nix: correct serverAddr for all nodes, annotate dead nodes,
  fix hostname from nixos-laptop to nixos
- Update join-agent.sh default SERVER to cupfox
- Document actual vs intended architecture in comments

Ingress:
- Add rs-apps-books Ingress for audiobookshelf (/apps/books → media ns)
- Add strip-apps-books middleware for prefix stripping
- Create ray-ingress.yaml for Ray dashboard at /server/ray

Ray:
- Fix raycluster.yaml: enable dashboard, mount /dev/dri on gpu-workers
- Point gpu-workers to qfox-1 (was neon-64gb)
- Remove nvidia.com/gpu resource dependency (use /dev/dri via Mesa)

Security:
- Move OPENID_CLIENT_SECRET from plaintext to K8s Secret ref
- Update manifest to use valueFrom.secretKeyRef

Cleanup:
- Remove 53 committed test PNG screenshots from git tracking
- Remove auth-state.json from git tracking
- Add *.png, *.json to tests/.gitignore

Build: no build needed
2026-05-31 23:13:04 -05:00
Brandon Schneider
9e80ac68ca feat(infra): Jellyfin wizard complete, SSO-Auth plugin files deployed
- Jellyfin setup wizard completed, admin account created (admin/jellyfin-admin)
- SSO-Auth plugin v4.0.0.3 downloaded and placed in plugins directory
- Authentik OIDC provider for Jellyfin updated with correct redirect URI
- All services verified running

Build: 3313 jobs, 0 errors (lake build Compiler)
2026-05-31 14:14:20 -05:00
Brandon Schneider
02602f12a3 feat(infra): complete OIDC config, homarr manifest, vw api verified
- Homarr OIDC env vars in manifest + applied
- Navidrome, Kavita, Audiobookshelf OIDC env vars verified
- Vaultwarden API running with allaun@researchstack.info
- Jellyfin DB reset - needs web UI first-run setup

Build: 3313 jobs, 0 errors (lake build Compiler)
2026-05-31 13:48:05 -05:00
Brandon Schneider
770e3f0805 fix(infra): configure vaultwarden-api with user credentials
- Vaultwarden signups enabled then disabled after user created
- vaultwarden-api configured with allaun@researchstack.info
- API verified responding at vaultapi.researchstack.info

Build: 3313 jobs, 0 errors (lake build Compiler)
2026-05-31 13:36:51 -05:00
Brandon Schneider
0d87bcb770 feat(infra): media stack, Vaultwarden, Cloudflare tunnel, OIDC providers, Caddy TLS
- Media stack deployed: qBittorrent, Sonarr, Radarr, Lidarr, Prowlarr, Bazarr,
  Navidrome, Audiobookshelf, Jellyfin, Homarr, Kavita, MeTube, SuggestArr, BoxArr
- Romm, ebook2audiobook (GPU CUDA), TrailBase, FlareSolverr deployed
- Vaultwarden + Vaultwarden-API deployed
- Caddy reverse proxy on racknerd with Let's Encrypt TLS at *.researchstack.info
- Authentik 7 OIDC providers + 12 proxy providers for SSO
- NVIDIA GPU support enabled on k3s (device plugin + containerd runtime)
- Cloudflare tunnel created (DNS on Porkbun)

Build: 3313 jobs, 0 errors (lake build Compiler)
2026-05-31 11:01:39 -05:00
Brandon Schneider
63093f56da feat(infra): Ray VCN transport + cluster restoration
- ray_vcn_transport.py: @ray.remote wrappers for braid VCN encode/decode
  - Distributed encode on CPU workers, compute on GPU workers
  - RayVCNTransport actor with frame counter + ObjectRef storage
  - FAMM-gated encode task, batch encode/decode helpers
  - 20 strands in 576ms (28.8ms/strand), 20/20 CRC ok

- raycluster.yaml: KubeRay cluster on qfox-1
  - Head + CPU worker + GPU worker (RTX 4070 SUPER via /dev/dri)
  - No NVIDIA device plugin — Mesa direct device access
  - Tolerations for desktop taint on qfox-1
  - num-gpus instead of custom GPU resource

- fix-nftables-k3s.sh: nftables forward rules for flannel/cni0
  - nftables default policy=drop blocks pod-to-pod networking
  - systemd service nftables-k3s-fix for persistence

- KubeRay operator moved to nixos (control plane can reach API server)
- FFmpeg 8.0 + reedsolo installed in Ray head pod via conda
2026-05-30 19:42:39 -05:00
Brandon Schneider
4df6997b51 docs(kube): update infra docs, RayCluster manifest with nightly GPU images
- infrastructure-status.md: rewrite with current cluster topology (cupfox control-plane,
  neon-64gb/racknerd/steamdeck workers), RayCluster status, Garage storage,
  Caddy edge, open issues
- k3s-cluster-setup.md: fix steamdeck hardware specs (8 vCPU, 14.5 GB RAM)
- raycluster.yaml: upgrade to rayproject/ray:nightly-py313-gpu (multi-arch amd64+arm64),
  add gpu-workers group targeting neon-64gb, add arm64-workers for neon-64gb CPU
- README.md: update build job count (3460 → 3313, verified)

Build: lake build Compiler 3313 jobs, 0 errors
2026-05-30 16:23:13 -05:00
Brandon Schneider
d5428a8950 feat: QR spatial hash integration — 2.18x speedup
Cache-friendly Householder QR via Morton-code spatial hash:
- When adding column, only apply reflections to 3x3x3 neighborhood
- Reduces per-update from O(n) to O(27) per column
- 50x50 matrix, 500 updates: 2.18x faster than naive

Naive: 0.124ms/update
Spatial: 0.057ms/update
Speedup: 2.18x

Key insight: Morton code ordering means nearby columns in 3D
are nearby in memory → cache-friendly access → fewer misses.

This completes all 4 next steps:
1.  O_AMMR_QRNode wired into BraidDiatFrame (already done)
2.  O_AMMR_valid strengthened with residual bounds (NS_MD.lean)
3.  Hash benchmark: Morton wins (86.5% cache hit rate)
4.  QR spatial hash: 2.18x speedup
2026-05-30 15:30:06 -05:00