Commit graph

630 commits

Author SHA1 Message Date
Brandon Schneider
aae680ad3c chore(infra): add internal routing smoke-test script
scripts/verify-internal.sh runs the exact verification sequence before
the edge is touched. Checks (in order):

  0. Port listeners — host Caddy owns :80, Traefik NodePort on :30080
  1. GET / — 200 Homer or 302 → auth.researchstack.info (not internal IP)
  2. /api/* bypass — none of /api/*/health may 302 to Authentik
  3. auth.researchstack.info — 200/302 from Authentik, no internal-IP loop
  4. Host header passthrough — Traefik :30080 /ping reachable; /api/jobs/health
     routes end-to-end through host Caddy → Traefik → Ingress
  5. X-Forwarded-Proto — https header survives the server-Caddy hop

Usage:
  # on nixos-laptop directly:
  bash 4-Infrastructure/k3s-flake/scripts/verify-internal.sh

  # from any tailnet node:
  bash 4-Infrastructure/k3s-flake/scripts/verify-internal.sh --remote

Exit 0 = safe to deploy edge. Exit 1 = fix red checks first.

Generated with Devin (https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-26 21:08:58 -05:00
Brandon Schneider
8d1d94b256 feat(k3s-edge): rewrite edge Caddy as dumb TLS forwarder + legacy 301s
Edge Caddy now does exactly three things:
1. Terminate TLS for researchstack.info + *.researchstack.info via
   Porkbun DNS-01 (wildcard cert covers all subdomains in one renewal)
2. 301-redirect legacy subdomains to canonical path equivalents:
     status.*  → /server/status/
     dash.*, home.*  → /
     media.*  → /apps/jellyfin/
     books.*  → /apps/books/
     music.*  → /apps/music/
     vault.*  → /server/vault/
     pulse.*  → /api/registry/
     apps.*  → /apps/
     *.* (wildcard fallback)  → /
3. Forward all other traffic to the internal router (host Caddy :80 on
   k3s-server over Tailscale) with X-Forwarded-* headers preserved.
   auth.* and mail/webmail.* are forwarded unchanged (stable subdomains).

No path routing logic on the edge. Traefik Ingress (k3s-server) owns
all path decisions. This commit has no effect until nixos-rebuild switch
is run on microvm-racknerd (deploy after k3s-server is verified).

Generated with Devin (https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-26 21:06:06 -05:00
Brandon Schneider
38f1022a20 feat(k3s-server): Traefik NodePort + host Caddy pass-through (internal-only)
Port conflict resolution:
- Add HelmChartConfig to pin Traefik web entrypoint to NodePort 30080
  (not host :80) so k3s Traefik and host Caddy do not race for the port
- Add host Caddy on :80 as a minimal pass-through to Traefik :30080;
  carries X-Forwarded-* headers so Traefik sees the real client IP and
  the correct Host. No TLS, no Porkbun, no subdomain logic — all of
  that stays on the edge Caddy (k3s-edge.nix)
- Caddy after= k3s.service so Traefik NodePort is ready before proxying

Authentik port fix:
- Change authentik server + worker services from NodePort 30080 to
  ClusterIP; Traefik reaches Authentik via the rs-auth Ingress and
  cluster DNS, no NodePort required

New manifests (internal, no public-traffic impact):
- manifests/ingress/: Traefik Ingress resources + Middleware CRDs
  (/apps/*, /server/* → forward_auth + strip-prefix; /api/* → strip only;
  / → Homer + forward_auth; auth.* → Authentik, no middleware)
- manifests/hermes/: placeholder chat/orchestrator service
- manifests/credential-server/: token-auth credential vault stub
- manifests/control-plane/: registry-api, jobs-api, blobs-api health stubs
- manifests/homer/configmap.yaml: updated dashboard links to canonical paths

Deploy order: rebuild k3s-server first, verify Traefik + Ingress
internally, then deploy k3s-edge (commit 3 / next step).

Generated with Devin (https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-26 21:03:54 -05:00
Brandon Schneider
f49548fe85 test(infra): add Playwright E2E routing test suite
Tests the full traffic path against live researchstack.info infrastructure:
  Edge Caddy (TLS) → Tailscale → Traefik Ingress → k3s services

Coverage:
- edge-tls-redirects: HTTPS reachability, cert validity, legacy subdomain
  301s (status/dash/home/media/books/music/vault/pulse/apps), stable
  subdomains (auth, mail), wildcard fallback
- path-routing: /apps/*, /server/*, /api/* routes; prefix stripping; SSO
  redirect vs token-auth isolation
- auth-integration: Authentik login page, OIDC discovery, forward_auth
  gating on protected paths, /api/* bypass

19/40 tests pass against current live infrastructure (pre-deploy). The 21
failures are "not yet deployed" signals, not design errors. Run after each
phase of the deployment plan to use as a regression gate.

Run: cd 4-Infrastructure/k3s-flake/tests && npm test

Generated with Devin (https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-26 21:03:27 -05:00
Allaun Silverfox
87f124f42a feat(mcp): route Lean-defined tools via external cmd map 2026-05-26 18:58:49 -05:00
Allaun Silverfox
59886851d3 feat(runtime): register mcp_surface_router module 2026-05-26 18:26:44 -05:00
Allaun Silverfox
c155a51373 feat(runtime): add generic Lean surface tool command router (CLAW_MCP_TOOL_CMD_MAP) 2026-05-26 18:25:03 -05:00
Allaun Silverfox
52bb3fe674 feat(runtime): register ene_context_tools shim module 2026-05-26 18:12:56 -05:00
Allaun Silverfox
c6d24cc5c4 feat(runtime): add ENE context tool dispatch shim (command-based) 2026-05-26 18:11:39 -05:00
Allaun Silverfox
d50699b619 docs(runtime): update Lean manifest cmd example to ENE surface toolsJson 2026-05-26 18:06:56 -05:00
Allaun Silverfox
48da4d058c feat(lean): define ENE context MCP surface (Lean-owned) 2026-05-26 18:06:00 -05:00
Allaun Silverfox
4b35b95351 chore(pending): move python ENE ContextStream MCP surface into pending quarantine 2026-05-26 17:59:51 -05:00
Allaun Silverfox
bc478a2a8d chore(pending): move python Notion+Linear MCP server into pending quarantine 2026-05-26 17:59:40 -05:00
Allaun Silverfox
a58090f356 chore(pending): quarantine python ENE ContextStream MCP surface (Lean-first unification) 2026-05-26 17:59:30 -05:00
Allaun Silverfox
f5d683242e chore(pending): quarantine python Notion+Linear MCP server (Lean-first unification) 2026-05-26 17:59:19 -05:00
Allaun Silverfox
f31752bbcc docs(pending): explain quarantine policy for non-Lean tool surfaces 2026-05-26 17:57:16 -05:00
Allaun Silverfox
4cd0cc27c2 chore(pending): move python MCP server into pending quarantine 2026-05-26 17:56:57 -05:00
Allaun Silverfox
ae967c36fd chore(pending): quarantine python MCP server (Lean-first unification) 2026-05-26 17:56:25 -05:00
Allaun Silverfox
7f5b7e92b7 feat(runtime): register lean_mcp_manifest module 2026-05-26 17:50:05 -05:00
Allaun Silverfox
0fa23e0a02 feat(runtime): source tools/list from Lean manifest env var when spec.tools empty 2026-05-26 17:49:55 -05:00
Allaun Silverfox
466e9b68ea feat(runtime): load MCP tools list from Lean manifest JSON (shim-only) 2026-05-26 17:42:33 -05:00
Allaun Silverfox
dcd3373874 feat(lean): define MCP surface manifest schema for JsonL connector tools 2026-05-26 17:32:55 -05:00
Allaun Silverfox
11d538e96b docs: add Lean-first boundary contract (Lean vs shims, receipts, float ban) 2026-05-26 17:31:54 -05:00
Allaun Silverfox
95806f7779 cleanup(ene): load node list from nodes.yaml instead of hardcoded hostnames 2026-05-26 17:16:02 -05:00
Allaun Silverfox
510ff80d9f cleanup(ene): use nodes.yaml inventory instead of hardcoded host resource map 2026-05-26 17:15:50 -05:00
Allaun Silverfox
f763372d03 cleanup(ene): make obsidian shim provenance configurable and schema-complete 2026-05-26 16:54:12 -05:00
Allaun Silverfox
29f93f9dfd cleanup(ene): make provenance node/tailscale configurable via env vars and CLI arg 2026-05-26 16:42:03 -05:00
Allaun Silverfox
791e75368a cleanup(ene): make provenance node/tailscale configurable via env vars and CLI arg 2026-05-26 16:40:35 -05:00
Allaun Silverfox
4914c4e921 cleanup(ene): make provenance node/lake_seed/tailscale_ip configurable via env vars 2026-05-26 16:39:43 -05:00
Allaun Silverfox
3e0aa86c17 cleanup(ene): read canonical SQL schema file for chat tables; remove embedded DDL string 2026-05-26 16:34:23 -05:00
Allaun Silverfox
f7d118a08f cleanup(ene): add canonical SQL schema for chat/session sync tables 2026-05-26 16:23:51 -05:00
Allaun Silverfox
f6c415bebd cleanup(ene): remove hardcoded schema path, drop inline schema fallback, add legacy shim strip receipt 2026-05-26 16:22:40 -05:00
Allaun Silverfox
7f63d2280f docs(shim): annotate alignment shim as legacy pending AVM port; add strip receipt metadata 2026-05-26 16:14:52 -05:00
Allaun Silverfox
e19a6a56c7 docs(shim): annotate as legacy shim pending AVM port; add strip receipt metadata 2026-05-26 16:13:43 -05:00
Allaun Silverfox
2609c37c07 feat(avm): add Lean-only strict-typed ISA skeleton (v1) 2026-05-26 16:05:08 -05:00
Allaun Silverfox
8d61cdd482 docs(avm): strengthen float prohibition and documentation requirement 2026-05-26 16:02:05 -05:00
Allaun Silverfox
7e1326e812 docs(avm): add stripping policy and bad-code elimination rules 2026-05-26 15:59:41 -05:00
Allaun Silverfox
5ba91c2a01 docs(avm): redefine AVM as Lean-only ISA with adapter shims/backends 2026-05-26 15:58:25 -05:00
Allaun Silverfox
de2be89666 fix(pist): correct repo root + fail-on-raw-disagreement check 2026-05-26 15:50:29 -05:00
Allaun Silverfox
fd28cace2f docs: add RRC PIST shape-alignment phase 2026-05-26 15:47:19 -05:00
Allaun Silverfox
7d07dd073f feat(pist): add RRC PIST shape-alignment calibration pass 2026-05-26 15:43:45 -05:00
Allaun Silverfox
92fc07f93e archive: remove duplicate PISTMachine search-space copy 2026-05-26 15:42:48 -05:00
Allaun Silverfox
308ce86f27 feat(pist): add RRC PIST validation report cleaner 2026-05-26 15:38:07 -05:00
Allaun Silverfox
689b8130e8 fix(lean): remove stale PISTMachine sorries 2026-05-26 15:31:59 -05:00
Allaun Silverfox
e61e46f9b2 docs: add receipt-density verification sequence 2026-05-26 15:19:07 -05:00
Allaun Silverfox
575b52cb80 feat(pist): add receipt-density sidecar readback validator 2026-05-26 15:18:11 -05:00
Allaun Silverfox
0275b43405 test(pist): add receipt-density injector regression harness 2026-05-26 15:16:40 -05:00
Allaun Silverfox
5033fcac0e feat(pist): use shared rds_connect for receipt-density writer 2026-05-26 15:13:35 -05:00
Brandon Schneider
86f8ff0b21 chore: remove unused import subprocess from v14a (handled by rds_connect) 2026-05-26 15:11:11 -05:00
Brandon Schneider
02f1c928d7 refactor(rds): consolidate 14 psycopg2 connect patterns into shared rds_connect module
Creates 4-Infrastructure/shim/rds_connect.py with a single connect_rds()
function that resolves connection parameters in priority order:
  1. explicit kwargs
  2. DATABASE_URL env var (postgres://user:pass@host:port/dbname?sslmode=...)
  3. individual RDS_* env vars (RDS_HOST, RDS_PORT, RDS_USER, etc.)
  4. built-in defaults

Auth resolution (when password is empty or RDS_IAM=1):
  1. RDS_IAM_TOKEN env var (pre-computed)
  2. boto3 SDK generate_db_auth_token (preferred)
  3. subprocess aws rds generate-db-auth-token (fallback)
  4. RDS_PASSWORD env var (non-IAM)

Replaces 8 connection pattern variants across 14 active shims:
  - subprocess + RDS_IAM_TOKEN fallback: pist_trace_classify_mcp, joint_classifier,
    pist_prove_and_classify, ingest_57_flexures
  - boto3 SDK: ene_wiki_body_reingest, ene_migrate_and_tag, dataset_ingest_rds
  - subprocess + RDS_PASSWORD: batch_embed_artifacts, sync_wiki_to_rds, seed_flexure_dataset
  - RDS_IAM_AUTH: pist_classify
  - bashrc parsed: credential_loader

v1.4a benchmark confirmed at 100% after refactor.
2026-05-26 15:09:34 -05:00