This squashes all local history (768 commits) onto the scrubbed PR #90
baseline. Individual commits were lost during filter-repo corruption;
the working tree content is preserved intact.
Build: N/A (working tree state only)
Document discrepancies between the aspirational architecture (in the
flake/docs) and what's actually deployed. Covers control plane location,
edge Caddy divergence, Traefik port, cross-namespace services, and
hostname mismatches. Load the research-stack-infrastructure-bootstrap
skill for the actual deployment procedure.
Build: no build needed
- Change middleware address from authentik-server.services.svc.cluster.local
to authentik-server.authentik.svc.cluster.local (Authentik lives in
authentik namespace, not services)
- Create ExternalName service authentik-server in services namespace so
the rs-auth Ingress can route to the cross-namespace service
Build: no build needed
Architecture alignment:
- Rewrite k3s-server.nix from aspirational role=server to actual role=agent
(cupfox is the real control-plane at 100.110.163.82:6443)
- Update flake.nix: correct serverAddr for all nodes, annotate dead nodes,
fix hostname from nixos-laptop to nixos
- Update join-agent.sh default SERVER to cupfox
- Document actual vs intended architecture in comments
Ingress:
- Add rs-apps-books Ingress for audiobookshelf (/apps/books → media ns)
- Add strip-apps-books middleware for prefix stripping
- Create ray-ingress.yaml for Ray dashboard at /server/ray
Ray:
- Fix raycluster.yaml: enable dashboard, mount /dev/dri on gpu-workers
- Point gpu-workers to qfox-1 (was neon-64gb)
- Remove nvidia.com/gpu resource dependency (use /dev/dri via Mesa)
Security:
- Move OPENID_CLIENT_SECRET from plaintext to K8s Secret ref
- Update manifest to use valueFrom.secretKeyRef
Cleanup:
- Remove 53 committed test PNG screenshots from git tracking
- Remove auth-state.json from git tracking
- Add *.png, *.json to tests/.gitignore
Build: no build needed
- Homarr OIDC env vars set (sso-homarr)
- Database created at /data/db.sqlite
- Config schema updated to v2
- Onboard wizard needs final UI completion
Build: 3313 jobs, 0 errors
- Libraries created via API (204 on creation + path update)
- Paths: /media/movies, /media/tv, /media/music
- SSO-Auth plugin Active, SSO button on login page
- Paths may need manual setting through Dashboard UI
Build: 3313 jobs, 0 errors
k3s-edge.nix:
- Added 'ca https://acme-v02.api.letsencrypt.org/directory' to porkbun_tls snippet
- Forces production LE instead of staging
fix-racknerd-tls.sh:
- Checks current Porkbun keys (redacted)
- Verifies production LE CA is configured
- Provides deploy and restart instructions
- Documents how to update Porkbun API keys via sops
- Revert builder image in embedded_surface Dockerfile to rust:bookworm for glibc compatibility with bookworm-slim runner.
- Replace racknerd-surface.json Git LFS pointer with the actual retrieved JSON config to resolve parse errors.
- Separated and imported regenerated docker images for stubs and credential-server on all cluster nodes.
- Fixed Ingress resources and middleware to target the correct authentik-server service name.
- Rewrote verification script checks to handle iptables-based ports and relative redirects robustly.
- Updated nixos-laptop IP in node topology documentation.
Build: 0 jobs, 0 errors
Edge Caddy now does exactly three things:
1. Terminate TLS for researchstack.info + *.researchstack.info via
Porkbun DNS-01 (wildcard cert covers all subdomains in one renewal)
2. 301-redirect legacy subdomains to canonical path equivalents:
status.* → /server/status/
dash.*, home.* → /
media.* → /apps/jellyfin/
books.* → /apps/books/
music.* → /apps/music/
vault.* → /server/vault/
pulse.* → /api/registry/
apps.* → /apps/
*.* (wildcard fallback) → /
3. Forward all other traffic to the internal router (host Caddy :80 on
k3s-server over Tailscale) with X-Forwarded-* headers preserved.
auth.* and mail/webmail.* are forwarded unchanged (stable subdomains).
No path routing logic on the edge. Traefik Ingress (k3s-server) owns
all path decisions. This commit has no effect until nixos-rebuild switch
is run on microvm-racknerd (deploy after k3s-server is verified).
Generated with Devin (https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Port conflict resolution:
- Add HelmChartConfig to pin Traefik web entrypoint to NodePort 30080
(not host :80) so k3s Traefik and host Caddy do not race for the port
- Add host Caddy on :80 as a minimal pass-through to Traefik :30080;
carries X-Forwarded-* headers so Traefik sees the real client IP and
the correct Host. No TLS, no Porkbun, no subdomain logic — all of
that stays on the edge Caddy (k3s-edge.nix)
- Caddy after= k3s.service so Traefik NodePort is ready before proxying
Authentik port fix:
- Change authentik server + worker services from NodePort 30080 to
ClusterIP; Traefik reaches Authentik via the rs-auth Ingress and
cluster DNS, no NodePort required
New manifests (internal, no public-traffic impact):
- manifests/ingress/: Traefik Ingress resources + Middleware CRDs
(/apps/*, /server/* → forward_auth + strip-prefix; /api/* → strip only;
/ → Homer + forward_auth; auth.* → Authentik, no middleware)
- manifests/hermes/: placeholder chat/orchestrator service
- manifests/credential-server/: token-auth credential vault stub
- manifests/control-plane/: registry-api, jobs-api, blobs-api health stubs
- manifests/homer/configmap.yaml: updated dashboard links to canonical paths
Deploy order: rebuild k3s-server first, verify Traefik + Ingress
internally, then deploy k3s-edge (commit 3 / next step).
Generated with Devin (https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Tests the full traffic path against live researchstack.info infrastructure:
Edge Caddy (TLS) → Tailscale → Traefik Ingress → k3s services
Coverage:
- edge-tls-redirects: HTTPS reachability, cert validity, legacy subdomain
301s (status/dash/home/media/books/music/vault/pulse/apps), stable
subdomains (auth, mail), wildcard fallback
- path-routing: /apps/*, /server/*, /api/* routes; prefix stripping; SSO
redirect vs token-auth isolation
- auth-integration: Authentik login page, OIDC discovery, forward_auth
gating on protected paths, /api/* bypass
19/40 tests pass against current live infrastructure (pre-deploy). The 21
failures are "not yet deployed" signals, not design errors. Run after each
phase of the deployment plan to use as a regression gate.
Run: cd 4-Infrastructure/k3s-flake/tests && npm test
Generated with Devin (https://cli.devin.ai/docs)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
