feat(k3s-server): Traefik NodePort + host Caddy pass-through (internal-only)

Port conflict resolution:
- Add HelmChartConfig to pin Traefik web entrypoint to NodePort 30080
  (not host :80) so k3s Traefik and host Caddy do not race for the port
- Add host Caddy on :80 as a minimal pass-through to Traefik :30080;
  carries X-Forwarded-* headers so Traefik sees the real client IP and
  the correct Host. No TLS, no Porkbun, no subdomain logic — all of
  that stays on the edge Caddy (k3s-edge.nix)
- Caddy after= k3s.service so Traefik NodePort is ready before proxying

Authentik port fix:
- Change authentik server + worker services from NodePort 30080 to
  ClusterIP; Traefik reaches Authentik via the rs-auth Ingress and
  cluster DNS, no NodePort required

New manifests (internal, no public-traffic impact):
- manifests/ingress/: Traefik Ingress resources + Middleware CRDs
  (/apps/*, /server/* → forward_auth + strip-prefix; /api/* → strip only;
  / → Homer + forward_auth; auth.* → Authentik, no middleware)
- manifests/hermes/: placeholder chat/orchestrator service
- manifests/credential-server/: token-auth credential vault stub
- manifests/control-plane/: registry-api, jobs-api, blobs-api health stubs
- manifests/homer/configmap.yaml: updated dashboard links to canonical paths

Deploy order: rebuild k3s-server first, verify Traefik + Ingress
internally, then deploy k3s-edge (commit 3 / next step).

Generated with Devin (https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

4-Infrastructure/k3s-flake/k3s-server.nix

@@ -2,18 +2,41 @@
 
 {
   ##########################################################################
-  # EDIT HERE — set your domain
+  # k3s-server.nix — Control plane + Traefik Ingress
-  # The domain below is used for TLS certificates via Porkbun DNS challenge.
+  #
-  # Set PORKBUN_API_KEY and PORKBUN_SECRET_KEY in /etc/caddy/porkbun.env
+  # This node runs k3s in server mode. Traefik (k3s built-in) handles all
-  # (sops-injected via porkbun-env.age).
+  # path-based routing inside the cluster via Ingress resources defined in
+  # manifests/ingress/.
+  #
+  # Port layout:
+  #   :80    — host Caddy pass-through (owns the port, Traefik cannot)
+  #   :30080 — Traefik web NodePort (ServiceLB, bound by k3s)
+  #
+  # Traffic flow (internal leg):
+  #   Edge Caddy (TLS, Porkbun, subdomain redirects) → Tailscale
+  #     → host Caddy :80 → Traefik :30080 → k3s services (via Ingress)
+  #
+  # All TLS, Porkbun DNS-01, subdomain 301s, and wildcard catch-alls are
+  # handled exclusively by the edge Caddy in k3s-edge.nix. Nothing here
+  # does TLS or subdomain routing — this is a plain HTTP bridge.
+  #
+  # Traefik NodePort is configured via HelmChartConfig so it does not race
+  # with the host Caddy for port 80.
+  #
+  # URL contract (defined in manifests/ingress/ingress.yaml):
+  #   /                    → Homer directory
+  #   /apps/chat/*         → Hermes (chat/orchestrator)
+  #   /apps/budget/*       → Actual Budget
+  #   /server/status/*     → Uptime Kuma
+  #   /server/dash/*       → Homarr
+  #   /server/vault/*      → Vaultwarden
+  #   /api/cred/*          → Credential Server
+  #   /api/registry/*      → Registry API (worker join/heartbeat)
+  #   /api/jobs/*          → Job Router
+  #   /api/blobs/*         → Blob Plane
+  #   auth.researchstack.info → Authentik (stable OIDC issuer, via rs-auth Ingress)
   ##########################################################################
 
-  sops.secrets.porkbun-env = {
-    sopsFile = ./secrets/porkbun-env.age;
-    format = "yaml";
-    path = "/etc/caddy/porkbun.env";
-  };
-
   sops.secrets.authentik-secrets = {
     sopsFile = ./secrets/authentik-secrets.age;
     format = "yaml";
@@ -24,7 +47,6 @@
     role = "server";
     clusterInit = true;
     extraFlags = [
-      "--disable=traefik"
       "--tls-san=100.102.173.61"
       "--tls-san=researchstack.info"
       "--tls-san=nixos-laptop"
@@ -34,69 +56,71 @@
     ];
   };
 
-  systemd.services.caddy.serviceConfig.EnvironmentFile = [ "/etc/caddy/porkbun.env" ];
+  # ── Traefik entrypoint config (HelmChartConfig CRD) ────────────────────
+  # k3s ships Traefik as a HelmChart. We override via HelmChartConfig to
+  # bind the web entrypoint to NodePort 30080 instead of the default :80,
+  # so the host Caddy can own :80 without a port conflict.
+  #
+  # k3s reads files placed under /var/lib/rancher/k3s/server/manifests/ and
+  # reconciles them against the live cluster on startup.
+  environment.etc."rancher/k3s/server/manifests/traefik-config.yaml".text = ''
+    apiVersion: helm.cattle.io/v1
+    kind: HelmChartConfig
+    metadata:
+      name: traefik
+      namespace: kube-system
+    spec:
+      valuesContent: |-
+        ports:
+          web:
+            port: 8000
+            nodePort: 30080
+            expose:
+              default: true
+            exposedPort: 30080
+            protocol: TCP
+          websecure:
+            port: 8443
+            nodePort: 30443
+            expose:
+              default: false
+            protocol: TCP
+        service:
+          type: NodePort
+        ingressRoute:
+          dashboard:
+            enabled: false
+  '';
 
+  # ── Host Caddy — plain HTTP pass-through (:80 → Traefik :30080) ────────
+  # Owns port 80 on the Tailscale interface. All routing logic lives in
+  # Traefik Ingress resources (manifests/ingress/). This Caddy instance is
+  # intentionally minimal — no TLS, no subdomain logic, no Porkbun.
   services.caddy = {
     enable = true;
     package = pkgs.caddy;
-    globalConfig = ''
-      auto_https off
-    '';
     extraConfig = ''
-      http://auth.${domain} {
+      :80 {
-        reverse_proxy 100.85.244.73:30080
+        reverse_proxy 127.0.0.1:30080 {
-      }
+          header_up Host {host}
-
+          header_up X-Real-IP {remote}
-      http://status.${domain} {
+          header_up X-Forwarded-For {remote}
-        reverse_proxy 127.0.0.1:30801
+          header_up X-Forwarded-Proto https
-      }
+          header_up X-Forwarded-Host {host}
-
+        }
-      http://apps.${domain} {
-        reverse_proxy 127.0.0.1:30802
-      }
-
-      http://home.${domain} {
-        reverse_proxy 127.0.0.1:30803
-      }
-
-      http://pulse.${domain} {
-        reverse_proxy 127.0.0.1:30804
-      }
-
-      http://dash.${domain} {
-        reverse_proxy 127.0.0.1:30805
-      }
-
-      http://media.${domain} {
-        reverse_proxy 127.0.0.1:30810
-      }
-
-      http://vault.${domain} {
-        reverse_proxy 10.43.130.188:80
-      }
-
-      http://books.${domain} {
-        reverse_proxy 127.0.0.1:30807
-      }
-
-      http://music.${domain} {
-        reverse_proxy 127.0.0.1:30809
-      }
-
-      http://mail.${domain} {
-        reverse_proxy 127.0.0.1:30808
-      }
-
-      http://webmail.${domain} {
-        reverse_proxy 127.0.0.1:30808
-      }
-
-      http://${domain} {
-        respond "k3s unified topology — Research Stack"
       }
     '';
   };
 
+  # Caddy starts after k3s so Traefik NodePort is ready before we proxy to it.
+  systemd.services.caddy = {
+    after = [ "k3s.service" "network-online.target" ];
+    wants = [ "k3s.service" "network-online.target" ];
+  };
+
+  networking.firewall.allowedTCPPorts = [ 80 ];
+
+  # ── Service deployment ──────────────────────────────────────────────────
   systemd.services.deploy-k3s-services = {
     description = "Deploy k3s topology services";
     after = [ "k3s.service" "tailscaled.service" "network-online.target" ];

4-Infrastructure/k3s-flake/manifests/authentik/helm-chart.yaml

@@ -20,6 +20,7 @@ spec:
             existingSecret: authentik-secrets
             secretKeys:
               adminPasswordKey: postgresql-password
+              userPasswordKey: postgresql-password
       primary:
         persistence:
           size: 8Gi
@@ -64,8 +65,9 @@ spec:
         limits:
           memory: 512Mi
       service:
-        type: NodePort
+        # ClusterIP — Traefik reaches Authentik via cluster DNS through the
-        nodePort: 30080
+        # rs-auth Ingress (manifests/ingress/ingress.yaml). No NodePort needed.
+        type: ClusterIP
     worker:
       nodeSelector:
         kubernetes.io/hostname: steamdeck
@@ -76,5 +78,4 @@ spec:
         limits:
           memory: 512Mi
       service:
-        type: NodePort
+        type: ClusterIP
-        nodePort: 30080

4-Infrastructure/k3s-flake/manifests/control-plane/Dockerfile.stub

@@ -0,0 +1,5 @@
+FROM python:3.12-slim
+RUN pip install --no-cache-dir fastapi uvicorn
+WORKDIR /app
+COPY app.py .
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]

4-Infrastructure/k3s-flake/manifests/control-plane/blobs-api/app.py

@@ -0,0 +1,35 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = ["fastapi", "uvicorn"]
+# ///
+"""Blobs API – binary object storage service."""
+
+from fastapi import FastAPI
+
+app = FastAPI(title="Blobs API")
+
+
+@app.get("/health")
+def health():
+    return {"status": "ok", "service": "blobs"}
+
+
+@app.put("/api/blobs/{key}")
+def put_blob(key: str):
+    return {"key": key, "stored": True}
+
+
+@app.get("/api/blobs/{key}")
+def get_blob(key: str):
+    return {"key": key, "exists": False}
+
+
+@app.get("/api/blobs/")
+def list_blobs():
+    return {"blobs": []}
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="0.0.0.0", port=8080)

4-Infrastructure/k3s-flake/manifests/control-plane/blobs-api/deployment.yaml

@@ -0,0 +1,28 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: blobs-api
+  namespace: services
+  labels:
+    app: blobs-api
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: blobs-api
+  template:
+    metadata:
+      labels:
+        app: blobs-api
+    spec:
+      containers:
+        - name: blobs-api
+          image: registry.researchstack.info/control-plane/blobs-api:latest
+          ports:
+            - containerPort: 8080
+          resources:
+            requests:
+              memory: "64Mi"
+              cpu: "50m"
+            limits:
+              memory: "128Mi"

4-Infrastructure/k3s-flake/manifests/control-plane/blobs-api/service.yaml

@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: blobs-api
+  namespace: services
+  labels:
+    app: blobs-api
+spec:
+  type: ClusterIP
+  selector:
+    app: blobs-api
+  ports:
+    - port: 8080
+      targetPort: 8080
+      protocol: TCP

4-Infrastructure/k3s-flake/manifests/control-plane/jobs-api/app.py

@@ -0,0 +1,35 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = ["fastapi", "uvicorn"]
+# ///
+"""Jobs API – job submission and status tracking service."""
+
+from fastapi import FastAPI
+
+app = FastAPI(title="Jobs API")
+
+
+@app.get("/health")
+def health():
+    return {"status": "ok", "service": "jobs"}
+
+
+@app.post("/api/jobs/submit")
+def submit_job():
+    return {"job_id": "placeholder", "status": "queued"}
+
+
+@app.get("/api/jobs/{job_id}")
+def get_job(job_id: str):
+    return {"job_id": job_id, "status": "pending"}
+
+
+@app.get("/api/jobs/")
+def list_jobs():
+    return {"jobs": []}
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="0.0.0.0", port=8080)

4-Infrastructure/k3s-flake/manifests/control-plane/jobs-api/deployment.yaml

@@ -0,0 +1,28 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: jobs-api
+  namespace: services
+  labels:
+    app: jobs-api
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: jobs-api
+  template:
+    metadata:
+      labels:
+        app: jobs-api
+    spec:
+      containers:
+        - name: jobs-api
+          image: registry.researchstack.info/control-plane/jobs-api:latest
+          ports:
+            - containerPort: 8080
+          resources:
+            requests:
+              memory: "64Mi"
+              cpu: "50m"
+            limits:
+              memory: "128Mi"

4-Infrastructure/k3s-flake/manifests/control-plane/jobs-api/service.yaml

@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: jobs-api
+  namespace: services
+  labels:
+    app: jobs-api
+spec:
+  type: ClusterIP
+  selector:
+    app: jobs-api
+  ports:
+    - port: 8080
+      targetPort: 8080
+      protocol: TCP

4-Infrastructure/k3s-flake/manifests/control-plane/kustomization.yaml

@@ -0,0 +1,9 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - registry-api/deployment.yaml
+  - registry-api/service.yaml
+  - jobs-api/deployment.yaml
+  - jobs-api/service.yaml
+  - blobs-api/deployment.yaml
+  - blobs-api/service.yaml

4-Infrastructure/k3s-flake/manifests/control-plane/registry-api/app.py

@@ -0,0 +1,35 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = ["fastapi", "uvicorn"]
+# ///
+"""Registry API – worker node registration and heartbeat service."""
+
+from fastapi import FastAPI
+
+app = FastAPI(title="Registry API")
+
+
+@app.get("/health")
+def health():
+    return {"status": "ok", "service": "registry"}
+
+
+@app.post("/api/registry/join")
+def join():
+    return {"accepted": True, "node_id": "placeholder"}
+
+
+@app.post("/api/registry/heartbeat")
+def heartbeat():
+    return {"ack": True}
+
+
+@app.get("/api/registry/nodes")
+def list_nodes():
+    return {"nodes": []}
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="0.0.0.0", port=8080)

4-Infrastructure/k3s-flake/manifests/control-plane/registry-api/deployment.yaml

@@ -0,0 +1,28 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: registry-api
+  namespace: services
+  labels:
+    app: registry-api
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: registry-api
+  template:
+    metadata:
+      labels:
+        app: registry-api
+    spec:
+      containers:
+        - name: registry-api
+          image: registry.researchstack.info/control-plane/registry-api:latest
+          ports:
+            - containerPort: 8080
+          resources:
+            requests:
+              memory: "64Mi"
+              cpu: "50m"
+            limits:
+              memory: "128Mi"

4-Infrastructure/k3s-flake/manifests/control-plane/registry-api/service.yaml

@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: registry-api
+  namespace: services
+  labels:
+    app: registry-api
+spec:
+  type: ClusterIP
+  selector:
+    app: registry-api
+  ports:
+    - port: 8080
+      targetPort: 8080
+      protocol: TCP

4-Infrastructure/k3s-flake/manifests/credential-server/deployment.yaml

@@ -0,0 +1,47 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: credential-server
+  namespace: services
+  labels:
+    app: credential-server
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: credential-server
+  template:
+    metadata:
+      labels:
+        app: credential-server
+    spec:
+      containers:
+        - name: credential-server
+          image: registry.researchstack.info/infra/credential-server:latest
+          ports:
+            - containerPort: 8444
+              name: http
+          env:
+            - name: RS_CREDENTIAL_CONFIG
+              value: /etc/credential-server/credentials.json
+            - name: RS_SURFACE_PORT
+              value: "8444"
+            - name: RS_SURFACE_HOST
+              value: "0.0.0.0"
+            - name: RUST_LOG
+              value: "info"
+          volumeMounts:
+            - name: config
+              mountPath: /etc/credential-server
+              readOnly: true
+          resources:
+            requests:
+              memory: 64Mi
+              cpu: 50m
+            limits:
+              memory: 128Mi
+      volumes:
+        - name: config
+          secret:
+            secretName: credential-server-config
+            optional: true

4-Infrastructure/k3s-flake/manifests/credential-server/kustomization.yaml

@@ -0,0 +1,5 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - deployment.yaml
+  - service.yaml

4-Infrastructure/k3s-flake/manifests/credential-server/service.yaml

@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: credential-server
+  namespace: services
+spec:
+  type: ClusterIP
+  selector:
+    app: credential-server
+  ports:
+    - port: 8444
+      targetPort: 8444
+      name: http

4-Infrastructure/k3s-flake/manifests/hermes/configmap.yaml

@@ -0,0 +1,27 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: hermes-placeholder
+  namespace: services
+data:
+  index.html: |
+    <!DOCTYPE html>
+    <html>
+    <head>
+      <title>Hermes — Research Stack</title>
+      <meta charset="utf-8">
+      <style>
+        body { font-family: system-ui, sans-serif; display: flex; justify-content: center; align-items: center; min-height: 100vh; margin: 0; background: #1a1a2e; color: #eee; }
+        .card { text-align: center; padding: 3rem; border-radius: 12px; background: #16213e; box-shadow: 0 4px 24px rgba(0,0,0,0.3); }
+        h1 { margin: 0 0 0.5rem; }
+        p { color: #aaa; }
+      </style>
+    </head>
+    <body>
+      <div class="card">
+        <h1>Hermes</h1>
+        <p>Chat / orchestrator — coming soon.</p>
+        <p style="font-size:0.8rem; color:#666;">This placeholder will be replaced with the Hermes container.</p>
+      </div>
+    </body>
+    </html>

4-Infrastructure/k3s-flake/manifests/hermes/deployment.yaml

@@ -0,0 +1,36 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: hermes
+  namespace: services
+  labels:
+    app: hermes
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: hermes
+  template:
+    metadata:
+      labels:
+        app: hermes
+    spec:
+      containers:
+        - name: hermes
+          image: nginx:alpine
+          ports:
+            - containerPort: 80
+              name: http
+          volumeMounts:
+            - name: placeholder-page
+              mountPath: /usr/share/nginx/html
+          resources:
+            requests:
+              memory: 32Mi
+              cpu: 10m
+            limits:
+              memory: 64Mi
+      volumes:
+        - name: placeholder-page
+          configMap:
+            name: hermes-placeholder

4-Infrastructure/k3s-flake/manifests/hermes/kustomization.yaml

@@ -0,0 +1,6 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - deployment.yaml
+  - service.yaml
+  - configmap.yaml

4-Infrastructure/k3s-flake/manifests/hermes/service.yaml

@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: hermes
+  namespace: services
+spec:
+  type: ClusterIP
+  selector:
+    app: hermes
+  ports:
+    - port: 80
+      targetPort: 80
+      name: http

4-Infrastructure/k3s-flake/manifests/homer/configmap.yaml

@@ -6,37 +6,63 @@ metadata:
 data:
   config.yml: |
     title: "Research Stack"
-    subtitle: "Unified Topology"
+    subtitle: "researchstack.info"
     header: true
     footer: false
     columns: "3"
     services:
-      - name: "Identity"
+      - name: "Applications"
+        icon: "fas fa-rocket"
+        items:
+          - name: "Hermes"
+            subtitle: "Chat / orchestrator"
+            url: "https://researchstack.info/apps/chat/"
+            target: "_blank"
+          - name: "Jellyfin"
+            subtitle: "Media streaming"
+            url: "https://researchstack.info/apps/jellyfin/"
+            target: "_blank"
+          - name: "Navidrome"
+            subtitle: "Music"
+            url: "https://researchstack.info/apps/music/"
+            target: "_blank"
+          - name: "Audiobookshelf"
+            subtitle: "Books & podcasts"
+            url: "https://researchstack.info/apps/books/"
+            target: "_blank"
+          - name: "Actual Budget"
+            subtitle: "Personal finance"
+            url: "https://researchstack.info/apps/budget/"
+            target: "_blank"
+
+      - name: "Server"
+        icon: "fas fa-server"
+        items:
+          - name: "Uptime Kuma"
+            subtitle: "Service status"
+            url: "https://researchstack.info/server/status/"
+            target: "_blank"
+          - name: "Homarr"
+            subtitle: "Dashboard"
+            url: "https://researchstack.info/server/dash/"
+            target: "_blank"
+          - name: "Vaultwarden"
+            subtitle: "Password vault"
+            url: "https://researchstack.info/server/vault/"
+            target: "_blank"
+
+      - name: "Identity & APIs"
         icon: "fas fa-id-card"
         items:
           - name: "Authentik"
-            logo: "https://auth.YOUR_DOMAIN/static/dist/assets/icons/authentik.svg"
             subtitle: "Single sign-on"
-            url: "https://auth.YOUR_DOMAIN"
+            url: "https://auth.researchstack.info"
             target: "_blank"
-
+          - name: "Registry API"
-      - name: "Monitoring"
+            subtitle: "Node join & heartbeat"
-        icon: "fas fa-heartbeat"
+            url: "https://researchstack.info/api/registry/"
-        items:
-          - name: "Uptime Kuma"
-            logo: "https://status.YOUR_DOMAIN/favicon.ico"
-            subtitle: "Service status & uptime"
-            url: "https://status.YOUR_DOMAIN"
             target: "_blank"
-          - name: "Pulse Receiver"
+          - name: "Job Router"
-            subtitle: "Edge node heartbeats"
+            subtitle: "Compute dispatch"
-            url: "https://pulse.YOUR_DOMAIN"
+            url: "https://researchstack.info/api/jobs/"
-            target: "_blank"
-
-      - name: "Applications"
-        icon: "fas fa-th"
-        items:
-          - name: "Heimdall"
-            subtitle: "Application dashboard"
-            url: "https://apps.YOUR_DOMAIN"
             target: "_blank"

4-Infrastructure/k3s-flake/manifests/ingress/ingress.yaml

@@ -0,0 +1,288 @@
+# Traefik Ingress — canonical path-based routing for researchstack.info
+#
+# This is the single source of truth for URL → service mapping inside k3s.
+# The public edge Caddy terminates TLS and forwards to Traefik's entrypoint.
+# Traefik handles path routing, prefix stripping, and forward-auth.
+#
+# Middleware annotation format: <namespace>-<middleware-name>@kubernetescrd
+#
+# Split into per-route Ingresses so each gets its own middleware chain:
+#   - /apps/*, /server/*  → forward_auth (Authentik) + strip-prefix
+#   - /api/*              → strip-prefix only (token-auth, no SSO)
+#   - /                   → forward_auth (Authentik), no strip
+#   - auth.*              → no middleware (Authentik itself)
+
+# ═══════════════════════════════════════════════════════════════════════════
+# /apps/chat/* → Hermes (SSO-gated)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-apps-chat
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-authentik-forward-auth@kubernetescrd,services-strip-apps-chat@kubernetescrd
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /apps/chat
+            pathType: Prefix
+            backend:
+              service:
+                name: hermes
+                port:
+                  number: 80
+
+# ═══════════════════════════════════════════════════════════════════════════
+# /apps/budget/* → Actual Budget (SSO-gated)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-apps-budget
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-authentik-forward-auth@kubernetescrd,services-strip-apps-budget@kubernetescrd
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /apps/budget
+            pathType: Prefix
+            backend:
+              service:
+                name: actual-budget
+                port:
+                  number: 5006
+
+# ═══════════════════════════════════════════════════════════════════════════
+# /server/status/* → Uptime Kuma (SSO-gated)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-server-status
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-authentik-forward-auth@kubernetescrd,services-strip-server-status@kubernetescrd
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /server/status
+            pathType: Prefix
+            backend:
+              service:
+                name: uptime-kuma
+                port:
+                  number: 3001
+
+# ═══════════════════════════════════════════════════════════════════════════
+# /server/dash/* → Homarr (SSO-gated)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-server-dash
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-authentik-forward-auth@kubernetescrd,services-strip-server-dash@kubernetescrd
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /server/dash
+            pathType: Prefix
+            backend:
+              service:
+                name: homarr
+                port:
+                  number: 7575
+
+# ═══════════════════════════════════════════════════════════════════════════
+# /server/vault/* → Vaultwarden (SSO-gated)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-server-vault
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-authentik-forward-auth@kubernetescrd,services-strip-server-vault@kubernetescrd
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /server/vault
+            pathType: Prefix
+            backend:
+              service:
+                name: vaultwarden
+                port:
+                  number: 80
+
+# ═══════════════════════════════════════════════════════════════════════════
+# /api/cred/* → Credential Server (token-auth, no SSO)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-api-cred
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-strip-api-cred@kubernetescrd
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /api/cred
+            pathType: Prefix
+            backend:
+              service:
+                name: credential-server
+                port:
+                  number: 8444
+
+# ═══════════════════════════════════════════════════════════════════════════
+# /api/registry/* → Registry API (token-auth, no SSO)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-api-registry
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-strip-api-registry@kubernetescrd
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /api/registry
+            pathType: Prefix
+            backend:
+              service:
+                name: registry-api
+                port:
+                  number: 8080
+
+# ═══════════════════════════════════════════════════════════════════════════
+# /api/jobs/* → Job Router (token-auth, no SSO)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-api-jobs
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-strip-api-jobs@kubernetescrd
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /api/jobs
+            pathType: Prefix
+            backend:
+              service:
+                name: jobs-api
+                port:
+                  number: 8080
+
+# ═══════════════════════════════════════════════════════════════════════════
+# /api/blobs/* → Blob Plane (token-auth, no SSO)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-api-blobs
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-strip-api-blobs@kubernetescrd
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /api/blobs
+            pathType: Prefix
+            backend:
+              service:
+                name: blobs-api
+                port:
+                  number: 8080
+
+# ═══════════════════════════════════════════════════════════════════════════
+# / → Homer landing page (SSO-gated, no strip)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-landing
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    traefik.ingress.kubernetes.io/router.middlewares: services-authentik-forward-auth@kubernetescrd
+    traefik.ingress.kubernetes.io/router.priority: "1"
+spec:
+  rules:
+    - host: researchstack.info
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: homer
+                port:
+                  number: 8080
+
+# ═══════════════════════════════════════════════════════════════════════════
+# auth.researchstack.info → Authentik (stable OIDC issuer, no middleware)
+# ═══════════════════════════════════════════════════════════════════════════
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: rs-auth
+  namespace: services
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+spec:
+  rules:
+    - host: auth.researchstack.info
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: authentik
+                port:
+                  number: 80

4-Infrastructure/k3s-flake/manifests/ingress/kustomization.yaml

@@ -0,0 +1,5 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - middleware.yaml
+  - ingress.yaml

4-Infrastructure/k3s-flake/manifests/ingress/middleware.yaml

@@ -0,0 +1,120 @@
+# Traefik middlewares for the Research Stack Ingress
+#
+# forward-auth: Authentik SSO gate (applied to /apps/*, /server/*)
+# strip-apps-chat: strip /apps/chat prefix before forwarding to Hermes
+# strip-apps-budget: strip /apps/budget prefix
+# strip-server-status: strip /server/status prefix
+# strip-server-dash: strip /server/dash prefix
+# strip-server-vault: strip /server/vault prefix
+# strip-api-cred: strip /api/cred prefix
+# strip-api-registry: strip /api/registry prefix
+# strip-api-jobs: strip /api/jobs prefix
+# strip-api-blobs: strip /api/blobs prefix
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: authentik-forward-auth
+  namespace: services
+spec:
+  forwardAuth:
+    address: http://authentik.services.svc.cluster.local/outpost.goauthentik.io/auth/caddy
+    authResponseHeaders:
+      - X-Authentik-Username
+      - X-Authentik-Email
+      - X-Authentik-Name
+      - X-Authentik-Uid
+      - X-Authentik-Jwt
+      - X-Authentik-Meta-Jwt
+      - X-Authentik-Meta-App
+      - X-Authentik-Meta-Version
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: strip-apps-chat
+  namespace: services
+spec:
+  stripPrefix:
+    prefixes:
+      - /apps/chat
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: strip-apps-budget
+  namespace: services
+spec:
+  stripPrefix:
+    prefixes:
+      - /apps/budget
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: strip-server-status
+  namespace: services
+spec:
+  stripPrefix:
+    prefixes:
+      - /server/status
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: strip-server-dash
+  namespace: services
+spec:
+  stripPrefix:
+    prefixes:
+      - /server/dash
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: strip-server-vault
+  namespace: services
+spec:
+  stripPrefix:
+    prefixes:
+      - /server/vault
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: strip-api-cred
+  namespace: services
+spec:
+  stripPrefix:
+    prefixes:
+      - /api/cred
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: strip-api-registry
+  namespace: services
+spec:
+  stripPrefix:
+    prefixes:
+      - /api/registry
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: strip-api-jobs
+  namespace: services
+spec:
+  stripPrefix:
+    prefixes:
+      - /api/jobs
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: strip-api-blobs
+  namespace: services
+spec:
+  stripPrefix:
+    prefixes:
+      - /api/blobs

4-Infrastructure/k3s-flake/manifests/kustomization.yaml

@@ -3,11 +3,24 @@ kind: Kustomization
 
 resources:
   - namespace.yaml
+  # Monitoring & dashboards
   - uptime-kuma
-  - heimdall
   - homer
-  - pulse-receiver
   - homarr
-  - authentik
+  # Apps
+  - hermes
   - actual-budget
+  - vaultwarden
+  # Identity
+  - authentik
+  # Control-plane APIs
+  - credential-server
+  - control-plane
+  # Ingress (Traefik path routing + middlewares)
+  - ingress
+  # Media (separate namespace)
+  - media
+  # Legacy (to be removed after migration)
+  - heimdall
+  - pulse-receiver
 

4-Infrastructure/k3s-flake/scripts/deploy-services.sh

@@ -9,9 +9,26 @@ set -euo pipefail
 #
 # Flow:
 #   1. Wait for k3s cluster to be healthy
-#   2. Create the K8s Secret for authentik from the sops-decrypted file
+#   2. Ensure namespaces exist
-#   3. Apply all manifests via kubectl
+#   3. Create the K8s Secret for authentik from the sops-decrypted file
-#   4. k3s built-in Helm controller picks up the HelmChart CRD for authentik
+#   4. Apply authentik HelmChart CRD
+#   5. Apply all service manifests (kustomize where available)
+#
+# URL contract (canonical paths served by internal Caddy router):
+#   /                    → Homer
+#   /apps/chat/*         → Hermes
+#   /apps/jellyfin/*     → Jellyfin
+#   /apps/books/*        → Audiobookshelf
+#   /apps/music/*        → Navidrome
+#   /apps/budget/*       → Actual Budget
+#   /server/status/*     → Uptime Kuma
+#   /server/dash/*       → Homarr
+#   /server/vault/*      → Vaultwarden
+#   /api/cred/*          → Credential Server
+#   /api/registry/*      → Registry API
+#   /api/jobs/*          → Job Router
+#   /api/blobs/*         → Blob Plane
+#   auth.researchstack.info → Authentik (stable OIDC issuer)
 ###########################################################################
 
 MANIFESTS_DIR="/etc/nixos/k3s-flake/manifests"
@@ -22,16 +39,14 @@ until kubectl cluster-info --request-timeout=5s >/dev/null 2>&1; do
 done
 echo "[deploy] cluster is healthy"
 
-echo "[deploy] ensuring CoreDNS has one schedulable replica per main workload node..."
+echo "[deploy] ensuring CoreDNS has schedulable replicas..."
 kubectl -n kube-system scale deployment/coredns --replicas=3
 
-echo "[deploy] ensuring services namespace exists..."
+echo "[deploy] ensuring namespaces exist..."
 kubectl get namespace services >/dev/null 2>&1 || kubectl create namespace services
+kubectl get namespace media >/dev/null 2>&1 || kubectl create namespace media
 
-# Authentik secrets from sops-decrypted file.
+# ── Authentik secrets from sops ──────────────────────────────────────────
-# The file format is an env file:
-#   secret-key=<value>
-#   postgresql-password=<value>
 if [ -n "${AUTHENTIK_SECRETS:-}" ] && [ -f "$AUTHENTIK_SECRETS" ]; then
   echo "[deploy] creating authentik-secrets from sops file..."
   kubectl delete secret --ignore-not-found -n services authentik-secrets
@@ -54,13 +69,13 @@ if [ -n "${AUTHENTIK_SECRETS:-}" ] && [ -f "$AUTHENTIK_SECRETS" ]; then
   fi
 fi
 
+# ── Apply manifests ──────────────────────────────────────────────────────
 echo "[deploy] applying authentik HelmChart..."
 kubectl apply -f "$MANIFESTS_DIR/authentik/helm-chart.yaml"
 
 echo "[deploy] applying all service manifests..."
 find "$MANIFESTS_DIR" -maxdepth 1 -type d | sort | while read -r dir; do
   name=$(basename "$dir")
-  # Skip root dir and authentik (managed via HelmChart)
   [ "$dir" = "$MANIFESTS_DIR" ] && continue
   [ "$name" = "authentik" ] && continue
   if [ -f "$dir/kustomization.yaml" ] || [ -f "$dir/kustomization.yml" ]; then
@@ -72,4 +87,4 @@ find "$MANIFESTS_DIR" -maxdepth 1 -type d | sort | while read -r dir; do
   fi
 done
 
-echo "[deploy] done"
+echo "[deploy] done — all services applied"