Research-Stack/audit/infrastructure_audit.md

# Infrastructure Audit Report

**Repository:** https://github.com/allaunthefox/Research-Stack  
**Branch:** main  
**Scope:** 4-Infrastructure/, 5-Applications/, 6-Kernel-Shim/, scripts/  
**Audit Date:** 2026-06-02  

---

## Summary

| Category | Count | Severity |
|----------|-------|----------|
| Total scoped files | 2,725 | - |
| **Secrets found** | **3** | 1 CRITICAL, 1 HIGH, 1 MEDIUM |
| **Security issues** | **6** | 2 HIGH, 3 MEDIUM, 1 LOW |
| **Dead code functions/regions** | **3** | 3 LOW |
| **Broken imports** | **2** | 1 MEDIUM, 1 LOW |
| **Path mismatches** | **3** | 1 MEDIUM, 2 LOW |
| **Build problems** | **2** | 1 MEDIUM, 1 LOW |
| **Config drift** | **2** | 1 MEDIUM, 1 LOW |

---

## CRITICAL Issues

### C1. Hardcoded Wolfram Alpha API Key (SECRET)
- **File:** `4-Infrastructure/shim/wolfram_verify.py`
- **Line:** `APPID = "HYJE3R3R63"`
- **Severity:** **CRITICAL**
- **Description:** A live Wolfram Alpha API key is hardcoded in plaintext and committed to version control. This key is used to make API calls to `api.wolframalpha.com`. Anyone with repository access can extract and abuse this key, incurring API charges and potentially violating terms of service.
- **Evidence:**
  ```python
  APPID = "HYJE3R3R63"
  # ...
  url = f"http://api.wolframalpha.com/v2/query?appid={APPID}&input=..."
  ```
- **Remediation:**
  1. **Immediately revoke** the exposed API key at Wolfram Alpha developer portal
  2. Replace with: `APPID = os.environ.get("WOLFRAM_APPID", "")`
  3. Add `WOLFRAM_APPID` to `.gitignore` and environment setup docs
  4. Add secret scanning to CI to prevent future commits
  5. Rotate any other keys that may have been exposed

---

## HIGH Severity Issues

### H1. Remote SSH Command Injection via shell=True
- **File:** `4-Infrastructure/shim/verify_all_shims.py`
- **Lines:** `subprocess.run(cmd, shell=True, ...)` (multiple instances)
- **Severity:** **HIGH**
- **Description:** The verification script constructs SSH commands via string concatenation and executes them with `shell=True`. While the current inputs appear hardcoded, the pattern is dangerous. The `run_remote()` function builds: `ssh {user}@{ip} "{cmd}"` where cmd comes from the SHIMS list but could be tampered with.
- **Evidence:**
  ```python
  def run_remote(cmd):
      ssh_cmd = f"ssh {CUPFOX_USER}@{CUPFOX_IP} \"{cmd}\""
      res = subprocess.run(ssh_cmd, shell=True, capture_output=True, text=True)
  ```
- **Remediation:**
  - Use `paramiko` or `asyncssh` library for SSH connections
  - If subprocess is required, use `shell=False` with list args
  - Validate all command components against an allowlist

### H2. Hardcoded Remote Infrastructure Credentials
- **File:** `4-Infrastructure/shim/verify_all_shims.py`
- **Lines:** `CUPFOX_IP = "46.232.249.226"`, `CUPFOX_USER = "root"`
- **Severity:** **HIGH**
- **Description:** Production infrastructure IP address and privileged username (root) are hardcoded. Combined with SSH key-based authentication (not visible in file but implied by SSH commands), this exposes production compute node details.
- **Evidence:**
  ```python
  CUPFOX_IP = "46.232.249.226"
  CUPFOX_USER = "root"
  ```
- **Remediation:**
  - Move to environment variables: `CUPFOX_IP`, `CUPFOX_USER`
  - Use non-privileged user for remote execution
  - Document SSH key management in secure vault

---

## MEDIUM Severity Issues

### M1. SQL Injection Pattern in Eigensolid Data Shim
- **File:** `4-Infrastructure/shim/ingest_eigensolid_data.py`
- **Function:** `generate_crossing_weights_insert()`, `generate_eigensolid_snapshot_insert()`, `generate_braid_strand_insert()`
- **Severity:** **MEDIUM**
- **Description:** SQL generator functions use f-string interpolation to construct INSERT statements. While the current `uuid.uuid4()` values are safe, the pattern is dangerous if extended to user-provided data. Note: the `insert_eigensolid_data()` function correctly uses parameterized queries - there is an inconsistency.
- **Evidence:**
  ```python
  def generate_crossing_weights_insert(...):
      return f"INSERT INTO ene.crossing_weights ... VALUES ('{cw_id}', '{recipe_id}', ...)"
  ```
- **Remediation:**
  - Remove unsafe SQL generator functions or use parameter placeholders (`%s`)
  - Ensure all SQL generation uses parameterized queries consistently

### M2. Broken Import Path Manipulation
- **File:** `4-Infrastructure/shim/qaoa_adapter.py`
- **Lines:** `sys.path.insert(0, ...)`, `from q16 import ...`
- **Severity:** **MEDIUM**
- **Description:** Module uses `sys.path.insert()` to find `q16.py` and `braid_search.py`. If these files are not present in the expected relative locations, the import will fail at runtime. Only partial graceful fallback exists.
- **Evidence:**
  ```python
  _sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "lib"))
  _sys.path.insert(0, str(Path(__file__).resolve().parent))
  from q16 import Q16_SCALE, from_q16, to_q16
  from braid_search import _q16_signed
  ```
- **Remediation:**
  - Package modules properly with `__init__.py`
  - Add try/except ImportError with fallback implementations

### M3. Kernel Module const-Cast Undefined Behavior
- **File:** `6-Kernel-Shim/pist_compress.c`
- **Function:** `pist_init_galois_table()`
- **Severity:** **MEDIUM**
- **Description:** The function casts away `const` from `galois_mul_table` to write to it at runtime: `(u8 *)galois_mul_table`. This is undefined behavior in C and potentially a security concern in kernel code.
- **Evidence:**
  ```c
  static const u8 galois_mul_table[256] = { ... };
  static void pist_init_galois_table(void) {
      u8 *tbl = (u8 *)galois_mul_table;  // const cast away
      for (...) tbl[i] = p;
  }
  ```
- **Remediation:**
  - Remove `const` qualifier from `galois_mul_table`
  - Or use a non-const working copy

### M4. Config Drift - Extracted Components Still Referenced
- **File:** `4-Infrastructure/AGENTS.md`
- **Severity:** **MEDIUM**
- **Description:** AGENTS.md states that `kubernetes/`, `k3s-flake/`, `kube/`, `shim/ray-actors/`, `shim/vcn_*.py`, `shim/hermes/` were extracted to `distributed-compute-fabric` repository. However, references to these paths may still exist in code and scripts, causing confusion.
- **Remediation:**
  - Audit all remaining references to extracted components
  - Update cross-references to point to new repository
  - Add deprecation notices where appropriate

### M5. Path Mismatch in Verification Script
- **File:** `4-Infrastructure/shim/verify_all_shims.py`
- **Line:** `local_path = os.path.join("4-Infrastructure/shim", shim)`
- **Severity:** **MEDIUM**
- **Description:** Uses relative path "4-Infrastructure/shim" which assumes script is run from repo root. If run from elsewhere, files won't be found.
- **Remediation:**
  ```python
  script_dir = Path(__file__).resolve().parent
  local_path = script_dir / shim
  ```

---

## LOW Severity Issues

### L1. Kernel Module Non-Destructive Statistics Read
- **File:** `6-Kernel-Shim/pist_neuromorphic.c`
- **Function:** `stats_show()`
- **Severity:** **LOW**
- **Description:** The `stats_show()` function destructively reads `byte_freq` by zeroing entries during the "top 10" loop. This is not thread-safe and corrupts statistics for concurrent readers.
- **Evidence:**
  ```c
  st->byte_freq[max_idx] = 0; /* zero out for next iter (destructive!) */
  ```
- **Remediation:** Copy array locally before processing. Use `spin_lock_irqsave()` to protect the read.

### L2. Dead Code in Kernel Compression Module
- **File:** `6-Kernel-Shim/pist_compress.c`
- **Functions:** `pist_shifter_hachimoji()`, `pist_shifter_aegis()`, `pist_entropy_approx()`
- **Severity:** **LOW**
- **Description:** Three shifter functions (Hachimoji, AEGIS, Natural DNA) and the entropy approximation function are defined but never called in the compression/decompression paths. Only `pist_shifter_mirror` is actually used in `pist_compress_page()`.
- **Remediation:** Remove unused functions or wire them into the compression engine via the shifter chain bitmask.

### L3. Dead Bit Flags in pist_compress.c
- **File:** `6-Kernel-Shim/pist_compress.c`
- **Severity:** **LOW**
- **Description:** Eight shifter bit flags are defined but only `PIST_SHIFT_PIST_MIRROR` is checked in the compression function.
- **Evidence:**
  ```c
  #define PIST_SHIFT_HACHIMOJI    BIT(0)  // unused
  #define PIST_SHIFT_AEGIS        BIT(1)  // unused
  #define PIST_SHIFT_NATURAL_DNA  BIT(2)  // unused
  #define PIST_SHIFT_PIST_MIRROR  BIT(3)  // used
  #define PIST_SHIFT_PNA          BIT(4)  // unused
  #define PIST_SHIFT_LNA          BIT(5)  // unused
  #define PIST_SHIFT_PRION        BIT(6)  // unused
  #define PIST_SHIFT_GALOIS       BIT(7)  // unused
  ```
- **Remediation:** Implement multi-shifter chain or remove unused flags.

### L4. Missing Output Directory Handling
- **File:** `5-Applications/scripts/rgflow_gpu_pipeline.py`
- **Line:** `save_results(flat, Path("out"))`
- **Severity:** **LOW**
- **Description:** Hardcoded output path "out" may not exist. While `out_dir.mkdir()` is called inside `save_results()`, the path is relative to cwd.
- **Remediation:** Use configurable output path with `mkdir(parents=True, exist_ok=True)`.

### L5. Heavy Monolithic requirements.txt
- **File:** `requirements.txt`
- **Severity:** **LOW**
- **Description:** Single requirements.txt includes GPU (wgpu), quantum (qiskit, cirq, perceval-quandela), distributed compute (ray), and database dependencies. Many will fail on systems without specialized hardware.
- **Remediation:**
  ```
  requirements.txt           # core only
  requirements-gpu.txt       # wgpu, CUDA
  requirements-quantum.txt   # qiskit, cirq, perceval
  requirements-db.txt        # psycopg2, boto3
  ```

### L6. AGENTS.md Storage Agent Description Drift
- **File:** `4-Infrastructure/AGENTS.md`
- **Severity:** **LOW**
- **Description:** AGENTS.md describes detailed trigger model for storage_agent.py with specific Q16_16 thresholds and receipt schema. The actual implementation may differ from documentation.
- **Remediation:** Add doc tests or sync documentation with code via automated extraction.

### L7. Potentially Unused Cloudflare WASM Functions
- **File:** `4-Infrastructure/cloudflare/src/lib.rs`
- **Functions:** `reset()`, `derive_scalar()`
- **Severity:** **LOW**
- **Description:** `reset()` and `derive_scalar()` are defined but may not be called from the JavaScript entry point (`index.js`).
- **Remediation:** Verify usage in `index.js` or remove if confirmed unused.

---

## Per-Component Breakdown

### 4-Infrastructure/ (1,420 Python files, 142 Rust files, 18 C files, 68 Shell files, 125 YAML files)

| Component | Files | Critical Issues | Warnings | Notes |
|-----------|-------|----------------|----------|-------|
| `shim/*.py` | ~50 | 1 (wolfram key) | 4 | Core adapter shims |
| `storage/` | ~10 | 0 | 1 | Storage agent subprocess calls |
| `cloudflare/` | 3 | 0 | 1 | WASM edge functions |
| `kernel/` | 2 C files | 0 | 3 | Kernel modules have dead code, const cast |
| `hardware/` | ~20 | 0 | 0 | FPGA/sparkle hardware bring-up |
| `NoDupeLabs/` | ~50 | 0 | 0 | Third-party package |

### 5-Applications/ (142 Rust files)

| Component | Files | Critical Issues | Warnings | Notes |
|-----------|-------|----------------|----------|-------|
| `parquet_compressor/` | ~30 | 0 | 1 | Cargo.toml dependency versions |
| `scripts/` | ~20 | 0 | 1 | Path handling in GPU pipeline |

### 6-Kernel-Shim/ (2 C files mirrored from 4-Infrastructure/kernel/)

| Component | Files | Critical Issues | Warnings | Notes |
|-----------|-------|----------------|----------|-------|
| `pist_compress.c` | 1 | 0 | 2 | Dead code, const cast |
| `pist_neuromorphic.c` | 1 | 0 | 1 | Destructive stats read |

**Note:** The C files in `6-Kernel-Shim/` appear to be mirrors of `4-Infrastructure/kernel/` files. Ensure they are kept in sync or use symlinks/submodules to avoid divergence.

---

## Security Analysis Details

### Python Security Patterns Found

| Pattern | Files | Risk Level | Notes |
|---------|-------|------------|-------|
| `subprocess.run(shell=True)` | verify_all_shims.py | HIGH | Remote SSH commands |
| `subprocess.run()` | storage_agent.py | MEDIUM | Calls to backup scripts |
| f-string SQL construction | ingest_eigensolid_data.py | MEDIUM | Generator functions |
| `eval()` / `exec()` | None found | - | Clean |
| `pickle.loads()` | None found | - | Clean |
| `input()` | None found | - | Clean |
| Hardcoded API key | wolfram_verify.py | CRITICAL | Wolfram Alpha APPID |

### Rust Safety Analysis

| File | unsafe Blocks | Notes |
|------|---------------|-------|
| `cloudflare/src/lib.rs` | 0 | Clean WASM bindings |
| `parquet_compressor/src/*.rs` | TBD (not in scope of read) | Likely has GPU unsafe code |

### C Kernel Safety Analysis

| File | Issue | Severity |
|------|-------|----------|
| `pist_compress.c` | const cast away | MEDIUM |
| `pist_compress.c` | int_sqrt() usage (kernel API) | LOW (verify availability) |
| `pist_neuromorphic.c` | Non-atomic stat read | LOW |
| `pist_neuromorphic.c` | PAGE_SIZE buffer in sysfs show | LOW (potential overflow) |

---

## Build System Analysis

### Cargo.toml (parquet_compressor)

```toml
[dependencies]
parquet = { version = "58.3", default-features = false, features = ["arrow", "zstd"] }
arrow = "58.3"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
rayon = "1.8"
clap = { version = "4.4", features = ["derive"] }
indicatif = "0.18"
byteorder = "1.5"
sysinfo = "0.30"
tokio = { version = "1.35", features = ["sync", "rt", "rt-multi-thread"] }
anyhow = "1.0"
divsufsort = "2.0"
itertools = "0.14"
flate2 = "1.0"
zstd = "0.13"
wgpu = "0.19"
bytemuck = { version = "1.14", features = ["derive"] }
pollster = "0.3"
chrono = "0.4"
glob = "0.3"
pathdiff = "0.2"
regex = "1.11"
ahash = "0.8"
dashmap = "6.2"
dirs = "5.0"
```

**Issues:**
- `itertools = "0.14"` is very recent (June 2026); may not be stable
- `divsufsort = "2.0"` may have build issues on Windows
- No `Cargo.lock` visible in the tree (should be committed for binaries)

### requirements.txt

**Issues:**
- Monolithic file with 20+ heavy dependencies
- GPU, quantum, and distributed compute deps all in one file
- No version pins (uses `>=` which can break with major version updates)
- `perceval-quandela>=1.2.3` requires proprietary SDK

---

## Remediation Priority Matrix

| Priority | Issue | Effort | File |
|----------|-------|--------|------|
| **P0** | Revoke and rotate Wolfram Alpha API key | 15 min | wolfram_verify.py |
| **P0** | Remove hardcoded key from source | 5 min | wolfram_verify.py |
| **P1** | Fix subprocess shell=True in verify_all_shims.py | 1 hour | verify_all_shims.py |
| **P1** | Move SSH credentials to environment | 30 min | verify_all_shims.py |
| **P2** | Fix SQL injection pattern | 30 min | ingest_eigensolid_data.py |
| **P2** | Fix const cast in kernel module | 15 min | pist_compress.c |
| **P2** | Add try/except for fragile imports | 30 min | qaoa_adapter.py |
| **P3** | Remove dead code from kernel modules | 1 hour | pist_compress.c |
| **P3** | Fix destructive stats read | 30 min | pist_neuromorphic.c |
| **P3** | Split requirements.txt | 30 min | requirements.txt |
| **P4** | Fix relative path assumptions | 30 min | Multiple files |
| **P4** | Sync AGENTS.md with code | 2 hours | AGENTS.md |

---

## Recommendations

1. **Immediate Actions (Today):**
   - Revoke the exposed Wolfram Alpha API key
   - Remove hardcoded IP/username from verify_all_shims.py
   - Run `git filter-repo` or similar to remove secrets from git history

2. **Short-term (This Week):**
   - Fix all subprocess shell=True usages
   - Fix SQL injection patterns
   - Fix kernel module const cast
   - Add secret scanning to CI (e.g., `truffleHog` or `git-secrets`)

3. **Medium-term (This Month):**
   - Remove dead code from kernel modules
   - Split requirements.txt into feature-specific files
   - Add proper error handling for missing files/paths
   - Sync documentation with actual code

4. **Long-term:**
   - Add automated security scanning to CI pipeline
   - Implement proper secret management (HashiCorp Vault, AWS Secrets Manager)
   - Add unit tests for kernel module sysfs interfaces
   - Create integration tests for shim verification pipeline