Research-Stack/scripts
Devin AI 6100565578 fix: regression test now actually exercises --staged in temp repo
Addresses Devin Review on PR #11 ("Staged regression test passes
vacuously — git diff runs in real repo, not temp repo").

Root cause:
  scripts/math-first/require_math_evidence.py hardcoded
  cwd=REPO_ROOT on the git subprocess. REPO_ROOT is computed from
  __file__, which always resolves to the REAL repo path -- even when
  the test invokes the script with cwd=tmp_path. So 'git diff
  --cached --name-only' queried the real (clean) repo, returned an
  empty list, and the script exited 0 via the noop short-circuit.
  The regression test then printed OK while never actually
  exercising the --staged classification logic.

Fix:
  Introduce _git_toplevel() in require_math_evidence.py that runs
  'git rev-parse --show-toplevel' from the *inherited* cwd. Both
  _files_from_staged() and _files_from_git_diff() now use that
  toplevel as their subprocess cwd. Pre-commit and CI both invoke
  the script from inside the repo root anyway, so behaviour in
  production is unchanged; only the test (and any other caller
  that runs the script from a non-Research-Stack directory) now
  works correctly.

Test hardening:
  test_require_math_evidence.py's staged regression now drives the
  script via plain subprocess with cwd=tmp_path (no runpy wrapper,
  no os.chdir trickery) and covers three sub-cases against the same
  temp repo:
    (a) math-track-only staged       -> expect exit 1 (negative)
    (b) math-track + receipt staged  -> expect exit 0 (the bug)
    (c) math-track + claims.yaml     -> expect exit 0 (registry)
  Sub-case (a) is the key addition: with the pre-fix script, this
  case wrongly returns 0 (real repo is clean -> noop), so the test
  fails loudly. Verified locally by temporarily reverting the cwd
  fix -- the test correctly reports
  'FAIL staged_regression_negative: expected exit 1, got 0'.

Docs: docs/math-first-tooling.md notes that
  receipt-required-for-math-content is a no-op in the CI
  pre-commit job (pre-commit's --from-ref/--to-ref mode does not
  touch the index, so --staged returns an empty list and the hook
  exits 0). PR-scope enforcement of the same rule lives in the
  dedicated require-evidence CI job. This addresses the Devin
  Review info comment on .pre-commit-config.yaml.
Co-Authored-By: Allaun Silverfox <bigdataiscoming+9i37y6j2@protonmail.com>
2026-05-12 04:51:57 +00:00
..
math-first fix: regression test now actually exercises --staged in temp repo 2026-05-12 04:51:57 +00:00
archive-and-delete-v2.sh initial: sovereign research stack (consolidated, weightless, and lfs-optimized) 2026-05-04 18:11:36 -05:00
bf4prover.py Consolidate research stack updates 2026-05-05 21:09:48 -05:00
clean-tailscale-refs.sh Fix Tauri builds: navigate API, unused imports, build config 2026-05-04 19:22:01 -05:00
finalize-monorepo.sh initial: sovereign research stack (consolidated, weightless, and lfs-optimized) 2026-05-04 18:11:36 -05:00
ingest_watcher.py Consolidate research stack updates 2026-05-05 21:09:48 -05:00
pist_orchestrator.py Consolidate research stack updates 2026-05-05 21:09:48 -05:00
populate-open-webui-knowledge-expanded.py Consolidate research stack updates 2026-05-05 21:09:48 -05:00
populate-open-webui-knowledge.py Consolidate research stack updates 2026-05-05 21:09:48 -05:00
remove-tailnet-nodes-api.sh Fix Tauri builds: navigate API, unused imports, build config 2026-05-04 19:22:01 -05:00
reset-tailnet.sh Fix Tauri builds: navigate API, unused imports, build config 2026-05-04 19:22:01 -05:00
review_agent.py Consolidate research stack updates 2026-05-05 21:09:48 -05:00
sciencehub_mcp.py Consolidate research stack updates 2026-05-05 21:09:48 -05:00
SCIENCEHUB_README.md Consolidate research stack updates 2026-05-05 21:09:48 -05:00
zotero_corpus_pipeline.py Consolidate research stack updates 2026-05-05 21:09:48 -05:00