mirror of
https://github.com/allaunthefox/Research-Stack.git
synced 2026-07-31 03:05:21 +00:00
docs: add LLM refinement protocol
This commit is contained in:
parent
b93603ed23
commit
c6e2d33ef3
1 changed files with 107 additions and 0 deletions
107
6-Documentation/docs/specs/LLM_REFINEMENT_PROTOCOL.md
Normal file
107
6-Documentation/docs/specs/LLM_REFINEMENT_PROTOCOL.md
Normal file
|
|
@ -0,0 +1,107 @@
|
|||
# LLM Refinement Protocol
|
||||
|
||||
Date captured: 2026-05-07
|
||||
Source prompt: How to Squeeze GENIUS out of LLMs: Notes from My QRI Retreat Skillshare, Andres Gomez-Emilsson, 2026-05-07.
|
||||
|
||||
Purpose: convert a rough research idea into a stable, auditable refinement loop.
|
||||
|
||||
## Core rule
|
||||
|
||||
Treat an LLM as a chart-transform instrument, not as an answer oracle.
|
||||
|
||||
The protocol forces the model to expose hidden dimensions, rotate lenses, lock vocabulary, compress the result into primitives, and audit residuals before accepting a claim.
|
||||
|
||||
## Refinement loop
|
||||
|
||||
1. Table expansion
|
||||
- Ask the model to infer the hidden dimensions of the concept.
|
||||
- Require columns for axis, observable proxy, uncertainty, failure mode, primitive mapping, and test hook.
|
||||
|
||||
2. Lens sweep
|
||||
- Run the same concept through multiple charts.
|
||||
- Default lenses: hostile domain scientist, patent examiner, systems engineer, formal-methods auditor, safety reviewer, and implementation engineer.
|
||||
- Preserve disagreements between lenses instead of smoothing them away.
|
||||
|
||||
3. Glossary lock
|
||||
- Extract every polysemic or overloaded term.
|
||||
- Give each term a local project definition.
|
||||
- Reuse the definitions block across future chats and repo notes.
|
||||
|
||||
4. Primitive compression
|
||||
- Collapse the concept into the four keeper primitives: field, shear, packet, spectral.
|
||||
- Reject concepts that cannot identify their governing primitive or residual.
|
||||
|
||||
5. Residual audit
|
||||
- Mark each subclaim as one of: metaphor, hypothesis, engineering delta, evidence-backed, or rejected.
|
||||
- Identify analogy leakage, missing evidence, hidden priors, and overclaim attractors.
|
||||
|
||||
6. Test hook
|
||||
- Produce the smallest falsifiable test.
|
||||
- The test must specify measurable input, output, control, failure threshold, and expected residual.
|
||||
|
||||
7. Warden pass
|
||||
- Ask what would make the claim false.
|
||||
- Ask what evidence would upgrade or downgrade the claim.
|
||||
- Ask whether the terminology is doing real work or hiding uncertainty.
|
||||
|
||||
## Standard prompt
|
||||
|
||||
```text
|
||||
Analyze the following concept using the LLM Refinement Protocol.
|
||||
|
||||
Concept:
|
||||
[PASTE CONCEPT]
|
||||
|
||||
Step 1: Expand the hidden dimensions as a table.
|
||||
Step 2: Run a lens sweep using hostile domain scientist, patent examiner, systems engineer, formal-methods auditor, safety reviewer, and implementation engineer.
|
||||
Step 3: Build a glossary for overloaded terms.
|
||||
Step 4: Collapse the result into field / shear / packet / spectral primitives.
|
||||
Step 5: Audit residuals and classify each subclaim as metaphor, hypothesis, engineering delta, evidence-backed, or rejected.
|
||||
Step 6: Propose the smallest falsifiable test.
|
||||
Step 7: Give the Warden pass: what would make this false, what would upgrade it, and what language is hiding uncertainty?
|
||||
```
|
||||
|
||||
## Output schema
|
||||
|
||||
```yaml
|
||||
concept: string
|
||||
one_sentence_claim: string
|
||||
hidden_dimensions:
|
||||
- axis: string
|
||||
observable_proxy: string
|
||||
uncertainty: string
|
||||
failure_mode: string
|
||||
primitive: field|shear|packet|spectral|mixed
|
||||
test_hook: string
|
||||
lens_sweep:
|
||||
- lens: string
|
||||
strongest_objection: string
|
||||
strongest_revision: string
|
||||
glossary:
|
||||
- term: string
|
||||
local_definition: string
|
||||
forbidden_drift: string
|
||||
primitive_compression:
|
||||
field: string
|
||||
shear: string
|
||||
packet: string
|
||||
spectral: string
|
||||
residual_audit:
|
||||
- subclaim: string
|
||||
status: metaphor|hypothesis|engineering_delta|evidence_backed|rejected
|
||||
evidence_needed: string
|
||||
smallest_test:
|
||||
input: string
|
||||
output: string
|
||||
control: string
|
||||
failure_threshold: string
|
||||
expected_residual: string
|
||||
warden_pass:
|
||||
falsifier: string
|
||||
upgrade_path: string
|
||||
uncertainty_language: string
|
||||
```
|
||||
|
||||
## Repository use
|
||||
|
||||
Use this protocol before promoting any new concept into a spec, patent-materials note, paper draft, Lean scaffold, or implementation ticket.
|
||||
Loading…
Add table
Reference in a new issue