Research-Stack/6-Documentation/docs/specs/OMNITOKEN_GCL_REDESIGN.md

# Omnitoken GCL Redesign

**Version:** 0.1  
**Status:** Canonical redesign draft  
**Scope:** Omnitoken as an OMNIsurface language for GCL, JupiterBox, swarm transport, and embedded node surfaces  
**Design center:** Works on Racknerd-class nodes and degrades down toward 8 KB RAM targets
**GCL revision reference:** `docs/specs/GCL_TOPOLOGY_REVISION_SPEC.md`

---

## Thesis

Omnitoken should not be "JSON over a transport", "UDP packets", "onion
messages", or "metrics packets with tags." Those are shell appearances.
Omnitoken is an OMNIsurface: it takes the local shape of whatever substrate it
touches while preserving one internal scalar/LUT meaning.

The redesigned Omnitoken is:

```text
a scale-invariant 1D scalar -> compressed LUT -> lawful GCL codon
```

The GCL topology revision fixes the authority boundary around that codon:
Omnitoken can project a shell into a finite codon, but only the GCL gate can
admit it as state. Builder, Warden, and Judge appear as topology phases inside
the same surface rather than as separate runtime authorities.

The revised field-equation extension is defined in
`GCL_FIELD_EQUATIONS_SPEC.md`. In that extension, the middle step becomes a
field-selected LUT:

```text
scale-invariant scalar
  -> surface/motif/informaton field lookup
  -> RGFlow persistent intersection
  -> compressed LUT
  -> lawful GCL codon
```

It is not a network, a blockchain container, a metrics format, or a transport
packet by itself. If it bumps into UDP, the shell may read as UDP. If it moves
through an onion network, the shell may read as onion. If `IPv923U` exists in
the future, it becomes another shell. The internal object remains the same:
a scalar-indexed LUT entry admitted by GCL.

---

## Problems in the Old Shape

The existing Omnitoken work contains the right ingredients, but they are mixed
at the wrong layer.

Observed issues:

1. Operation names are often open strings.
2. Metrics, transport, routing, and execution containers share vocabulary but
   not one canonical frame.
3. CRC is treated as useful integrity, but not clearly separated from
   authentication or replay protection.
4. UDP-like carriers are tolerated, but ordering, loss, replay, and corruption
   are not made first-class protocol obligations.
5. JupiterBox is treated as a transport enhancement rather than the bounded
   carrier envelope for Omnitoken/GCL state.
6. Tiny targets cannot afford JSON, float64, variable dictionaries, or large tag
   blocks.
7. Large targets and tiny targets do not yet share the same operation language.

The redesign fixes this by making Omnitoken a layered codon protocol.

---

## Layer Model

```text
OT0  Codon vocabulary       finite ops, fields, status codes
OT1  Frame                  binary packet with sequence, window, crc
OT2  Fragment/manifest      bounded reassembly and payload hash
OT3  JupiterBox envelope    carrier box, mode, phase, route identity
OT4  Carrier binding        serial, TCP, WebSocket, Tailscale, I2P, ICMP, UDP
OT5  MNN routing            morphic adaptive routing based on goal, state, carrier
OT6  GCL dispatch           admit, attest, route, recover, execute/refuse
```

Only OT4 knows the substrate. OT0-OT3 must be portable across all carriers.

OT5 (MNN routing) is the Morphic Neural Network layer that adapts routing based on:
- Packet goal (health, attest, compress, route, recover)
- Local node state (memory, CPU, recovery mode, trust score)
- Carrier conditions (latency, loss rate, bandwidth)
- Historical outcomes (adaptive learning)

The MNN decides whether to execute locally, defer to the atlas, or reject the operation.

Hosted adapters that need JSON-L use `HUMAN_SURFACE_JSONL_SPEC.md` above this
layer. The JSON-L envelope is for human/tool surfaces; Omnitoken remains the
finite admitted projection.

---

## Design Invariants

Omnitoken MUST:

1. Use finite operation IDs, not open operation strings.
2. Treat carrier delivery as untrusted.
3. Include sequence and replay-window material in every nontrivial frame.
4. Separate accidental corruption checks from authentication.
5. Admit or reject work before expansion.
6. Support tiny targets without JSON, malloc, filesystem, or TCP/IP.
7. Support large targets through the same operation vocabulary.
8. Be routeable by GCL before shell-specific payload interpretation.
9. Fail closed on malformed, replayed, unordered-critical, or unauthenticated
   privileged frames.
10. Preserve recovery mode as a plain, minimal subset.

Omnitoken SHOULD:

1. Prefer compact fixed fields over variable maps.
2. Use manifests for large payloads.
3. Use JupiterBox for multiplexed and phase-aware transport.
4. Allow WebSocket only as a compatibility carrier, not as the inner protocol.
5. Maintain monotonic capability tiers.

---

## OMNI Core: Scale-Invariant Scalar LUT

The compressed internal form is a 1D scalar `s` plus a small context domain.
The scalar selects a LUT entry; the entry expands to a finite GCL codon.

```text
shell surface       local carrier appearance: udp, onion, serial, ipv923u
scalar s           scale-invariant 1D coordinate
domain d           finite context/LUT bank
lut[d][s]          compressed action entry
expanded codon     finite GCL/Omnitoken operation
```

Scale invariance means the shell can change resolution without changing
semantic identity. A 2-byte pulse, an 8-byte tiny frame, a WebSocket binary
message, and an onion payload may all project to the same `(d, s)` pair.
An IBM-II-class software Ethernet controller may also project the same pair
from an Ethernet-looking frame shell.

The shell is descriptive. The scalar/LUT pair is decisive.

```text
shell decides how it looks locally
scalar decides what it means internally
GCL decides whether it may become state
```

For tiny profiles, the preferred pulse is two bytes:

```text
u8 domain
u8 scalar
```

Example recovery pulse:

```text
domain = 0x0D  recovery bank
scalar = 0x01  recovery_subset_boot_ok
lut[0x0D][0x01] -> recover, recovery_allowed, admitted candidate
```

The 2-byte OISC is therefore not a carrier. It is an ultracompressed LUT
execution surface under Omnitoken.

### IBM-II Software Ethernet Intake

An early or intentionally tiny machine can host Omnitoken without a full network
stack by treating Ethernet as a shell around the scalar/LUT pair.

```text
Ethernet-looking frame
  -> shell validation: length, destination, EtherType, FCS
  -> payload[0..1]
  -> (domain, scalar)
  -> LUT admission
  -> GCL codon
```

The controller does not need to understand IP, UDP, TCP, onion routing, or
future `IPv923U`. It only needs to recognize the local shell boundary and pass
the two-byte OMNI core into the LUT. Hosted observers may wrap the admitted
result as JSON-L after the fact.

For an IBM-II-class target, the expected minimum shape is:

```text
RX ring          4 slots
RX slot budget   160 bytes
shell check      destination, EtherType, FCS
inner payload    u8 domain, u8 scalar
admission        finite LUT only
```

This proves the OMNI claim: Ethernet is just one appearance. The invariant
object is still the scale-invariant 1D scalar inside its LUT bank.

### AngrySphinx Default

The default dynamic LUT slot is `angry_sphinx`.

Unknown workloads, unknown shells, and unsafe AMMR routes MUST NOT expand into
large registries by default. They enter the AngrySphinx frustration range:

```text
unknown shell/workload
  -> AVMR mountain signal
  -> S3C partial-computation gate
  -> AMMR safety gate
  -> angry_sphinx slot
  -> proof-of-defense challenge or quarantine
```

This preserves the OMNI invariant while forcing reverse-engineering pressure to
pay solve cost before it can learn useful structure. Recovery remains the only
low-cost escape hatch.

### S3C Partial Computation

S3C is the partial-computation layer between mountain selection and LUT
expansion. A tiny node may not know enough to expand the full workload table, so
it evaluates only the shell coordinates of the proposed token:

```text
(lut_slot, domain, scalar)
  -> shell decomposition
  -> contact A / contact C
  -> bounded J-like score
  -> emit or fall back
```

If the S3C gate emits, AMMR may allow the slot to expand. If it does not emit,
the token SHOULD fall back to AngrySphinx unless it is already in the recovery
profile. This makes partial computation productive: the node spends just enough
work to choose a safe mountain range, not enough to reveal or load the whole
registry.

## OT0: Codon Vocabulary

The core operation table is one byte.

| ID | Op | Class | Meaning |
|----|----|-------|---------|
| 0x00 | `nop` | control | No operation / keepalive |
| 0x01 | `health` | control | Minimal liveness |
| 0x02 | `status` | control | Node state summary |
| 0x03 | `metrics` | control | Bounded telemetry |
| 0x04 | `attest` | trust | Hash/sign state |
| 0x05 | `admit` | gcl | RGFlow/admission check |
| 0x06 | `compress` | data | Compress or report method |
| 0x07 | `route` | gcl | Route or refuse |
| 0x08 | `manifest` | data | Fragment manifest |
| 0x09 | `fragment` | data | Fragment payload |
| 0x0A | `ack` | control | Acknowledge frame/window |
| 0x0B | `nack` | control | Reject/missing frame |
| 0x0C | `snapshot` | recovery | Persist tiny state |
| 0x0D | `recover` | recovery | Enter or operate recovery mode |
| 0x0E | `mark_good` | recovery | Mark image/state as good |
| 0x0F | `refuse` | gcl | Explicit lawful refusal |

Profile-specific operations start at `0x40`.

```text
0x40-0x5F swarm profile
0x60-0x7F storage profile
0x80-0x9F chain/container profile
0xA0-0xBF diagnostic profile
0xC0-0xEF experimental/private profile
0xF0-0xFF reserved
```

---

## OT1: Core Frame

The smallest standard frame is 16 bytes plus payload.

```text
u8    magic          0x4F
u8    version        0x01
u8    op             OT0 operation id
u8    flags          bitfield
u16le source_id      local or atlas-assigned short id
u16le route_id       route/window/domain id
u16le seq            sender sequence
u16le ack            receiver acknowledgment
u16le len            payload bytes
u16le crc16          frame header+payload corruption check
bytes payload        0..profile_max
```

Flag bits:

```text
0x01 payload_compressed
0x02 payload_authenticated
0x04 payload_fragmented
0x08 privileged
0x10 recovery_allowed
0x20 response_required
0x40 carrier_unreliable
0x80 reserved
```

CRC16 is for accidental corruption only. Authenticated profiles add MAC or
signature material in the payload or trailer.

Tiny profiles MAY use an 8-byte pulse frame for `health` only:

```text
u8 magic
u8 version
u8 op
u8 status
u16le source_id
u16le crc16
```

---

## OT2: Fragment and Manifest

UDP weakness research makes fragmentation explicit. No carrier fragment is
trusted as semantic state.

A manifest binds the payload:

```text
payload_hash      16 or 32 bytes
total_len         u32
fragment_count    u16
fragment_size     u16
codec             u8
admission_class   u8
route_hint        u16
```

Each fragment binds itself:

```text
manifest_id       16 bytes
fragment_index    u16
fragment_count    u16
offset            u32
chunk_hash        8 or 16 bytes
chunk_payload     bytes
```

Receiver rule:

```text
carrier arrival != state
valid fragment != state
complete manifest + hashes + replay check + GCL admission == candidate state
```

Reassembly MUST have a byte cap and timeout.

---

## OT3: JupiterBox Envelope

JupiterBox is the bounded carrier envelope. It supplies route identity, phase,
mode, and recovery metadata around Omnitoken frames.

```text
box_id            u16
mode              u8      0..13 for grounded phase
phase             u8      grounded, seismic, flame
priority          u8
ttl               u8
frame_count       u8
box_flags         u8
frames            Omnitoken frames
box_crc           u16
```

Phase behavior:

| Phase | Modes | Behavior |
|-------|-------|----------|
| grounded | 14 | Full multiplexing |
| seismic | 7 | Reduced multiplexing, prefer redundancy |
| flame | 1 or 0 | Do not rely on Jupiter-only path |

If phase is `flame`, GCL MUST either reject, route through a non-Jupiter
manifest shell, or require redundant carriers.

---

## OT4: Carrier Binding

Carriers are dumb pipes. Omnitoken must survive carrier differences.

| Carrier | Role | Rule |
|---------|------|------|
| serial | tiny/recovery | preferred for 8 KB targets |
| TCP | simple reliable carrier | still use Omnitoken seq/replay |
| WebSocket | compatibility | carries Omnitoken frames as binary |
| Tailscale | private mesh | normal large-node carrier |
| I2P | sovereign manifest path | prefer for manifest-heavy payloads |
| ICMP | ghost/sideband | heartbeat or small codon only |
| UDP | unreliable datagram | allowed only with OT2/OT3 protections |

UDP-specific rule:

```text
Raw UDP is never the protocol. It is only a carrier for Omnitoken frames.
```

Every UDP-carried frame MUST set `carrier_unreliable`.

---

## OT5: GCL Dispatch

GCL receives candidate Omnitoken frames and decides:

```text
drop
refuse
ack
nack
admit
route
execute local action
enter recovery
```

Dispatch order:

1. Parse fixed frame.
2. Check length and CRC.
3. Check replay window.
4. Check authentication if required.
5. Check operation privilege.
6. Run RGFlow/admission policy.
7. Expand/decompress only if admitted.
8. Execute or route.
9. Attest state transition.

No payload expansion before admission.

---

## Authentication and Replay

CRC is not security.

Security tiers:

| Tier | Name | Requirement |
|------|------|-------------|
| S0 | public pulse | CRC only, health/nop only |
| S1 | local trusted | CRC + replay window |
| S2 | peer trusted | MAC or keyed hash |
| S3 | atlas trusted | signature or attested session |
| S4 | recovery privileged | signature + physical/local policy |

Replay state for small nodes:

```text
source_id
last_seq
window_bitmap 16 or 32 bits
last_route_id
```

8 KB targets SHOULD use a 16-bit replay window. Larger targets SHOULD use 32 or
64 bits.

---

## Compression

Compression is an admission decision, not a default.

| Tier | RAM | Compression |
|------|-----|-------------|
| T0 | 2 KB | codon substitution only |
| T1 | 8 KB | RLE, tiny LZSS, static dictionary |
| T2 | 32 KB | larger LZSS/dictionary, fragments |
| T3 | 128 KB | stronger dictionary, object snapshots |
| T4 | 512 KB+ | zstd/lz4 profiles, WebSocket bridge |

Omnitoken frame headers are never compressed. Payloads may be compressed only
after admission policy says the CPU and memory trade is lawful.

---

## Capability Tiers

### T0: Pulse Node, about 2 KB RAM

Can:

```text
health
nop
refuse
recover-lite
```

Cannot:

```text
fragment
route tables
payload compression
dynamic peer state
```

### T1: GCL Nerve Ending, about 8 KB RAM

Can:

```text
health
status
attest-lite
admit
route-small
snapshot-small
recover
bounded replay
tiny compression
```

This is the reference minimum.

### T2: Embedded Node, about 32 KB RAM

Adds:

```text
fragment reassembly
small peer table
manifest verification
JupiterBox reduced mode
```

### T3: Rich Embedded Node, about 128 KB RAM

Adds:

```text
object spool
stronger hashes
better recovery shell
dictionary updates
```

### T4: Hosted Surface, 512 KB RAM and up

Adds:

```text
WebSocket carrier
JSON compatibility adapter
zstd/lz4
GDrive mount control
large-node swarm bridge
```

---

## Compatibility Adapters

Old packet styles should become adapters, not the core protocol.

### Metrics Adapter

Old:

```text
metric_name + float64 + timestamp + JSON tags + CRC32
```

New:

```text
OT1 op=metrics
payload = compact metric tuple or profile-specific metric block
```

### Transport Adapter

Old:

```text
transport action encoded as Omnitoken metric-like packet
```

New:

```text
OT1 op=route/admit/manifest
OT3 JupiterBox optional
OT4 carrier selected by MIMO
```

### Container Adapter

Old:

```text
OmniTokenAction container with idempotency, KOT cost, fragments, compliance
```

New:

```text
OT profile 0x80-0x9F
container_id -> manifest_id
idempotency -> replay/session field
compliance -> attestation payload
fragments -> OT2 fragments
```

---

## Failure Semantics

Omnitoken must have explicit failure language.

Refusal codes:

| Code | Meaning |
|------|---------|
| 0x00 | unknown |
| 0x01 | malformed |
| 0x02 | crc_failed |
| 0x03 | replay |
| 0x04 | auth_required |
| 0x05 | auth_failed |
| 0x06 | op_not_supported |
| 0x07 | privilege_required |
| 0x08 | admission_failed |
| 0x09 | route_unavailable |
| 0x0A | memory_budget_exceeded |
| 0x0B | fragment_timeout |
| 0x0C | manifest_hash_failed |
| 0x0D | jupiter_phase_flame |
| 0x0E | recovery_locked |
| 0x0F | carrier_untrusted |

Refusal is not failure if it preserves GCL invariants.

---

## Canonical Tiny Frame Example

A T1 health frame:

```text
4F 01 01 00  2A 00  01 00  10 00  0F 00  00 00  CRC CRC
```

Meaning:

```text
magic=0x4F
version=1
op=health
flags=0
source_id=42
route_id=1
seq=16
ack=15
len=0
crc16=...
```

No JSON. No strings. No heap.

---

## Redesign Migration Plan

1. Freeze this spec as Omnitoken v5 draft.
2. Keep old v4 metrics codec as `compat.metrics.v4`.
3. Implement a tiny reference codec in C or Rust with no allocation.
4. Implement a Python adapter for hosted nodes and Docker tests.
5. Replace open-string operations with OT0 IDs.
6. Route all old Omnitoken/MIMO/Jupiter paths through OT1/OT2/OT3.
7. Add Lean definitions for finite ops, frame validity, replay acceptance, and
   admission-before-expansion.
8. Only then deploy to Racknerd as a hosted carrier.

---

## The New Meaning of Omnitoken

Omnitoken is the language by which one observer chart asks another observer
chart to accept a lawful state transition.

Short form:

```text
Omnitoken = finite GCL codons + replay-aware frames + route-admissible state
```

It wants to be the invariant packet language of the atlas. This redesign makes
that explicit.