mirror of https://github.com/allaunthefox/Research-Stack.git synced 2026-07-31 03:05:21 +00:00

Brandon Schneider 5f88abf618 initial: sovereign research stack (consolidated, weightless, and lfs-optimized)

2026-05-04 18:11:36 -05:00

20 KiB

Raw Blame History

Human Surface JSON-L Spec

Version: 0.1
Status: Draft surface envelope
Scope: JSON-L event format for every human-facing surface that feeds Omnitoken, GCL, ENE, RGFlow, and embedded node surfaces
Depends on: docs/UNIFIED_JSONL_SCHEMA.md, docs/specs/OMNITOKEN_GCL_REDESIGN.md, docs/specs/EMBEDDED_NODE_SURFACE_SPEC.md

Thesis

The atlas needs one append-only language for human surfaces.

A human surface is any interface where a person, tool, model, device, or storage substrate exposes state that a node may observe, route, compress, attest, or recover.

Examples:

Chat messages
Documents
Tasks
Code changes
Terminal sessions
Browser/search traces
Email and calendar events
Files and mounted drives
Images, audio, video, maps, sensors
APIs, databases, ledgers, model outputs
Recovery pulses and node health events

The surface layer is JSON-L because hosted nodes and human tools can inspect it. Tiny nodes do not parse JSON-L directly. They receive the admitted result as Omnitoken/GCL codons.

human surface -> JSON-L envelope -> GCL admission -> Omnitoken frame

Design Invariants

Every observed surface event MUST fit in one JSON object per line.
The line MUST be append-only. Corrections are new lines.
Decision fields MUST be finite enums.
Future surfaces MUST have a place to land without changing the core enum.
Raw adapter payloads MAY be preserved, but GCL MUST admit before expansion.
Large content SHOULD be referenced by hash or URI, not embedded.
Hosted JSON-L and tiny Omnitoken MUST share the same operation intent.
Unknown surfaces MUST be preservable and routeable as unknown, not discarded.
Privacy and patent-hold status MUST travel with the event.
Recovery mode MUST use the same envelope with a smaller required subset.

Layer Position

HS0  Surface classes       finite human/computational surface classes
HS1  JSON-L envelope       append-only hosted event line
HS2  Admission projection  finite op, caps, cost, privacy, genome
HS3  Omnitoken mapping     scalar/LUT, OT0 op id, flags, route id, manifest
HS4  Shell/storage         Ethernet, GDrive, disk, WS, serial, TCP, onion, future labels

HS1 is intentionally richer than an embedded node can afford. HS2 is the projection boundary. Anything not admitted by HS2 remains archived data, not state.

Default unknown-shell behavior is AngrySphinx quarantine. Unknown surfaces are preserved as archive data, but their Omnitoken projection SHOULD use the angry_sphinx LUT slot until a hosted registry admits a more specific workload profile.

Tiny projections MAY include S3C partial-computation metadata such as s3c_emit and s3c_score. These fields describe whether the proposed (lut_slot, domain, scalar) had enough shell contact to expand; they are not a replacement for GCL admission.

Mandatory Root Fields

Field	Type	Meaning
`v`	String	Schema version, currently `hs-jsonl-0.1`
`id`	String	Stable event id
`t`	Number	Unix timestamp, seconds with optional fraction
`op`	Enum	Surface operation intent
`surface`	Object	Surface class, kind, instance, and capabilities
`actor`	Object	Human, agent, service, or node that caused the event
`object`	Object	Thing being observed or changed
`content`	Object	Hash, URI, MIME, text, or compact payload pointer
`gcl`	Object	Admission, lawful state, cost, refusal, and invariants
`omni`	Object	Omnitoken projection metadata
`provenance`	Object	Origin, chain, node, source time, and prior event
`privacy`	Object	Disclosure, retention, and patent-hold policy

Optional root fields:

Field	Type	Meaning
`genome`	Object	6D RGFlow routing signature
`links`	Array	Typed edges to other events or objects
`storage`	Object	Local, GDrive, content-addressed, or remote storage pointer
`raw`	Object	Adapter-native payload preserved after admission boundary
`render`	Object	Human presentation hints, never decision-critical

Finite Operation Enum

op is a finite enum. Adapters MUST map their native actions into one of these values.

Op	Meaning	Typical OT0 Projection
`observe`	Read or witness state without claiming change	`status`, `metrics`
`assert`	Add or upsert a state claim	`admit`
`revise`	Correct or supersede a previous claim	`admit`
`delete`	Tombstone or remove availability	`admit`
`attest`	Hash, sign, or witness state	`attest`
`route`	Ask the atlas to move or locate state	`route`
`recover`	Enter or report recovery mode	`recover`
`snapshot`	Persist a compact state image	`snapshot`
`render`	Produce a human-viewable representation	profile op
`reply`	Respond to an interaction	profile op
`link`	Create an edge between objects	`admit`
`refuse`	Preserve a lawful refusal	`refuse`

Profile-specific operations MAY appear in raw.native_op, but never in op.

Surface Classification

The surface decision field is surface.class. It is finite.

Class	Meaning
`chat`	Human or agent conversation
`doc`	Document, note, page, paper, wiki
`task`	Issue, ticket, todo, project state
`code`	Repository, diff, review, build artifact
`terminal`	Shell, command, log, process interaction
`file`	File object, mount object, object-store blob
`drive`	Mounted cloud or remote drive surface
`email`	Mail message or mailbox state
`calendar`	Calendar event, availability, schedule
`browser`	Web page, search result, bookmark, tab state
`api`	HTTP/RPC/tool call boundary
`db`	Database row, query, schema, materialized view
`sensor`	Hardware or environmental reading
`image`	Image or visual artifact
`audio`	Audio artifact or stream
`video`	Video artifact or stream
`map`	Geospatial object or route
`finance`	Price, trade, account, market observation
`chain`	Blockchain, ledger, proof, block, transaction
`model`	Model prompt, output, embedding, eval
`node`	Runtime, health, topology, recovery surface
`unknown`	Preserved unknown surface

The open extension field is surface.kind, a namespaced string such as notion.page, linear.issue, gdrive.file, github.pull_request, jupiterbox.pulse, or racknerd.health.

The rule is:

surface.class decides
surface.kind explains
raw preserves

Surface Object

{
  "class": "doc",
  "kind": "notion.page",
  "instance": "notion:workspace:research",
  "caps": ["read", "write", "link", "attest"],
  "adapter": "notion-v1",
  "adapter_version": "0.1"
}

caps is finite:

read write delete link attest route recover render stream mount execute

Adapters MAY add human labels in render, but capability decisions use caps.

Actor Object

{
  "type": "agent",
  "id": "node:qfox",
  "display": "qfox",
  "authority": "local_admin"
}

actor.type is finite:

human agent service node model device unknown

authority is finite:

none local_user local_admin root service_token delegated recovery

Object Object

{
  "type": "page",
  "id": "notion:page:7a27bd85",
  "stable_id": "sha256:...",
  "parent": "notion:database:research-stack",
  "version": "2026-04-25T16:40:00Z"
}

object.type is adapter-specific descriptive text. It does not decide admission. Admission uses surface.class, op, caps, privacy, and gcl.

Content Object

The content object should prefer pointers over bulk payloads.

{
  "mime": "text/markdown",
  "hash": "sha256:...",
  "size": 2048,
  "uri": "gdrive://research-stack/docs/specs/example.md",
  "summary": "Surface event for a reusable embedded node spec.",
  "text": "Optional short text payload."
}

Allowed fields:

Field	Meaning
`mime`	Media type
`hash`	Content hash
`size`	Byte size before compression
`compressed_size`	Byte size after compression
`codec`	`none`, `zstd`, `brotli`, `gzip`, `heatshrink`, `lz4`, `custom`
`uri`	Storage or retrieval URI
`summary`	Short human summary
`text`	Small text payload
`bytes_b64`	Small binary payload only
`refs`	Content references

Large payloads MUST become Omnitoken manifests or storage references before transport to constrained nodes.

GCL Object

{
  "admission": "candidate",
  "lawful": true,
  "cost_q16_16": 65536,
  "invariant": "admission_before_expansion",
  "refusal_code": "none",
  "capability_tier": "T4"
}

gcl.admission is finite:

unknown candidate admitted refused quarantined archived

gcl.refusal_code maps to the Omnitoken refusal table:

none unknown malformed replay auth_required auth_failed op_not_supported
privilege_required admission_failed route_unavailable memory_budget_exceeded
fragment_timeout manifest_hash_failed recovery_locked carrier_untrusted

capability_tier is finite:

T0_pulse T1_8kb T2_32kb T3_128kb T4_hosted T5_full

Omnitoken Object

{
  "op_id": "0x05",
  "source_id": 42,
  "route_id": 1,
  "seq": 16,
  "flags": ["response_required"],
  "manifest_id": null,
  "frame_profile": "OT1"
}

omni.flags is the symbolic form of OT1 flags:

payload_compressed payload_authenticated payload_fragmented privileged
recovery_allowed response_required carrier_unreliable

Hosted JSON-L MAY omit seq before projection. Once projected to Omnitoken, sequence and replay-window material are required for nontrivial frames.

Provenance Object

{
  "origin": "notion:workspace:research",
  "node": "qfox",
  "observed_t": "2026-04-25T16:40:00Z",
  "ingested_t": "2026-04-25T16:40:02Z",
  "prev_id": null,
  "attestation_hash": "sha256:..."
}

prev_id links corrections, replies, and causal chains. attestation_hash SHOULD be the cumulative hash of the normalized event plus the previous attestation hash.

Privacy Object

{
  "tier": "patent_hold",
  "retention": "archive",
  "export": "deny",
  "redaction": "none"
}

privacy.tier is finite:

public internal private secret patent_hold local_only

privacy.export is finite:

allow redact deny local_only

privacy.retention is finite:

ephemeral cache archive legal_hold delete_requested

No adapter may downgrade privacy. A forwarded event may only keep or strengthen the tier.

Storage Object

{
  "class": "drive",
  "uri": "gdrive://research-stack/surfaces/2026/04/event.jsonl",
  "mount": "/mnt/gdrive",
  "content_address": "sha256:...",
  "local_cache": "/var/lib/atlas/cache/sha256/..."
}

storage.class is finite:

none local drive object_store db content_addressed remote ephemeral

For underpowered nodes, storage SHOULD be an external mount or content-addressed cache. The node surface does not need a full application stack if a drive can hold the heavy state.

Links

[
  {
    "rel": "supersedes",
    "target": "hs:old-event-id"
  },
  {
    "rel": "derived_from",
    "target": "gdrive://research-stack/raw/source.md"
  }
]

rel is finite:

parent child supersedes corrects derived_from caused_by blocks blocked_by
mentions replies_to attests_to stores routes_to recovers_from mirrors

Genome

The optional genome field is the same 6D quantized routing signature used by the existing unified JSON-L schema.

{
  "mu": 0,
  "rho": 0,
  "c": 0,
  "m": 0,
  "ne": 0,
  "sig": 0
}

The genome is not a hash. It is a lossy routing projection.

Minimal Recovery Line

Recovery mode may emit only the required recovery subset:

{
  "v": "hs-jsonl-0.1",
  "id": "node:racknerd-510bd9c:recover:0001",
  "t": 1777135200.0,
  "op": "recover",
  "surface": {
    "class": "node",
    "kind": "racknerd.health",
    "instance": "racknerd-510bd9c",
    "caps": ["read", "recover"],
    "adapter": "embedded-surface",
    "adapter_version": "0.1"
  },
  "actor": {
    "type": "node",
    "id": "node:racknerd-510bd9c",
    "display": "racknerd-510bd9c",
    "authority": "recovery"
  },
  "object": {
    "type": "runtime",
    "id": "node:racknerd-510bd9c",
    "stable_id": "node:racknerd-510bd9c",
    "parent": null,
    "version": "recovery"
  },
  "content": {
    "mime": "application/json",
    "hash": "sha256:...",
    "size": 64,
    "summary": "Recovery heartbeat."
  },
  "gcl": {
    "admission": "admitted",
    "lawful": true,
    "cost_q16_16": 1,
    "invariant": "recovery_subset",
    "refusal_code": "none",
    "capability_tier": "T1_8kb"
  },
  "omni": {
    "op_id": "0x0D",
    "source_id": 42,
    "route_id": 1,
    "seq": 1,
    "flags": ["recovery_allowed"],
    "manifest_id": null,
    "frame_profile": "OT1"
  },
  "provenance": {
    "origin": "node:racknerd-510bd9c",
    "node": "racknerd-510bd9c",
    "observed_t": "2026-04-25T16:40:00Z",
    "ingested_t": "2026-04-25T16:40:00Z",
    "prev_id": null,
    "attestation_hash": "sha256:..."
  },
  "privacy": {
    "tier": "internal",
    "retention": "cache",
    "export": "deny",
    "redaction": "none"
  }
}

Example: Chat Surface

{
  "v": "hs-jsonl-0.1",
  "id": "chat:research:2026-04-25:0001",
  "t": 1777135201.5,
  "op": "assert",
  "surface": {
    "class": "chat",
    "kind": "codex.session",
    "instance": "research-stack",
    "caps": ["read", "write", "link", "attest"],
    "adapter": "codex",
    "adapter_version": "0.1"
  },
  "actor": {
    "type": "human",
    "id": "human:allaun",
    "display": "allaun",
    "authority": "local_admin"
  },
  "object": {
    "type": "message",
    "id": "chat:research:msg:0001",
    "stable_id": "sha256:...",
    "parent": "chat:research",
    "version": "1"
  },
  "content": {
    "mime": "text/plain",
    "hash": "sha256:...",
    "size": 92,
    "summary": "Request for a JSON-L spec that can cover every human surface.",
    "text": "Use every human surface possible and have space for more."
  },
  "gcl": {
    "admission": "candidate",
    "lawful": true,
    "cost_q16_16": 65536,
    "invariant": "surface_event_append_only",
    "refusal_code": "none",
    "capability_tier": "T4_hosted"
  },
  "omni": {
    "op_id": "0x05",
    "source_id": 42,
    "route_id": 1,
    "seq": null,
    "flags": ["response_required"],
    "manifest_id": null,
    "frame_profile": "hosted-jsonl"
  },
  "provenance": {
    "origin": "codex:session",
    "node": "qfox",
    "observed_t": "2026-04-25T16:40:01.500Z",
    "ingested_t": "2026-04-25T16:40:02.000Z",
    "prev_id": null,
    "attestation_hash": "sha256:..."
  },
  "privacy": {
    "tier": "patent_hold",
    "retention": "archive",
    "export": "deny",
    "redaction": "none"
  }
}

Example: GDrive Mounted File

{
  "v": "hs-jsonl-0.1",
  "id": "gdrive:file:research-stack:spec:0001",
  "t": 1777135222.0,
  "op": "observe",
  "surface": {
    "class": "drive",
    "kind": "gdrive.file",
    "instance": "gdrive:research-stack",
    "caps": ["read", "write", "mount", "attest"],
    "adapter": "rclone",
    "adapter_version": "1"
  },
  "actor": {
    "type": "service",
    "id": "service:rclone",
    "display": "rclone",
    "authority": "service_token"
  },
  "object": {
    "type": "file",
    "id": "gdrive://research-stack/docs/specs/HUMAN_SURFACE_JSONL_SPEC.md",
    "stable_id": "sha256:...",
    "parent": "gdrive://research-stack/docs/specs",
    "version": "2026-04-25T16:40:22Z"
  },
  "content": {
    "mime": "text/markdown",
    "hash": "sha256:...",
    "size": 12000,
    "uri": "gdrive://research-stack/docs/specs/HUMAN_SURFACE_JSONL_SPEC.md",
    "summary": "Reusable human surface JSON-L envelope."
  },
  "storage": {
    "class": "drive",
    "uri": "gdrive://research-stack/docs/specs/HUMAN_SURFACE_JSONL_SPEC.md",
    "mount": "/mnt/gdrive",
    "content_address": "sha256:...",
    "local_cache": null
  },
  "gcl": {
    "admission": "candidate",
    "lawful": true,
    "cost_q16_16": 4096,
    "invariant": "external_storage_pointer",
    "refusal_code": "none",
    "capability_tier": "T4_hosted"
  },
  "omni": {
    "op_id": "0x08",
    "source_id": 42,
    "route_id": 2,
    "seq": null,
    "flags": ["payload_fragmented"],
    "manifest_id": "sha256:...",
    "frame_profile": "OT2"
  },
  "provenance": {
    "origin": "gdrive:research-stack",
    "node": "qfox",
    "observed_t": "2026-04-25T16:40:22Z",
    "ingested_t": "2026-04-25T16:40:23Z",
    "prev_id": null,
    "attestation_hash": "sha256:..."
  },
  "privacy": {
    "tier": "internal",
    "retention": "archive",
    "export": "redact",
    "redaction": "hash_only"
  }
}

Example: Terminal Command Result

{
  "v": "hs-jsonl-0.1",
  "id": "terminal:qfox:cmd:0001",
  "t": 1777135244.0,
  "op": "attest",
  "surface": {
    "class": "terminal",
    "kind": "bash.command",
    "instance": "qfox",
    "caps": ["read", "execute", "attest"],
    "adapter": "local-shell",
    "adapter_version": "0.1"
  },
  "actor": {
    "type": "agent",
    "id": "agent:codex",
    "display": "codex",
    "authority": "local_user"
  },
  "object": {
    "type": "command",
    "id": "terminal:qfox:cmd:0001",
    "stable_id": "sha256:...",
    "parent": "terminal:qfox",
    "version": "exit:0"
  },
  "content": {
    "mime": "text/plain",
    "hash": "sha256:...",
    "size": 128,
    "summary": "Command completed successfully."
  },
  "gcl": {
    "admission": "admitted",
    "lawful": true,
    "cost_q16_16": 1024,
    "invariant": "attested_command_result",
    "refusal_code": "none",
    "capability_tier": "T4_hosted"
  },
  "omni": {
    "op_id": "0x04",
    "source_id": 42,
    "route_id": 3,
    "seq": null,
    "flags": ["payload_authenticated"],
    "manifest_id": null,
    "frame_profile": "hosted-jsonl"
  },
  "provenance": {
    "origin": "terminal:qfox",
    "node": "qfox",
    "observed_t": "2026-04-25T16:40:44Z",
    "ingested_t": "2026-04-25T16:40:44Z",
    "prev_id": null,
    "attestation_hash": "sha256:..."
  },
  "privacy": {
    "tier": "private",
    "retention": "cache",
    "export": "deny",
    "redaction": "none"
  }
}

Legacy Mapping

The existing docs/UNIFIED_JSONL_SCHEMA.md remains the ENE substrate ingestion schema. This spec is the wider surface envelope.

Legacy src maps into surface as follows:

Legacy `src`	Surface Class	Surface Kind
`notion`	`doc`	`notion.page` or `notion.database`
`linear`	`task`	`linear.issue`
`ene`	`db`	`ene.package`
`rgflow`	`chain`	`rgflow.block`
`swarm`	`node`	`swarm.topology`

Legacy op maps as follows:

Legacy `op`	Human Surface `op`
`upsert`	`assert`
`delete`	`delete`
`snapshot`	`snapshot`
`correct`	`revise`
`attest`	`attest`

Admission Rules

Parse the JSON object.
Validate mandatory fields and finite enums.
Verify privacy cannot be downgraded.
Refuse privileged operations without authority.
Hash or manifest large content before payload expansion.
Compute or propagate genome.
Evaluate GCL admission and cost.
Project admitted intent to Omnitoken OT0.
Assign sequence/window state when sending over a carrier.
Archive refused lines with op: "refuse" or gcl.admission: "refused".

The key rule is:

archive can store arbitrary surface data
state can only contain admitted finite projections

Tiny Target Projection

Tiny nodes do not implement this full JSON-L schema.

Target	Receives
`T0_pulse`	Health/recovery pulse only
`T1_8kb`	OT1 frames, no JSON, tiny fixed buffers
`T2_32kb`	OT1 plus bounded manifests
`T3_128kb`	OT1/OT2 plus small dictionaries
`T4_hosted`	Full JSON-L envelope
`T5_full`	Full adapters, search, render, storage

The JSON-L envelope is the human-readable front porch. Omnitoken is the nerve signal.

Future Surface Rule

When a new human surface appears:

Choose the nearest existing surface.class.
Add a namespaced surface.kind.
Preserve native fields under raw.
Map native operations into finite op.
Define capability and privacy behavior.
Define GCL admission behavior.
Define Omnitoken projection.

No core schema change is required unless the new surface cannot be truthfully represented by any existing class.

20 KiB Raw Blame History