Content storage and module formats

tricu separates executable content, human-facing names, and build provenance. A tree's identity comes from its canonical representation. Module manifests name exported trees; aliases are mutable conveniences; receipts record build inputs.

Hashes and paths

Compiler/CLI objects use domain-separated SHA-256:

hash = lowercase_hex(SHA256(UTF8(domain) || 0x00 || payload))
path = STORE/objects/hash[0:3]/hash

Filesystem digests are exactly 64 lowercase hexadecimal characters. Typed reads verify both the expected domain and digest before decoding. Cached verified objects preserve the bytes verified at read time as immutable snapshots.

The HTTP registry has a different identity and layout:

hash = lowercase_hex(SHA256(whole_bundle_bytes))
path = REGISTRY/objects/hash[0:3]/hash.arboricx

Use the whole-file hash for browser downloads and registry URLs. Use a manifest hash for --toolchain and stored execution. Individual executable objects have their own hashes for object-level operations.

The registry computes upload identities and publishes whole files atomically. Its direct read API validates the digest's spelling before reading; clients such as the browser authenticate downloaded bytes against their selected pin. The registry store and its directory layout are trusted configuration.

Executable representations

Domain	Payload
arboricx.tree-term.v1	Canonical preorder bytes: Leaf 00; Stem 01 child; Fork 02 left right.
arboricx.indexed-tree.v1	A name-neutral single-root Arboricx bundle, exporting root with the tree ABI.
arboricx.merkle.node.v1	One node: tag 00, 01 plus one raw 32-byte child digest, or 02 plus two child digests.
arboricx.module-manifest.v1	The text manifest below.

Flat tree-term bytes enumerate logical occurrences, including repeated branches. Indexed objects preserve physical sharing and are the CLI import default. Root extraction prunes unrelated nodes and removes source export names from object identity. Loading checks the graph, digest, object kind, and ABI before rebuilding shared values. Tree inspection still counts logical occurrences.

Native module publication and imports use indexed objects, matching CLI import. Flat objects remain available for explicit raw-tree transport and retained older toolchains. The current native module reader requires indexed objects. The host's Merkle-node representation is a separate object graph used by its tree store. Each representation has its own identity even when it denotes the same tree.

Bounded flat import

arboricx import --flat computes aggregate expanded bytes across all exports before expansion or object writes. The default budget is 64 MiB; a positive --max-flat-bytes BYTES changes it. Saturating DAG accounting handles tiny graphs whose unfolded trees would be enormous. The budget bounds flat output. Input size, process memory, and execution time require separate limits.

Bundle export accepts indexed and flat references, preserving the declared kind. Bare hashes select a format from the wire magic, then authenticate its complete payload before decoding. Indexed packing retains sharing. All-flat packing uses a streaming reader; mixed packing decodes both formats into shared trees and can retain a flat input payload while decoding it.

Module manifest v1

Canonical encoding is UTF-8 text with a final LF. The header is:

arboricx.module-manifest.v1

Each following row is tab-separated:

reference<TAB>alias<TAB>kind<TAB>hash
export<TAB>name<TAB>kind<TAB>hash<TAB>abi

Fields escape % as %25, TAB as %09, LF as %0A, and CR as %0D. Row order is part of the hashed bytes. Export names are unique. Executable readers require arboricx.abi.tree.v1 and a supported object kind.

The Haskell format supports reference rows and typed export rows. The native tricu module profile accepts canonical executable export rows with ASCII source names and the indexed-tree kind; re-encoding checks its exact canonical spelling. Its module producer requires a nonempty interface. The Haskell decoder accepts a broader text/name profile, with typed constraints applied by consumers.

Contracts are ordinary trees. An annotated export carries its checking code inside the executable; a separately exported contract is invoked by its consumer. See Contracts.

Aliases

Aliases point to a kind and hash using two LF-terminated lines:

kind: OBJECT_DOMAIN
hash: OBJECT_HASH

The host stores name, module, and package aliases under aliases/names, aliases/modules, and aliases/packages. Names are literal filename components. Empty names, ./.., separators, colons, and control characters are rejected; qualified and symbolic names are allowed.

The native module API uses ASCII identifier-like module aliases and verifies the referenced manifest before atomic alias publication. The Haskell alias writer uses ordinary file replacement. CLI import validates the complete alias set before changing aliases; a later IO failure may still leave partial publication.

Publication and trust

The native object layer uses atomic no-replace publication and verifies an existing or racing winner. Individual aliases and checkpoints use atomic replacement. The Haskell filesystem adapter checks existing bytes and uses same-store temporary files and rename for object writes. These paths have different durability and concurrency behavior; select the relevant API explicitly.

Store directories, ancestors, and symlink layout must be trusted. Content hashes establish identity and detect mismatched bytes. Trust in the producer and execution permissions is a separate decision. Multi-object builds can leave intermediate objects after failure.

Verified native tree transport

The Haskell IO driver provides these typed transport operations:

Action	Tag	Result
readTreeTerm path digest	34	Verified, decoded tree.
hashTreeTerm tree	35	Canonical tree-term digest.
writeTreeTermIfAbsent path digest tree	36	ok true for a new file; ok false for an identical existing file.
readIndexedTree path digest	50	Verified tree from a name-neutral indexed object.
hashIndexedTree tree	51	Canonical arboricx.indexed-tree.v1 digest.
writeIndexedTreeIfAbsent path digest tree	52	No-replace indexed publication, with the same Boolean Result as tag 36.

The indexed operations use exactly the single-root object format produced by indexed bundle import. Encoding memoizes physical sharing and interns exact constructor/child-ID keys; equal finite trees retain canonical bytes regardless of their physical sharing. Both codecs treat the supplied tree as data. Tags 40 and 41 remain the existing state operations.

Reads require read permission, a canonical digest, and a valid payload under the selected format's fixed hash domain. Tree-term bytes must end exactly at the end of the encoded tree. Shared decoding interns constructors without evaluating the program. Writes require read and write permission, verify the supplied digest, use a unique same-directory temporary file and atomic no-replace link, sync the file and directory, and compare an existing winner's bytes. A sync error after publication may leave the new name visible. tricu owns directory creation.

These operations avoid constructing a tricu byte list for large executable transport. The portable tree codec remains an independent implementation.

Build receipts and cache

sourceBuildGraph memoizes completed units within a call. Across calls it writes cache/source-build/NAME after publishing the manifest and a separate arboricx.build-receipt.v1 object.

The receipt records the source digest, declared compiler identity, options digest, output manifest, ordered local manifest/receipt references, and exact imported or re-exported tree pins. Its source digest covers the raw source bytes. Receipts identify those bytes; retain source snapshots alongside artifacts for rebuilding.

The arboricx.source-build-key.v1 key covers the same inputs except output. A matching cache index must reference an authenticated receipt with the expected inputs and outputs, a valid manifest, and verified imported/exported trees. Missing indices or changed keys rebuild. Malformed indices or corrupt matching objects fail. Indices are trusted local acceleration metadata. Checking referenced bytes establishes integrity; establishing compiler reproduction requires a fresh rebuild.

Cache granularity is the complete interface and dependency provenance. Changing an unused export or a dependency's source-only receipt can invalidate consumers. The index retains the latest result per module name; historical CAS objects can outlive that index. These rules preserve namespace checks and exact dependency identity through reuse.

Retained inputs and independent reproduction

tricu provenance retains inputs for the native arboricx.build-receipt.v1 graph, preserving its receipt format and executable identities. See the capture/verify/rebuild workflow.

A portable input pack contains a build-inputs descriptor and a standard CAS under store/objects. The descriptor is authenticated with the domain arboricx.build-inputs.v1; retain/share its full hash independently of its path. Its canonical LF/tab format records an entry name, the native root receipt, a compiler manifest, sorted unit-name/receipt pairs, and optional project metadata pins. The inventory must exactly match the reachable native local-receipt graph; unit paths are derived as NAME.tri.

Native receipts remain authoritative for each unit's raw source digest, declared compiler identity, options, output, ordered local edges and imported tree pins. The pack supplies the corresponding preimages: raw sources under arboricx.source.v1 and the supported profile string under arboricx.build-options.v1. Source snapshots must match the existing receipt exactly. Each receipt's complete pin group must match its referenced module interface, and local output/receipt links are checked recursively. An explicit --receipt selects a receipt directly. Otherwise, the native cache locates it, and the cache key/output fields must agree with the authenticated receipt.

The first profile is the indexed, stable flat-workspace compiler route with explicit toolchain selection. Its native compiler identity is the indexed object hash of sourceBuildGraph; the retained compiler manifest must select that exact export. The descriptor retains a runnable compiler facade. The producer’s original facade and transport bundle remain unverified claims. Default packaged-compiler builds currently record a whole-bundle identity instead; rebuild with an explicit toolchain before capturing them in this profile. Other option profiles/frontends are rejected.

Optional project declaration/lock bytes have domains arboricx.project-declaration.v1 and arboricx.project-lock.v1. They record capture-time selection context; native receipts separately identify compiler-read inputs. Their selections must agree, and reached generated binding snapshots must match the lock. Unused project selections are retained as context. Rebuilds require libraries only for reached selections.

Claimed outputs are retained as comparison evidence, separate from executable inputs. Rebuilds seed only immutable external imports into a new application CAS and the selected compiler into a separate tool CAS. Producer caches, aliases, receipts, checkpoints, and output-only objects stay outside that application store. An output that is also a genuine external input (such as a header-only re-export) is necessarily available as an input. The compiler-returned receipt and its new local graph are authenticated after execution; root receipt/module identity and export structures are compared against the pack.

Verification establishes byte integrity and receipt-link consistency. A successful fresh rebuild establishes a local observation that the selected compiler reproduced the native receipt and module. Publisher authentication and signed attestations require separate mechanisms; source history can be ambiguous. Several source histories or retained contexts may produce the same executable. Pack hashes and native receipt hashes remain separate from module and distribution-bundle identities.

Export checkpoints

lib/source/export-cache.tri offers a separate per-export checkpoint profile. Its scope fixes compiler identity and rewritten source, including immutable import/re-export manifests. Export names are separately domain-hashed for paths. Each index publishes after verified object publication; restart authenticates and decodes the entire tree before reuse. Invalid indices or corrupt objects fail.

Restart still stages, parses, orders, resolves, and constructs the definition environment. It skips materializing completed exports. Module manifests and receipts publish after all required exports complete. See Compiler rebuilding for closure audits and acceptance.

Exact package release index and locks

Package release v1 identifies a release by its distribution bundle and indexed native module manifest. At REGISTRY/packages/NAME/VERSION, the canonical UTF-8 file contains two LF-terminated lines (fields separated by TAB):

arboricx.package-release.v1
NAME<TAB>VERSION<TAB>MODULE_MANIFEST_HASH<TAB>WHOLE_BUNDLE_HASH

The bundle's ordered exports must import into exactly the advertised native manifest. The publisher checks this without evaluating exports. Metadata derives its authority from trust in the registry. Signature verification requires a separate mechanism. Richer package metadata requires an explicitly versioned format; this profile’s module pin identifies a native module manifest.

A client lock is a separate, canonical, three-line file:

arboricx.package-lock.v1
REGISTRY_NAME<TAB>ORIGIN_URL
NAME<TAB>VERSION<TAB>MODULE_MANIFEST_HASH<TAB>WHOLE_BUNDLE_HASH

The package publisher uses same-directory temporary files, file sync, hard-link no-replace publication, byte comparison against a racing winner, and parent sync. Locks and generated workspace bindings use atomic replacement. The Haskell CAS adapter retains its own durability/concurrency guarantees. Atomicity applies to individual files; serialize lock updates and use trusted directories and ancestors. Power-loss and crash-injection behavior require separate validation. See package usage.