commit 08a004a8043e5926cd0724e53eb1ceb58b3055a6
parent 316961cf1b3e5af5f54b0b6bd23a6caca8c27aa9
Author: James Eversole <james@eversole.co>
Date: Sat, 17 Oct 2026 16:43:19 -0500
Merge branch 'rewrite-arboricx-bundle-format-docs'
Diffstat:
| M | docs/arboricx-bundle-format.md | | | 59 | +++++++++++++++++++++++++++-------------------------------- |
| M | docs/compiler-rebuilding.md | | | 235 | ++++++++++++++++++++++++++++++++++++++++++------------------------------------- |
| M | docs/modules.md | | | 466 | +++++++++++++++++++++++++++++++++++++++++-------------------------------------- |
| M | docs/storage.md | | | 134 | +++++++++++++++++++++++++++++++++++++++++-------------------------------------- |
| M | docs/tri-tools.md | | | 199 | ++++++++++++++++++++++++++++++++++++++++++------------------------------------- |
| M | docs/validation.md | | | 928 | +++++++++++++++++++++++++++++++++++++++++-------------------------------------- |
6 files changed, 1054 insertions(+), 967 deletions(-)
diff --git a/docs/arboricx-bundle-format.md b/docs/arboricx-bundle-format.md
@@ -2,7 +2,7 @@
**Container:** 1.0. **Manifest:** 1.1. Indexed nodes.
-The Arboricx Portable Bundle is a self-contained binary format for distributing Tree Calculus programs. It uses topological indexing instead of cryptographic hashing for node identity, making it writable from pure Tree Calculus and verifiable via structural inspection.
+The Arboricx Portable Bundle is a self-contained binary format for distributing Tree Calculus programs. Nodes are identified by their positions in a topologically ordered array. Pure Tree Calculus programs can write bundles, and readers verify them through structural inspection.
## Table of Contents
@@ -23,13 +23,13 @@ The Arboricx Portable Bundle is a self-contained binary format for distributing
## 1. Design Principles
-- **No cryptographic primitives required.** Node identity is topological (array index), not a SHA-256 hash.
-- **Self-contained.** A bundle includes all nodes reachable from its exports. No external references.
+- **Indexed nodes.** Each node is identified by its array position; child references point to earlier entries.
+- **Self-contained.** A bundle includes all nodes reachable from its exports.
- **Deterministic.** Canonical writers produce identical bytes for identical ordered named roots and metadata.
- **Compact.** Leaf, Stem, and Fork records occupy 5, 9, and 13 bytes, respectively, including their length prefix.
-- **Verifiable via structure.** Bounds checking and acyclicity verification replace hash recomputation.
+- **Structurally verifiable.** Readers check record layouts, reference bounds, and acyclicity.
-Global artifact identity (for registries, lockfiles, or content-addressed caches) is achieved by hashing the complete canonical bundle file externally. The bundle format itself knows nothing about this hash.
+Distribution systems establish artifact identity and integrity using a digest of the complete canonical bundle file. Registries, lockfiles, and content-addressed caches use this file-level identity.
---
@@ -81,8 +81,6 @@ Array of `N` entries, each exactly **32 bytes**.
- Compression must be `0` (none).
- Reserved fields must be zero.
-**Note:** No per-section digest is stored. Integrity is verified at the distribution layer (e.g. SHA-256 of the complete bundle file) rather than inside the container.
-
---
## 5. Section: Manifest (type 1)
@@ -206,7 +204,7 @@ NodeEntry =
payload byte[payloadLen]
```
-There is **no hash field**. The node is identified solely by its position in the array.
+Each node’s zero-based position in `entries` is its identity.
---
@@ -253,7 +251,7 @@ The t operator is left associative.
3c. t (t a b) c (t u v) -> c u v
```
-**Closure:** The bundle declares `closure = 0` (complete), meaning all nodes reachable from export roots are present in the nodes section. No external references exist.
+**Closure:** The bundle declares `closure = 0` (complete), meaning all nodes reachable from export roots are present in the nodes section.
---
@@ -298,21 +296,19 @@ value = (byte[0] << 56) | ... | byte[7]
7. **Acyclicity:** Guaranteed by the `child < parent` rule above.
8. **Closure:** Traverse from all root/export indices; confirm every reached index is valid.
-No hash computation is required.
-
---
## 11. Canonicalization
A bundle is **canonical** iff:
-1. **Maximal deduplication.** No two entries represent structurally identical subtrees.
+1. **Maximal deduplication.** Each structurally distinct subtree has exactly one entry.
2. **Topological order.** Children precede parents.
3. **Deterministic post-order traversal.** Nodes are emitted in the order discovered by a left-to-right recursive post-order walk.
-4. **No trailing bytes** in any section.
+4. **Exact section lengths.** Each section ends immediately after its encoded content.
5. **Reserved fields are zero.**
-Canonical bundles produce deterministic bytes and can be file-level hashed for global identity.
+Canonical writers produce identical bytes for identical ordered named roots and metadata.
---
@@ -354,54 +350,53 @@ Bundles use the `.arboricx` file extension. Plain source files use `.tri`.
precedence. Reader profiles differ at the container boundary.
- Binary input is a proper list of canonical octets. Prefix reads validate only
- consumed cells; a zero-length read does not inspect its input. Exact reads and
+ consumed cells; a zero-length read returns its input unchanged and uninspected. Exact reads and
expectations roll failures back to the start of that operation. Counted record
parsers propagate the failing field's remainder. `orParser` deliberately
- retries the original input; it is not the consumption-aware parser protocol.
+ retries the original input after failure, including a failure that consumed input.
- Container boundaries validate the complete byte-list shape, header 1.0, zero
- header flags and directory offset 32. Section version 1, flags 0/1, no compression
+ header flags and directory offset 32. Section version 1, flags 0/1, compression value 0
and zero reserved fields are supported. Unknown critical sections, duplicate
IDs, ranges before the directory, out-of-bounds ranges, and overlapping nonempty
sections are rejected. Unknown **noncritical version-1** sections are allowed,
but still receive range/profile checks.
- `readManifest` is a **prefix parser** returning the actual suffix. Its value is
- `pair core (pair metadata extensions)`, retaining both TLV lists instead of
- losing the extensions and remainder. `readManifestComplete` rejects a suffix.
+ `pair core (pair metadata extensions)`, retaining both TLV lists. `readManifestComplete` rejects a suffix.
Manifest version 1.1, UTF-8 core strings/roles/export fields, the supported
semantics/evaluation/ABI, empty capabilities, complete closure, nonempty roots
and unique nonempty export names are enforced at their appropriate parsing or
bundle-validation boundaries. Every export must declare `term` and the tree
ABI. Known metadata text tags require UTF-8; unknown metadata/extension values
- remain opaque. No Unicode normalization or filesystem-name interpretation occurs.
+ remain opaque. Text retains its encoded Unicode form and is treated as manifest data.
- Nodes require exact payload lengths, canonical octets, strictly backward child
indices and unique payloads. All records, advertised roots, and exports are
validated before choosing an executable, including unselected ones. Payload
errors retain precedence over edge/duplicate checks. Reconstruction uses a
- persistent bit-indexed table rather than repeated prefix association searches.
+ persistent bit-indexed table.
- `readArboricxBundle` validates both required sections; executable selection
- reuses the parsed sections instead of reparsing the entire container. Container
- remainders still mean **bytes after the directory**, not bytes after the whole
- file. Random-access containers have no streaming EOF convention here. Trailing
- bytes outside advertised sections are not a second manifest/node-section suffix.
+ reuses the parsed sections. Container remainders are **bytes after the
+ directory**. Sections are accessed by their advertised offsets and lengths;
+ section-level suffix checks apply within those ranges. Bytes outside those
+ ranges are permitted.
Additional numeric error codes: invalid binary input 15, invalid count 16,
unsupported section profile 17, invalid section range 18. Manifest magic/version/
trailing-data errors use 10/11/12; unsupported manifest contracts use 13. Existing
missing/duplicate node errors remain 9/7. `eof?` on nonempty input reports
-unexpected bytes (2), not unexpected EOF (1).
+unexpected bytes (2).
Validation is in memory. Directory overlap and export-name checks are quadratic
in their small lists. Counted parsers and logical-tree observation still need
-caller-selected resource limits; structural validity alone does not bound work.
+caller-selected resource limits, including for structurally valid inputs.
## Host reader profiles
The Haskell `decodeBundle` / `verifyBundle` path validates required sections,
manifest execution semantics and export ABIs, roots, backward edges, and duplicate
-nodes/exports. Its container decoder accepts a broader header/directory profile:
-minor version, header flags, section versions/reserved fields, and general section
-overlap are not all constrained as in the portable reader above. Decode and
+nodes/exports. Its container decoder accepts a broader header/directory profile, with weaker
+constraints on minor version, header flags, section versions/reserved fields, and
+general section overlap than the portable reader above. Decode and
semantic verification are separate API calls; CLI loading uses both.
The browser reader uses strict header/directory checks, contiguous sections,
@@ -411,6 +406,6 @@ and compiler download profiles are documented in the
Canonical writers emit the format described here. Reordered roots/exports or
changed metadata affect file identity; canonicalization is deterministic for an
-ordered named export set and metadata, rather than identifying all bundles with
-extensionally equivalent programs. Distribution authentication and execution
+ordered named export set and metadata. Programs with equal application results
+can have distinct representations and bundle identities. Distribution authentication and execution
permissions are separate from structural parsing.
diff --git a/docs/compiler-rebuilding.md b/docs/compiler-rebuilding.md
@@ -24,8 +24,8 @@ Select these inputs explicitly:
| `SOURCE` | Source checkout to build. |
An accepted-pair recovery transfer provides the completed second generation at
-`recovery/seed` and the adapter at `recovery/tools/io-adapter.arboricx`. Identify
-other inputs from their recorded provenance, rather than directory names.
+`recovery/seed` and the adapter at `recovery/tools/io-adapter.arboricx`.
+Identify other inputs from their recorded provenance.
```sh
python3 test/bootstrap_generations.py "$PREVIOUS" "$NEXT" --tricu "$TRICU" \
@@ -52,10 +52,10 @@ A full `source_map` generation produces:
- `io-adapter.arboricx`: inherited standard adapter.
Use `--backend stable` explicitly for new generations. The runner defaults to
-stable; historical records without a backend mean optimized. Generation selection
-uses `sourceMapBuildStable` or `sourceMapBuild`, while application selection uses
-the chosen application bundle. Neither path silently substitutes another backend
-or the recovery frontend.
+stable; historical records without a backend mean optimized. Generation
+selection uses `sourceMapBuildStable` or `sourceMapBuild`, while application
+selection uses the chosen application bundle. Both paths require the selected
+backend and entrypoint.
### Execution-host selection
@@ -168,9 +168,10 @@ The JSON manifest has schema `tricu.bootstrap-kit.v1`, backend `stable`, and an
Each role also records its lowercase whole-file `sha256`. Select the manifest
through a trusted release or verified artifact preparation. The runner verifies
both files before creating output, captures their bytes, imports them indexed,
-and records the manifest digest alongside the actual executable identities.
-It neither reads a previous store nor falls back to another compiler. A
-`--validate-only` run checks source/map staging inputs without loading executables.
+and records the manifest digest alongside the actual executable identities. The
+kit supplies the complete compiler selection, independently of previous stores.
+A `--validate-only` run checks source/map staging inputs without loading
+executables.
Use generation one's completed output as `PREVIOUS` for generation two with
`--expect-identical`. Kit selection cannot be combined with previous-generation,
@@ -189,13 +190,13 @@ these candidate checks from reference-artifact tests and acceptance.
Several reviewable candidate commits may precede a generation checkpoint. Record
the candidate identities, checks, pending freshness, and acceptance scope in
`STATUS.md`. Keep accepted fixtures and the rollback toolchain unchanged until
-publication. Run full generations at the agreed checkpoint, rather than after
-each rewrite, draft, or repair.
+publication. Use focused checks between drafts and run full generations at the
+agreed checkpoint.
Verification failures block acceptance and further feature work. Diagnose and
-repair with bounded checks. A failed generation gate remains unaccepted until a
-fresh pair passes; cached equality, partial audits, and fixture refresh do not
-replace it.
+repair with bounded checks. A failed generation gate requires a passing fresh
+pair for acceptance, including complete independent audits. Cached equality and
+refreshed fixtures serve only as supporting evidence.
## Mandatory compiler hygiene gate
@@ -241,16 +242,17 @@ An explicitly agreed checkpoint may promote the audited fresh pair after the
smokes and [rolling retention](#rolling-development-retention) while deferring
fixture publication, the broad downstream sweep, and recovery packaging. Record
**verified compiler checkpoint, publication deferred**, with source commit, tool
-identities, coverage, and remaining obligations. Retain
-the previous accepted compiler for rollback. This promotes a development compiler,
-not the entire support cohort or its fixture publication.
+identities, coverage, and remaining obligations. Retain the previous accepted
+compiler for rollback. Promotion covers the development compiler; support-cohort
+acceptance and fixture publication remain pending.
### Publish the accepted batch
When replacing the shipped `bootstrap/build-stable.arboricx`, update its trusted
-whole-file SHA256 in `src/PortableCompiler.hs` (`packagedCompilerSha256`) alongside
-`bootstrap/stable-compiler.json`. The packaged frontend rejects mismatched bytes;
-its receipt identity is this bundle digest, not its indexed transport object hash.
+whole-file SHA256 in `src/PortableCompiler.hs` (`packagedCompilerSha256`)
+alongside `bootstrap/stable-compiler.json`. The packaged frontend rejects
+mismatched bytes; its receipt identity is this bundle digest. Indexed transport
+objects have their own separately recorded hashes.
Publish genuinely rebuilt compiler and support fixtures using the procedures in
[Testing](validation.md.html#compiler-transport-and-publication). Capture matching source
@@ -263,31 +265,32 @@ nix flake check
```
Reference snapshots authenticate the bytes actually used to compile an artifact.
-Changing fingerprints to describe an old executable as newly compiled is not a
-publication procedure. Support-only changes need candidate/publication checks,
-but do not require a new compiler generation merely because they edit `.tri`.
+Publish freshly compiled artifacts with fingerprints of their actual inputs.
+Support-only `.tri` changes require candidate/publication checks; compiler
+generations are required for compiler-affecting changes.
Full compiler publication also includes the browser/site assets. Commit the
accepted compiler/support fixtures first (the site authenticates their full Git
-origin), then complete the [site compiler-promotion refresh](../ext/demo-site/README.md.html#compiler-promotion-refresh)
-and commit its matching assets and validation changes. Updating the native
-compiler does **not** update the site's separately pinned compiler. A deferred
-site refresh is an incomplete full publication, not a completed promotion.
-Finish [rolling retention](#rolling-development-retention) after this refresh;
-retain matching previous site/dependency outputs for rollback. Local preparation
-and validation do not authorize public deployment.
+origin), then complete the [site compiler-promotion
+refresh](../ext/demo-site/README.md.html#compiler-promotion-refresh) and commit
+its matching assets and validation changes. Update the site’s separately pinned
+compiler explicitly. Full publication remains incomplete while the site refresh
+is deferred. Finish [rolling retention](#rolling-development-retention) after
+this refresh; retain matching previous site/dependency outputs for rollback.
+Public deployment requires separate authorization after local preparation and
+validation.
## Inputs, restart, and resource controls
Full jobs use eight GHC capabilities, at most 64 GiB, and a two-hour whole-job
-ceiling. Select `--capabilities 8 --memory-mib 65536 --wall-seconds 7200` explicitly;
-CLI defaults are smaller. Use lower bounds for focused probes. More capabilities
-do not guarantee faster compilation.
+ceiling. Select `--capabilities 8 --memory-mib 65536 --wall-seconds 7200`
+explicitly; CLI defaults are smaller. Use lower bounds for focused probes.
+Measure the effect of capability counts on the selected workload.
-On an interactive workstation, run heavy jobs sequentially with low priority.
-If selecting CPU affinity, choose available CPUs with a caller-provided `CPU_SET`.
-Nix daemon builders do not inherit caller affinity; use Nix's job/core settings
-or separately configured builder limits when necessary.
+On an interactive workstation, run heavy jobs sequentially with low priority. If
+selecting CPU affinity, choose available CPUs with a caller-provided `CPU_SET`.
+Configure Nix daemon builders separately with Nix’s job/core settings or builder
+limits; caller affinity applies to the caller’s processes.
| Option | Meaning |
| --- | --- |
@@ -298,26 +301,28 @@ or separately configured builder limits when necessary.
| `--initial-seconds`, `--extended-seconds` | No-publication boundaries. |
| `--wall-seconds` | Whole-job budget including preparation, audits, and packing; default 3600, maximum 7200. |
-Native transport commands also have a 600-second timeout. Direct-child RSS is not
-an aggregate process-group memory limit, and sampled peaks can miss short-lived
-allocations. Keep the enclosing process alive beyond the configured guards.
+Native transport commands also have a 600-second timeout. RSS sampling covers
+the direct child; descendants require separate aggregate limits. Sampled peaks
+can miss short-lived allocations. Keep the enclosing process alive beyond the
+configured guards.
-A `native` map row preserves source bytes and declared import/export policy.
-Its ordered dependencies must match local imports, re-exports, and build-only
-edges, deduplicated in first-occurrence order. Legacy `unit` and `reexport` rows
-translate checked headers. Compiler dependency names are map aliases; use a staged
-workspace or immutable manifests rather than assuming directory search.
+A `native` map row preserves source bytes and declared import/export policy. Its
+ordered dependencies must match local imports, re-exports, and build-only edges,
+deduplicated in first-occurrence order. Legacy `unit` and `reexport` rows
+translate checked headers. Compiler dependency names are map aliases; supply
+them through a staged workspace or immutable manifests.
Sources must remain unchanged throughout a generation. The runner checks them
-again before accepting outputs, but does not create an adversarial filesystem
-snapshot. `inputs.json`, `source-map.v1`, `invocation.json`, logs, and `status.json`
-record input identities, limits, comparisons, and completion.
+again before accepting outputs. Snapshot consistency relies on trusted, stable
+filesystem contents throughout the job. `inputs.json`, `source-map.v1`,
+`invocation.json`, logs, and `status.json` record input identities, limits,
+comparisons, and completion.
Rerunning a diagnostic output directory reuses verified checkpoints. Changed
-source/compiler identities invalidate keys, and an output lock prevents concurrent
-writers. Interrupted builds remain incomplete. Resumption helps diagnosis and
-recovery; the acceptance pair requires fresh stores. The map-builder has completed-
-module checkpoints, not per-export checkpoints.
+source/compiler identities invalidate keys, and an output lock prevents
+concurrent writers. Interrupted builds remain incomplete. Resumption helps
+diagnosis and recovery; the acceptance pair requires fresh stores. The
+map-builder checkpoints completed modules as whole units.
## Transitioning older inputs
@@ -337,16 +342,17 @@ python3 test/bootstrap_generations.py "$PREVIOUS" "$NEXT" --tricu "$TRICU" \
Worker preparation selects only the requested backend and its reachable local
definitions. It audits every transported support export. Inspect `--plan-only`
before preparing a retained flat compiler; its source compiler export can exceed
-the normal 64 MiB flat limit. Select an explicit `--support-flat-bytes BYTES` for
-that verified support transport. This does not change normal import defaults.
-Worker compilation has a 900-second guard within the selected whole-job limit.
+the normal 64 MiB flat limit. Select an explicit `--support-flat-bytes BYTES`
+for that verified support transport. Ordinary flat imports retain their default
+64 MiB limit. Worker compilation has a 900-second guard within the selected
+whole-job limit.
For a pre-consolidation header parser, explicitly add `--legacy-compiler-header`
-to worker preparation. It changes the bridge wrapper's header, not the native
-language or failed-compilation behavior. Subsequent generations use the rebuilt
-closure directly. Historical custom four-argument adapters are not substitutes
-for the standard adapter. Test wrappers also use retained portable compilation,
-not a host source frontend.
+to worker preparation. It changes only the bridge wrapper’s header; native
+language semantics and failed-compilation behavior remain unchanged. Subsequent
+generations use the rebuilt closure directly. Both routes require the standard
+six-argument adapter. Test wrappers also compile through retained portable
+tools.
## Rolling development retention
@@ -366,7 +372,7 @@ Before pruning an older store:
without the old stores, within the selected audit budget.
4. Preserve compact sources, bundles, receipts/manifests, and reports. Record the
exact removed object directories and mark their parents incomplete. Unlink
- obsolete paths; do not edit shared hardlinked objects.
+ obsolete paths, preserving the bytes of shared hardlinked objects.
5. Re-audit retained stores and smoke-test both compilers after deletion.
Check for sibling-worktree users before pruning. Removing generation one means
@@ -376,20 +382,21 @@ scratch cleanup.
### Checkpoint cleanup command
-Pair acceptance and smokes do not finish the operational checkpoint. Run
-`test/bootstrap_retention.py` after them, before declaring completion or starting
-the next feature batch. This is an opt-in Linux/Haskell workflow requiring Python,
-Nix and `bwrap`; it never builds another full compiler generation or runs Nix GC.
-An existing verified checkpoint can satisfy its outstanding retention obligation
-without building a replacement pair.
+Checkpoint completion requires pair acceptance, smokes, and rolling retention.
+Run `test/bootstrap_retention.py` after acceptance and smokes, before declaring
+completion or starting the next feature batch. This opt-in Linux/Haskell
+workflow requires Python, Nix and `bwrap`. It validates retained toolchains and
+prunes selected scratch objects; full compiler generation and Nix GC are
+separate operations. An existing verified checkpoint can satisfy its outstanding
+retention obligation without building a replacement pair.
Create an explicit local retention registry, for example
-`$SCRATCH/retained-toolchains.json`. Replace the example paths and runtime hashes
-with recorded provenance; do not select compilers by directory age. `current` and
-`rollback` are required unique roles. Additional uniquely named toolchains or
-protected scratch directories are allowed for active consumers and release evidence.
-Prefer an independently useful prior compiler revision for rollback: two builds
-with identical executable bundles do not provide an executable revision rollback.
+`$SCRATCH/retained-toolchains.json`. Replace the example paths and runtime
+hashes with recorded provenance as the basis for compiler selection. `current`
+and `rollback` are required unique roles. Additional uniquely named toolchains
+or protected scratch directories are allowed for active consumers and release
+evidence. Prefer an independently useful prior compiler revision for executable
+rollback. Two builds with identical bundles retain the same executable revision.
```json
{
@@ -422,20 +429,20 @@ with identical executable bundles do not provide an executable revision rollback
```
All scratch paths are normalized, root-relative, and protected by explicit real
-**directories**. The current command reruns full-pair acceptance, so both retained
-first generations must also be protected and present. It does not automate their
-retirement. Acceptance must select the actual retained second-generation path;
+**directories**. The current command reruns full-pair acceptance, so both
+retained first generations must also be protected and present throughout
+retention. Acceptance must select the actual retained second-generation path;
for a relocated pair, create relocated evidence with `bootstrap_acceptance.py`
-and its explicit source override rather than rewriting old acceptance records.
-The selected standard adapters remain inside the protected generations.
+and its explicit source override. Preserve the original acceptance records. The
+selected standard adapters remain inside the protected generations.
The support preparation must contain the complete selected candidate cohort in
-one directory. The command reproduces current support bundles, metadata, receipts
-and full native manifests through the existing publisher, then checks source
-freshness. It does not publish or edit fixtures. `relocate_patterns` selects the
-current V3 publication procedure. Protect any other preparation or store used by
-active work before planning. The two confirmation fields are operator obligations,
-not automatic discovery of every external user.
+one directory. The command reproduces current support bundles, metadata,
+receipts and full native manifests through the existing publisher, then checks
+source freshness. Published fixtures remain unchanged. `relocate_patterns`
+selects the current V3 publication procedure. Protect any other preparation or
+store used by active work before planning. The operator must check external
+users and stop uncoordinated jobs before setting the two confirmation fields.
```sh
SCRATCH="$PWD/.bootstrap-run"
@@ -453,20 +460,22 @@ python3 test/bootstrap_retention.py check --root "$SCRATCH" \
--directory "$CHECKPOINT"
```
-Planning registers indirect Nix GC roots for each exact accepted Haskell runtime.
-It inventories allocated blocks by inode, excludes hardlinks still referenced by
-retained or external files from reclaim estimates, and preserves compact
-header-bearing CAS documents with whole-file hashes before any deletion.
-Only selected `objects/` directories inside this `.bootstrap-run` are candidates.
-Sources, portable bundles, original reports and archives outside that scope are
-not deleted. Each removed CAS gets a `PRUNED-CAS.json` marker; old completion
-records become historical evidence, not a resumable or complete store.
+Planning registers indirect Nix GC roots for each exact accepted Haskell
+runtime. It inventories allocated blocks by inode, excludes hardlinks still
+referenced by retained or external files from reclaim estimates, and preserves
+compact header-bearing CAS documents with whole-file hashes before any deletion.
+Only selected `objects/` directories inside this `.bootstrap-run` are
+candidates. Sources, portable bundles, original reports and archives outside
+that scope are preserved. Each removed CAS gets a `PRUNED-CAS.json` marker; its
+old completion records serve as historical evidence, and its store is marked
+incomplete.
Application holds an exclusive scratch-root lock. Generation runners take shared
locks on previous/output scratch roots, so they refuse concurrent cleanup.
Existing generation output locks are checked too. Stop other tools and external
-users separately: arbitrary host invocations do not participate in this lock.
-The command assumes trusted local filesystems, not adversarial writers or mounts.
+users separately: lock coordination covers the generation runners and retention
+command. Use trusted local filesystems and keep adversarial writers or mounts
+outside the workflow.
Before deletion, `bwrap` hides the entire scratch root, exposing only registered
protected directories and the writable checkpoint evidence directory. Each
@@ -479,13 +488,13 @@ gate succeed. Verification logs, reports, registry and harness snapshots remain
beside the plan. Keep those evidence directories and their runtime roots.
Deletion is journaled and resumable. A failed precheck deletes nothing. A failed
-postcheck leaves retention incomplete even if unlinking finished. For resumption,
-review the updated plan and approve its **current** SHA256; changed protected inputs,
-candidates, compact evidence or deletion harness block resumption. The completion
-checker can read archived harness identities after tooling evolves, but an in-flight
-delete must use its reviewed harness. Record blockers and outstanding bytes rather
-than calling an incomplete checkpoint finished. Never wait until disk pressure to
-perform the next rotation.
+postcheck leaves retention incomplete even if unlinking finished. For
+resumption, review the updated plan and approve its **current** SHA256; changed
+protected inputs, candidates, compact evidence or deletion harness block
+resumption. The completion checker can read archived harness identities after
+tooling evolves, but an in-flight delete must use its reviewed harness. Report
+an incomplete checkpoint with its blockers and outstanding bytes. Perform
+rotations at each checkpoint, while sufficient disk space remains.
Safety controls run in the fast Nix check without compiling tricu programs:
@@ -527,14 +536,16 @@ checks. Follow the transfer's own README when restoring. Verify
`TRANSFER-SHA256SUMS` before extraction and internal `SHA256SUMS` afterward.
Existing transfers remain immutable when documentation changes.
-Archive creation preserves existing evidence; it runs no additional compiler
-generation. Same-host runtime reimport does not establish empty-Nix-store or
-cross-platform recovery. The runtime closure is not an offline Haskell build
-closure. Archived SCP is not an execution or recovery prerequisite.
+Archive creation packages and validates existing accepted evidence. Runtime
+reimport is checked on the same host; empty-Nix-store and cross-platform
+recovery require separate validation. The retained Nix closure supports running
+the host; an offline Haskell rebuild requires its build dependencies as well.
+Archived SCP is retained solely as historical material.
Copy the whole transfer directory, including checksum and validation sidecars,
to independent off-machine storage, and verify restoration before relying on it.
-A local archive alone is not a backup. Retain the original recovery transfers.
+Maintain an independently stored backup and retain the original recovery
+transfers.
## Focused validation
@@ -547,8 +558,10 @@ python3 test/verified_tree_loading.py "$TRICU" "$PREVIOUS" \
```
Use new writable diagnostic directories. The CLI probe checks cold/warm/change,
-interruption, corruption, backend selection, and receipt options. An older closure
-can use an explicit `--worker-bundle`. The transport probe compiles an adapter
-through the selected portable toolchain and checks unit exports and loading.
-Its diagnostic checkpoints do not constitute an ordinary completed generation.
-See [Storage](storage.md.html#verified-native-tree-transport) for the transport contract.
+interruption, corruption, backend selection, and receipt options. An older
+closure can use an explicit `--worker-bundle`. The transport probe compiles an
+adapter through the selected portable toolchain and checks unit exports and
+loading. Its checkpoints support diagnosis; ordinary generation completion
+requires the generation runner and acceptance workflow above. See
+[Storage](storage.md.html#verified-native-tree-transport) for the transport
+contract.
diff --git a/docs/modules.md b/docs/modules.md
@@ -31,12 +31,13 @@ math<TAB>libraries/math.tri
prelude<TAB>libraries/prelude.tri
```
-Then `apps/main.tri` can import `math` and `prelude` without copying files beside
-it. Paths are relative to the declaration, must end in `.tri`, and can include
-`..` for explicitly shared sources. LF endings and a final newline are required;
-comments and blank lines are not supported. Declarations are limited to 64 KiB;
-names to 128 ASCII characters. All mapped files must exist. Direct symlinks and
-nonregular input files are rejected; use stable, trusted directories.
+Then `apps/main.tri` can import `math` and `prelude` without copying files
+beside it. Paths are relative to the declaration, must end in `.tri`, and can
+include `..` for explicitly shared sources. LF endings and a final newline are
+required; every line must contain the header or a mapping row. Declarations are
+limited to 64 KiB; names to 128 ASCII characters. All mapped files must exist.
+Direct symlinks and nonregular input files are rejected; use stable, trusted
+directories.
The repository declaration provides its native library locations, including
`native_math` and `arboricx_server`. Compile the application directly:
@@ -48,24 +49,25 @@ result/bin/tricu arboricx compile --file tricu-apps/arboricxServer.tri \
```
Sibling modules remain available. A sibling and mapping with the same name must
-refer to the same file; conflicting and case-colliding names fail rather than
-silently shadowing one another. Parent declarations are not merged. A nearer
+refer to the same file; conflicting and case-colliding names fail. The nearest
+declaration supplies the complete mapping. A nearer
header-only `tricu.workspace` establishes an isolated flat workspace; the
self-contained module demos use this boundary.
The host copies the selected source locations into a temporary flat workspace
-and removes it after compilation, including on failure. Source text is unchanged;
-the portable compiler still reads headers, discovers the reached dependency
-closure, rejects cycles, and owns compilation and cache semantics. Only reached
-modules are compiled, though all mapped and validly named sibling files are
-copied. This is not registry fetching, recursive library discovery, source
-rewriting, or a new compiler backend. Keep sources stable during the copy/build.
-Full-hash imports and package-generated binding modules work as before.
+and removes it after compilation, including on failure. Source text is
+unchanged; the portable compiler still reads headers, discovers the reached
+dependency closure, rejects cycles, and owns compilation and cache semantics.
+Only reached modules are compiled, though all mapped and validly named sibling
+files are copied. Source discovery is limited to the declared mappings and
+siblings; registry fetching is a separate preparation step. Keep sources stable
+during the copy/build. Full-hash imports and package-generated binding modules
+work as before.
`eval`, `arboricx compile`, and provenance capture share these location rules.
-Capture retains reached source bytes under their module names; replay needs no
-original directory layout or workspace declaration. Workspace paths themselves
-are not executable or receipt identities. Use [Build and run](build-and-run.md.html)
+Capture retains reached source bytes under their module names; replay uses that
+flat layout. Executable and receipt identities depend on content, independently
+of the workspace paths. Use [Build and run](build-and-run.md.html)
to select the stable compiler.
## Headers
@@ -93,14 +95,15 @@ The workspace explicitly supplies every dependency, including the prelude.
Repeated imports under different qualifiers share one dependency build. Local
edges are deduplicated in first header-occurrence order. Duplicate build-only
edges, redundant build-only/import edges, name collisions, missing dependencies,
-and cycles are errors. Bodies contain definitions rather than a final expression.
+and cycles are errors. Bodies contain definitions; expose the entrypoint through
+an export.
## Resolution and identity
Workspace names resolve through the selected workspace. A source map assigns
paths, aliases, frontends, and ordered dependencies for a larger build. Full
hashes select immutable manifests. Graph builds resolve workspace names through
-that map rather than mutable store aliases.
+that map. Store aliases are managed separately.
Imports hydrate and verify the complete public interface. This includes unused
exports: namespace validation and cache keys depend on the interface as a whole.
@@ -143,10 +146,11 @@ supports CLI artifact operations; source compilation uses the portable compiler.
## Exact-version packages from trusted registries
The initial package profile is **one compiled native module per release**.
-Versions are exact opaque labels, not SemVer ranges. Names and versions contain
-1–128 ASCII letters, digits, dots, underscores or hyphens, starting with a letter
-or digit. Registry policy lives in the Haskell CLI; the native server serves the
-index and bundles, and native compilation continues to consume local CAS objects.
+Versions are matched as exact opaque labels. Names and versions contain 1–128
+ASCII letters, digits, dots, underscores or hyphens, starting with a letter or
+digit. Registry policy lives in the Haskell CLI; the native server serves the
+index and bundles, and native compilation continues to consume local CAS
+objects.
### Publish
@@ -163,15 +167,15 @@ result/bin/tricu package publish --root "$REGISTRY_ROOT" \
Publication validates the bundle and its native interface, writes the bundle
before the index record, and refuses to replace an existing name/version with
different bytes. Identical retries are allowed, including concurrent publishers.
-Failure may leave an unreferenced bundle. This is a local administrator operation,
-not an unauthenticated network publishing endpoint.
+Failure may leave an unreferenced bundle. Publication requires local
+administrator access to the registry store.
-The current `lib/arboricx/server.tri` serves
-`GET /_arboricx/package/NAME/VERSION` from that same store. Compile the updated
-server source before using this route; historical server bundles do not gain it
-automatically. Use an HTTPS reverse proxy for non-loopback service. A read-only
-registry deployment should restrict public methods/routes and grant the server
-only read access; the older bundle-upload route still exists.
+The current `lib/arboricx/server.tri` serves `GET
+/_arboricx/package/NAME/VERSION` from that same store. Compile the updated
+server source and deploy the resulting bundle before using this route. Use an
+HTTPS reverse proxy for non-loopback service. A read-only registry deployment
+should restrict public methods/routes and grant the server only read access; the
+older bundle-upload route still exists.
### Resolve, fetch, bind
@@ -191,43 +195,47 @@ result/bin/tricu package bind --lock "$WORK/library.lock" \
Configuration rows are tab-separated registry name and origin URL. Names must be
unique. URLs are HTTPS origins without trailing slash, path, credentials, query
or fragment; HTTP is allowed only for `localhost`, `127.0.0.1` and `[::1]`.
-There is no registry fallback or automatic trust discovery. Redirects are rejected.
+Each request uses the explicitly configured registry. Redirects are rejected.
-`resolve` alone consults the index. It records registry name/origin, exact version,
-module-manifest hash and whole-bundle hash. Changing a selection requires
-`--update`; a changed mapping for the same locked registry/origin/name/version
-is rejected even with that flag. This detects changes relative to the current
-lock, not every historical registry claim.
+`resolve` alone consults the index. It records registry name/origin, exact
+version, module-manifest hash and whole-bundle hash. Changing a selection
+requires `--update`; a changed mapping for the same locked
+registry/origin/name/version is rejected even with that flag. This detects
+changes relative to the current lock. Retain older locks separately to track
+historical registry claims.
-`fetch` uses only the lock's bundle pin, verifies the download and its advertised
-module identity before importing into the application CAS, and creates no mutable
-module aliases. Its configured origin must still match the lock. A complete,
-verified local module needs no network request. Corrupt or incomplete objects
-under an existing manifest fail rather than silently repairing them.
+`fetch` uses only the lock's bundle pin, verifies the download and its
+advertised module identity before importing into the application CAS, while
+leaving mutable module aliases unchanged. Its configured origin must still match
+the lock. A complete, verified local module is reused offline. Corrupt or
+incomplete objects under an existing manifest fail and require explicit repair.
`bind` is offline. It verifies the local module and writes a normal workspace
module containing `!export * from HASH`. An application beside that file can use
`!import library as L` and call `L.someExport`. Use the same application store
-for compilation. Commit the lock and generated binding; after an explicit update,
-fetch and regenerate the binding. The compiler reads the binding, **not the
-package lock**; it never resolves names against the registry or updates locks.
-For several dependencies, use the project declaration and consolidated lock below
-instead of assembling separate package lock files.
-
-Trusting a registry accepts its name/version assertions. It neither proves build
-provenance nor grants downloaded programs IO permissions. TLS uses the host client
-trust configuration. Metadata downloads are bounded to 4 KiB, bundles to 64 MiB,
-and requests to a 60-second overall deadline (30-second response timeout). These
-are transport limits, not fixed heap bounds or an execution sandbox. The native
-server still reads/encodes complete files. Remote credentials, signatures, version
-solving, multi-module package records and provenance are outside this profile.
+for compilation. Commit the lock and generated binding; after an explicit
+update, fetch and regenerate the binding. The compiler reads the **source
+binding** and local CAS. Registry resolution and lock updates belong to the
+package commands. For several dependencies, use the project declaration and
+consolidated lock below to manage them in one lock.
+
+Trusting a registry accepts its name/version assertions. Establish build
+provenance separately and grant program IO permissions explicitly. TLS uses the
+host client trust configuration. Metadata downloads are bounded to 4 KiB,
+bundles to 64 MiB, and requests to a 60-second overall deadline (30-second
+response timeout). These limits bound transport. Set heap limits and execution
+isolation separately; the native server reads/encodes complete files. This
+profile supports explicitly trusted public-read registries and exact one-module
+releases. Credential handling, signatures, version solving, multi-module
+records, and provenance require a separate package profile.
## Project dependencies
-`tricu project` prepares a whole dependency set while keeping registry resolution,
-fetching, workspace binding and compilation separate. It does not add a version
-solver, implicit imports, transitive source dependencies, or network access to the
-compiler. Local source modules still use the ordinary workspace rules above.
+`tricu project` prepares a whole dependency set while keeping registry
+resolution, fetching, workspace binding and compilation separate. Declare each
+dependency and exact version explicitly, fetch it, and import its generated
+source binding. Compilation uses local inputs under the ordinary workspace rules
+above.
A `tricu.project` file declares **local binding, registry label, package name,
exact version**, separated by tabs, with LF endings and a final newline:
@@ -239,15 +247,15 @@ fileio<TAB>home<TAB>tricu-file-io<TAB>0.1.0
inspection<TAB>home<TAB>arboricx-inspection<TAB>0.1.0
```
-`<TAB>` above denotes one actual tab. The checked-in
-[`inspector declaration`](../demos/package-inspector/tricu.project) is ready to
-copy. Declaration order does not affect selection; generated locks are sorted
-by binding name. Bindings use the flat workspace name grammar above, at most 128
+`<TAB>` above denotes one actual tab. The checked-in [`inspector
+declaration`](../demos/package-inspector/tricu.project) is ready to copy.
+Selection is independent of declaration order; generated locks are sorted by
+binding name. Bindings use the flat workspace name grammar above, at most 128
ASCII characters, excluding names that would parse as full manifest hashes.
Duplicate and case-colliding names are rejected, even on case-sensitive systems.
-Project metadata is bounded to 64 KiB and 128 direct dependencies. Empty projects
-are valid. Versions are opaque exact labels: even `1.x` would be a literal label,
-not a range. Comments and blank rows are not part of this metadata format.
+Project metadata is bounded to 64 KiB and 128 direct dependencies. Empty
+projects are valid. Versions are opaque exact labels: `1.x` selects the literal
+version `1.x`. Each row must contain the header or a dependency declaration.
With an explicitly trusted registry configuration whose `home` entry serves those
releases, prepare a fresh workspace (the site releases must be deployed or served
@@ -268,11 +276,12 @@ result/bin/tricu arboricx compile --file "$WORK/inspector/main.tri" \
```
From within that workspace, omit `--project`: it defaults to `tricu.project`.
-The lock defaults to `tricu.lock` beside the declaration; `--lock FILE` overrides
-it. Bind/check default to that same directory; `--directory WORKSPACE` overrides
-where generated modules live. They do not copy authored sources. Explicit paths
-are relative to the caller's working directory. Local or global `--store` selects
-the application CAS, just as with the other CLI operations.
+The lock defaults to `tricu.lock` beside the declaration; `--lock FILE`
+overrides it. Bind/check default to that same directory; `--directory WORKSPACE`
+overrides where generated modules live. Authored sources stay in their existing
+locations. Explicit paths are relative to the caller's working directory. Local
+or global `--store` selects the application CAS, just as with the other CLI
+operations.
### Frozen resolution and deliberate updates
@@ -286,18 +295,18 @@ the application CAS, just as with the other CLI operations.
the option for several, or use **`--all`** explicitly. Edit the declaration first
to request a new version, add a dependency, or remove one. Unselected bindings
must match the declaration and configured origins exactly; otherwise the command
- fails before any GET. A removal needs no registry request. A rename is a removal
+ fails before any GET. Removals are handled locally. A rename is a removal
plus an addition: select both names, or use `--all`. For a case-only rename,
remove the old dependency and bind before adding the new spelling.
- Every successful update prints before/after selections, full module and bundle
- hashes, and whether each identity changed. A label change need not change code;
- a changed module or bundle does not establish behavioral differences.
+ hashes, and whether each identity changed. Labels, artifact identities, and
+ observed behavior can vary independently; assess behavioral changes separately.
- All resolutions and validation must succeed before one atomic lock publication;
their failure leaves the previous lock intact. A post-publication filesystem
sync failure may leave the new complete lock visible: retry and verify, as in
- the publication rules above. No executable objects are imported at this stage. A changed assertion for an origin/name/version in the **current** lock is
- rejected, even across binding or registry-label renames. There is no historical
- claim database; preserve older locks in version control.
+ the publication rules above. Executable import occurs during `fetch`. A changed assertion for an origin/name/version in the **current** lock is
+ rejected, even across binding or registry-label renames. Historical
+ claims can be tracked by preserving older locks in version control.
For example, after changing only the foundation version in the declaration:
@@ -312,47 +321,52 @@ result/bin/tricu project check --project "$PROJECT" --store "$APP_STORE"
### Fetching, binding and recovery
`fetch` first checks the whole declaration/lock and every configured origin. It
-uses bundle hashes only, never the release index. Verified complete local modules
-need no download. Transport and executable verification use the same package
-controls above; an interrupted fetch may leave valid immutable objects available
-for retry. Corrupt/incomplete existing modules fail rather than being repaired
-silently. This verifies local executable availability, not possession of the
-original distribution bundle's bytes.
+fetches by locked bundle hashes. Verified complete local modules are reused
+offline. Transport and executable verification use the same package controls
+above; an interrupted fetch may leave valid immutable objects available for
+retry. Corrupt/incomplete existing modules fail and require explicit repair.
+This verifies local executable availability. Retaining the original distribution
+bundle’s bytes is a separate responsibility.
`bind` verifies **all** required local interfaces and preflights **all** output
paths before changing bindings. It writes normal `!export * from HASH` modules.
-A derived `.arboricx-project-bindings` ledger tracks which files it owns, allowing
-safe updates and removal of obsolete bindings. It never adopts/overwrites an
-unmanaged file, even one with matching bytes. Edited owned files, direct symlinks,
-directories and input/output collisions are errors. Unrelated authored files are
-left alone. To migrate manually generated bindings, explicitly remove those
-files after preserving any edits, then bind into the clean paths.
-
-Multi-file binding is **not atomic**. A `.arboricx-project-bindings.pending`
-journal is published before any changes. After interruption, rerun `bind` with
-the same project/lock; it accepts files from the prior or pending generation but
-refuses user edits. Finish recovery before another dependency update. If the lock
-has already changed, restore the interrupted generation's declaration/lock first.
-Use exclusively owned, trusted workspaces and keep their contents stable throughout
-bind/check/compile; concurrent updates or hostile directory mutation are unsupported.
+A derived `.arboricx-project-bindings` ledger tracks which files it owns,
+allowing safe updates and removal of obsolete bindings. Output paths must be
+fresh or already owned by the ledger; unmanaged files cause rejection even when
+their bytes match. Edited owned files, direct symlinks, directories and
+input/output collisions are errors. Unrelated authored files are left alone. To
+migrate manually generated bindings, explicitly remove those files after
+preserving any edits, then bind into the clean paths.
+
+Multi-file binding uses **journaled, per-file updates**. A
+`.arboricx-project-bindings.pending` journal is published before any changes;
+interruption can leave a mixture of prior and pending files. After interruption,
+rerun `bind` with the same project/lock; it accepts files from the prior or
+pending generation but refuses user edits. Finish recovery before another
+dependency update. If the lock has already changed, restore the interrupted
+generation's declaration/lock first. Use exclusively owned, trusted workspaces
+and keep their contents stable throughout bind/check/compile; concurrent updates
+or hostile directory mutation are unsupported.
`check` is read-only and offline. It rejects stale declarations, mismatched or
edited bindings, pending transactions, missing modules and corrupt objects. The
-compiler still reads ordinary source bindings, **not** the project lock: run check
-before compilation when lock/binding agreement matters. It is not a source
-inventory, a compiler pin, a build receipt or an execution sandbox.
+compiler reads ordinary source bindings: run check before compilation when
+lock/binding agreement matters. The check covers dependency selections and
+bindings. Track source inventories, compiler pins, and build receipts
+separately, and configure execution isolation at the host boundary.
Commit the declaration and lock. Either regenerate bindings in fresh workspaces,
-or commit generated modules **and their ownership ledger together**. Do not commit
-an unfinished journal. The one consolidated lock stores registry labels/origins,
-exact releases, module hashes and bundle hashes; the ledger is merely derived
-local file-ownership state, not another dependency resolution mechanism.
+or commit generated modules **and their ownership ledger together**. Complete
+journal recovery before committing. The one consolidated lock stores registry
+labels/origins, exact releases, module hashes and bundle hashes; the ledger
+records derived local file ownership.
### Explain selections and compare locks
-These commands are **offline and read-only**. They do not initialize a store,
-fetch missing objects, consult registry configuration, regenerate bindings, write
-comparison caches, or execute exports:
+These commands are **offline and read-only**. They inspect the supplied project
+metadata and existing local store as data. Store contents, bindings, and caches
+remain unchanged; missing evidence is reported as unavailable. Registry
+resolution, fetching, and export execution are separate operations:
```sh
result/bin/tricu project explain --project "$PROJECT" --store "$APP_STORE"
@@ -362,70 +376,73 @@ result/bin/tricu project diff --before initial.lock --after current.lock --store
`explain` uses the same declaration/lock/directory defaults as bind/check. It
shows declared and locked selections, exact registry origins, full module and
bundle pins, binding-file agreement, ownership/journal status, and verified or
-unavailable local interfaces/exports. A stale declaration or generated file does
-not hide the other evidence. A missing or malformed declaration/lock is an input
-error, not an implicit resolve request. It reports the lock's registry claim, not
-current registry trust, publisher authentication or build provenance.
+unavailable local interfaces/exports. Other evidence remains visible even when a
+declaration or generated file is stale. A missing or malformed declaration/lock
+produces an input error. Registry claims come from the lock; current registry
+trust, publisher authentication, and build provenance require independent
+checks.
`diff` needs only two consolidated locks and a local store containing evidence
-for either/both. It separates selection additions/removals/changes, export and ABI
-changes, object identities, and **ordered tree structure** for common exports.
-Unchanged selections are included too. Comparison of unavailable evidence is
-**unknown**, never unchanged merely because the locks contain equal hashes.
-Every relevant local object is authenticated and decoded, including unchanged
-pins. An absent dependency on one side is distinct from a selected module whose
-objects are missing.
+for either/both. It separates selection additions/removals/changes, export and
+ABI changes, object identities, and **ordered tree structure** for common
+exports. Unchanged selections are included too. Comparison of unavailable
+evidence is **unknown**, including when the locks contain equal hashes. Every
+relevant local object is authenticated and decoded, including unchanged pins. An
+absent dependency on one side is distinct from a selected module whose objects
+are missing.
Structural comparison operates on validated indexed DAG records. Bottom-up
-interning of constructor/child-ID tuples in a shared table gives exact structural
-equality without probabilistic subtree hashes, recursive expanded-tree equality,
-reconstruction or execution. It ignores physical node numbering, unreachable
-records and irrelevant packaging metadata, while retaining constructor and child
-order. Work for an export pair scales with stored records (with logarithmic map
-operations), not the number of logical occurrences in the unfolded tree. Normal native package import already canonicalizes exports into name-neutral
-indexed objects; the inspector acceptance case therefore also has unchanged
-common-export object hashes. The comparison does not assume that every readable
-indexed encoding is canonical. It does
-**not** normalize programs or establish behavioral equivalence. Distinct trees
-can have identical observed results; tricu can also observe their structure.
+interning of constructor/child-ID tuples in a shared table gives exact
+structural equality directly over stored records. It ignores physical node
+numbering, unreachable records and irrelevant packaging metadata, while
+retaining constructor and child order. Work for an export pair scales with
+stored records (with logarithmic map operations), independently of the number of
+logical occurrences in the unfolded tree. Normal native package import already
+canonicalizes exports into name-neutral indexed objects; the inspector
+acceptance case therefore also has unchanged common-export object hashes. The
+comparison accepts readable noncanonical indexed encodings and compares their
+tree structure directly. Normalization and behavioral equivalence require
+separate analysis. Distinct trees can have identical observed results; tricu can
+also observe their structure.
The first report profile supports closed nonempty native package modules with
-indexed-tree exports and `arboricx.abi.tree.v1`. Unknown kinds/ABIs are explicitly
-unsupported; available manifest interfaces can still be compared. Corrupt bytes,
-malformed graphs, missing files, nonregular CAS paths and unsupported profiles do
-not become successful evidence. Display fields percent-escape non-ASCII/control
-bytes and `%`, so package metadata cannot insert terminal control sequences.
+indexed-tree exports and `arboricx.abi.tree.v1`. Unknown kinds/ABIs are
+explicitly unsupported; available manifest interfaces can still be compared.
+Corrupt bytes, malformed graphs, missing files, nonregular CAS paths and
+unsupported profiles are reported as invalid or unavailable evidence. Display
+fields percent-escape non-ASCII/control bytes and `%`, so package metadata
+cannot insert terminal control sequences.
Report commands return **0** when all requested evidence is available/verified
-(and explain's declaration/bindings/ownership state agree), even when a valid diff
-finds changes. They return **1** for incomplete/invalid/unsupported evidence or
-stale project state, printing the available report to stdout. Invalid input files
-produce a diagnostic instead. Human-readable report wording is not a versioned
-machine protocol.
-
-Reads are bounded to 64 KiB per module manifest and 64 MiB of retained authenticated
-executable bytes per module. Repeated object references within a module share
-inspection results; report text is materialized per dependency before proceeding
-to the next. A budget-limited export is reported unavailable. These encoded-byte
-limits are **not heap/RSS guarantees**: parsed graphs and comparison tables take
-additional space. Use stable, trusted stores and external time/heap controls for
-large inputs; this is not a hostile-filesystem sandbox. Original distribution
-bundle bytes are not required or authenticated by these commands—bundle pins are
-reported as locked claims, while executable evidence comes from verified CAS
-modules and objects.
+(and explain's declaration/bindings/ownership state agree), even when a valid
+diff finds changes. They return **1** for incomplete/invalid/unsupported
+evidence or stale project state, printing the available report to stdout.
+Invalid input files produce a diagnostic instead. Report wording is intended for
+human readers and may change between releases.
+
+Reads are bounded to 64 KiB per module manifest and 64 MiB of retained
+authenticated executable bytes per module. Repeated object references within a
+module share inspection results; report text is materialized per dependency
+before proceeding to the next. A budget-limited export is reported unavailable.
+These limits cover encoded bytes; parsed graphs and comparison tables take
+additional heap space. Use stable, trusted stores and external time/heap
+controls for large inputs. Filesystem safety relies on that trust and stability.
+Bundle pins are reported as locked claims; authenticated executable evidence
+comes exclusively from CAS modules and objects. Authenticate original
+distribution bundles separately when their bytes are needed.
The inspector acceptance example compares its retained `initial.lock` with the
updated `consumer/tricu.lock`: foundation gained `revision`; its three common
exports remained structurally identical; fileio and inspection stayed pinned.
The acceptance harness separately records whether the rebuilt inspector itself
-remained byte-identical. A dependency-lock diff alone cannot infer the identity
-of a downstream build.
+remained byte-identical. Determining a downstream build’s identity requires
+inspecting its own artifacts.
## Retained build inputs and reproduction
-The native receipt already records build inputs and outputs, but it does not
-retain source bytes or make a clean rebuild possible on its own. `provenance`
-retains the required closure and keeps three assurances distinct: **verified
+The native receipt records input and output identities. A clean rebuild also
+requires the corresponding source bytes and executable inputs. `provenance`
+retains that closure and keeps three assurances distinct: **verified
bytes**, **recorded producer claims**, and **a reproduced build**.
First compile with an explicit trusted toolchain, as in
@@ -443,13 +460,15 @@ result/bin/tricu provenance capture --file "$WORK/inspector/main.tri" \
```
Capture prints the input-pack hash. It discovers the native root receipt through
-the latest cache locator, or accepts an explicit `--receipt HASH` if that locator
-is absent or another historical build is desired. Every reached source file must
-still match its receipt. `--project` is optional (omit it for a library workspace);
-its lock defaults to `tricu.lock` beside the declaration, with `--lock FILE` as an
-explicit override. It does not capture registry credentials, the whole checkout,
-or unrelated source files. Paths are caller-relative; output directories must be
-new. A failed capture can leave an incomplete directory without a descriptor.
+the latest cache locator, or accepts an explicit `--receipt HASH` if that
+locator is absent or another historical build is desired. Every reached source
+file must still match its receipt. `--project` is optional (omit it for a
+library workspace); its lock defaults to `tricu.lock` beside the declaration,
+with `--lock FILE` as an explicit override. Capture is limited to reached
+sources, required executable inputs, receipts, and optional project metadata;
+registry credentials and unrelated checkout files stay outside the pack. Paths
+are caller-relative; output directories must be new. A failed capture can leave
+an incomplete directory without a descriptor.
Copy the pack directory to another machine or isolated working directory and
transfer its hash through the channel whose assertions you intend to accept:
@@ -458,57 +477,57 @@ transfer its hash through the channel whose assertions you intend to accept:
INPUTS_HASH=PASTE_THE_PRINTED_HASH
result/bin/tricu provenance verify --pack "$PACK" --pin "$INPUTS_HASH"
-# This executes the retained compiler. Do not blindly trust a hash merely
-# because an untrusted pack prints it: select a compiler you actually approve.
+# This executes the retained compiler. Select its hash from an independently
+# trusted source and authorize only a compiler you approve.
result/bin/tricu provenance rebuild --pack "$PACK" --pin "$INPUTS_HASH" \
--trust-compiler "$COMPILER_MANIFEST" --output "$WORK/fresh-rebuild" \
--compile-timeout 900
```
-`verify` is read-only and never executes compiler or application exports. It
-checks the authenticated descriptor and its referenced closure, not unrelated
-files someone might have placed alongside it. `rebuild` verifies first, requires
-explicit compiler authorization, and creates new `workspace`, `application` and
-`tools` directories. It uses only retained sources and executable inputs. There
-is no registry lookup, source fallback, package resolution, or producer-cache
-reuse. The host's packaged compiler is not needed for this explicit route.
+`verify` is read-only and checks the authenticated descriptor and its referenced
+closure as data. Its verification scope is that closure; unrelated files
+alongside it remain unverified. `rebuild` verifies first, requires explicit
+compiler authorization, and creates new `workspace`, `application` and `tools`
+directories. It uses only retained sources and executable inputs. The retained
+compiler and pack supply all compilation inputs, independently of the registry,
+producer cache, and host’s packaged compiler.
A successful result reports exact native module and receipt identities plus
structural comparison of exported trees, and saves `rebuild.txt`. The resulting
module is runnable from the new application CAS using `arboricx run`. An exact
reproduction returns 0; a mismatch reports available structural differences and
-returns nonzero. Tree equality alone does not count as exact reproduction. A
-compiler/validation failure also returns nonzero and may leave partial local
-artifacts, but never a success report. Retry into a new directory, not a warm
-store. The application exports being compared are never executed. This workspace
-is an input snapshot, not a restored project binding-ownership transaction; a
+returns nonzero. Exact reproduction requires matching module and receipt
+identities as well as export structure. A compiler/validation failure also
+returns nonzero and may leave partial local artifacts. Retry into a new
+directory. Application exports are compared as tree data. The workspace restores
+input snapshots; project binding ownership must be established separately. A
project may also declare unused dependencies intentionally absent from the pack.
-This reproduces the **native module**, not arbitrary export filtering or metadata
-of an original distribution bundle. It retains the compiled compiler, not the
-compiler's own source provenance, the Haskell host executable, OS environment,
-or repository history. A compatible Haskell host is still required. It is a
-file-transferable pack, not yet registry provenance discovery, HTTP fetching,
-a signature system or a remote attestation service. Review source/toolchain
-contents for secrets and licensing requirements before distributing them; the
-pack does not automatically collect repository license or origin metadata.
-
-Use trusted, stable directories. Internal CAS directories and input files must
-not be symlinks; concurrent mutation is unsupported. Limits are 128 reached source
-units, 1 MiB per source/receipt, 64 KiB per descriptor/project file/module manifest,
-64 MiB per executable object, and 256 MiB of retained closure bytes. The shared
-inspection reader additionally bounds retained executable bytes per module.
-These are encoded-byte bounds, not heap/RSS guarantees. Rebuilding executes the
-selected compiler with the existing host IO model and compiler deadline, **not**
-an OS/network sandbox; use external resource/isolation controls as appropriate.
-See [storage](storage.md.html#retained-inputs-and-independent-reproduction) for
-the identities and [validation](validation.md.html#retained-provenance-opt-in)
-for the real-library and false-claim acceptance tests.
+The reproduction target is the **native module**. Recreating a distribution
+bundle’s export selection and metadata requires those settings separately. The
+pack retains the compiled compiler; preserve its source provenance, host
+executable, OS environment, and repository history separately as needed. Supply
+a compatible Haskell host and transfer the pack as files. Registry discovery,
+HTTP transport, signatures, and remote attestations require separate services.
+Before distribution, review source/toolchain contents for secrets and licensing
+requirements, and supply repository license and origin metadata explicitly.
+
+Use trusted, stable directories. Symlinked internal CAS directories and input
+files are rejected. Keep contents stable throughout each operation. Limits are
+128 reached source units, 1 MiB per source/receipt, 64 KiB per
+descriptor/project file/module manifest, 64 MiB per executable object, and 256
+MiB of retained closure bytes. The shared inspection reader additionally bounds
+retained executable bytes per module. These bounds cover encoded bytes; runtime
+heap/RSS requires separate controls. Rebuilding executes the selected compiler
+under the existing host IO model and compiler deadline. OS/network isolation
+requires external controls. See
+[storage](storage.md.html#retained-inputs-and-independent-reproduction) for the
+identities and [validation](validation.md.html#retained-provenance-opt-in) for
+the real-library and false-claim acceptance tests.
## First published libraries: an offline bundle inspector
-The initial reusable library set has deliberately small interfaces, not a promise
-that every definition in the underlying `lib/` modules is public:
+The initial reusable libraries expose the following public interfaces:
| Release name | Authored interface | Exports |
| --- | --- | --- |
@@ -517,9 +536,9 @@ that every definition in the underlying `lib/` modules is public:
| `arboricx-inspection` | `lib/arboricx/inspect.tri` | `inspectBundle`, `renderSummary`, `describeError` |
Their closed executable exports include the required binary, numeric and parser
-helpers. Consumers do not need those implementation libraries' sources or a
-transitive package resolver. Importing an IO constructor does not grant IO
-permissions. Each release uses the ordinary one-module package profile above.
+helpers, so consumers can use the closed exports directly. Grant IO permissions
+separately when executing IO trees. Each release uses the ordinary one-module
+package profile above.
[`demos/package-inspector/main.tri`](../demos/package-inspector/main.tri) is a real
consumer of all three releases. Its `tricu.project` selects `tricu-base` as
@@ -529,7 +548,7 @@ that directory using the application store containing the fetched modules; the
registry may be offline. The compiler itself still needs its selected toolchain.
After importing the resulting inspector bundle indexed into `APP_STORE`, run its
-manifest (not a release bundle hash):
+module manifest:
```sh
result/bin/tricu arboricx run --store "$APP_STORE" \
@@ -547,26 +566,27 @@ export<TAB>name<TAB>root index<TAB>kind<TAB>ABI
```
The report preserves export order. Fields percent-escape `%` and bytes outside
-printable ASCII, including UTF-8, tabs, newlines and terminal controls. It hashes
-the same in-memory byte snapshot it validates; it does not reopen the input for
-hashing. It validates every export and stored node, including unused ones, but
-never reconstructs/applies an exported tree or expands shared logical occurrences.
-Malformed bundles and file permission/read failures produce a nonzero process
-exit and no successful report. No writes or network permissions are needed.
+printable ASCII, including UTF-8, tabs, newlines and terminal controls. It
+hashes the same in-memory byte snapshot it validates. Validation covers every
+export and stored node, including unused ones, directly over encoded records.
+Exported trees remain inert, with sharing preserved. Malformed bundles and file
+permission/read failures produce a nonzero process exit and suppress the success
+report. Execution requires only read permission for the input bundle.
`inspectBundle` returns the library Result protocol: failure is a numeric codec
-diagnostic; success is `(stored node count, export records)`, with an empty Result
-remainder. An export record is `(name bytes, (u32 root bytes, (kind bytes, ABI bytes)))`.
-`renderSummary` accepts that validated success payload, not an arbitrary tree;
-`describeError` gives a human-readable diagnostic category. Success hides the internal decoder cursor; failure retains the codec remainder
-as opaque diagnostic context, not an incremental parsing continuation. Internal
-helper definitions are not exported.
-
-This is structural validation and byte identity, **not** publisher authentication,
-build provenance, or a promise that a program is safe to execute. The first
-inspector reads/parses the whole file in memory. It is not a streaming inspector
-or a hostile-input resource sandbox; use external process/time/heap limits for
+diagnostic; success is `(stored node count, export records)`, with an empty
+Result remainder. An export record is `(name bytes, (u32 root bytes, (kind
+bytes, ABI bytes)))`. `renderSummary` requires that validated success payload;
+`describeError` gives a human-readable diagnostic category. Success hides the
+internal decoder cursor; failure retains the codec remainder as opaque
+diagnostic context. Incremental parsing requires a separate parser interface.
+The three listed exports form the complete public interface.
+
+The inspector establishes structural validity and byte identity. Authenticate
+publishers, verify build provenance, and assess execution safety separately. It
+reads/parses the whole file in memory; use external process/time/heap limits for
untrusted large files. Current formats and supported ABIs follow the underlying
-native Arboricx reader. The [acceptance workflow](validation.md.html#published-library-consumer-opt-in)
-checks the actual compiled libraries against an independent Haskell decoder,
-including inspection of the published libraries and the inspector itself.
+native Arboricx reader. The [acceptance
+workflow](validation.md.html#published-library-consumer-opt-in) checks the
+actual compiled libraries against an independent Haskell decoder, including
+inspection of the published libraries and the inspector itself.
diff --git a/docs/storage.md b/docs/storage.md
@@ -15,7 +15,7 @@ path = STORE/objects/hash[0:3]/hash
Filesystem digests are exactly 64 lowercase hexadecimal characters. Typed reads
verify both the expected domain and digest before decoding. Cached verified
-objects are immutable snapshots, rather than live views of later file changes.
+objects preserve the bytes verified at read time as immutable snapshots.
The HTTP registry has a different identity and layout:
@@ -25,8 +25,8 @@ path = REGISTRY/objects/hash[0:3]/hash.arboricx
```
Use the whole-file hash for browser downloads and registry URLs. Use a manifest
-hash for `--toolchain` and stored execution. Neither is interchangeable with an
-individual executable-object hash.
+hash for `--toolchain` and stored execution. Individual executable objects have
+their own hashes for object-level operations.
The registry computes upload identities and publishes whole files atomically.
Its direct read API validates the digest's spelling before reading; clients such
@@ -50,17 +50,18 @@ shared values. Tree inspection still counts logical occurrences.
Native module publication and imports use indexed objects, matching CLI import.
Flat objects remain available for explicit raw-tree transport and retained older
-toolchains; they are not accepted by the current native module reader. The host's
-Merkle-node representation is a separate object graph used by its tree store.
-Each representation has its own identity even when it denotes the same tree.
+toolchains. The current native module reader requires indexed objects. The
+host's Merkle-node representation is a separate object graph used by its tree
+store. Each representation has its own identity even when it denotes the same
+tree.
### Bounded flat import
`arboricx import --flat` computes aggregate expanded bytes across all exports
before expansion or object writes. The default budget is 64 MiB; a positive
-`--max-flat-bytes BYTES` changes it. Saturating DAG accounting handles tiny graphs
-whose unfolded trees would be enormous. The budget bounds flat output, not input
-size, process memory, or execution time.
+`--max-flat-bytes BYTES` changes it. Saturating DAG accounting handles tiny
+graphs whose unfolded trees would be enormous. The budget bounds flat output.
+Input size, process memory, and execution time require separate limits.
Bundle export accepts indexed and flat references, preserving the declared kind.
Bare hashes select a format from the wire magic, then authenticate its complete
@@ -145,11 +146,12 @@ The Haskell IO driver provides these typed transport operations:
The indexed operations use exactly the single-root object format produced by
indexed bundle import. Encoding memoizes physical sharing and interns exact
constructor/child-ID keys; equal finite trees retain canonical bytes regardless
-of their physical sharing. Neither codec applies the supplied tree as a program.
+of their physical sharing. Both codecs treat the supplied tree as data.
Tags 40 and 41 remain the existing state operations.
Reads require read permission, a canonical digest, and a valid payload under the
-selected format's fixed hash domain. Tree-term bytes allow no trailing data. Shared decoding interns constructors without evaluating the
+selected format's fixed hash domain. Tree-term bytes must end exactly at the end
+of the encoded tree. Shared decoding interns constructors without evaluating the
program. Writes require read and write permission, verify the supplied digest,
use a unique same-directory temporary file and atomic no-replace link, sync the
file and directory, and compare an existing winner's bytes. A sync error after
@@ -173,8 +175,9 @@ The `arboricx.source-build-key.v1` key covers the same inputs except output. A
matching cache index must reference an authenticated receipt with the expected
inputs and outputs, a valid manifest, and verified imported/exported trees.
Missing indices or changed keys rebuild. Malformed indices or corrupt matching
-objects fail. Indices are trusted local acceleration metadata; checking their
-referenced bytes alone does not prove the claimed compiler produced the output.
+objects fail. Indices are trusted local acceleration metadata. Checking
+referenced bytes establishes integrity; establishing compiler reproduction
+requires a fresh rebuild.
Cache granularity is the complete interface and dependency provenance. Changing
an unused export or a dependency's source-only receipt can invalidate consumers.
@@ -184,61 +187,61 @@ identity through reuse.
### Retained inputs and independent reproduction
-`tricu provenance` wraps the existing native `arboricx.build-receipt.v1` graph;
-it does not create a competing receipt format or change executable identity.
-See the [capture/verify/rebuild workflow](modules.md.html#retained-build-inputs-and-reproduction).
+`tricu provenance` retains inputs for the native `arboricx.build-receipt.v1`
+graph, preserving its receipt format and executable identities. See the
+[capture/verify/rebuild
+workflow](modules.md.html#retained-build-inputs-and-reproduction).
A portable input pack contains a `build-inputs` descriptor and a standard CAS
under `store/objects`. The descriptor is authenticated with the domain
-`arboricx.build-inputs.v1`; retain/share its full hash independently of its path.
-Its canonical LF/tab format records an entry name, the native root receipt,
-a compiler manifest, sorted unit-name/receipt pairs, and optional project
-metadata pins. The inventory must exactly match the reachable native local-receipt
-graph; unit paths are derived as `NAME.tri`, never accepted as arbitrary paths.
+`arboricx.build-inputs.v1`; retain/share its full hash independently of its
+path. Its canonical LF/tab format records an entry name, the native root
+receipt, a compiler manifest, sorted unit-name/receipt pairs, and optional
+project metadata pins. The inventory must exactly match the reachable native
+local-receipt graph; unit paths are derived as `NAME.tri`.
Native receipts remain authoritative for each unit's raw source digest, declared
compiler identity, options, output, ordered local edges and imported tree pins.
-The pack supplies previously missing preimages: raw sources under
+The pack supplies the corresponding preimages: raw sources under
`arboricx.source.v1` and the supported profile string under
-`arboricx.build-options.v1`. Source snapshots must match the existing receipt,
-not a freshly invented claim about changed source. Each receipt's complete pin
-group must match its referenced module interface, and local output/receipt links
-are checked recursively. Capturing a selected `--receipt` does not need a cache;
-without that option, the native cache is only a locator and its key/output fields
-must agree with the authenticated receipt.
+`arboricx.build-options.v1`. Source snapshots must match the existing receipt
+exactly. Each receipt's complete pin group must match its referenced module
+interface, and local output/receipt links are checked recursively. An explicit
+`--receipt` selects a receipt directly. Otherwise, the native cache locates it,
+and the cache key/output fields must agree with the authenticated receipt.
The first profile is the indexed, stable flat-workspace compiler route with
explicit toolchain selection. Its native compiler identity is the indexed object
-hash of `sourceBuildGraph`; the retained compiler manifest must select that exact
-export. The descriptor retains a runnable compiler facade, not evidence that the
-producer originally used that particular facade or transport bundle. Default
-packaged-compiler builds currently record a whole-bundle identity instead; rebuild
-with an explicit toolchain before capturing them in this profile. Other option
-profiles/frontends are rejected rather than silently interpreted as stable builds.
+hash of `sourceBuildGraph`; the retained compiler manifest must select that
+exact export. The descriptor retains a runnable compiler facade. The producer’s
+original facade and transport bundle remain unverified claims. Default
+packaged-compiler builds currently record a whole-bundle identity instead;
+rebuild with an explicit toolchain before capturing them in this profile. Other
+option profiles/frontends are rejected.
Optional project declaration/lock bytes have domains
-`arboricx.project-declaration.v1` and `arboricx.project-lock.v1`. They are
-capture-time selection context, not compiler-read inputs retroactively recorded
-by native receipts. Their selections must agree, and reached generated binding
-snapshots must match the lock. Unused project selections are retained as context
-but their libraries are not required inputs to this entry's rebuild.
+`arboricx.project-declaration.v1` and `arboricx.project-lock.v1`. They record
+capture-time selection context; native receipts separately identify
+compiler-read inputs. Their selections must agree, and reached generated binding
+snapshots must match the lock. Unused project selections are retained as
+context. Rebuilds require libraries only for reached selections.
Claimed outputs are retained as comparison evidence, separate from executable
inputs. Rebuilds seed only immutable external imports into a new application CAS
-and the selected compiler into a separate tool CAS. No producer cache, alias,
-receipt, checkpoint, or output-only object is copied into that application store.
-An output that is also a genuine external input (such as a header-only re-export)
-is necessarily available as an input. The compiler-returned receipt and its new
-local graph are authenticated after execution; root receipt/module identity and
-export structures are compared against the pack.
-
-Verification establishes byte integrity and receipt-link consistency, **not** that
-the compiler actually produced the claimed result. A successful fresh rebuild is
-a local observation that the selected compiler reproduced the native receipt and
-module. Neither authenticates a publisher, signs an attestation, nor proves unique
-source history. Several source histories or retained contexts may produce the
-same executable. Pack hashes and native receipt hashes remain separate from
-module and distribution-bundle identities.
+and the selected compiler into a separate tool CAS. Producer caches, aliases,
+receipts, checkpoints, and output-only objects stay outside that application
+store. An output that is also a genuine external input (such as a header-only
+re-export) is necessarily available as an input. The compiler-returned receipt
+and its new local graph are authenticated after execution; root receipt/module
+identity and export structures are compared against the pack.
+
+Verification establishes **byte integrity and receipt-link consistency**. A
+successful fresh rebuild establishes a local observation that the selected
+compiler reproduced the native receipt and module. Publisher authentication and
+signed attestations require separate mechanisms; source history can be
+ambiguous. Several source histories or retained contexts may produce the same
+executable. Pack hashes and native receipt hashes remain separate from module
+and distribution-bundle identities.
### Export checkpoints
@@ -255,8 +258,8 @@ receipts publish after all required exports complete. See
## Exact package release index and locks
-Package release v1 uses the existing bundle transport and indexed native module
-identity; it does not introduce another executable or package CAS object. At
+Package release v1 identifies a release by its distribution bundle and indexed
+native module manifest. At
`REGISTRY/packages/NAME/VERSION`, the canonical UTF-8 file contains two
LF-terminated lines (fields separated by TAB):
@@ -266,10 +269,10 @@ NAME<TAB>VERSION<TAB>MODULE_MANIFEST_HASH<TAB>WHOLE_BUNDLE_HASH
```
The bundle's ordered exports must import into exactly the advertised native
-manifest. The publisher checks this without evaluating exports. Metadata is a
-trusted registry assertion, not a signed statement. Future package profiles must
-use an explicitly versioned format rather than interpreting this module pin as
-a richer package manifest.
+manifest. The publisher checks this without evaluating exports. Metadata derives
+its authority from trust in the registry. Signature verification requires a
+separate mechanism. Richer package metadata requires an explicitly versioned
+format; this profile’s module pin identifies a native module manifest.
A client lock is a separate, canonical, three-line file:
@@ -280,9 +283,10 @@ NAME<TAB>VERSION<TAB>MODULE_MANIFEST_HASH<TAB>WHOLE_BUNDLE_HASH
```
The package publisher uses same-directory temporary files, file sync, hard-link
-no-replace publication, byte comparison against a racing winner, and parent sync.
-Locks and generated workspace bindings use atomic replacement. This does not
-change the Haskell CAS adapter's existing durability/concurrency guarantees.
-Directories and ancestors remain trusted; there is no multi-file transaction,
-concurrent lock-update merge, or power-loss/crash-injection acceptance claim.
-See [package usage](modules.md.html#exact-version-packages-from-trusted-registries).
+no-replace publication, byte comparison against a racing winner, and parent
+sync. Locks and generated workspace bindings use atomic replacement. The Haskell
+CAS adapter retains its own durability/concurrency guarantees. Atomicity applies
+to individual files; serialize lock updates and use trusted directories and
+ancestors. Power-loss and crash-injection behavior require separate validation.
+See [package
+usage](modules.md.html#exact-version-packages-from-trusted-registries).
diff --git a/docs/tri-tools.md b/docs/tri-tools.md
@@ -35,11 +35,11 @@ SHA256.
lets applications start with an empty store. Explicit overrides retain their
verified executable-object hash as compiler identity in receipts.
-With no file, `eval` reads a complete native module from stdin; it does not accept
-a recovery-language bare expression. Stdin uses an isolated temporary workspace:
-use manifest imports, or a file for workspace-relative dependencies. Multiple input
-files are rejected; use native imports instead. `tree write` also uses the packaged
-compiler and selects an **exported** definition.
+When the file argument is omitted, `eval` requires a complete native module on
+stdin, including its exported entrypoint. Stdin uses an isolated temporary
+workspace: use manifest imports, or a file for workspace-relative dependencies.
+Multiple input files are rejected; use native imports instead. `tree write` also
+uses the packaged compiler and selects an **exported** definition.
`--compile-timeout SECONDS` bounds tool loading and module builds. Its default is
120 seconds, with a range of 1 to 86400. Completed immutable objects can remain
@@ -79,29 +79,30 @@ with success/error labels. `--repeat` repeats an invocation after loading once;
pure applications may reuse an already evaluated value, so this is unsuitable
for independent fresh-run benchmarks.
-For stored IO commands that print their own output, `--io --no-result` suppresses
-the trailing host-rendered return value. Program output and errors are unchanged;
-`--no-result` without `--io`, or combined with `--result`, fails before execution.
-Repeat/profiling markers retain their existing behavior. Observation phases still
-occur with `--no-result`, but do not render or force the suppressed return value.
-Explicit `--profile-full-result` forcing remains independent of result rendering.
+For stored IO commands that print their own output, `--io --no-result`
+suppresses the trailing host-rendered return value. Program output and errors
+are unchanged; `--no-result` without `--io`, or combined with `--result`, fails
+before execution. Repeat/profiling markers retain their existing behavior.
+Observation phases still occur with `--no-result`, leaving the suppressed return
+value unrendered and lazy. Explicit `--profile-full-result` forcing remains
+independent of result rendering.
## Read-only interaction-tree inspection
-Inspect a materialized host IO tree without applying it, running effects, importing
-it into a store, or loading a compiler:
+Inspect a materialized host IO tree as read-only data directly from a bundle:
```sh
result/bin/tricu arboricx inspect --file "$WORK/app.arboricx" --name main --as io
result/bin/tricu arboricx inspect --file "$WORK/app.arboricx" --name main --as io --json
```
-`--as io` is required and selects the `tricuIO` version 1 envelope and host action
-protocol, **not** the pure contract-effect layer. `--name` defaults to `main`.
-Inspection does not apply arguments: a function that would produce IO must first
-be materialized elsewhere. Only explicit child actions are followed; continuations
-and environment-transforming functions remain opaque, labeled by their roles.
-No original source names or future control flow are inferred.
+`--as io` is required and selects the `tricuIO` version 1 envelope and host
+action protocol. Pure contract effects use a separate protocol. `--name`
+defaults to `main`. Supply an already materialized IO tree; apply any arguments
+needed to produce that tree before inspection. Only explicit child actions are
+followed; continuations and environment-transforming functions remain opaque,
+labeled by their roles. Names come from bundle export aliases, and the outline
+covers currently explicit control flow.
For a small example with a named continuation:
@@ -121,8 +122,9 @@ root: tricuIO v1 aliases: "main"
root.action.resume: continuation (not applied) aliases: "afterRead"
```
-The demo spells out the IO constructors so it needs no library imports. Compiling
-and inspecting it does not read `settings.txt`; running its IO tree would.
+The demo spells out the IO constructors directly. Reading `settings.txt` occurs
+only when the IO tree is executed; compilation and inspection treat its path as
+data.
### Inspect the actual Arboricx server
@@ -160,9 +162,10 @@ Four configured values match directly: `storeRoot`, `createRootDirectory`,
`ensureRootDirectory`, and `initializeStore`. The remaining labels are
**author-named structural projections** of the materialized action. Projection
extracts existing subtrees as data; all four continuations remain opaque and
-unapplied. Aliases establish exact tree identity; their names convey the author's
-interpretation. Matching requires the specialized tree itself, rather than its
-general library helper or an observationally equivalent expression.
+unapplied. Aliases establish exact tree identity; their names convey the
+author's interpretation. Matching requires exact structural equality with the
+specialized tree. General library helpers and observationally equivalent
+expressions can have different trees.
The independently materialized `serveView` exposes `serveRegistry`, its
`createSocket` action, and the opaque `afterSocketResult` continuation.
@@ -186,47 +189,51 @@ application IO remains unexecuted.
### Identities, output, and limits
-The bundle's whole-file SHA-256 fingerprints the inspected bytes; it does not
-establish their provenance or authenticate them against an external expected
-hash. The complete indexed graph is structurally validated, including unused
-nodes. Export aliases are exact matches to displayed trees within that bundle.
-Several names may match one tree, and names lost through specialization cannot
-be recovered. `n123` identifies a **bundle-local tree node**, not a CAS address or
-an execution occurrence. A generated path such as `root.action.resume` identifies
-an occurrence; equal tree IDs do not imply an effect executes only once.
+The bundle’s whole-file SHA-256 fingerprints the inspected bytes. Authenticate
+that fingerprint against an independently selected pin and verify provenance
+separately. The complete indexed graph is structurally validated, including
+unused nodes. Export aliases are exact matches to displayed trees within that
+bundle. Several names may match one tree, and names lost through specialization
+cannot be recovered. `n123` identifies a **bundle-local tree node**. CAS objects
+have separate addresses. A generated path such as `root.action.resume`
+identifies an occurrence; multiple occurrences of one tree can execute effects
+independently.
`--json` emits `tricu.interaction-inspection.v1`: bundle fingerprint, selected
export, supported protocol version, limits, stored-node count, and a depth-first
`occurrences` array. Each occurrence includes its path, depth, role, tree ID,
export aliases, omitted-alias count, display summary, status, and optional
reference to a previously described action occurrence. Summaries are display
-text, not source or a stable payload serialization. Status is `decoded`, `opaque`,
-`invalid`, `unknown`, `truncated`, or `reference`. `opaque` is a deliberate boundary,
-not a certification that a continuation is callable or a value is well-typed.
+text whose wording may change; retain the artifact for exact payload data.
+Status is `decoded`, `opaque`, `invalid`, `unknown`, `truncated`, or
+`reference`. `opaque` marks a value whose internals remain unexamined.
+Callability and type validity require separate checks.
| Option | Default | Meaning |
| --- | --- | --- |
| `--max-bytes` | 67108864 | Input byte cap, adjustable downward to 1 byte. |
| `--max-depth` | 24 | Occurrence depth, 0–256; root depth is 0. |
-| `--max-nodes` | 500 | Displayed occurrences, 1–100000; **not** stored nodes. |
+| `--max-nodes` | 500 | Displayed occurrences, 1–100000; stored-node count is separate. |
| `--preview-items` | 80 | Characters, bytes, or aliases per occurrence, 1–4096. |
-Numbers are inspected up to 64 encoded bits. String/byte previews do not certify
-unvisited suffixes. Depth and preview truncation appear on the affected occurrence;
-`occurrenceLimitReached` reports an omitted traversal remainder. Alias omission
-is reported separately. Payload previews may reveal secrets already present in
-the artifact; inspection output is not redacted.
+Numbers are inspected up to 64 encoded bits. String/byte previews validate only
+the visited prefix; suffixes remain unexamined. Depth and preview truncation
+appear on the affected occurrence; `occurrenceLimitReached` reports an omitted
+traversal remainder. Alias omission is reported separately. Payload previews may
+reveal secrets already present in the artifact; treat the unredacted inspection
+output accordingly.
Input must be a regular file. Validation and name indexing process the stored
-DAG, never the exponentially larger unfolded tree. Output limits do not bound
-validation heap or elapsed time; the input byte cap is not a heap cap. Use external
-resource limits when inspecting untrusted artifacts.
+DAG directly. Display limits bound report size, and the input cap bounds file
+bytes. Validation heap and elapsed time require external resource limits when
+inspecting untrusted artifacts.
Unknown versions/tags and encountered malformed protocol shapes produce a report
-and a nonzero exit. Invalid bundles, missing exports, and input/option errors fail
-before a report. Deliberate opacity and budget truncation are successful reports,
-**not** proof that the complete program is valid or safe to execute. A global
-`--store` option is ignored by inspection; no store is read or created.
+and a nonzero exit. Invalid bundles, missing exports, and input/option errors
+fail before a report. Deliberate opacity and budget truncation are successful
+reports with explicitly partial protocol coverage. Complete-program validity and
+execution safety require separate analysis. Inspection uses the bundle alone and
+ignores `--store`.
## IO and permissions
@@ -236,8 +243,8 @@ access and belongs only in trusted, deliberately selected runs.
Compilation grants reads to the selected source directory and application store,
and writes to the application store. Application execution uses its own grants.
-Use trusted programs and store directories; path grants are capabilities rather
-than a hostile-filesystem sandbox.
+Path grants authorize access within trusted programs and store directories.
+Hostile-filesystem containment requires separate isolation.
## Host artifact checks
@@ -272,32 +279,35 @@ result/bin/tricu eval demos/source-patterns/app.tri \
+RTS -T -s -RTS 2>"$WORK/profile.log"
```
-JSON samples report monotonic `wall_ns`, allocation, live heap, and GC counters to
-stderr. Differences between timestamps give stage timings. Sampling does not
-force collection; allocation/live values reflect the most recent collection.
+JSON samples report monotonic `wall_ns`, allocation, live heap, and GC counters
+to stderr. Differences between timestamps give stage timings. Samples read
+existing counters; allocation/live values reflect the most recent collection.
`-s` adds process totals. Stored execution provides `--profile` with `+RTS -T`.
Report preparation, loading, compilation, execution, publication, and result
observation separately when comparing workloads.
Stored `--profile` follows the ordinary renderer's demand. `invoke-end-N` marks
-completion of the IO driver (or creation of a still-lazy pure result), not full
-pure evaluation. `observe-begin-N` / `observe-end-N` bracket rendering and output;
-`observe-end-N` follows flushing stdout. Measure `invoke-begin-N` through
-`observe-end-N` for invocation plus selected output, including rendering/IO costs.
-Merged logs receive a separating stderr newline before the final JSON sample.
+completion of the IO driver or creation of a still-lazy pure result. Pure
+evaluation can continue during observation. `observe-begin-N` / `observe-end-N`
+bracket rendering and output; `observe-end-N` follows flushing stdout. Measure
+`invoke-begin-N` through `observe-end-N` for invocation plus selected output,
+including rendering/IO costs. Merged logs receive a separating stderr newline
+before the final JSON sample.
For an explicitly different workload, add `--profile-full-result` (requires
-`--profile`). Separate `force-begin-N` / `force-end-N` samples bracket full-result
-forcing before rendering. This traversal memoizes completed physical nodes within
-one invocation, so repeated DAG branches are not expanded. It still demands fields
-that `--result` discards: infinite or partial results can diverge or exhaust
-resources. No forced collection, reduction counter, or bounded-memory guarantee
-is implied.
+`--profile`). Separate `force-begin-N` / `force-end-N` samples bracket
+full-result forcing before rendering. This traversal memoizes completed physical
+nodes within one invocation, preserving sharing during traversal. It still
+demands fields that `--result` discards: infinite or partial results can diverge
+or exhaust resources. Samples read existing GC counters; reduction counts and
+peak memory require separate measurement and resource controls.
Previously, `--profile` implicitly forced the complete expanded result before
-`invoke-end`. Those historical intervals are not comparable to either new mode.
-Active workload probes label invocation-through-output measurements explicitly;
-use fresh processes, not `--repeat`, to avoid reusing an evaluated application.
+`invoke-end`. Those historical intervals measured a different workload; compare
+samples within the same profiling mode. Active workload probes label
+invocation-through-output measurements explicitly; use fresh processes for
+independent application evaluation. `--repeat` permits reuse of an evaluated
+application.
## `.tri` equation alignment
@@ -328,7 +338,7 @@ cancellation fail before replacement. See the
## Library workspaces and demos
The repository `tricu.workspace` maps native library aliases to source paths;
-ordinary CLI compilation needs no manually staged directory. Bootstrap source
+ordinary CLI compilation stages mapped sources automatically. Bootstrap source
maps remain a separate compiler-rebuilding protocol. To reuse already compiled
dependency manifests from a completed generation in a new `DEMO_DIRECTORY`:
@@ -346,39 +356,42 @@ full-cohort work uses explicitly selected limits from [Testing](validation.md.ht
## Interactive sessions
The default REPL and `tricu repl` use the portable compiler. The `repl` command
-accepts the same compiler override/deadline options as `eval`. There is no implicit
-prelude. Simple `x = 7` definitions persist; `x` displays a result without changing
-session source. Redefinition replaces the entire previous equation group and
-recompiles all session definitions, so dependents see the new definition: after `x = 7`,
-`f ignored = x`, and `x = 8`, `f t` returns `8` rather than retaining `7`.
+accepts the same compiler override/deadline options as `eval`. Import the
+prelude explicitly when needed. Simple `x = 7` definitions persist; `x` displays
+a result without changing session source. Redefinition replaces the entire
+previous equation group and recompiles all session definitions, so dependents
+see the new definition: after `x = 7`, `f ignored = x`, and `x = 8`, `f t`
+returns `8`.
For multiline definitions, enter `!def NAME`, the complete definition/equation
group, then `!end`. The compiled local export set must match the named session
groups exactly: a block for `x` defining only `y`, or also defining an unrelated
-name, is rejected without changing the session. For multiline or ambiguous expressions, use `!expr` through
-`!end`. EOF cancels an unfinished block. `replResult` is reserved for the session.
-The convenience single-line classifier is not a second language parser; the
-portable compiler validates every submission.
-
-`!import` accepts native import syntax, with workspace dependencies resolved from
-the current directory. `!load FILE` compiles the file and opens its exported
-interface by immutable manifest. `!use MANIFEST [NAMESPACE]` imports a pinned
-interface; a module alias is resolved to its current pin when submitted. Import
-collisions fail rather than silently replacing definitions. Use qualified imports
+name, is rejected without changing the session. For multiline or ambiguous
+expressions, use `!expr` through `!end`. EOF cancels an unfinished block.
+`replResult` is reserved for the session. The single-line classifier chooses a
+submission form; the portable compiler parses and validates every submission.
+
+`!import` accepts native import syntax, with workspace dependencies resolved
+from the current directory. `!load FILE` compiles the file and opens its
+exported interface by immutable manifest. `!use MANIFEST [NAMESPACE]` imports a
+pinned interface; a module alias is resolved to its current pin when submitted.
+Import collisions fail, preserving existing definitions. Use qualified imports
when names overlap. Temporary session modules are removed after each submission;
the current directory must be writable. CAS objects and receipts may remain.
Compilation, import and result-rendering errors preserve prior successful source
-and completion state. This is not rollback of immutable CAS publications or any
-external effects; expression results are displayed, not executed as IO actions. Expression results are not bound automatically. `!reset`
-clears source/imports; `!env` lists local exported definitions. `!format`, `!output`,
-`!store`, `!clear`, `!help`, and `!exit` remain available. Changing `!store` does not
-copy pinned dependencies; a subsequent build can fail if the new store lacks them.
-The legacy `!name` tree injection operation is retired; import an explicit module
-interface instead. The historical host frontend has been removed; CLI operations
-and source tests use the portable compiler.
-
-Focused edited-host coverage (not a generation or fixture publication):
+and completion state. Immutable CAS publications and external effects persist.
+Expression results are displayed as values; IO execution requires a separate
+run, and retaining a result requires an explicit definition. `!reset` clears
+source/imports; `!env` lists local exported definitions. `!format`, `!output`,
+`!store`, `!clear`, `!help`, and `!exit` remain available. After changing
+`!store`, ensure the new store contains the pinned dependencies before building.
+The legacy `!name` tree injection operation is retired. Import explicit module
+interfaces to introduce existing trees into the session. CLI operations and
+source tests use the portable compiler.
+
+Focused CLI and REPL validation, using the selected compiler and existing
+fixtures:
```sh
python3 test/default_frontend.py result/bin/tricu
diff --git a/docs/validation.md b/docs/validation.md
@@ -1,34 +1,34 @@
# Testing
-Use the Haskell host only; build/link/test through Nix, never direct `cabal`.
-See [build/run](build-and-run.md.html) for guide selection and the
-[language-feature inventory](language-features.md.html) before `.tri` edits.
-For representation-sensitive work, see [Compiler architecture](compiler.md.html).
+Use the Haskell host and route builds, linking, and tests through Nix. See
+[build/run](build-and-run.md.html) for guide selection and the [language-feature
+inventory](language-features.md.html) before `.tri` edits. For
+representation-sensitive work, see [Compiler architecture](compiler.md.html).
## Fast gates
The Haskell lexer/parser/compiler/source loader, recovery AST, and recovery
workspace resolver have been removed. Executables, tests and benchmarks share
the source-free runtime; the Nix boundary check rejects reintroduction of the
-retired frontend files. `megaparsec` is no longer a host/test dependency.
+retired frontend files. Host/test dependencies follow this source-free boundary.
Runtime-presentation tests use independent constructor-display goldens; runtime
-demand tests exercise the same reducer used by production. Historical Haskell
-lexer/parser/lambda-elimination tests and self-hosting differential comparisons
-are retired, not maintained as language oracles. Portable source tests retain
-their explicit expected values, locations, diagnostics and negative controls.
-Representation tests concern retained portable artifacts, not old-host parity.
+demand tests exercise the same reducer used by production. Portable source tests
+provide the active language oracles: explicit expected values, locations,
+diagnostics, and negative controls. Representation tests cover retained portable
+artifacts. Historical Haskell frontend and self-hosting differential tests are
+retired.
```sh
nix flake check
python3 test/unit.py
```
-The historical executable preparation paths in `test/stable_bundle.py` and
-`test/stable_costs.py` are retired: they depended on implicit recovery compilation
-and have no verified native-support input route. Their CLIs fail before writes;
-`stable_bundle`'s pure importable wire helpers remain supported. Do not bypass the
-guard by trusting old support/cache stamps. Those retired harnesses have no replacement route. The separate artifact-based
-Criterion runner below does not revive them or establish new performance measurements.
+The CLIs in `test/stable_bundle.py` and `test/stable_costs.py` are retired and
+fail before writes. Their preparation relied on implicit recovery compilation;
+keep those guards intact, including when old support/cache stamps are present.
+`stable_bundle`’s pure importable wire helpers remain supported. For current
+benchmarks, use the separately defined artifact-based Criterion workload below
+and collect fresh measurements.
### Artifact-based Criterion benchmarks (opt-in)
@@ -43,35 +43,38 @@ result-bench/bin/tricu-bench result-benchmark-workers/workers.arboricx \
`benchmarkWorkers` builds `bench/workers.tri` and its explicit current-source
library dependencies with the packaged stable compiler. It retains the staged
-sources, compiler metadata, CAS/receipts, bundle and whole-bundle SHA256. Preparation
-is bounded to 900 seconds compiling, 930 seconds externally, and an 8 GiB managed
-heap. It is an application build, not a compiler generation or fixture publication.
-
-The Haskell runner requires an explicit bundle and expected digest; it never reads
-library source or invokes either compiler. Loading validates bundle structure,
-execution ABI and the exact export interface. A 120-second deadline covers loading
-and checking all 27 cases against independent complete expected values. Workers,
-inputs and expected values are fully constructed before timing; use external
-process/heap limits for actual timing runs. A digest authenticates selected bytes,
-not the behavior of arbitrary caller-selected workers.
-
-The contract group measures precompiled application plus complete result equality;
-the registry group observes canonical Booleans in the same way. Parsing, compilation
-and publication are excluded. Historical contract benchmarks included recovery
-parsing/evaluation and are **not comparable**. This is a new workload definition,
-not a speedup claim. `benchmarkSmoke` checks expected results, wrong hashes,
-malformed bundles, incorrect interfaces, Criterion listing, and a one-iteration
-invocation of every case; it is not a statistically meaningful performance run.
+sources, compiler metadata, CAS/receipts, bundle and whole-bundle SHA256.
+Preparation is bounded to 900 seconds compiling, 930 seconds externally, and an
+8 GiB managed heap. Preparation builds benchmark applications; compiler
+generations and fixture publication follow their own workflows.
+
+The Haskell runner consumes an explicit precompiled bundle and expected digest.
+Loading validates bundle structure, execution ABI and the exact export
+interface. A 120-second deadline covers loading and checking all 27 cases
+against independent complete expected values. Workers, inputs and expected
+values are fully constructed before timing; use external process/heap limits for
+actual timing runs. The digest authenticates selected bytes. Behavioral
+correctness depends on the independent expected-value checks.
+
+The contract group measures precompiled application plus complete result
+equality; the registry group observes canonical Booleans in the same way.
+Parsing, compilation and publication are excluded. Historical contract
+benchmarks included recovery parsing/evaluation; compare timings only within the
+same workload definition. `benchmarkSmoke` checks expected results, wrong
+hashes, malformed bundles, incorrect interfaces, Criterion listing, and a
+one-iteration invocation of every case. Use repeated Criterion measurements for
+statistical performance analysis.
### Interaction-tree inspection
`nix flake check` includes the Haskell `Interaction inspection` unit group and
-`test/interaction_inspection.py` in the host-policy check. The CLI fixtures build
-indexed bundles independently of the inspector, check outline/JSON identities
-and exit behavior, retain malformed/unknown/oversized input controls, and verify
-that effectful trees neither mutate files nor create a store. The unit group
-covers all current host IO action tags, opaque continuations, exact export aliases,
-preview/depth/occurrence limits, and reachable exponentially shared values.
+`test/interaction_inspection.py` in the host-policy check. The CLI fixtures
+build indexed bundles independently of the inspector, check outline/JSON
+identities and exit behavior, retain malformed/unknown/oversized input controls,
+and verify that inspection of effectful trees preserves files and leaves absent
+stores absent. The unit group covers all current host IO action tags, opaque
+continuations, exact export aliases, preview/depth/occurrence limits, and
+reachable exponentially shared values.
For the focused CLI check after building the Haskell host:
@@ -80,23 +83,24 @@ nix build .#
PYTHONDONTWRITEBYTECODE=1 python3 test/interaction_inspection.py result/bin/tricu
```
-The compiled-source smoke example is in
-[read-only interaction-tree inspection](tri-tools.md.html#read-only-interaction-tree-inspection).
-It checks the native compiler-to-inspector path separately from independent wire
-fixtures; inspection itself never loads a compiler or executes an exported tree.
+The compiled-source smoke example is in [read-only interaction-tree
+inspection](tri-tools.md.html#read-only-interaction-tree-inspection). It checks
+the native compiler-to-inspector path separately from independent wire fixtures;
+inspection reads the exported tree as data directly from its bundle.
### Host loading and compiler supervision
The default Haskell tests directly exercise the production loader with flat and
indexed objects: digest and ABI rejection, malformed and missing objects,
-manifest rejection, retained sharing, profiling-stage order, and selected exports
-that do not read unrelated objects. Whole-interface loading retains manifest order.
+manifest rejection, retained sharing, profiling-stage order, and selected
+exports with reads confined to their selected objects. Whole-interface loading
+retains manifest order.
-Supervisor tests use constructed ABI workers, not fresh compiler bootstraps.
-They check explicit selection, option validation, Result and publication
-validation, source-write denial, and a deadline that interrupts a sleeping worker.
-The default-frontend and portable-CLI probes additionally exercise real source
-compilation, REPL transactions, cache identities and compiler profiling:
+Supervisor tests use constructed ABI workers. They check explicit selection,
+option validation, Result and publication validation, source-write denial, and a
+deadline that interrupts a sleeping worker. The default-frontend and
+portable-CLI probes additionally exercise real source compilation, REPL
+transactions, cache identities and compiler profiling:
```sh
nix build .#
@@ -106,7 +110,8 @@ python3 test/portable_cli.py result/bin/tricu \
--compiler-bundle bootstrap/build-stable.arboricx
```
-These checks do not regenerate the compiler or publish fixtures.
+These checks exercise the selected compiler and leave published fixtures
+unchanged.
### IO permissions and run lifetime
@@ -116,7 +121,8 @@ without forcing unpublished trees. Listening-socket rebinding checks run after
normal completion, abort and timeout; blocked accept is also cancelled. FIFO
writers establish that asynchronous OS work has started, stop draining until the
run ends, then verify that the writer cannot finish its payload after run exit.
-These are resource/effect observations, not just scheduler-state assertions.
+These checks observe actual resource lifetime and effects alongside scheduler
+state.
See [the host ABI](host-abi.md.html#haskell-io-run-ownership) for cancellation
semantics and the limits of path-based permissions and timeout enforcement.
@@ -131,7 +137,7 @@ PYTHONDONTWRITEBYTECODE=1 python3 test/io_dependencies.py result/bin/tricu
This opt-in probe compiles current `io`, `base`, and `native_math` sources with
only those modules present in a temporary workspace. It checks IO constructor
and conditional results and rejects an explicit import of the absent `patterns`
-module. It does not rebuild or publish compiler/support fixtures. The fast host
+module. Published compiler/support fixtures remain unchanged. The fast host
policy tests separately check that the working compiler map is an exact closure
and that IO depends only on `base`.
@@ -139,65 +145,66 @@ and that IO depends only on `base`.
Agree on a substantial related module group, its intended interface/behavior
changes and a focused-test budget before starting. Compile candidates with the
-accepted stable toolchain, linking affected consumers against the candidate
-rather than quietly testing retained dependency kernels. Reuse authenticated
-unchanged inputs; do not rebuild the entire support cohort on every iteration.
-Commit reviewable slices with explicit **unpublished candidate** status. Keep
-published fixtures unchanged until the batch's agreed acceptance checkpoint.
-
-Start with a few complete-result/demand/effect probes and representative callers;
-expand coverage where the edit changes behavior or exposes a defect. Bound each
-compilation/run and the combined probe job. Initial estimates are planning guides,
-not roadblocks: modest, justified adjustments are allowed and should be recorded.
-Unexpectedly costly or repeatedly exhausted jobs require diagnosis, not unlimited
-retries or a claimed pass. Broader matrices and repeated performance studies need
-an explicit reason and budget; full-generation resource ceilings still apply.
-
-Differential mismatches require classification, not automatic compatibility.
-Preserve meaningful requirements and intensional behavior; deliberately replace
-insecure/incoherent behavior or poor interfaces, updating callers and independent
-expected results with a stated rationale. Do not erase historical oracles, drop
-negative controls, or bless candidate output as its own specification. A source
-rollback is easy in Git, but does not establish behavioral correctness.
-
-The full compiler gate is an explicit batch checkpoint, not an inter-draft
-barrier. A verified compiler checkpoint may explicitly defer publication: retain
-the audited fresh pair and generation-two native-import/recursion/program smoke
-coverage, leaving reference fixtures and working-source freshness pending. This
-permits subsequent candidate work with the verified compiler but is not final
-publication of the compiler/support cohort. See [compiler rebuilding](compiler-rebuilding.md.html#development-batches-and-acceptance-checkpoints).
+accepted stable toolchain and link affected consumers against that candidate.
+Reuse authenticated unchanged inputs to keep iteration focused on affected
+modules. Commit reviewable slices with explicit **unpublished candidate**
+status. Keep published fixtures unchanged until the batch's agreed acceptance
+checkpoint.
+
+Start with a few complete-result/demand/effect probes and representative
+callers; expand coverage where the edit changes behavior or exposes a defect.
+Bound each compilation/run and the combined probe job. Record modest, justified
+adjustments to initial budget estimates. Unexpectedly costly or repeatedly
+exhausted jobs require diagnosis before another bounded attempt; exhausted jobs
+remain failed checks. Broader matrices and repeated performance studies need an
+explicit reason and budget; full-generation resource ceilings still apply.
+
+Classify differential mismatches against intended requirements. Preserve
+meaningful requirements and intensional behavior; deliberately replace
+insecure/incoherent behavior or poor interfaces, updating callers and
+independent expected results with a stated rationale. Retain historical oracles
+and negative controls, and derive expectations independently of candidate
+output. Git provides source rollback; behavioral correctness requires tests.
+
+Run the full compiler gate at the agreed batch checkpoint, using focused checks
+between drafts. A verified compiler checkpoint may explicitly defer publication:
+retain the audited fresh pair and generation-two native-import/recursion/program
+smoke coverage, leaving reference fixtures and working-source freshness pending.
+This permits subsequent candidate work with the verified compiler. Final
+compiler/support-cohort publication remains pending. See [compiler
+rebuilding](compiler-rebuilding.md.html#development-batches-and-acceptance-checkpoints).
## Three distinct checks
1. **Reference-artifact tests** (`nix flake check`): current Haskell runtime,
retained compiled compiler/support implementations, and their authenticated
- source snapshots. They do **not** claim coverage of edited `.tri` implementations.
+ source snapshots. Edited `.tri` implementations require candidate tests.
2. **Candidate tests**: explicitly compile the changed implementation with the
retained stable compiler and run focused semantics/integrity/integration checks
against the resulting artifact. Record candidate identity and preparation scope.
3. **Final batch acceptance**: for compiler-affecting batches, the fresh stable
generation/self-rebuild gate; genuinely rebuilt current fixtures, explicit
working-source freshness and the fast suite. Support-only batches still need
- candidate/publication validation, but do not acquire a compiler-generation
- requirement solely because they contain `.tri` edits.
+ candidate/publication validation; compiler generations are required for
+ compiler-affecting changes.
The Haskell gate loads fingerprinted, full compiler/support artifacts. Source
bytes are stored by their existing `arboricx.source.v1` digest under
-`test/fixtures/reference-source/`; the `.v1` fixture metadata remains authoritative.
-The loader authenticates those bytes, including the headers used for import
-binding. Missing/corrupt snapshots
-fail: there is no fallback to working-tree library source. Unmapped test programs
-remain live test inputs. Bundle, receipt, namespace, kernel, and ABI checks remain.
+`test/fixtures/reference-source/`; the `.v1` fixture metadata remains
+authoritative. The loader authenticates those bytes, including the headers used
+for import binding. Missing/corrupt snapshots fail. Library inputs come
+exclusively from authenticated snapshots. Unmapped test programs remain live
+test inputs. Bundle, receipt, namespace, kernel, and ABI checks remain.
`test/ReferenceFixtures.hs` owns authenticated artifact loading and direct
published-interface selection. `test/LibraryValueTests.hs` invokes byte-list and
HTTP exports with host-constructed inputs, without source wrappers or recovery
-compilation. `test/ParserValueTests.hs` does the same for binary and finite-stream
-parser protocols, including consumed/committed flags, rollback, zero-width
-repetition, and poisoned-field controls. HTTP framing/error expectations and no-socket-access controls remain
-independent of the compiled implementations. Mixed-cohort reuse verification in
-`SupportFixture` uses complete provider identity; it has no recovery parser or
-free-variable-analysis dependency.
+compilation. `test/ParserValueTests.hs` does the same for binary and
+finite-stream parser protocols, including consumed/committed flags, rollback,
+zero-width repetition, and poisoned-field controls. HTTP framing/error
+expectations and no-socket-access controls remain independent of the compiled
+implementations. Mixed-cohort reuse verification in `SupportFixture` compares
+complete provider identities directly.
`test/IODriverTests.hs` uses explicit ABI trees for primitive/scheduler controls
and portable-compiled programs for library sequencing, state, concurrency and
@@ -206,20 +213,20 @@ socket integration. Its source scope contains only published `io`, `prelude` and
order and complete replies without real filesystem/network access; live IO tests
retain their permission and failure checks.
-This avoids recompiling libraries on every unit run without silently treating old
-executable trees as new source. Compiler conformance wrappers use
-`test/PortableTestCompiler.hs`: the retained stable unit compiler receives explicit
-published fixture imports and a definition body. This test-only header adapter is
-not native module resolution; it returns local definitions, not imported names.
-Preparation has a 30-second deadline and checks Result/list/name structure without
-forcing unused definition values. Invalid source and import shadowing fail without
-fallback. Useful library program tests use the same bounded portable compilation route
-with an explicit published prelude. `div` and `mod` zero-divisor expectations
-follow the checked numeric contract, not the retired host cohort's zero result.
-No test uses the historical frontend or reconstructs its ambient import scopes.
+Unit runs reuse authenticated library artifacts with their recorded source
+identities. Compiler conformance wrappers use `test/PortableTestCompiler.hs`:
+the retained stable unit compiler receives explicit published fixture imports
+and a definition body. This test-only header adapter binds explicit fixture
+imports and returns only local definitions. Native module resolution has
+separate coverage. Preparation has a 30-second deadline and checks
+Result/list/name structure without forcing unused definition values. Invalid
+source and import shadowing fail without fallback. Useful library program tests
+use the same bounded portable compilation route with an explicit published
+prelude. `div` and `mod` zero-divisor expectations follow the checked numeric
+contract. All source tests use portable compilation with explicit imports.
```sh
-# Reference-source/bundle fingerprints only; no working-tree freshness claim.
+# Verify reference-source/bundle fingerprints; check working-source freshness below.
python3 test/fixture_sources.py verify
# Required at final batch acceptance; exits nonzero and lists stale/missing
@@ -227,11 +234,12 @@ python3 test/fixture_sources.py verify
python3 test/fixture_sources.py current
```
-A pending freshness check during an explicitly scoped development batch is not
-an indication of a candidate behavioral regression. It blocks final publication
-acceptance, but not an explicitly scoped verified compiler checkpoint. Failed reference integrity, candidate correctness or generation
-verification still blocks further feature work until repaired. Reference tests
-alone never validate a compiler edit.
+Working-source freshness and candidate behavior are separate checks. Pending
+freshness blocks final publication acceptance; an explicitly scoped verified
+compiler checkpoint may defer it. Failed reference integrity, candidate
+correctness, or generation verification blocks further feature work until
+repaired. Compiler edits require candidate tests and the generation acceptance
+gate.
To capture reference sources after a verified fixture publication (or to recover
missing snapshots), select the source root that matches the fixture metadata:
@@ -241,17 +249,18 @@ python3 test/fixture_sources.py capture --source-root "$FIXTURE_SOURCE"
```
Capture authenticates **all** inputs before writing, only adds digest-named
-files, and refuses changed source or an existing corrupt snapshot. It neither
-rewrites fingerprints nor compiles/publishes a new artifact. Keep the snapshots
-with fixture publications. Old digest-named files are not automatically pruned.
-When sources differ, perform genuine candidate compilation/fixture publication;
-do not update hashes merely to make freshness pass.
+files, and refuses changed source or an existing corrupt snapshot. Existing
+fingerprints and compiled artifacts remain unchanged. Keep the snapshots with
+fixture publications and manage retirement of old digest-named files explicitly.
+Changed sources require genuine candidate compilation and fixture publication
+with fingerprints matching the actual inputs.
`nix flake check` also runs cheap artifact-only CLI checks and `unit.py`
provenance/header, selector/reexport, RSS, prepared-session, native-scope,
fixture-projection and reference/freshness separation controls. Its filtered
-source excludes `archive/`. It builds through Nix but runs no self-hosted compiler
-generations or library source compilation.
+source excludes `archive/`. It builds through Nix and tests retained compiler
+and library artifacts. Library-source compilation and compiler-generation checks
+are opt-in workflows.
## Focused checks
@@ -266,11 +275,11 @@ checks below supply candidate coverage; final batch acceptance follows the
### Digest and dependency analysis
-The fast digest-hex checks compare the reference fixture worker with the immutable
-pre-optimization implementation in `test/fixtures/hash-hex-reference.arboricx`,
-including all byte values and malformed/noncanonical inputs. Do not refresh that
-reference from a candidate. Select a genuinely compiled candidate object module
-for focused checks of edits:
+The fast digest-hex checks compare the reference fixture worker with the
+immutable pre-optimization implementation in
+`test/fixtures/hash-hex-reference.arboricx`, including all byte values and
+malformed/noncanonical inputs. Preserve that reference unchanged. Select a
+genuinely compiled candidate object module for focused checks of edits:
```sh
python3 test/digest_hex.py result/bin/tricu \
@@ -279,8 +288,8 @@ python3 test/digest_hex.py result/bin/tricu \
The directory must be fresh. This runner is artifact-only.
-Focused name/dependency-analysis checks select compiled candidate modules, not
-working source or silently substituted reference implementations:
+Focused name/dependency-analysis checks require explicitly selected, precompiled
+candidate modules:
```sh
python3 test/source_order.py result/bin/tricu \
@@ -293,11 +302,11 @@ python3 test/source_order.py result/bin/tricu \
The directory must be fresh. Prepare those full-export modules with the retained
stable compiler, linking `source_stable` and `source_unit` against the changed
`source_order` manifest (and unit against the relinked stable manifest). Record
-source hashes and dependency manifests during preparation. This runner performs
-no compilation: it compares complete order/diagnostic/graph results and, with the
-optional unit bundle, complete stable compiled results for 38 cases. Reference
-sources and artifacts are authenticated independently; the report records every
-selected artifact identity.
+source hashes and dependency manifests during preparation. This artifact-only
+runner compares complete order/diagnostic/graph results and, with the optional
+unit bundle, complete stable compiled results for 38 cases. Reference sources
+and artifacts are authenticated independently; the report records every selected
+artifact identity.
### HTTP syntax and framing
@@ -309,16 +318,17 @@ python3 test/http_headers.py "$TRICU" --subject "$HTTP_CANDIDATE" \
```
The candidate must contain the full unqualified HTTP exports. The default corpus
-checks strict CRLF lines, complete header blocks, token names, value byte classes,
-SP/HTAB trimming, exact error progress, opaque body remainders, and shared
-request-line scanning. Deliberate semantic changes carry explicit reasons and
-independent complete expected trees; unaffected cases still compare against the
-immutable pre-rewrite reference. `--legacy-behavior` reproduces the earlier
-readability-only corpus against historical artifacts, not the corrected parser.
-The byte domain is canonical naturals in 0..255. Historical out-of-domain cases
-are recorded separately rather than used as current byte-parser expectations.
-All byte classes and exact progress/remainder checks remain active. The frozen
-reference is independent of the candidate.
+checks strict CRLF lines, complete header blocks, token names, value byte
+classes, SP/HTAB trimming, exact error progress, opaque body remainders, and
+shared request-line scanning. Deliberate semantic changes carry explicit reasons
+and independent complete expected trees; unaffected cases still compare against
+the immutable pre-rewrite reference. `--legacy-behavior` reproduces the earlier
+readability-only corpus against historical artifacts. Use the default corpus for
+the corrected parser. The byte domain is canonical naturals in 0..255.
+Historical out-of-domain cases have separate historical records; current
+expectations use the canonical byte domain. All byte classes and exact
+progress/remainder checks remain active. The frozen reference is independent of
+the candidate.
Content-Length/body-length selection has a separate artifact-only check:
@@ -378,7 +388,7 @@ and memory together, including concurrent workloads when assessing throughput.
identical publishers) and `test/package_registry.py` against a controlled
loopback HTTP peer. This checks explicit trust, lock identity and mutation
controls, redirects, metadata bounds, corrupt transport, module mismatches,
-offline reuse/binding, and local corruption. It does not compile source.
+offline reuse/binding, and local corruption using prebuilt wire fixtures.
For the full current-source vertical slice on the Haskell host:
@@ -404,16 +414,18 @@ python3 test/package_registry.py result/bin/tricu \
--native-server /tmp/package-registry-candidate/server.arboricx
```
-That module must export `main root port`, returning the IO-wrapped server action.
-These checks use real loopback HTTP, not live HTTPS/certificate infrastructure,
-and do not claim hostile-filesystem containment or crash durability testing.
+That module must export `main root port`, returning the IO-wrapped server
+action. These checks cover real loopback HTTP. HTTPS/certificate infrastructure,
+hostile-filesystem containment, and crash durability require separate
+validation.
## Registry server pipelines (opt-in)
Prepare `tests.httpEffects` and `tests.registryEffects` explicitly through the
stable native-library preparation route. The registry adapter reuses the same
-fuel-bounded Pure/Bind interpreter; neither test module belongs to the support
-fixture cohort. Select full unqualified registry and optional demo artifacts:
+fuel-bounded Pure/Bind interpreter. Both modules are prepared as test-only
+artifacts, separately from the support fixture cohort. Select full unqualified
+registry and optional demo artifacts:
```sh
python3 test/registry_server.py "$TRICU" --subject "$SERVER_CANDIDATE" \
@@ -422,84 +434,86 @@ python3 test/registry_server.py "$TRICU" --subject "$SERVER_CANDIDATE" \
--directory /tmp/registry-server-check
```
-The runner performs no source compilation or real filesystem/socket actions.
-Without a driver it checks pure hash/contract/path/prefix operations only. With
-it, independent complete trees check attempted action payloads/order, errors,
-unused replies, initialization, uploads/downloads, routes, the actual HTTP
-adapter and listener startup. Optional demo artifacts check initialization
-failure prevents listening, and successful initialization proceeds to listening.
-`--baseline` checks explicit historical expectations against an old subject
-instead, including **inert** unsafe write requests; it does not weaken candidate
-oracles. Reference comparisons characterize each intentional behavior change.
-Wrong-subject selection must fail. Reports record selected artifact identities;
-preparation provenance and publication are separate checks.
+The runner executes selected artifacts through pure/scripted effects. Without a
+driver it checks pure hash/contract/path/prefix operations only. With it,
+independent complete trees check attempted action payloads/order, errors, unused
+replies, initialization, uploads/downloads, routes, the actual HTTP adapter and
+listener startup. Optional demo artifacts check initialization failure prevents
+listening, and successful initialization proceeds to listening. `--baseline`
+checks explicit historical expectations against an old subject including
+**inert** unsafe write requests. Candidate runs retain their independent
+current-behavior oracles. Reference comparisons characterize each intentional
+behavior change. Wrong-subject selection must fail. Reports record selected
+artifact identities; preparation provenance and publication are separate checks.
The server validates exact 64-character lowercase hexadecimal hashes before
filesystem IO, including the digest reply used for upload. The old public
unchecked hash-taking writer and shared `<hash>.tmp` staging are removed. Upload
-publication uses `atomicWriteBytes`, whose host-owned unique temporary file is in
-the destination directory. Existing objects are still replaced, not preserved
-under write-if-absent policy; arbitrary uploaded bytes are still accepted without
-bundle-format validation. A post-publication sync error may leave the new file
-visible. Initialization now propagates errors and stops immediately. Bind/listen
-errors attempt close once; close errors do not replace the original error.
+publication uses `atomicWriteBytes`, whose host-owned unique temporary file is
+in the destination directory. Publication replaces existing objects. Uploads
+accept arbitrary opaque bytes; consumers must validate bundle format before use.
+A post-publication sync error may leave the new file visible. Initialization now
+propagates errors and stops immediately. Bind/listen errors attempt close once
+and preserve the original error even if close fails.
The fast Haskell gate separately exercises bounded malformed-tree rejection
(including unevaluated oversized suffixes), complete primitive traces, and small
-real-filesystem checks: initialization, binary round-trip, replacement, concurrent
-same-object uploads and absence of temporary residue. These use authenticated
-reference fixtures, not edited source. No live-network, scheduler fairness,
-crash-injection or terminal-abort/host-exception cleanup claim is implied.
-
-Store root remains trusted configuration. Identifier validation is not a private
-IO capability or general symlink/race containment: use appropriate host permissions
-and a trusted store filesystem. Pure path helpers
+real-filesystem checks: initialization, binary round-trip, replacement,
+concurrent same-object uploads and absence of temporary residue. These use
+authenticated reference fixtures. Edited-source coverage, live-network behavior,
+scheduler fairness, crash injection, and terminal-abort/host-exception cleanup
+require separate checks.
+
+Store root remains trusted configuration. Identifier validation checks spelling;
+host permissions control IO access, and symlink/race safety requires a trusted,
+stable store filesystem. Pure path helpers
remain unguarded constructors. Hash predicates retain arbitrary-tree rejection;
-`stripPrefix` now requires a proper list prefix and returns the exact suffix,
-rather than preserving malformed legacy Maybe results.
+`stripPrefix` requires a proper list prefix and returns the exact suffix.
## Support fixture publication
Support fixture v3 retains immutable original compiler/support snapshots and
per-module provenance. Its 39-module reference cohort has stable-rebuilt native
-receipts, full manifests, and a separate compiled-root snapshot. Original snapshots
-are not relabeled as outputs of another compiler. Rebuilt modules may change
-bodies and exports: their source fingerprints must match the actual compiled
-source/receipt, and their full export sets must match their new manifests, not
-the historical interface. Tests still check receipt identities and
-published/rebuilt kernel equality. Support-provider manifests in receipts must
-match the selected cohort; an unchanged consumer body is not sufficient evidence
-for reusing an artifact linked against an older provider.
+receipts, full manifests, and a separate compiled-root snapshot. Original
+snapshots preserve their recorded compiler identities. Rebuilt modules may
+change bodies and exports: their source fingerprints must match the actual
+compiled source/receipt, and their full export sets must match their new
+manifests. Tests still check receipt identities and published/rebuilt kernel
+equality. Support-provider manifests in receipts must match the selected cohort.
+Reusing an artifact requires matching provider interfaces as well as an
+unchanged consumer body.
Origin-retained modules require unchanged source fingerprints/output trees and
identical **complete interfaces of every imported or reexported provider**.
-Comparison uses exact constructor/child interning across the bundles, not local
-node indices or probabilistic tree hashes. Provider membership, every export name
-and every executable tree must match; this also preserves namespace and collision
+Comparison uses exact constructor/child interning across the bundles,
+independently of local node indices. Provider membership, every export name and
+every executable tree must match; this also preserves namespace and collision
outcomes. Unrelated providers may change. Changes to unused or shadowed imports
-now require rebuilding: the former free-name-based reuse optimization is retired.
-
-This is an identity proof relative to an authenticated, accepted original cohort,
-not a source validator or a new compilation receipt. The verifier recognizes the
-fixture's simple named-provider imports and reexports; unsupported directives,
-including build-only `!source` dependencies, fail closed and require rebuilding.
-No body parser or scope inference is involved. Prefer rebuilding affected consumers
-to relaxing these checks. The current 39-module cohort is entirely rebuilt, so
-this stricter reuse policy does not require regenerating its fixtures. Future
-source changes still need candidate coverage and genuine publication at acceptance.
-
-For a justified narrow repair, `test/repair_support_dependencies.py` materializes
-full original dependency manifests (and asserts their original identities), and
-`test/native_library_migration.py` builds the affected aliases with stable
-lowering. For a full support rebuild, explicitly select all current support
-modules and let their dependencies compile in the fresh candidate store. Use
-`--backend stable --capabilities 8 --memory-mib 65536`; candidate commands default
-to a 20-minute guard, and `--batch-seconds 7200` permits a bounded full cohort.
-Neither candidate preparation nor fixture packaging constitutes compiler
+now require rebuilding: the former free-name-based reuse optimization is
+retired.
+
+This proves identity relative to an authenticated, accepted original cohort.
+Source validation and new compilation receipts require compilation. The verifier
+recognizes the fixture's simple named-provider imports and reexports;
+unsupported directives, including build-only `!source` dependencies, fail closed
+and require rebuilding. The verifier compares recorded provider interfaces
+directly. Prefer rebuilding affected consumers to relaxing these checks. The
+current 39-module cohort is entirely rebuilt and already satisfies this policy.
+Future source changes still need candidate coverage and genuine publication at
acceptance.
-`test/repair_support_fixture.py` creates a **new candidate directory**, never
-writes over published fixtures, and reproduces every candidate's complete native
+For a justified narrow repair, `test/repair_support_dependencies.py`
+materializes full original dependency manifests (and asserts their original
+identities), and `test/native_library_migration.py` builds the affected aliases
+with stable lowering. For a full support rebuild, explicitly select all current
+support modules and let their dependencies compile in the fresh candidate store.
+Use `--backend stable --capabilities 8 --memory-mib 65536`; candidate commands
+default to a 20-minute guard, and `--batch-seconds 7200` permits a bounded full
+cohort. For compiler-affecting work, acceptance additionally requires the
+generation workflow below.
+
+`test/repair_support_fixture.py` creates a **new candidate directory**,
+preserves published fixtures, and reproduces every candidate’s complete native
manifest before accepting its kernels against the compilation receipt:
```sh
@@ -510,17 +524,18 @@ python3 test/repair_support_fixture.py "$TRICU" "$SUPPORT_CANDIDATES" \
The publisher also accepts multiple preparation directories. To replace a prior
candidate explicitly, supply its directory first, the new preparation later, and
-repeat `--replace ALIAS` for each duplicate alias. It checks the retained records
-against their own source bytes, compiler identities, and receipts; no compilation
-is implied by repacking them. Include genuinely rebuilt affected consumers in
-the selection: stale support dependency pins fail before metadata is published.
-Cohort membership/alias/path changes remain a separate operation; source and
-export compatibility are not requirements for rebuilt modules. The explicit
-`--relocate-patterns` operation adds only `patterns` at `lib/patterns.tri`, requires
-its genuine native rebuild, and refuses publication while it still belongs to
-the compiler fixture. V3 records this relocation without changing the immutable
-origin membership. Use that option when publishing the current 39-module support
-set; ordinary candidate replacement still cannot add arbitrary modules.
+repeat `--replace ALIAS` for each duplicate alias. It checks the retained
+records against their own source bytes, compiler identities, and receipts.
+Repacking preserves those original compilation claims. Include genuinely rebuilt
+affected consumers in the selection: stale support dependency pins fail before
+metadata is published. Cohort membership/alias/path changes remain a separate
+operation. Rebuilt modules may deliberately change source and exports, with
+callers and expectations updated. The explicit `--relocate-patterns` operation
+adds only `patterns` at `lib/patterns.tri`, requires its genuine native rebuild,
+and refuses publication while it still belongs to the compiler fixture. V3
+records this relocation without changing the immutable origin membership. Use
+that option when publishing the current 39-module support set; ordinary
+candidate replacement preserves cohort membership.
Current indexed native-manifest reproduction uses indexed import and exact
manifest comparison. Reproducing retained flat manifests expands trees: its
@@ -533,20 +548,21 @@ publication job is bounded to 20 minutes. `verification.json` records the actual
checks. Failed candidates remain unpublished.
After successful candidate validation, install only `support-library.arboricx`,
-`support-rebuilt.arboricx`, `support-library.v1`, and `support-receipts/`, preserving
-the immutable origin files. Capture the matching reference sources, then run
-`python3 test/fixture_sources.py current` and `nix flake check` before accepting
-the fixture.
-Never change compiler identities or original snapshots merely to clear a failure.
-Backend-sensitive tree-shape goldens must be labeled honestly and checked
-independently, not treated as extensional equivalence claims.
+`support-rebuilt.arboricx`, `support-library.v1`, and `support-receipts/`,
+preserving the immutable origin files. Capture the matching reference sources,
+then run `python3 test/fixture_sources.py current` and `nix flake check` before
+accepting the fixture. Preserve actual compiler identities and original
+snapshots when repairing failures. Label backend-sensitive tree-shape goldens by
+their backend and check them independently. Extensional equivalence requires
+separate evidence.
## SCP: archived experiment
-SCP sources, commands, tests, demos, and the mixed workspace dispatcher are under
-[`archive/scp/`](../archive/scp/README.md.html). They are not installed, tested or
-required by active tricu builds. The former SCP Nix targets have been removed.
-Historical commands in [the SCP reference](../archive/scp/docs/usage.md.html) require a historical checkout.
+SCP sources, commands, tests, demos, and the mixed workspace dispatcher are
+under [`archive/scp/`](../archive/scp/README.md.html) as historical material.
+Active builds and tests use the native module tooling. Historical commands in
+[the SCP reference](../archive/scp/docs/usage.md.html) require a historical
+checkout.
The original fixture projection is recorded in `test/fixtures/support-projection.json`.
Its immutable origin fingerprints remain checked inputs; rebuilt outputs are
@@ -577,37 +593,39 @@ artifacts. `verified_tree_loading.py` covers verified transport and bounded
loading/publication measurements, outside the default checks.
Header-only equivalence uses the explicit `--native-header-equivalence` option
-of `refresh_compiler_fixture.py`. It authenticates exact staged bytes and previous
-receipts; it does not accept body/dependency changes or establish a new generation.
-For a support change, rebuild only the justified affected cohort and prove reuse
-for the rest; never rewrite a compiler pin to avoid a genuine dependency change.
-Archive creation consumes existing accepted artifacts and does not require a new
-fixture migration simply because another snapshot is being made.
-
-`refresh_compiler_fixture.py` and `refresh_support_fixture.py` are publication
-utilities, not tests. `refresh_compiler_fixture.py --output-directory DIR` stages
-a compiler candidate without replacing the reference cohort, allowing pre-
+of `refresh_compiler_fixture.py`. It authenticates exact staged bytes and
+previous receipts and requires unchanged bodies and dependencies. It records
+header equivalence; new generations follow the generation runner workflow. For a
+support change, rebuild only the justified affected cohort and prove reuse for
+the rest. Record actual compiler pins and rebuild consumers whose dependency
+identities changed. Archive creation consumes existing accepted artifacts and
+their matching fixtures.
+
+`refresh_compiler_fixture.py` and `refresh_support_fixture.py` publish artifacts
+after validation. `refresh_compiler_fixture.py --output-directory DIR` stages a
+compiler candidate without replacing the reference cohort, allowing pre-
publication comparisons. Publish the compiler before the final support publisher
run: support metadata binds the compiler fixture present during publication.
-They authenticate completed preparation receipts, current
-source fingerprints, full dependency manifests, and the packed fixture. Explicit
-`--completed-only` / `--replace ALIAS` are reviewable policies, not silent failure
-recovery. Streaming publication retains tighter 600-second / 1 GiB RSS / 512 MiB
-managed-heap guards.
-
-`native_library_migration.py` prepares explicit candidates, preserving definition
-bodies and dependency/import policy. Bare terminal-selector conversion and
-import-only reexports require explicit flags; arbitrary expressions and namespace
-widening fail. Candidate jobs / batches default to 20 minutes; an explicitly selected full cohort may use a two-hour batch ceiling.
-`--memory-mib` sets both the sampled direct-child RSS guard and host heap ceiling,
-up to 64 GiB. Full dependency manifests are mandatory; the helper does not modify
-source or publish fixtures.
+They authenticate completed preparation receipts, current source fingerprints,
+full dependency manifests, and the packed fixture. Explicit `--completed-only` /
+`--replace ALIAS` select explicit, reviewable publication policies; failed
+preparation requires separate repair. Streaming publication retains tighter
+600-second / 1 GiB RSS / 512 MiB managed-heap guards.
+
+`native_library_migration.py` prepares explicit candidates, preserving
+definition bodies and dependency/import policy. Bare terminal-selector
+conversion and import-only reexports require explicit flags; arbitrary
+expressions and namespace widening fail. Candidate jobs / batches default to 20
+minutes; an explicitly selected full cohort may use a two-hour batch ceiling.
+`--memory-mib` sets both the sampled direct-child RSS guard and host heap
+ceiling, up to 64 GiB. Full dependency manifests are mandatory; authored source
+and published fixtures remain unchanged.
## Research and costs
Native math checks, formatter tooling, stable abstraction/bundles, and staging
experiments remain opt-in; use their retained compiler and support artifacts.
-Do not cold-compile current compiler libraries with a legacy host frontend.
+Compile current compiler libraries through the selected portable toolchain.
```sh
python3 test/staging_experiment.py result/bin/tricu \
@@ -623,15 +641,16 @@ python3 test/staged_conditional.py result/bin/tricu \
```
Both research harnesses cap whole runs and children at 1200 seconds. Prepared
-fixture selections reuse authenticated trees; generated case data is batched,
-not compiled once per export. Explicit scalar fixture data supplies Unicode
+fixture selections reuse authenticated trees; generated case data is compiled
+in batches. Explicit scalar fixture data supplies Unicode
codepoints directly; native source-file loading remains restricted to ASCII.
Run formatter integration separately.
Report preparation, loading, compilation, publication, allocation, RSS, logical
-size and DAG size separately. Direct-child RSS is not a descendant total.
-Guard stops / missing measurements remain failures, not successful coverage.
-Research-profile contracts are in [Compiler architecture](compiler.md.html#experimental-staging-profiles).
+size and DAG size separately. Direct-child RSS covers only the child process;
+measure descendants separately. Guard stops and missing measurements are
+failures. Research-profile contracts are in [Compiler
+architecture](compiler.md.html#experimental-staging-profiles).
## Local-control candidates (opt-in)
@@ -640,7 +659,7 @@ Select genuinely rebuilt `source_unit` and `source_program` candidate bundles,
plus a stable-prepared `tests.localControlEffects` driver and fresh output
directories. The driver depends on the existing `tests.httpEffects` interpreter;
prepare it through the native library preparation tool with that dependency
-selected explicitly. No test driver enters the production support cohort.
+selected explicitly. Keep test drivers in a separate test-only preparation.
```sh
python3 test/local_control.py "$TRICU" --unit "$UNIT" --program "$PROGRAM" \
@@ -651,13 +670,14 @@ python3 test/local_control_consumers.py "$TRICU" --unit "$UNIT" \
The first check compares complete located AST replies, complete diagnostic
records, independent values, and a wrong-compiler negative control. Compiler
-private diagnostic remainders are retained but not given a language-level oracle.
-The second compiles and serializes small HTTP/registry replacement functions
-before invoking them, compares both candidate and authenticated retained kernels
-to independent complete values/effect traces, and records expanded wire-tree
-sizes and guarded whole-process costs. Authenticate driver source, receipts, and
-dependencies during preparation. The default Nix check runs cheap harness controls;
-run these compilations explicitly.
+private diagnostic remainders are retained as implementation-level evidence;
+language-level oracles cover the public diagnostics. The second compiles and
+serializes small HTTP/registry replacement functions before invoking them,
+compares both candidate and authenticated retained kernels to independent
+complete values/effect traces, and records expanded wire-tree sizes and guarded
+whole-process costs. Authenticate driver source, receipts, and dependencies
+during preparation. The default Nix check runs cheap harness controls; run these
+compilations explicitly.
## Arboricx reader rewrites (opt-in)
@@ -681,8 +701,8 @@ identity before refreshing fixtures.
continuation and supports `do` directly. The
full `readManifest` entrypoint historically returns unit in both its extension
slot and successful remainder, even with trailing input. This is explicitly
-characterized separately from the remainder-preserving field readers, not
-silently corrected as part of a readability rewrite.
+characterized separately from the remainder-preserving field readers. The
+current suffix-preserving behavior is covered by `bundle_batch.py` below.
## Socket convenience cleanup (opt-in)
@@ -692,12 +712,13 @@ python3 test/socket_conveniences.py "$TRICU" \
--driver "$REGISTRY_EFFECT_DRIVER" --directory "$FRESH_PROBE_DIRECTORY"
```
-Checks complete primitive payloads/order and exact Result remainders. The retained
-reference predates cleanup: successful acquisition followed by failed bind, listen
-or connect intentionally adds one close; the original error survives close failure.
-Acquisition failure adds no close; success transfers ownership. Terminal aborts
-and host exceptions are not ordinary Result cleanup paths. The scripted test does
-not open sockets. The Haskell gate separately exercises a real ephemeral listener
+Checks complete primitive payloads/order and exact Result remainders. The
+retained reference predates cleanup: successful acquisition followed by failed
+bind, listen or connect intentionally adds one close; the original error
+survives close failure. Acquisition failure returns directly; success transfers
+ownership. Terminal aborts and host exceptions follow host lifetime handling,
+separately from ordinary Result cleanup. The scripted test records socket
+actions as data. The Haskell gate separately exercises a real ephemeral listener
and successful loopback connection, plus exact scripted cleanup paths.
## Parser control rewrites (opt-in)
@@ -708,16 +729,17 @@ python3 test/parser_control.py "$TRICU" \
--driver "$PARSER_CONTROL_DRIVER" --directory "$FRESH_PROBE_DIRECTORY"
```
-Compile `test/parserControl.tri` explicitly through the stable portable toolchain.
-Its higher-order wrappers receive the selected compiled parser export *after*
-wrapper construction. Independent full replies cover repetition, zero-width
-success, failure progress, bind/lexer flag propagation, and non-Leaf truthiness.
-Divergent ignored fields and unselected handlers must remain unforced. A
-same-arity wrong-export control uses `parserOptional` in place of `parserMany`.
-Record/authenticate driver source, receipts, compiler/options, dependencies and
-complete manifest separately. Parser edits are compiler-affecting: these focused
-checks do not replace the fresh stable generation/self-rebuild and publication
-requirements in [compiler rebuilding](compiler-rebuilding.md.html).
+Compile `test/parserControl.tri` explicitly through the stable portable
+toolchain. Its higher-order wrappers receive the selected compiled parser export
+*after* wrapper construction. Independent full replies cover repetition,
+zero-width success, failure progress, bind/lexer flag propagation, and non-Leaf
+truthiness. Divergent ignored fields and unselected handlers must remain
+unforced. A same-arity wrong-export control uses `parserOptional` in place of
+`parserMany`. Record/authenticate driver source, receipts, compiler/options,
+dependencies and complete manifest separately. Parser edits are
+compiler-affecting: acceptance also requires the fresh stable
+generation/self-rebuild and publication steps in [compiler
+rebuilding](compiler-rebuilding.md.html).
## Foundational list workload comparisons (opt-in)
@@ -743,17 +765,19 @@ cases, ignored-field demand, and a wrong-subject control. Its whole-job budget i
The second compiles complete reference/current base modules and linked drivers,
then checks independent edge/demand/effect cases and a smaller workload screen.
Its whole-job budget is five minutes, compilation 60 seconds, and ordinary calls
-20 seconds. Effects are interpreted interaction trees, not network traffic.
+20 seconds. Effects are interpreted interaction trees with scripted replies.
Use fresh processes, interleaved variants, discarded warmups, and recorded raw
-samples. `arboricx run --repeat` can reuse an evaluated application and is not a
-fresh-invocation benchmark. Loading-to-WHNF, invocation-through-rendered-output,
-and explicit full-result forcing measure different work; the latter two include
-deferred evaluation and their respective observer costs. See the
-[profile phase definitions](tri-tools.md.html#profiling); ordinary `--profile`
-no longer forces discarded result fields. Report preparation separately. Sampled RSS can miss short-lived peaks, and small
-timing differences do not establish general speedups. Partial matrices remain
-incomplete. These probes neither generate compilers nor publish fixtures.
+samples. `arboricx run --repeat` can reuse an evaluated application; use a fresh
+process for each independent invocation. Loading-to-WHNF,
+invocation-through-rendered-output, and explicit full-result forcing measure
+different work; the latter two include deferred evaluation and their respective
+observer costs. See the [profile phase
+definitions](tri-tools.md.html#profiling); ordinary `--profile` leaves discarded
+result fields lazy. Report preparation separately. Sampled RSS can miss
+short-lived peaks, and small timing differences require broader evidence before
+generalizing. Partial matrices remain incomplete. These probes leave accepted
+compiler and support fixtures unchanged.
### Pattern lowering comparison (opt-in)
@@ -764,10 +788,11 @@ python3 test/pattern_lowering.py result/bin/tricu \
```
The probe authenticates the shipped compiler and complete unchanged reference
-modules. It stages the source lowerer and rebuilds its affected
-compiler consumers through `source_unit`; receipts, source bytes, exported objects,
-and exact dependency pins are audited before execution. Only unchanged dependency
-boundaries use full retained-module facades. It never replaces accepted artifacts.
+modules. It stages the source lowerer and rebuilds its affected compiler
+consumers through `source_unit`; receipts, source bytes, exported objects, and
+exact dependency pins are audited before execution. Only unchanged dependency
+boundaries use full retained-module facades. Accepted artifacts remain
+unchanged.
The Haskell control driver is compiled through Nix. Checks reuse the 75-case
independent equation corpus, exercise constructor/equality workloads on finite
@@ -782,25 +807,27 @@ local-control suite also runs on the candidate.
Select an authenticated retained `source_unit` bundle with unqualified exports
for `--reference-unit`; its digest is recorded separately from the shipped
-compiler used to construct the candidate. `--prepare-worker` additionally rebuilds
-all affected consumers through `source_map` and emits a pruned stable transition
-`worker.arboricx`. This is candidate preparation, not a full generation or
-fixed-point acceptance. `--resume` permits diagnostic checkpoint reuse only.
+compiler used to construct the candidate. `--prepare-worker` additionally
+rebuilds all affected consumers through `source_map` and emits a pruned stable
+transition `worker.arboricx` for subsequent generation acceptance. `--resume`
+permits diagnostic checkpoint reuse only.
Fresh-process cost probes interleave reference/candidate executions, discard a
warmup, and retain three samples per role. Identical explicit recursion and full
output oracles isolate dispatch costs. Separate small compiler probes force the
complete returned definition trees. Both use explicit full-result profiling:
reduction plus DAG traversal, excluding load and rendering. Allocation is
-GC-sampled, not peak residency or a reduction counter. Reports retain raw profiles,
-source/compiler/artifact identities, receipts, controls, and medians.
-
-Limits: 20 minutes overall, 600 seconds for native compilation with an 8 GiB heap,
-180 seconds / 4 GiB for the Haskell controls, and 20-second / 512 MiB workload
-calls (30 seconds / 1 GiB for construction probes). `--resume` authenticates native
-checkpoints and preserves the previous report; it is diagnostic reuse, not a fresh
-generation. `--skip-costs` retains semantic/integrity controls without timing runs.
-These checks do not replace the compiler generation/self-rebuild acceptance gate.
+GC-sampled; peak residency and reduction counts require separate measurements.
+Reports retain raw profiles, source/compiler/artifact identities, receipts,
+controls, and medians.
+
+Limits: 20 minutes overall, 600 seconds for native compilation with an 8 GiB
+heap, 180 seconds / 4 GiB for the Haskell controls, and 20-second / 512 MiB
+workload calls (30 seconds / 1 GiB for construction probes). `--resume`
+authenticates native checkpoints and preserves the previous report for
+diagnostic reuse. `--skip-costs` retains semantic/integrity controls without
+timing runs. Compiler acceptance additionally requires a fresh
+generation/self-rebuild pair.
### Sharing characterization (opt-in)
@@ -810,32 +837,33 @@ python3 test/sharing_probes.py result/bin/tricu \
```
Use a fresh directory. The probe compiles tiny import-free programs with the
-shipped stable compiler and runs only the Haskell host. It compares a bound shared
-computation with two separately written calls, checks their complete small outputs
-against an independent oracle, and compares a shared DAG with itself using an
-ordinary TC structural comparator (not Haskell `Eq` or an optimized equality
-primitive). Input DAGs contain depth+1 physical nodes and 2^(depth+1)-1 logical
-occurrences. It also checks that ordinary profiling ignores a divergent Result
-remainder while explicit full-result forcing hits a resource bound.
+shipped stable compiler and runs only the Haskell host. It compares a bound
+shared computation with two separately written calls, checks their complete
+small outputs against an independent oracle, and compares a shared DAG with
+itself using an ordinary TC structural comparator. Input DAGs contain depth+1
+physical nodes and 2^(depth+1)-1 logical occurrences. It also checks that
+ordinary profiling ignores a divergent Result remainder while explicit
+full-result forcing hits a resource bound.
Runs use fresh processes, three samples by default, a 512 MiB child heap cap,
20-second commands (three seconds for the expected noncompletion), and a
-180-second whole-job budget. `report.json` retains commands, source/artifact hashes,
-manifest pins, raw profiles, and medians. The measured interval is `force-begin`
-through `force-end`: reduction plus sharing-aware traversal, excluding loading
-and rendering. Allocation counters are GC-sampled; they are not reduction counts
-or peak residency. The probes characterize these examples, not universal binding
-sharing, eager compatibility, or a general compiler speedup. They change no
-published compiler/support artifacts.
+180-second whole-job budget. `report.json` retains commands, source/artifact
+hashes, manifest pins, raw profiles, and medians. The measured interval is
+`force-begin` through `force-end`: reduction plus sharing-aware traversal,
+excluding loading and rendering. Allocation counters are GC-sampled; reduction
+counts and peak residency require separate measurements. Results characterize
+these particular examples. Universal binding sharing, eager compatibility, and
+general compiler speedups require broader evidence. Published compiler/support
+artifacts remain unchanged.
## Collections, contracts, and ordinary cleanup candidates (opt-in)
-The three library batch runners below require `REFERENCE_FIXTURES`: authenticated
-**pre-rewrite** fixtures, including their reference-source snapshots, selected
-from frozen `1f24085:test/fixtures` (or the identical foundations publication).
-Do not point historical correction/API-delta controls at the newly published
-fixtures. Sources under test still come from the current checkout; the reference
-flag selects only historical controls and cost/API comparisons.
+The three library batch runners below require `REFERENCE_FIXTURES`:
+authenticated **pre-rewrite** fixtures, including their reference-source
+snapshots, selected from frozen `1f24085:test/fixtures` (or the identical
+foundations publication). Historical correction/API-delta controls require those
+pre-rewrite fixtures. Sources under test still come from the current checkout;
+the reference flag selects only historical controls and cost/API comparisons.
```sh
python3 test/collections_batch.py "$TRICU" \
@@ -915,25 +943,27 @@ recorded sources and published executable tree, without generating a compiler.
### Module/object candidate (opt-in)
-`test/source_modules.py "$TRICU" --compiler "$STABLE_COMPILER"`
-`--seed "$LOWERING_CANDIDATE_DIRECTORY" --directory "$NEW_DIRECTORY"` exercises
-current codecs, native headers, demand, and scripted complete IO traces. The seed
-is optional, explicit, and reauthenticated. The shared `compiler_candidate.py` links immutable CAS objects only, copies mutable
-checkpoints and reaudits all staged current sources/receipts/exports after the
-build. Existing directories require `--resume`. Limits remain 1500s overall,
-900s per compilation and 65536 MiB sampled child/heap. It never modifies the seed.
+`test/source_modules.py "$TRICU" --compiler "$STABLE_COMPILER"` `--seed
+"$LOWERING_CANDIDATE_DIRECTORY" --directory "$NEW_DIRECTORY"` exercises current
+codecs, native headers, demand, and scripted complete IO traces. The seed is
+optional, explicit, and reauthenticated. The shared `compiler_candidate.py`
+links immutable CAS objects only, copies mutable checkpoints and reaudits all
+staged current sources/receipts/exports after the build. Existing directories
+require `--resume`. Limits remain 1500s overall, 900s per compilation and 65536
+MiB sampled child/heap. The seed remains unchanged.
### Build/publication candidates (opt-in)
-`test/source_building.py "$TRICU" --compiler "$STABLE_COMPILER"`
-`--seed "$MODULE_CANDIDATE_DIRECTORY" --directory "$NEW_DIRECTORY"` checks complete
-bundle bytes against a Python postorder-interning model, integer field boundaries,
-lock rows, source maps, receipts, and empty/memo/cycle demand. It emits an audited
-`source_map-selected.arboricx` exposing both explicit backends. Use it as the
-`--worker-bundle` for `test/bootstrap_generation_cli.py` with a verified previous
-generation, selected standard adapter and `--backend stable`; this separately
-checks live publication, invalidation, interruption, corruption, and no fallback.
+`test/source_building.py "$TRICU" --compiler "$STABLE_COMPILER"` `--seed
+"$MODULE_CANDIDATE_DIRECTORY" --directory "$NEW_DIRECTORY"` checks complete
+bundle bytes against a Python postorder-interning model, integer field
+boundaries, lock rows, source maps, receipts, and empty/memo/cycle demand. It
+emits an audited `source_map-selected.arboricx` exposing both explicit backends.
+Use it as the `--worker-bundle` for `test/bootstrap_generation_cli.py` with a
+verified previous generation, selected standard adapter and `--backend stable`;
+this separately checks live publication, invalidation, interruption, corruption,
+and failure of unavailable compiler selections.
`test/source_export_cache.py` uses the same options, with the build candidate as
seed, and checks complete scripted cache-hit/miss/storage/error/selection traces.
@@ -956,11 +986,12 @@ done
The core group covers 20 construction/application/diagnostic/demand cases; the
sequence group covers eight slice/repetition cases. Whole-job and sampled child
RSS guards remain active. A single unsharded run can exceed the whole-job budget
-because each distinct program is separately published; use shards, not a larger
-limit. `complete.json` binds selected/completed names and compiler/harness hashes.
-Unicode sequence fixtures contain explicit scalar cells: this tests codepoint
-operations, not UTF-8 native-source transport. Bounded expected divergence is
-not a proof of nontermination.
+because each distinct program is separately published; shard the corpus while
+keeping the limit fixed. `complete.json` binds selected/completed names and
+compiler/harness hashes. Unicode sequence fixtures contain explicit scalar cells
+and test codepoint operations. UTF-8 native-source transport has separate
+coverage below. Bounded expected divergence records noncompletion within the
+selected budget; proving nontermination requires separate analysis.
### Native source UTF-8 boundary (opt-in focused probe)
@@ -974,23 +1005,24 @@ support closure. Use a Nix-built Haskell host and a new output directory. It
compiles current read/stage and map consumers with that retained compiler and
authenticated support exports. The published frontend is checked separately by
`test/default_frontend.py`, including Unicode source, raw-byte receipts, stdin,
-and malformed/BOM rejection. It checks exact input/overlay octets,
-source-domain fingerprints, ASCII parity, multibyte strings/comments, scalar
-coordinates, and malformed UTF-8/BOM rejection. Host unit tests separately cover
-binary IO parity, permissions and invalid encoder scalars. Existing scalar-list
-fixtures do not establish file decoding. Limits are 8192 MiB, 900 seconds for
-compilation, 180 seconds per smoke and 1500 seconds for the whole job. This is
-not a generation or fixture-publication gate.
+and malformed/BOM rejection. It checks exact input/overlay octets, source-domain
+fingerprints, ASCII parity, multibyte strings/comments, scalar coordinates, and
+malformed UTF-8/BOM rejection. Host unit tests separately cover binary IO
+parity, permissions and invalid encoder scalars. Existing scalar-list fixtures
+cover scalar operations; file decoding requires the byte-level cases above.
+Limits are 8192 MiB, 900 seconds for compilation, 180 seconds per smoke and 1500
+seconds for the whole job. Generation acceptance and fixture publication follow
+their separate workflows.
## Published-library consumer (opt-in)
Builds three curated libraries through the shipped stable compiler, publishes
exact releases in a new **local** registry directory, fetches them into an empty
-consumer CAS over a controlled GET peer using one project declaration/lock, closes
-that server, and binds/checks/compiles
-`demos/package-inspector/main.tri` offline. No producer source or CAS is copied
-into the consumer workspace. The native registry route itself is covered by the
-separate package-registry checks above.
+consumer CAS over a controlled GET peer using one project declaration/lock,
+closes that server, and binds/checks/compiles `demos/package-inspector/main.tri`
+offline. The consumer receives library inputs exclusively through fetched
+packages. The native registry route itself is covered by the separate
+package-registry checks above.
```sh
nix build .#
@@ -1001,72 +1033,78 @@ nix shell --inputs-from . nixpkgs#python3 -c python3 test/library_inspector.py \
--directory "$WORK/inspection"
```
-`inspectorOracle` is a test-only Haskell executable using `Wire.decodeBundle` and
-`verifyBundle`; it never calls the compiled inspector or evaluates inspected
-exports. Complete reports must match, including whole-file hash, record count,
-export order, root indices, kind/ABI and percent escaping. Fixtures cover invalid
-headers, truncation, malformed manifests, invalid unselected roots/ABIs, duplicate
-exports/nodes, forward edges, terminal-control/UTF-8 names, and 41 stored nodes
-representing more than a trillion logical occurrences. An IO export that aborts
-if executed is inspected successfully. The published libraries and inspector
-itself are inputs too. Missing files and denied reads must fail nonzero.
+`inspectorOracle` is a test-only Haskell executable using `Wire.decodeBundle`
+and `verifyBundle` to inspect export records as data independently of the
+compiled inspector. Complete reports must match, including whole-file hash,
+record count, export order, root indices, kind/ABI and percent escaping.
+Fixtures cover invalid headers, truncation, malformed manifests, invalid
+unselected roots/ABIs, duplicate exports/nodes, forward edges,
+terminal-control/UTF-8 names, and 41 stored nodes representing more than a
+trillion logical occurrences. An IO export that aborts if executed is inspected
+successfully. The published libraries and inspector itself are inputs too.
+Missing files and denied reads must fail nonzero.
On the validation host, the full 18-input publication/consumer check took about
-five minutes: offline consumer compilation was about 1.5 seconds, while inspecting
-the roughly 221 KiB inspection library or consumer bundle took about a minute
-each. These are observations, not performance guarantees. The in-memory portable
-parser is significantly more expensive than the native Haskell report oracle.
+five minutes: offline consumer compilation was about 1.5 seconds, while
+inspecting the roughly 221 KiB inspection library or consumer bundle took about
+a minute each. These timings describe that validation host and workload; measure
+other environments independently. The in-memory portable parser is significantly
+more expensive than the native Haskell report oracle.
It then publishes a **local test-only** foundation revision with one additional
export, changes only the `foundation` selection, and explicitly updates that one
-binding. The other two lock rows must stay byte-identical. Stale checks and binding
-before fetch fail. After fetching, the registry is stopped again and the unchanged
-consumer is rebound, checked and recompiled offline; its output must still match
-the independent oracle. The update diff and resulting inspector identity are
-recorded; this test release is not added to the site's public index. With the
-registry still closed, `project explain` must report verified local evidence and
-`project diff` must identify the added `revision` export and structurally identical
-common foundation exports. Both human-readable reports are retained in the JSON.
-
-The preparation retains artifacts, the initial/current project locks and generated
-bindings/ownership state, separate producer and
-consumer stores, compiler/source hashes, command logs and outcomes in
-`report.json`. This is local validation evidence, not signed/distributed provenance.
-Current bounds: 8 GiB GHC heap per tricu child, 900-second compile deadline,
-180-second inspection deadline and 2400-second overall command budget. Heap bounds
-are not total process RSS bounds. It remains opt-in; default checks exercise cheap
-CLI `--no-result` controls and site-staging immutability without recompilation.
+binding. The other two lock rows must stay byte-identical. Stale checks and
+binding before fetch fail. After fetching, the registry is stopped again and the
+unchanged consumer is rebound, checked and recompiled offline; its output must
+still match the independent oracle. The update diff and resulting inspector
+identity are recorded; the test release stays confined to the local registry.
+With the registry still closed, `project explain` must report verified local
+evidence and `project diff` must identify the added `revision` export and
+structurally identical common foundation exports. Both human-readable reports
+are retained in the JSON.
+
+The preparation retains artifacts, the initial/current project locks and
+generated bindings/ownership state, separate producer and consumer stores,
+compiler/source hashes, command logs and outcomes in `report.json` as local
+validation evidence. Signing and distributing provenance requires a separate
+workflow. Current bounds: 8 GiB GHC heap per tricu child, 900-second compile
+deadline, 180-second inspection deadline and 2400-second overall command budget.
+Heap bounds cover managed heap; total process RSS requires separate controls. It
+remains opt-in; default checks exercise cheap CLI `--no-result` controls and
+site-staging immutability without recompilation.
`test/project_dependencies.py` runs in the default host-policy check with small
wire fixtures and a controlled GET peer. It covers frozen/no-network resolution,
scoped updates/additions/removals, late-resolution failure preserving the whole
-lock, registry equivocation (including renames), origin mismatch, missing/corrupt
-CAS objects, authored/edited/symlink binding collisions, journal recovery, removal
-of owned bindings only, input/output collisions and offline retries. No library
-compilation or public network access is involved; each CLI child has a 1 GiB GHC
-heap limit and a 30-second deadline. Haskell tests separately check the metadata
-codecs, canonicalization, name bounds and duplicate/conflicting selections.
+lock, registry equivocation (including renames), origin mismatch,
+missing/corrupt CAS objects, authored/edited/symlink binding collisions, journal
+recovery, removal of owned bindings only, input/output collisions and offline
+retries. Inputs are prebuilt wire fixtures and networking is confined to the
+controlled peer; each CLI child has a 1 GiB GHC heap limit and a 30-second
+deadline. Haskell tests separately check the metadata codecs, canonicalization,
+name bounds and duplicate/conflicting selections.
`test/project_inspection.py` is another default host-policy check. Independently
assembled wire/CAS fixtures prove ordered structural equality despite changed
physical numbering/object hashes, unequal trees, 81-node exponential DAGs,
unreachable records, interface/ABI/kind changes, terminal-safe metadata, and
-missing/corrupt/malformed evidence even under unchanged pins. Files and mtimes are
-snapshotted around every report; absent stores must remain absent. Each CLI child
-has a 512 MiB GHC heap guard and a 30-second deadline. Small Haskell comparisons
-also agree with independently evaluated constructor equality; no large DAG is
-unfolded to produce an expected answer. A focused byte-budget test confirms
+missing/corrupt/malformed evidence even under unchanged pins. Files and mtimes
+are snapshotted around every report; absent stores must remain absent. Each CLI
+child has a 512 MiB GHC heap guard and a 30-second deadline. Small Haskell
+comparisons also agree with independently evaluated constructor equality;
+large-DAG expected answers preserve sharing. A focused byte-budget test confirms
that repeated object references share inspection results and additional objects
become unavailable when the retained-byte allowance is exhausted.
-The default command never changes checked-in site files. To additionally stage
+The default command preserves checked-in site files. To additionally stage
accepted library releases for the existing static-site/dependency-pack pipeline,
-add `--stage-site ext/demo-site` to a fresh run. Staging authenticates the bundle
-bytes, preserves existing versions, refuses conflicts, writes bundles before
-index entries, and does not deploy or contact a public registry. `--version LABEL`
-selects a new exact release label for the library set; changing executable bytes
-under an existing name/version is rejected. Use an exclusively owned local site
-workspace; staging is not a concurrent index-merging protocol.
+add `--stage-site ext/demo-site` to a fresh run. Staging authenticates the
+bundle bytes, preserves existing versions, refuses conflicts, writes bundles
+before index entries, and leaves public deployment to a separately authorized
+operation. `--version LABEL` selects a new exact release label for the library
+set; changing executable bytes under an existing name/version is rejected. Use
+an exclusively owned local site workspace and serialize staging operations to
+preserve index consistency.
## Retained provenance (opt-in)
@@ -1085,33 +1123,36 @@ nix shell --inputs-from . nixpkgs#python3 -c python3 test/provenance.py \
The harness copies producer workspaces/stores to a disposable location, captures
four self-contained input packs, **deletes that producer copy**, and rebuilds in
fresh directories from the packs only. It runs outside the checkout, with the
-packaged compiler data path deliberately unavailable. No registry is started.
+packaged compiler data path deliberately unavailable and the registry offline.
Each native module and compiler-returned receipt must reproduce exactly. The
initial validation host took about 3.5 minutes total: roughly 31/35/139 seconds
for foundation/file IO/inspection libraries and 2 seconds for the inspector.
-These are observations, not performance guarantees.
+These timings describe that validation host and workload; measure other
+environments independently.
An additional genuine build adds an ignored source comment: source, receipt and
pack identities must change while the executable stays identical. That history
must also reproduce after its disposable producer is deleted.
A separate negative fixture changes the inspector's source and native receipt to
-claim the old output, authenticates every changed object, and includes a matching
-poisoned producer cache. Byte/receipt verification must succeed as **claims only**;
-a fresh rebuild must ignore that cache, produce different code, and report a
-mismatch. This guards the distinction between integrity and actual reproduction.
-Artifacts, pack pins, commands, timings and outcomes are retained in `report.json`.
-Children use an 8 GiB GHC heap ceiling, 900-second compiler deadline and bounded
-subprocess deadlines within a 2400-second overall budget; these are not RSS bounds.
-
-The default `test/provenance.py` run (wired into `nix flake check`) does not compile
-source. Independent wire/CAS/native-receipt fixtures cover no-execution verification
-with a non-compiler tree, stale source refusal, explicit receipt selection without
-a cache, immutable capture destinations, incorrect pins, missing/corrupt sources,
-options, receipts and executables, malformed inventories, path traversal,
-symlinked CAS directories, an unauthorized compiler, and verification after all
-producer files have been removed. Verification snapshots assert no file/mtime
-changes. Each child has a 512 MiB GHC heap ceiling and a 30-second deadline.
+claim the old output, authenticates every changed object, and includes a
+matching poisoned producer cache. Byte/receipt verification must succeed as
+**claims only**; a fresh rebuild must ignore that cache, produce different code,
+and report a mismatch. This guards the distinction between integrity and actual
+reproduction. Artifacts, pack pins, commands, timings and outcomes are retained
+in `report.json`. Children use an 8 GiB GHC heap ceiling, 900-second compiler
+deadline and bounded subprocess deadlines within a 2400-second overall budget.
+Total RSS requires separate controls.
+
+The default `test/provenance.py` run (wired into `nix flake check`) uses
+independent wire/CAS/native-receipt fixtures. These cover no-execution
+verification with a non-compiler tree, stale source refusal, explicit receipt
+selection without a cache, immutable capture destinations, incorrect pins,
+missing/corrupt sources, options, receipts and executables, malformed
+inventories, path traversal, symlinked CAS directories, an unauthorized
+compiler, and verification after all producer files have been removed.
+Verification snapshots require unchanged files and mtimes. Each child has a 512
+MiB GHC heap ceiling and a 30-second deadline.
## CLI workspace source locations
@@ -1137,24 +1178,25 @@ exercises its server behavior.
This check rebuilds `source_stable` and `source_unit` from the working source
with the packaged compiler, authenticating unchanged fixture dependencies. It
-checks the source implementation, not just the installed compiler's behavior:
+checks the rebuilt source implementation using independent expectations:
```sh
nix develop -c python3 test/stable_constructor_k.py result/bin/tricu \
--directory "$WORK/constructor-k"
```
-Use a fresh output directory. GHC and its dependencies are selected through
-Nix. The harness adds probe-only exports in its disposable workspace; cached
-residual helpers are not exported by the production module. The independent
-Haskell oracle uses full free-variable sets rather than the implementation's
-minimum-index summary. Checks cover successive annotated abstractions, sparse
-indices, constructor and unsafe-call classification, complete application
-results against a no-shortcut baseline, opaque arguments, closed-divergence
-construction, and diagnostic/remainder preservation. It also runs the local
-control corpus against the rebuilt unit compiler and audits native dependency
-pins and receipts. Children have explicit heap and time bounds.
-
-These finite controls are not a general termination or demand-equivalence
-proof. Compiler publication additionally requires the full reproduction,
-cross-host checks, and retention workflow in [compiler rebuilding](compiler-rebuilding.md.html).
+Use a fresh output directory. GHC and its dependencies are selected through Nix.
+The harness adds probe-only exports in its disposable workspace; cached residual
+helpers remain private in production. The independent Haskell oracle uses full
+free-variable sets; the implementation uses a minimum-index summary. Checks
+cover successive annotated abstractions, sparse indices, constructor and
+unsafe-call classification, complete application results against a no-shortcut
+baseline, opaque arguments, closed-divergence construction, and
+diagnostic/remainder preservation. It also runs the local control corpus against
+the rebuilt unit compiler and audits native dependency pins and receipts.
+Children have explicit heap and time bounds.
+
+These finite controls establish behavior for the tested cases. General
+termination and demand equivalence require separate proof. Compiler publication
+additionally requires the full reproduction, cross-host checks, and retention
+workflow in [compiler rebuilding](compiler-rebuilding.md.html).