commit 5058a648a94004651804ac9c8cda0795f9b25f66
parent d2a35355cce35f384ebb7a13b595252a2b6bbebb
Author: James Eversole <james@eversole.co>
Date: Fri, 16 Oct 2026 14:23:30 -0500
Polish runtime and cross-host compiler documentation for publication
Describe current behavior, usage, resource limits and validation directly. Remove progress framing, defensive scope disclaimers and speculative architecture commentary; retain ownership, permission and resource requirements. Apply the same wording discipline to CLI help and measurement descriptions.
Diffstat:
9 files changed, 220 insertions(+), 260 deletions(-)
diff --git a/docs/build-and-run.md b/docs/build-and-run.md
@@ -2,9 +2,10 @@
Run these commands from the extracted tricu directory. You need [Nix](https://nixos.org/download/)
with flakes enabled. Native development uses the Haskell host; builds and tests
-run through Nix. The experimental [Zig runtime and owned-handle C ABI](../ext/zig/README.md)
-have separate opt-in Nix validation targets; they are not compiler-toolchain acceptance.
-For that stronger check, use the [isolated cross-runtime reproduction experiment](compiler-rebuilding.md.html#isolated-cross-runtime-fixed-point-reproduction).
+run through Nix. The [Zig runtime and owned-handle C ABI](../ext/zig/README.md)
+have their own Nix validation targets. The
+[cross-runtime reproduction workflow](compiler-rebuilding.md.html#isolated-cross-runtime-fixed-point-reproduction)
+verifies identical compiler generations on Haskell and Zig.
## Build the host
diff --git a/docs/compiler-rebuilding.md b/docs/compiler-rebuilding.md
@@ -57,35 +57,33 @@ uses `sourceMapBuildStable` or `sourceMapBuild`, while application selection use
the chosen application bundle. Neither path silently substitutes another backend
or the recovery frontend.
-### Experimental execution-host selection
+### Execution-host selection
`--execution-host zig --zig-library /nix/store/.../lib/libarboricx.so` selects
-Zig execution through the owned ABI. `--tricu` still selects the Haskell runtime
-for authenticated input transport and output packaging; compilation and
-publication execute only on Zig. The default execution host remains Haskell.
-The lowering `--backend` is an independent selection and is never inferred from
-the execution host. Missing libraries/exports fail without backend fallback.
-
-Provenance records the execution runtime and transport runtime separately.
-Historical records lacking `execution_host` describe Haskell execution. Zig's
-recorded runtime is the shared library, not the Python execution launcher; the
-launcher is separately fingerprinted. The normal shared supervisor retains source
-consistency checks, independent store audits, publication deadlines, whole-job
-limits, and incomplete status on failure. Zig additionally uses an address-space
-ceiling, CPU ceiling, and an 8 MiB native stack; it receives no GHC RTS flags.
-Both generation execution routes grant trusted compiler jobs broad file IO;
-these runners are not an untrusted-program filesystem sandbox.
-
-Bounded cross-host validation uses `test/cross_host_generation.py`, documented
-in the [Zig runtime guide](../ext/zig/README.md#small-compiler-probe). This validates
-the shared runner on small closures. Ordinary pair acceptance and retention
-toolchain roles remain Haskell-only. Cross-host reproduction has the separate
-verifier below; selecting Zig execution does not bypass acceptance or retention.
+Zig execution through the owned ABI. `--tricu` selects the Haskell runtime for
+authenticated input transport and output packaging. The selected execution host
+performs compilation and publication; the default is Haskell. `--backend`
+independently selects the lowering backend. Missing libraries or exports fail
+preparation.
+
+Provenance records the execution and transport runtimes separately. Historical
+records lacking `execution_host` describe Haskell execution. Zig records separate
+fingerprints for its shared library and Python launcher. The shared supervisor
+checks source consistency, independently audits the store, enforces publication
+and whole-job deadlines, and records completion status. Zig also applies an
+address-space ceiling, CPU ceiling, and an 8 MiB native stack.
+
+Both execution routes grant broad file IO to trusted compiler jobs. Use the
+isolated workflow below to restrict filesystem access. Small-closure validation
+uses `test/cross_host_generation.py`, documented in the
+[Zig runtime guide](../ext/zig/README.md#small-compiler-probe). Ordinary pair
+acceptance and retention toolchain roles use Haskell generations; the cross-host
+verifier compares Haskell and Zig generations.
### Isolated cross-runtime fixed-point reproduction
-The opt-in Linux experiment builds two Haskell generations, then forks the second
-compiler into a Haskell control and a Zig-executed third generation:
+The Linux workflow builds two Haskell generations, then uses the second compiler
+to build a Haskell control and a Zig-executed third generation:
```text
seed --Haskell--> H1 --Haskell--> H2 --Haskell--> H3
@@ -98,27 +96,26 @@ nix develop -c python3 test/bootstrap_cross_host.py \
--tricu "$TRICU" --zig-library "$ZIG/lib/libarboricx.so"
```
-Select exact Nix runtime paths. The output directory must not exist. The source
-commit supplies the frozen map, sources, kit and inner generation runner. Each job
-uses a fresh store in a filesystem sandbox without previous stores or the live
-checkout. Only the selected authenticated compiler/adapter kit crosses between
-jobs. Haskell performs common transport and packing; the selected execution host
-performs all compilation and publication. Jobs run sequentially with 32 GiB and
-one hour per job; Haskell uses eight capabilities. No budget increases or backend
-fallbacks are automatic. See the [isolated probe](../ext/zig/README.md#isolated-mapped-library-probe)
-for namespace requirements and bounded validation before a full run.
-
-`experiment.json` freezes source/archive hashes, mapped inputs, runtime/Python
+Select exact Nix runtime paths and a new output directory. The source commit
+supplies the frozen map, sources, kit and inner generation runner. Each job uses
+a fresh store in a filesystem sandbox containing its source snapshot, selected
+compiler/adapter kit, Nix dependencies and writable work directory. Haskell
+performs common transport and packing; the selected execution host performs
+compilation and publication. Jobs run sequentially with fixed limits of 32 GiB
+and one hour per job; Haskell uses eight capabilities. See the
+[isolated probe](../ext/zig/README.md#isolated-mapped-library-probe) for namespace
+requirements and bounded validation before a full run.
+
+`experiment.json` records source/archive hashes, mapped inputs, runtime/Python
identities, tooling and comparison boundaries. Logs, invocations and runtime
-provenance deliberately differ. The independent verifier requires ordinary
-H1/H2 and H2/H3 acceptance, verifies that Z3 used H2's exact compiler, and requires
-H2/H3/Z3 to match in complete object inventories, module checkpoints/receipts and
-all three packaged compiler artifacts. It does not trust recorded comparison
-booleans. `accepted-cross-host.json` records this reproduction result, not completed
-promotion or cleanup. `--verify-only` rechecks existing results without compilation.
+provenance identify each execution separately. The independent verifier checks
+H1/H2 and H2/H3 acceptance, verifies that Z3 used H2's exact compiler, and compares
+the actual H2/H3/Z3 object inventories, module checkpoints/receipts and all three
+packaged compiler artifacts. `accepted-cross-host.json` records the verified
+reproduction result. `--verify-only` rechecks existing artifacts and provenance.
-Post-generation validation includes the ordinary H2 smoke, then complete result
-observation on Zig:
+Post-generation validation runs the H2 program smoke and checks its complete
+results on Zig:
```sh
nix develop -c python3 test/bootstrap_smoke.py "$TRICU" \
@@ -132,17 +129,17 @@ nix develop -c python3 test/bootstrap_cross_host_controls.py "$EXPERIMENT" \
```
The program smoke covers native imports, recursive lists, and invoice totals and
-decimal text against independent complete trees. Its program compilation is
-Haskell-executed using the bit-identical accepted compiler; result execution is
-checked on both hosts. Negative controls mutate independent copies only and reject
-false host/runtime/freshness/compiler claims, corrupt objects, changed packaged
-artifacts and additional valid objects. Restored copies must pass again.
-
-Finish ordinary rolling retention with H2 as the current Haskell toolchain and an
-independently useful rollback. Explicitly protect H3/Z3, frozen inputs, cross-host
-reports, and a registered Nix root for the Zig runtime as additional evidence.
-This milestone demonstrates cross-runtime reproduction, not Haskell-free setup,
-transport, packaging, or distribution.
+decimal text against independent complete trees. Haskell executes the program's
+compilation using the bit-identical accepted compiler; both hosts execute the
+resulting program. Negative controls mutate independent copies and require the
+verifier to reject false host/runtime/freshness/compiler claims, corrupt objects,
+changed packaged artifacts and additional valid objects. Restored copies must
+pass again.
+
+Finish rolling retention with H2 as the current Haskell toolchain and an
+independently useful rollback. Protect H3/Z3, frozen inputs, cross-host reports,
+and a registered Nix root for the Zig runtime as additional evidence. Common
+preparation, transport and packaging use Haskell throughout this workflow.
### Explicit bootstrap kit
diff --git a/ext/zig/README.md b/ext/zig/README.md
@@ -1,7 +1,7 @@
# Zig runtime and C ABI
-This is the experimental Zig tree-calculus runtime. The supported compiler
-workflow remains the Haskell host. Zig runtime validation is opt-in:
+The Zig runtime evaluates tree-calculus programs and provides IO through an
+owned-handle C ABI. Build and validate it with Nix:
```sh
nix build .#tricu-zig .#tricu-zig-tests --no-link
@@ -9,22 +9,13 @@ nix build .#tricu-js-tests .#tricu-php-tests --no-link
```
The executable and shared library build in **ReleaseSafe**, with Zig runtime
-safety checks enabled. Reducer/collector and libuv scheduler-root tests run in
-both Debug and ReleaseSafe. These tests do not establish compiler self-hosting
-acceptance or a performance comparison with Haskell.
-
-Full compiler reproducibility uses the separate
-[isolated cross-runtime fixed-point experiment](../../docs/compiler-rebuilding.md#isolated-cross-runtime-fixed-point-reproduction).
-It verifies Haskell and Zig generations against each other with independent
-complete-object audits, post-generation smokes and explicit retention. Haskell
-still supplies common artifact transport and packaging; this is not a
-Haskell-free installation or distribution workflow.
-
-The libuv header import locally disables `_FORTIFY_SOURCE` during C-to-Zig
-translation: the pinned importer otherwise diagnoses glibc's variadic
-`open`/`openat` inline wrapper bodies in optimized builds. The driver calls
-`uv_fs_open`, not those libc wrappers. This does not change the separately built
-libuv library or C callers' hardening, and does not disable Zig safety checks.
+safety checks enabled. Reducer, collector and libuv scheduler-root tests run in
+both Debug and ReleaseSafe.
+
+[Cross-runtime compiler reproduction](../../docs/compiler-rebuilding.md#isolated-cross-runtime-fixed-point-reproduction)
+compares complete compiler generations executed by Haskell and Zig. The workflow
+includes independent object audits, program smokes and retention checks. Haskell
+supplies common artifact transport and packaging.
## Basic performance smoke (Linux)
@@ -34,28 +25,23 @@ PERF=$(nix path-info .#tricu-zig-performance-smoke)
cat "$PERF/report.json"
```
-This opt-in target checks complete results for string identity, append of two
+This target checks complete results for string identity, append of two
independently constructed inputs, deep constant-selection chains, distribution
chains, and repeated calls in one context. It records three fresh-process samples
per case, phase timings, fixture hashes, CPU information and memory observations.
-The C driver is optimized; the linked runtime is **ReleaseSafe**, as above.
-The driver records the runtime optimization mode supplied by its Nix build in
-every sample, and the report rejects mixed-mode samples.
+The C driver is optimized and the linked runtime is ReleaseSafe. Each sample
+records the runtime optimization mode; mixed-mode samples fail validation.
Construction, WHNF reduction, complete observation and explicit collection are
timed separately. Lazy work may occur during observation, and automatic GC time
is included in whichever phase triggers it. Explicit collection runs every 16
-iterations, at the midpoint, and at the end. Peak memory is Linux `VmHWM` for the
-executable's address space (not `getrusage`'s potentially inherited pre-exec
-high-water mark); midpoint/end RSS comes from `/proc/self/statm` after collection.
-These are process-memory observations, not exact live-node counts.
+iterations, at the midpoint, and at the end. Peak process RSS comes from Linux
+`VmHWM`; midpoint/end RSS comes from `/proc/self/statm` after collection.
Each child applies a 512 MiB address-space limit, 8 MiB stack limit and 60-second
CPU limit before initializing the runtime; the harness enforces a 65-second wall
-limit. A wrong result, crash, exhausted limit or timeout fails the smoke. There
-are no machine-dependent timing thresholds, historical-speedup claims or compiler
-workloads. Three samples on a shared machine are descriptive, not a controlled
-benchmark.
+limit. A wrong result, crash, exhausted limit or timeout fails the smoke. Timings
+are recorded for comparison across runs.
Nix may reuse the stored measurement report. For fresh measurements with the
same built runner:
@@ -68,12 +54,11 @@ nix develop -c python3 ext/zig/tests/performance_smoke.py \
## Small compiler probe
`test/cross_host_generation.py` runs a two-module native map-builder job through
-the shared generation runner on both Haskell and Zig, using
-the authenticated `bootstrap/rebuild` kit. It does not rebuild the compiler,
-modify retained toolchains, or promote a generation. Haskell authenticates the
-captured input bundles; Zig applies the retained compiler and adapter and drives
-IO. Successful publication must pass the independent store audit and produce
-`41` when executed by Haskell, followed by a fresh-context warm-reuse check.
+the shared generation runner on both Haskell and Zig, using the authenticated
+`bootstrap/rebuild` kit. Haskell authenticates the captured input bundles. Each
+execution host applies the retained compiler and adapter and drives IO.
+Publication passes an independent store audit, and the resulting program returns
+`41` on Haskell. A second invocation checks warm reuse in a fresh runtime context.
```sh
nix build .#tricu-zig .# --no-link
@@ -84,99 +69,89 @@ nix develop -c python3 test/cross_host_generation.py \
--directory /tmp/zig-compiler-probe-new
```
-The output directory must not exist. Limits are 1 GiB per child and 120 seconds
-per generation invocation, with an 8 MiB stack for Zig. Cold and warm runs must
-publish identical complete receipt and object sets across hosts. Inputs, runtime hashes, logs and
-incomplete/success status are recorded there. This is an opt-in compatibility
-probe, not a compiler generation, bootstrap acceptance, or a comparative
-performance benchmark. Unsupported actions during a pending IO continuation
-produce an error result identifying the action tag rather than a successful
-compilation.
+Choose a new output directory. Limits are 1 GiB per child and 120 seconds per
+generation invocation, with an 8 MiB stack for Zig. Cold and warm runs must
+publish identical complete receipt and object sets across hosts. The output
+records inputs, runtime hashes, logs and completion status. Unsupported IO actions
+produce an error result identifying the action tag.
-Add `--compatibility` for reexports/multiple exports, direct and local recursion,
+Add `--compatibility` for reexports, multiple exports, direct and local recursion,
patterns, sequencing, lazy discard, UTF-8 comments, dependency changes, warm
reuse, compiler error tags, corrupt checkpoints/executables, guarded interruption
-and restart, and missing-input/no-fallback controls. BOM/header failures are
-supervisor preflight checks, not evidence of executing the compiler's source
-reader. Source/indexed IO permission denial and malformed-input handling are
-covered separately by the runtime's Debug/ReleaseSafe tests. The execution child
-records string diagnostics or a bounded numeric prefix for structured diagnostics;
-it is not a replacement for a complete diagnostic renderer.
+and restart, and missing-input controls. The supervisor checks BOMs and headers
+before execution. Runtime Debug/ReleaseSafe tests cover source/indexed IO
+permission denial and malformed inputs. The execution child records string
+diagnostics or a bounded numeric prefix for structured diagnostics.
-### Indexed tree transport
+## Indexed tree transport
-The existing IO actions `readIndexedTree` (50), `hashIndexedTree` (51), and
+The IO actions `readIndexedTree` (50), `hashIndexedTree` (51), and
`writeIndexedTreeIfAbsent` (52) implement the
[verified native transport contract](../../docs/storage.md#verified-native-tree-transport).
-They use the fixed `arboricx.indexed-tree.v1` domain and Haskell-compatible
-canonical bytes. No additional protocol tags or C ABI entrypoints are required.
-
-The reusable `src/indexed.zig` codec has no filesystem, permission, or compiler
-policy. It uses iterative traversal, memoizes physical node identities, and
-interns exact constructor/child-ID keys. Physical sharing affects cost, not
-identity; equal finite trees produce the same bytes. Serialization demands the
-constructor structure but never applies the supplied tree as a program.
-The validated reader is shared with C ABI/CLI bundle loading and retains DAG
-sharing. Parsing alone does not authenticate an object: transport checks its
+They use the `arboricx.indexed-tree.v1` domain and Haskell-compatible canonical
+bytes.
+
+The `src/indexed.zig` codec uses iterative traversal, memoizes physical node
+identities, and interns exact constructor/child-ID keys. Equal finite trees
+produce the same bytes regardless of physical sharing. Serialization observes
+the tree's constructor structure. The validated reader is shared with C ABI/CLI
+bundle loading and retains DAG sharing. Transport authenticates each object's
domain-separated digest before decoding.
-The IO adapters compose that codec with byte-buffer hashing and generic native
-filesystem operations, without expanding serialized bytes into tricu lists.
-Reads need read permission; no-replace publication needs both read and write
-permission, checked before demanding the supplied tree. Writes verify the
-expected digest before publication and compare the winning file byte-for-byte,
-including an existing winner. Atomic link publication and file/directory syncing
-use the same primitives as byte writes. A post-publication failure can leave a
-valid object visible. tricu still chooses paths, creates directories, and owns
-manifests, caches and publication order.
+The IO adapters combine the codec with byte-buffer hashing and native filesystem
+operations. Reads require read permission; atomic no-replace publication requires
+both read and write permission, checked before demanding the supplied tree.
+Writes verify the expected digest before publication and compare the winning
+file byte-for-byte, including an existing winner. Atomic link publication and
+file/directory syncing use the same primitives as byte writes. A post-publication
+failure can leave a valid object visible. tricu chooses paths, creates directories,
+and manages manifests, caches and publication order.
-These operations occupy a synchronous scheduler turn and retain in-memory
-serialized buffers; they are neither streaming nor request-wide fuel-bounded.
-A public buffer-handle protocol is not introduced merely to implement these
-adapters: it would require separate resource ownership and lifetime semantics.
+These operations occupy a synchronous scheduler turn and retain complete
+serialized buffers in memory. Apply external time and memory limits to the
+containing IO program.
-### Source-text IO
+## Source-text IO
`readSourceFile` (37) requires read permission and returns a Result containing
-both the exact raw octets and a Unicode scalar list. It rejects malformed UTF-8
-and an initial BOM; it does not strip BOMs, normalize Unicode, or fold newlines.
-Paths use Unicode scalars encoded as UTF-8, with embedded NUL rejected.
-`encodeSourceText` (38) requires no filesystem permission and returns the exact
-UTF-8 bytes of a canonical scalar list. Surrogates, out-of-range values,
-noncanonical numeral trees and malformed lists are errors. Encoding U+FEFF is
-allowed; initial-BOM rejection belongs to source-file reading.
-
-These operations do not change byte-oriented ABI string/binary codecs or
-`readFile`. Source reads, like native CAS filesystem operations, run
-synchronously within one scheduler turn; they are not streaming or globally
-fuel-bounded. Allocation failures propagate to the host rather than becoming a
-successful or partial source result.
+both the exact raw octets and a Unicode scalar list. It preserves Unicode and
+newline sequences and rejects malformed UTF-8 and an initial BOM. Paths use
+Unicode scalars encoded as UTF-8, with embedded NUL rejected.
+
+`encodeSourceText` (38) is permission-free and returns the exact UTF-8 bytes of a
+canonical scalar list. Surrogates, out-of-range values, noncanonical numeral
+trees and malformed lists are errors. Encoding U+FEFF is allowed; initial-BOM
+rejection belongs to source-file reading.
+
+ABI string/binary codecs and `readFile` operate on bytes. Source reads run
+synchronously within one scheduler turn. Allocation failures propagate to the
+host.
## ABI v2 ownership
-`include/arboricx.h` is the interface. This deliberately replaces the original
-bare-node-index ABI; callers must rebuild and migrate. `arb_abi_version()` is 2.
-The JS, PHP and Python test bindings check this version before using handles.
+`include/arboricx.h` defines ABI v2. `arb_abi_version()` returns 2; the JS, PHP and
+Python test bindings check this version before using handles.
-* A handle is an opaque 64-bit integer, not a node address or serialization.
- Each successful handle output owns one root. Constructors borrow their inputs.
+* A handle is an opaque 64-bit integer identifying an owned root. Each successful
+ handle output owns one root. Constructors borrow their inputs.
* `arb_retain` returns a **different**, independently owned handle to the same
- node. `arb_release` invalidates just that handle. Context destruction releases
- everything, including handles the caller did not explicitly release.
-* Inspection returns owned child handles. Releasing a parent does not invalidate
- an independently held child. Comparing handles does not compare tree values.
+ node. `arb_release` invalidates that handle. Context destruction releases all
+ remaining handles and allocations.
+* Inspection returns independently owned child handles. Child handles remain
+ valid after releasing the parent. Handle equality compares handle identities;
+ tree equality requires structural comparison.
* Stale, zero, and foreign-context handles return `ARB_INVALID_HANDLE`.
Slot reuse has a fresh process-wide generation; generation exhaustion returns
- `ARB_HANDLE_EXHAUSTED` rather than wrapping. Handles are positive signed-64-bit
- representable values; JavaScript bindings expose them as `bigint`, never Number.
+ `ARB_HANDLE_EXHAUSTED`. Handles fit in a positive signed 64-bit integer;
+ JavaScript bindings expose them as `bigint`.
* Operations return `ARB_OK` or an explicit status. Failure leaves output
parameters unchanged. Multi-handle output publication is all-or-nothing.
* Reduction updates the shared graph in place. Aliases observe evaluation
progress, including partial progress after fuel or allocation failure.
- The returned result handle is independently owned; it does not consume input.
-* Contexts are single-threaded and non-reentrant. Pointers must be valid, and
- output parameters must be distinct. Handles are misuse detection, not a
- security boundary against native code forging values or corrupting memory.
+ The returned result handle and the input handle are independently owned.
+* Contexts require single-threaded, non-reentrant use. Pointers must be valid,
+ and output parameters must be distinct. Callers are responsible for native
+ memory safety; handle validation detects stale and foreign references.
* Decoded byte/string buffers have separate ownership: free them with
`arboricx_free_buf` before destroying their context. `arboricx_init` returns
null on allocation failure; other tree operations use status/out parameters.
@@ -196,7 +171,7 @@ if (arb_leaf(ctx, &leaf) != ARB_OK ||
arboricx_free(ctx);
return 1;
}
-/* result still owns Stem Leaf, including the child released above. */
+/* result owns Stem Leaf, including the child released above. */
int is_stem;
if (arb_is_stem(ctx, result, &is_stem) != ARB_OK || !is_stem) {
arboricx_free(ctx);
@@ -211,82 +186,74 @@ arboricx_free(ctx);
## Reduction and collection contract
-The reducer is a heap-framed, in-place, call-by-need WHNF machine. Constructor
-children and discarded arguments are not eagerly evaluated. Suspended function,
-discriminator, argument and update demands do not use the native call stack.
+The reducer is a heap-framed, in-place, call-by-need WHNF machine. It evaluates
+terms on demand. Suspended function, discriminator, argument and update demands
+use heap frames.
The arena uses non-moving mark/sweep and reuses dead slots. Collection occurs at
reducer loop safepoints, with the current term and continuation redexes rooted.
The ABI supplies live handles and its kernel root; the CLI supplies the invocation
root. The IO scheduler snapshots all tasks' current terms, environments, state,
-results and continuation frames for each turn. Async callbacks only allocate
-and link nodes; they do not force terms. Root scopes unwind on errors.
+results and continuation frames for each turn. Async callbacks allocate and link
+nodes. Root scopes unwind on errors.
-Construction never collects automatically: this permits allocating/linking
-children without intermediate root registration. An allocation-heavy client
-that does not evaluate should call `arb_collect` periodically. Releasing a handle
-does not itself collect. Evaluation and IO no longer copy graphs through scratch
-arenas. Internally, raw node IDs must not escape their registered root lifetime,
-and pointers into the growable node array must not survive allocations.
+Construction allocates and links nodes; collection runs during reduction or an
+explicit `arb_collect` call. Construction-only clients should call `arb_collect`
+periodically. Graphs unreachable from the remaining roots become eligible
+for reclamation at the next collection. Evaluation and IO operate directly on the shared graph. Internally,
+keep raw node IDs within their registered root lifetime and reacquire pointers
+into the growable node array after allocations.
-### Limits
+### Resource limits
-* This is not a hard memory limit. Arena capacity and collector work buffers can
- reach a high-water mark; freeing nodes does not promise lower resident memory.
+* Arena capacity and collector work buffers retain their high-water allocations.
Retaining handles retains their reachable graphs. Root-table rebuilding scans
- handle slots at ABI observation/evaluation entry.
-* Fuel counts application entries for one reducer invocation, not an entire
- request. Codec observations still use separate budgets; IO evaluation is not
- governed by a single request-wide fuel budget. Use external resource limits.
-* Reducer stack safety does not imply stack safety for every codec, structural
- comparison, parser or renderer. Full observations can diverge on infinite
- results. Native pointer misuse is outside the ABI's validation contract.
-* This runtime does not implement global application memoization, packed nodes,
- moving collection, or full compiler/toolchain acceptance.
-
-Tests include all kernel rules, independent finite-reference comparisons,
+ handle slots at ABI observation/evaluation entry. Apply an external memory
+ limit to bound process usage.
+* Fuel counts application entries for one reducer invocation. Codec observations
+ use separate budgets. Apply an external deadline to bound a complete IO program.
+* Codecs, structural comparisons, parsers and renderers may use the native stack.
+ Complete observations require finite results; observing infinite results can
+ diverge.
+
+Tests cover all kernel rules, independent finite-reference comparisons,
undemanded divergent terms, shared updates, deep evaluation, allocation-failure
cleanup, GC slot reuse, scheduler roots, stale/foreign handles, independent
-ownership, and C/Python/JS/PHP fixture programs. `tests/helpers.h` contains
-checked constructors for fixture tests only; it is not a compatibility ABI.
+ownership, and C/Python/JS/PHP fixture programs. `tests/helpers.h` supplies checked
+constructors for fixture tests.
## Paired Haskell/Zig runtime measurements
```sh
nix build .#reducerComparison --no-link
-# Run after builds finish, without other intentional benchmark/build workloads.
nix run .#reducerComparison -- /tmp/reducer-comparison.json
```
-This opt-in Linux harness compares the real Haskell reducer (GHC `-O2`, one RTS
-capability) with Zig ReleaseSafe. Both use the same retained identity/append
-bundles, canonical byte inputs, and constant-selection/distribution workloads.
-Every iteration checks its complete result. Each case has five fresh-process
-samples per host, alternating host order, pinned to one allowed CPU. Results
-include binary/fixture hashes, compiler version, raw samples, timing ranges and
-process peak RSS. Package checks include deliberately wrong workers that must
-fail result validation.
-
-The primary reported timing combines WHNF reduction and full output decoding
-and checking. This is **not isolated rewrite throughput**: codecs and automatic
-GC are included. Build and explicit-GC phases are also recorded, along with their
-combined total. Haskell input normal forms are forced before application timing;
-opaque suspension boxes prepare unevaluated applications without specializing
-away known constant/identity rules. Harness-only anti-sharing flags and varying
-input bytes prevent reusing an already-evaluated result across rounds. Production
-Haskell runtime modules compile normally. Zig's construction includes owned-handle
-C ABI calls; Haskell's includes input forcing and suspension preparation. Neither
-construction cost should be mistaken for reduction cost.
-
-GC placement differs: Haskell may collect during construction whereas Zig
-collects automatically at reduction safepoints. Compare both evaluation/observation
-and combined totals before drawing conclusions. Initialization is excluded from
-phase timings but included in process RSS. Common guards are 512 MiB sampled RSS,
-60 CPU seconds, 65 wall seconds and an 8 MiB native stack per child; Zig also has
-a 512 MiB address-space cap, and Haskell has a 512 MiB managed-heap cap and an 8 MiB
-RTS stack cap. Small successful samples should remain far below these limits.
-These measurements are descriptive host smokes, not compiler or bootstrap speed
-comparisons, and no timing threshold is a test gate.
+Run measurements after builds finish, with other benchmark/build workloads idle.
+This Linux harness compares the Haskell reducer (GHC `-O2`, one RTS capability)
+with Zig ReleaseSafe. Both use the same retained identity/append bundles,
+canonical byte inputs, and constant-selection/distribution workloads. Every
+iteration checks its complete result. Each case has five fresh-process samples
+per host, alternating host order, pinned to one allowed CPU. Results include
+binary/fixture hashes, compiler version, raw samples, timing ranges and process
+peak RSS. Package checks require deliberately wrong workers to fail result
+validation.
+
+The primary timing combines WHNF reduction and full output decoding and checking,
+including codecs and automatic GC. Build and explicit-GC phases are recorded
+separately and as part of the combined total. Haskell input normal forms are
+forced before application timing; opaque suspension boxes prepare unevaluated
+applications. Harness-only anti-sharing flags and varying input bytes give each
+round fresh work. Production Haskell runtime modules compile normally. Zig's
+construction includes owned-handle C ABI calls; Haskell's includes input forcing
+and suspension preparation.
+
+Haskell may collect during construction; Zig collects automatically at reduction
+safepoints. Compare both evaluation/observation and combined totals. Initialization
+contributes to process RSS and has its own timing. Common guards are 512 MiB
+sampled RSS, 60 CPU seconds, 65 wall seconds and an 8 MiB native stack per child.
+Zig also has a 512 MiB address-space cap; Haskell has a 512 MiB managed-heap cap
+and an 8 MiB RTS stack cap.
### Isolated mapped-library probe
@@ -301,21 +268,17 @@ nix develop -c python3 test/bootstrap_cross_host.py --probe \
--tricu "$HASKELL/bin/tricu" --zig-library "$ZIG/lib/libarboricx.so"
```
-Each sandbox mounts only the read-only source snapshot and kit, Nix dependencies,
+Each sandbox mounts the read-only source snapshot and kit, Nix dependencies,
an initially empty writable work directory, and ephemeral system directories.
-Prior generation stores and the live checkout are absent. The ordinary generation
-supervisor runs inside the sandbox, so its RSS monitor observes the execution
-child rather than the outer isolation wrapper. The probe compares complete
-independently audited object inventories. Its experiment definition records the
-source commit/archive, mapped-input hashes, runtimes, Python, tooling, limits and
-comparison boundaries. Historical `ext/zig/result` is the sole excluded archive
-entry: a tracked absolute build-output symlink, not a compiler input.
-
-The launcher first creates a private PID namespace with a matching procfs mount,
-then starts the filesystem sandbox. This avoids assuming that the caller's `/proc`
-uses its PID numbering. Both isolation tool identities are recorded. A full
-experiment stopped before a job creates its generation directory can explicitly
-use `--continue-experiment`: frozen sources/runtimes and completed generations
-are reverified, the failed launch and original tooling are preserved, and the
-pending job still starts fresh. Incomplete executed generations are never resumed
-by this acceptance route.
+The generation supervisor runs inside the sandbox, where its RSS monitor observes
+the execution child. The probe compares complete independently audited object
+inventories. Its experiment definition records the source commit/archive,
+mapped-input hashes, runtimes, Python, tooling, limits and comparison boundaries.
+Snapshot extraction excludes the tracked build-output symlink `ext/zig/result`.
+
+The launcher creates a private PID namespace with a matching procfs mount,
+then starts the filesystem sandbox. Both isolation tool identities are recorded.
+For a full experiment stopped before a job creates its generation directory,
+`--continue-experiment` reverifies frozen sources, runtimes and completed
+generations, preserves the failed launch and original tooling, and starts the
+pending job fresh. A partially executed generation requires a new experiment.
diff --git a/test/bootstrap_cross_host.py b/test/bootstrap_cross_host.py
@@ -2,9 +2,8 @@
"""Opt-in isolated H1 -> H2 -> {H3,Z3} reproduction, or bounded native_math probe.
Every job sees only /source, /kit, /work, /nix/store and ephemeral /tmp,/proc,/dev.
-The ordinary supervisor runs INSIDE isolation and monitors the actual execution
-child. Haskell transport is common; prior stores are never mounted. This does
-not perform toolchain promotion, smokes or mandatory rolling retention.
+The generation supervisor runs inside isolation and monitors the execution
+child. Haskell supplies common artifact transport and packaging.
"""
import argparse
import json
@@ -99,9 +98,9 @@ def main():
p.add_argument('--source-commit',required=True)
p.add_argument('--tricu',type=Path,required=True)
p.add_argument('--zig-library',type=Path,required=True)
- p.add_argument('--probe',action='store_true',help='native_math on both hosts, 1 GiB/120s; no full generations')
+ p.add_argument('--probe',action='store_true',help='native_math on both hosts, 1 GiB/120s per job')
p.add_argument('--verify-only',action='store_true')
- p.add_argument('--continue-experiment',action='store_true',help='re-audit completed jobs; continue only jobs that never entered execution')
+ p.add_argument('--continue-experiment',action='store_true',help='re-audit completed jobs and retry pre-execution launch failures')
args=p.parse_args()
directory=args.directory.resolve()
runtime,library=args.tricu.resolve(),args.zig_library.resolve()
@@ -227,7 +226,8 @@ def main():
raise ValueError('experiment tooling changed during execution')
report['complete']=True
save(directory/'report.json',report)
- print('Experiment verified; full-run promotion/smokes/retention remain separate: '+str(directory))
+ label='Isolated mapped-library probe' if args.probe else 'Cross-runtime compiler reproduction'
+ print(label+' verified: '+str(directory))
if __name__=='__main__':
diff --git a/test/bootstrap_cross_host_controls.py b/test/bootstrap_cross_host_controls.py
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
-"""Negative controls on a completed experiment; mutate only independent copies.
-No compiler execution, no publication and no changes to accepted stores.
+"""Check verifier rejection of corrupted copies of a completed experiment.
+Restore the copies and verify acceptance again.
"""
import argparse
import hashlib
diff --git a/test/cross_host_generation.py b/test/cross_host_generation.py
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
-"""Opt-in bounded parity checks through the shared generation runner.
-No compiler generation or toolchain promotion. Haskell transports and verifies;
-only the selected execution host may perform compilation and publication.
+"""Bounded compiler compatibility checks through the shared generation runner.
+Haskell supplies transport and result execution; each selected execution host
+compiles and publishes the test modules. Independent audits verify the stores.
"""
import argparse
import hashlib
diff --git a/test/reducer_comparison.py b/test/reducer_comparison.py
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
-"""Small paired runtime measurements, not compiler/bootstrapping acceptance."""
+"""Paired Haskell/Zig runtime measurements with complete-result checks."""
import hashlib
import json
import os
@@ -28,12 +28,12 @@ report = dict(
haskell_rts=['-N1','-M512m','-K8m'], zig_address_space_mib=512),
notes=[
'Identical normal-form fixture functions and canonical input values; fresh application graphs each iteration.',
- 'Primary metric is WHNF reduction plus complete output decoding/checking. Not isolated rewrite throughput.',
+ 'Primary metric includes WHNF reduction, complete output decoding/checking and automatic GC.',
'Input construction is separate; Haskell forces input normal forms and uses opaque suspension boxes to prepare unevaluated applications.',
'Native allocations, FFI, codec implementations and automatic GC differ. All automatic GC remains charged to its triggering phase.',
- 'Explicit GC follows identical round cadence; total includes build/reduce/observe/explicit GC, not initialization or input-byte generation.',
- 'Memory is complete-process VmHWM, including runtime initialization; it is not live-tree size.',
- 'Fresh processes, sequential samples, alternating host order, single CPU affinity. Five samples describe this machine, not statistical significance.'
+ 'Explicit GC follows identical round cadence; total sums build, reduction, observation and explicit GC phases.',
+ 'Memory is complete-process VmHWM, including runtime initialization.',
+ 'Five fresh-process samples per case, sequential execution, alternating host order, single CPU affinity.'
], cases=[])
def save():
Path(output).write_text(json.dumps(report, indent=2)+'\n')
diff --git a/test/zig_generation_executor.py b/test/zig_generation_executor.py
@@ -1,11 +1,10 @@
#!/usr/bin/env python3
-"""Execution-only child for bootstrap_generations; no transport or compiler policy.
+"""Zig execution child for bootstrap_generations.
-The supervisor authenticates/imports captured bundles, supplies the exact six
-arguments and independently audits all publications. This child rechecks captured
-bundle digests, runs the selected entrypoint via the owned ABI, and never falls
-back to Haskell. IO permissions match the generation runner's unsafe-IO scope;
-this is for trusted compiler jobs, not an untrusted-program sandbox.
+The supervisor authenticates/imports captured bundles, supplies six arguments
+and independently audits all publications. This child rechecks captured bundle
+digests and runs the selected entrypoint through the owned ABI. It grants broad
+file IO to trusted compiler jobs, matching the generation runner's unsafe-IO scope.
"""
import argparse
import ctypes as C
diff --git a/test/zig_observe.py b/test/zig_observe.py
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""Bounded test observer: invoke a named bundle export and emit its entire tree.
-Flat test data only; this is not a production transport or diagnostic renderer.
+Reads a flat argument tree and writes the complete result in the same format.
"""
import argparse
import ctypes as C
