Zig runtime and C ABI

The Zig runtime evaluates tree-calculus programs and provides IO through an owned-handle C ABI. Build and validate it with Nix:

nix build .#tricu-zig .#tricu-zig-tests --no-link
nix build .#tricu-js-tests .#tricu-php-tests --no-link

The executable and shared library build in ReleaseSafe, with Zig runtime safety checks enabled. Reducer, collector and libuv scheduler-root tests run in both Debug and ReleaseSafe.

Cross-runtime compiler reproduction compares complete compiler generations executed by Haskell and Zig. The workflow includes independent object audits, program smokes and retention checks. Haskell supplies common artifact transport and packaging.

The WASM adapter reuses this reducer without native IO. The demo site selects it alongside JS and Haskell, with browser IO implemented by the owning session. Browser work uses explicit unbounded reducer/Unicode/preview entry points and hard Worker termination; existing finite-fuel native entry points retain their semantics.

Basic performance smoke (Linux)

nix build .#tricu-zig-performance-smoke --no-link
PERF=$(nix path-info .#tricu-zig-performance-smoke)
cat "$PERF/report.json"

This target checks complete results for string identity, append of two independently constructed inputs, deep constant-selection chains, distribution chains, and repeated calls in one context. It records three fresh-process samples per case, phase timings, fixture hashes, CPU information and memory observations. The C driver is optimized and the linked runtime is ReleaseSafe. Each sample records the runtime optimization mode; mixed-mode samples fail validation.

Construction, WHNF reduction, complete observation and explicit collection are timed separately. Lazy work may occur during observation, and automatic GC time is included in whichever phase triggers it. Explicit collection runs every 16 iterations, at the midpoint, and at the end. Peak process RSS comes from Linux VmHWM; midpoint/end RSS comes from /proc/self/statm after collection.

Each child applies a 512 MiB address-space limit, 8 MiB stack limit and 60-second CPU limit before initializing the runtime; the harness enforces a 65-second wall limit. A wrong result, crash, exhausted limit or timeout fails the smoke. Timings are recorded for comparison across runs.

Nix may reuse the stored measurement report. For fresh measurements with the same built runner:

nix develop -c python3 ext/zig/tests/performance_smoke.py \
  "$PERF/bin/performance-smoke" test/fixtures /tmp/tricu-zig-performance-smoke.json

Small compiler probe

test/cross_host_generation.py runs a two-module native map-builder job through the shared generation runner on both Haskell and Zig, using the authenticated bootstrap/rebuild kit. Haskell authenticates the captured input bundles. Each execution host applies the retained compiler and adapter and drives IO. Publication passes an independent store audit, and the resulting program returns 41 on Haskell. A second invocation checks warm reuse in a fresh runtime context.

nix build .#tricu-zig .# --no-link
ZIG=$(nix path-info .#tricu-zig)
HASKELL=$(nix path-info .#)
nix develop -c python3 test/cross_host_generation.py \
  --library "$ZIG/lib/libarboricx.so" --tricu "$HASKELL/bin/tricu" \
  --directory /tmp/zig-compiler-probe-new

Choose a new output directory. Limits are 1 GiB per child and 120 seconds per generation invocation, with an 8 MiB stack for Zig. Cold and warm runs must publish identical complete receipt and object sets across hosts. The output records inputs, runtime hashes, logs and completion status. Unsupported IO actions produce an error result identifying the action tag.

Add --compatibility for reexports, multiple exports, direct and local recursion, patterns, sequencing, lazy discard, UTF-8 comments, dependency changes, warm reuse, compiler error tags, corrupt checkpoints/executables, guarded interruption and restart, and missing-input controls. The supervisor checks BOMs and headers before execution. Runtime Debug/ReleaseSafe tests cover source/indexed IO permission denial and malformed inputs. The execution child records string diagnostics or a bounded numeric prefix for structured diagnostics.

Evaluator allocation headroom

The default collector runs after max(1048576, last_live_nodes) allocations since the last collection. It recomputes this threshold after each collection, so a transient live-heap peak does not permanently delay later collections. Construction still never collects automatically. These thresholds count nodes, not bytes, and are not RSS limits.

C ABI clients can choose the memory/speed tradeoff per context:

/* More allocation headroom for a memory-rich compiler job. Check the status. */
arb_status_t status = arb_set_gc_policy(ctx, 4194304, 4);

The two positive uint32_t arguments are the minimum allocation interval and live-node multiplier. The threshold is their maximum after multiplying the most recently measured live-node count (with saturating arithmetic). Changing policy does not allocate, collect, or reset the allocation counter. The next reducer safepoint checks the new threshold. Zero arguments return ARB_INVALID_DATA without changing the policy. arb_collect still requests an explicit collection regardless of the threshold. Older ABI-v2 libraries may lack this additive entrypoint; callers requiring it must select a library that exports it, not silently fall back.

The generation runner accepts the same explicit policy and records it in input provenance, status, invocation and the execution report. For a bounded native math compilation, using the runtime paths from the small compiler probe:

nix develop -c python3 test/bootstrap_generations.py /tmp/zig-math-headroom-new \
  --bootstrap-kit bootstrap/rebuild --tricu "$HASKELL/bin/tricu" \
  --execution-host zig --zig-library "$ZIG/lib/libarboricx.so" \
  --root native_math --fresh --memory-mib 4096 --wall-seconds 180 \
  --zig-gc-policy 4194304 4

Use a fresh directory for each sample. Omit --zig-gc-policy to measure the library default; it is also accepted by test/cross_host_generation.py for cold/warm parity and compatibility checks. The existing isolated 1 GiB probe continues to use the default. Larger headroom may need several GiB, even on a single module; increasing it further need not improve execution time. Keep external resource guards and measure both time and peak RSS on the actual job. The collector retains backing capacity, including its mark/work buffers, and the reducer retains continuation-buffer capacity. Collection trims dead node array tails without moving live IDs, but does not promise to return memory to the OS.

For comparisons, alternate original/candidate order on one allowed CPU after builds finish. Compare execution.json execution times and status.json child peak RSS, not just supervisor wall time. Require independent audits and identical complete cas/objects inventories. A native-math measurement is not a full compiler-generation measurement; use the cross-runtime reproduction workflow for compiler fixed-point acceptance.

Indexed tree transport

The IO actions readIndexedTree (50), hashIndexedTree (51), and writeIndexedTreeIfAbsent (52) implement the verified native transport contract. They use the arboricx.indexed-tree.v1 domain and Haskell-compatible canonical bytes.

The src/indexed.zig codec uses iterative traversal, memoizes physical node identities, and interns exact constructor/child-ID keys. Equal finite trees produce the same bytes regardless of physical sharing. Serialization observes the tree's constructor structure. The validated reader is shared with C ABI/CLI bundle loading and retains DAG sharing. Transport authenticates each object's domain-separated digest before decoding.

The IO adapters combine the codec with byte-buffer hashing and native filesystem operations. Reads require read permission; atomic no-replace publication requires both read and write permission, checked before demanding the supplied tree. Writes verify the expected digest before publication and compare the winning file byte-for-byte, including an existing winner. Atomic link publication and file/directory syncing use the same primitives as byte writes. A post-publication failure can leave a valid object visible. tricu chooses paths, creates directories, and manages manifests, caches and publication order.

These operations occupy a synchronous scheduler turn and retain complete serialized buffers in memory. Apply external time and memory limits to the containing IO program.

Source-text IO

readSourceFile (37) requires read permission and returns a Result containing both the exact raw octets and a Unicode scalar list. It preserves Unicode and newline sequences and rejects malformed UTF-8 and an initial BOM. Paths use Unicode scalars encoded as UTF-8, with embedded NUL rejected.

encodeSourceText (38) is permission-free and returns the exact UTF-8 bytes of a canonical scalar list. Surrogates, out-of-range values, noncanonical numeral trees and malformed lists are errors. Encoding U+FEFF is allowed; initial-BOM rejection belongs to source-file reading.

ABI string/binary codecs and readFile operate on bytes. Source reads run synchronously within one scheduler turn. Allocation failures propagate to the host.

ABI v2 ownership

include/arboricx.h defines ABI v2. arb_abi_version() returns 2; the JS, PHP and Python test bindings check this version before using handles.

• A handle is an opaque 64-bit integer identifying an owned root. Each successful handle output owns one root. Constructors borrow their inputs.
• arb_retain returns a different, independently owned handle to the same node. arb_release invalidates that handle. Context destruction releases all remaining handles and allocations.
• Inspection returns independently owned child handles. Child handles remain valid after releasing the parent. Handle equality compares handle identities; tree equality requires structural comparison.
• Stale, zero, and foreign-context handles return ARB_INVALID_HANDLE. Slot reuse has a fresh process-wide generation; generation exhaustion returns ARB_HANDLE_EXHAUSTED. Handles fit in a positive signed 64-bit integer; JavaScript bindings expose them as bigint.
• Operations return ARB_OK or an explicit status. Failure leaves output parameters unchanged. Multi-handle output publication is all-or-nothing.
• Reduction updates the shared graph in place. Aliases observe evaluation progress, including partial progress after fuel or allocation failure. The returned result handle and the input handle are independently owned.
• Contexts require single-threaded, non-reentrant use. Pointers must be valid, and output parameters must be distinct. Callers are responsible for native memory safety; handle validation detects stale and foreign references.
• Decoded byte/string buffers have separate ownership: free them with arboricx_free_buf before destroying their context. arboricx_init returns null on allocation failure; other tree operations use status/out parameters.

For example (check every status in application code):

arb_ctx_t *ctx = arboricx_init();
if (!ctx) return 1;
arb_handle_t leaf, application, result;
if (arb_leaf(ctx, &leaf) != ARB_OK ||
    arb_app(ctx, leaf, leaf, &application) != ARB_OK ||
    arb_release(ctx, leaf) != ARB_OK ||
    arb_reduce(ctx, application, 1000, &result) != ARB_OK ||
    arb_release(ctx, application) != ARB_OK ||
    arb_collect(ctx) != ARB_OK) {
    arboricx_free(ctx);
    return 1;
}
/* result owns Stem Leaf, including the child released above. */
int is_stem;
if (arb_is_stem(ctx, result, &is_stem) != ARB_OK || !is_stem) {
    arboricx_free(ctx);
    return 1;
}
if (arb_release(ctx, result) != ARB_OK) {
    arboricx_free(ctx);
    return 1;
}
arboricx_free(ctx);

Reduction and collection contract

The reducer is a heap-framed, in-place, call-by-need WHNF machine. It evaluates terms on demand. Constructor demands dispatch directly; only suspended function, discriminator, argument and update demands use heap frames. Already-normal observations avoid borrowing continuation storage unless collection is due. Distribution and fork-argument triage also fuse immediately demanded constructor/constant applications, without allocating their private intermediate thunks. These are kernel-rule reductions, not source-name intrinsics: they do not demand discarded arguments, and charge the same application-entry fuel as the unfused rules. Low-fuel cases fall back to the unfused path. Tests compare single-invocation fuel boundaries and resumed results against a literal unfused evaluator.

The arena uses non-moving mark/sweep and reuses dead slots. Collection occurs at reducer loop safepoints, with the current term and continuation redexes rooted. The ABI supplies live handles and its kernel root; the CLI supplies the invocation root. The IO scheduler snapshots all tasks' current terms, environments, state, results and continuation frames for each turn. Async callbacks allocate and link nodes. Root scopes unwind on errors.

Construction allocates and links nodes; collection runs during reduction or an explicit arb_collect call. Construction-only clients should call arb_collect periodically. Graphs unreachable from the remaining roots become eligible for reclamation at the next collection. Evaluation and IO operate directly on the shared graph. Internally, keep raw node IDs within their registered root lifetime and reacquire pointers into the growable node array after allocations.

Resource limits

• Arena capacity and collector work buffers retain their high-water allocations. Retaining handles retains their reachable graphs. Root-table rebuilding scans handle slots at ABI observation/evaluation entry. Apply an external memory limit to bound process usage.
• Fuel counts application entries for one reducer invocation. Codec observations use separate budgets. Apply an external deadline to bound a complete IO program.
• Codecs, structural comparisons, parsers and renderers may use the native stack. Complete observations require finite results; observing infinite results can diverge.

Tests cover all kernel rules, independent finite-reference comparisons, undemanded divergent terms, shared updates, deep evaluation, allocation-failure cleanup, GC slot reuse, scheduler roots, stale/foreign handles, independent ownership, and C/Python/JS/PHP fixture programs. tests/helpers.h supplies checked constructors for fixture tests.

Paired Haskell/Zig runtime measurements

nix build .#reducerComparison --no-link
nix run .#reducerComparison -- /tmp/reducer-comparison.json

Run measurements after builds finish, with other benchmark/build workloads idle. This Linux harness compares the Haskell reducer (GHC -O2, one RTS capability) with Zig ReleaseSafe. Both use the same retained identity/append bundles, canonical byte inputs, and constant-selection/distribution workloads. Every iteration checks its complete result. Each case has five fresh-process samples per host, alternating host order, pinned to one allowed CPU. Results include binary/fixture hashes, compiler version, raw samples, timing ranges and process peak RSS. Package checks require deliberately wrong workers to fail result validation.

The primary timing combines WHNF reduction and full output decoding and checking, including codecs and automatic GC. Build and explicit-GC phases are recorded separately and as part of the combined total. Haskell input normal forms are forced before application timing; opaque suspension boxes prepare unevaluated applications. Harness-only anti-sharing flags and varying input bytes give each round fresh work. Production Haskell runtime modules compile normally. Zig's construction includes owned-handle C ABI calls; Haskell's includes input forcing and suspension preparation.

Haskell may collect during construction; Zig collects automatically at reduction safepoints. Compare both evaluation/observation and combined totals. Initialization contributes to process RSS and has its own timing. Common guards are 512 MiB sampled RSS, 60 CPU seconds, 65 wall seconds and an 8 MiB native stack per child. Zig also has a 512 MiB address-space cap; Haskell has a 512 MiB managed-heap cap and an 8 MiB RTS stack cap.

Isolated mapped-library probe

test/bootstrap_cross_host.py --probe runs the frozen native_math closure on both hosts, with identical kits and a 1 GiB/120-second budget per job. It requires Linux bwrap and util-linux unshare with working user/PID/mount namespaces, and a Nix Python environment:

nix develop -c python3 test/bootstrap_cross_host.py --probe \
  --source-commit COMMITTED_SOURCE --directory /tmp/cross-host-probe-new \
  --tricu "$HASKELL/bin/tricu" --zig-library "$ZIG/lib/libarboricx.so"

Each sandbox mounts the read-only source snapshot and kit, Nix dependencies, an initially empty writable work directory, and ephemeral system directories. The generation supervisor runs inside the sandbox, where its RSS monitor observes the execution child. The probe compares complete independently audited object inventories. Its experiment definition records the source commit/archive, mapped-input hashes, runtimes, Python, tooling, limits and comparison boundaries. Snapshot extraction excludes the tracked build-output symlink ext/zig/result.

The launcher creates a private PID namespace with a matching procfs mount, then starts the filesystem sandbox. Both isolation tool identities are recorded. For a full experiment stopped before a job creates its generation directory, --continue-experiment reverifies frozen sources, runtimes and completed generations, preserves the failed launch and original tooling, and starts the pending job fresh. A partially executed generation requires a new experiment.