Native Haskell embedding

The Haskell shared library exposes tricu trees and the native IO driver through include/tricu.h, embedding ABI 1. It uses the same Runtime.Tree, value codecs, bundle verifier and IO driver as the Haskell executable. Native consumers use the Python, JavaScript, and PHP bindings.

Build and validate

nix build .#tricu-haskell-ffi --no-link
EMBED=$(nix path-info .#tricu-haskell-ffi)

# Compiled IO copier and a two-module portable compiler job
nix build .#tricu-haskell-ffi-integration --no-link
CHECK=$(nix path-info .#tricu-haskell-ffi-integration)
cat "$CHECK/integration.json" "$CHECK/compiler.json"

# Common value and IO conformance through all three installed native bindings
nix build .#tricu-native-bindings-integration --no-link
BINDINGS=$(nix path-info .#tricu-native-bindings-integration)
cat "$BINDINGS/report.json"

The library build runs a linked C client, Haskell-side exception/demand controls, and fresh-process Python ctypes checks before and after installation. nix flake check includes this basic gate and consumer-level Python/JS/PHP conformance. Source-compiling acceptance is opt-in. Shared JSON fixtures supply independent expected values; each language runs malformed-input, ownership and IO-outcome controls. The opt-in binding integration additionally executes the compiled copier, failure cleanup and transferred-resource cancellation in all three languages.

Native embedding is validated on x86-64 Linux with the pinned Nix/GHC toolchain. The native CLI, embedding and development compiler select GHC 9.14.1; the static-musl and browser/WASM targets retain their separate toolchain selections. The artifact installs lib/libtricu-haskell.so, include/tricu.h, and share/tricu/tricu.py. On Linux its Haskell shared-library dependencies are installed beside it; native dependencies retain their Nix runtime paths. Keep the complete Nix closure available. Copying just the main .so is insufficient. Other native targets require their own acceptance. Browser embedding has its Haskell WASM contract.

Compiler-workflow embedding checks

nix build .#tricu-generation-ffi-test --no-link
CHECK=$(nix path-info .#tricu-generation-ffi-test)
cat "$CHECK/compatibility/report.json" "$CHECK/programs/report.json" "$CHECK/scoped/report.json"

The generation runner accepts --execution-host haskell-ffi --haskell-library PATH with explicit IO kit selection. A fresh execution process authenticates the captured compiler/adapter, six arguments, runtime, executor and binding identities before compiler effects. Its one-capability RTS is independent of CLI transport capabilities. External CPU/address-space/RSS/wall guards bound the job; compiler failures remain distinct from opaque terminal task outcomes.

The opt-in gate compares cold/warm publication, source changes, corruption, interruption/restart and complete receipts/objects against the CLI. It also checks twelve complete import/recursion/invoice results against independent oracles, plus scoped native_math generations and request-tampering rejection. These are application/library workflows; compiler-pair acceptance and retention use the compiler rebuilding procedure.

Full-host linkage variants

Both native artifacts expose the complete ABI in include/tricu.h: reduction, value codecs, authenticated bundle loading and native IO. tricu-haskell-ffi links Haskell dependencies dynamically; tricu-haskell-ffi-archive incorporates PIC Haskell archives and the threaded RTS in one shared object, with an ELF export map and unused-section collection. The archive variant is opt-in on x86-64 Linux. Its PIC compiler, boot libraries and dependency archives share one package set. The first build may compile that entire dependency graph. Native system libraries retain their Nix runtime paths; keep the complete runtime closure available for either variant.

nix build .#tricu-haskell-ffi-archive .#tricu-haskell-ffi-archive-integration --no-link
CHECK=$(nix path-info .#tricu-haskell-ffi-archive-integration)
cat "$CHECK/integration.json" "$CHECK/compiler.json"

Both variants run C ownership/codec/reduction checks, Haskell demand/exception/ concurrency controls, Linux loader-lifetime tests, Python ownership/IO checks and installed-header/client checks. The archive variant additionally checks the exact public symbol set and absence of Haskell shared dependencies. Its opt-in integration runs the compiled IO copier and cold two-module compiler comparison against the CLI. Default bindings select the dynamic full host.

Select one artifact per process: both own process-wide GHC initialization and expose the same symbols. Handles belong to the selected library instance. Internally, EmbeddingCore isolates reducer/value operations and the registry; Embedding adds bundle verification and IO using that same boundary.

Footprint and linkage measurements

nix build .#tricu-haskell-ffi .#tricu-haskell-ffi-archive --no-link
python3 ext/haskell/footprint.py \
  --host "$(nix path-info .#tricu-haskell-ffi)" \
  --host-archive "$(nix path-info .#tricu-haskell-ffi-archive)" \
  --output /tmp/tricu-runtime-footprint.json
nix build .#tricu-haskell-linkage-smoke --no-link
SMOKE=$(nix path-info .#tricu-haskell-linkage-smoke)
cat "$SMOKE/summary.json"

The footprint report distinguishes library bytes, installed package file/NAR sizes, runtime closure NAR size, package tar.gz size and gzip-compressed Nix closure export size (compression level 6). Package archives require external runtime paths; closure exports are Nix distributions. Closure totals include whole transitive native packages.

The linkage smoke calls the same public C API in separate processes. Each workload constructs fresh constant/distribution application chains, releases temporary roots, demands the result and checks that it is Leaf. Graph-building and weak-head-demand timings are separate, with one warmup and seven measured rounds per size. The report records peak RSS and timings for these workloads. Installed share/tricu/build.json identifies the compiler and linkage.

Nix-free deployment prototype

tricu-portable packages the archive-linked full host and non-glibc native libraries into a relocatable Linux x86-64 tarball. ELF dependencies use $ORIGIN; system glibc and the consumer's language runtime remain external. See the included transfer instructions for the layout, compatibility scope and paired source-delivery requirements.

nix build .#tricu-portable .#tricu-portable-test --no-link
PORTABLE=$(nix path-info .#tricu-portable)
CHECK=$(nix path-info .#tricu-portable-test)
cat "$CHECK/report.json"
# Distribute the runtime and matching source materials together:
ls "$PORTABLE/tricu-linux-x86_64.tar.gz" "$PORTABLE/tricu-source-materials.tar.gz"
cat "$PORTABLE/release.json"

The opt-in Linux acceptance requires working unprivileged user/mount namespaces. It extracts the actual archive at two locations, including spaces, and runs Python ABI/scoped-handle controls, compiled IO copying and the installed filecheck worker with independent digest oracles. Repeated successful/error calls check descriptor stability; CLI controls check record/verify and mutation. An empty mount root exposes only a relocated Python/glibc test OS, the release, fixtures and writable scratch. The test asserts /nix/store is absent, checks loaded mappings, and rejects a missing bundled GMP dependency. The test OS's LD_LIBRARY_PATH names only its synthetic system-library directory; release libraries resolve through their own relative paths. This isolated test checks Nix-free execution. portable.json records the required GLIBC symbol floor.

Notices, source materials and replacement

The redistribution contract describes the paired runtime and source archives, license choices and modification/replacement rights. tricu-redistribution resolves the link map's archive/object inputs and bundled ELF dependencies to actual source owners. The versioned component policy rejects unreviewed source distributions and missing required notices. It preserves named license texts and relevant leading C-family copyright/permission blocks, including vendored crypto implementations. The source archive includes upstream inputs, patches, Cabal revisions, complete tricu/Nixpkgs build recipes, native flake helper sources, and file/symlink hash inventories. Source-only notice texts are retained conservatively and identified as such.

tricu-portable verifies source ownership and original library hashes before relocation. It installs the notice corpus, pairs the runtime with the source manifests, and verifies both archive contents before emitting release.json. Deliver both archives with that receipt.

nix build .#tricu-portable .#tricu-portable-replacement-test --no-link
CHECK=$(nix path-info .#tricu-portable-replacement-test)
cat "$CHECK/replacement.json" "$CHECK/report.json"

The replacement check adds a non-allocated ELF metadata section to each bundled LGPL library (libgmp, libelf, libdw, libnuma), verifies changed hashes and runs the same Nix-free workload at both relocation paths. This checks loading and execution with metadata-modified, ABI-compatible shared objects. The fast Nix gate covers owner ambiguity/missing owners, embedded notices, changed source/notice files, symlink manifests and archive pairing.

Cross-distribution container acceptance

The tricu-distro-test runner separates networked image preparation from offline acceptance. It uses distro-provided Python, system glibc and the actual transfer archive. Target userspaces are Ubuntu 24.04, Debian 13 and current Fedora; Ubuntu 22.04 is an expected GLIBC-version rejection control. Python/userspace acceptance has covered Ubuntu 24.04 with glibc 2.39/Python 3.12, Debian 13 with glibc 2.41/Python 3.13, and Fedora 44 with glibc 2.43/Python 3.14. The matrix records image, OS, glibc, Python and host-kernel identities for reproduction.

Use a working local rootless Podman or Docker engine on Linux x86-64. First test the proposed baseline and older-glibc control, choosing new local directories:

nix run .#tricu-distro-test -- prepare --engine podman \
  --directory /tmp/tricu-distro-images --distro ubuntu24 --distro ubuntu22
nix run .#tricu-distro-test -- run --engine podman \
  --lock /tmp/tricu-distro-images/images.json --output /tmp/tricu-distro-results
cat /tmp/tricu-distro-results/report.json

Select --engine docker for Docker. Omit both --distro arguments during preparation to select all four targets. Preparation pulls linux/amd64 images, resolves registry digests, writes digest-pinned Dockerfiles and installs Python inside derived images. The local images.json pins the resulting image content IDs and hashes the Dockerfiles/package inventories. Package repositories can change, so rebuilding a prepared image is a new observation. Keep the prepared images and evidence for replay; engine save/load can transfer those exact images. The runner preserves prepared images and evidence for replay.

Acceptance verifies local image identities, uses --pull never, disables network access and runs as an unprivileged container user with a read-only root, dropped capabilities, no-new-privileges, 1-GiB memory and bounded process/time/scratch limits. The container mount contains a copied test kit with compiled identity/append/copier/filecheck workers and the shared workload. The suite covers two archive extraction locations, complete file hashes, repeated success/error calls, CLI mutation detection and fresh-process ABI/IO/scope checks. Ubuntu 22.04 requires a ctypes loader error identifying a specifically unavailable newer GLIBC symbol version to pass its rejection control.

Each distro produces a log and JSON report. A combined report.json is written only after every selected case passes, and includes image identities, image-lock and transfer hashes and the complete test-kit file hashes. Partial failures retain per-distro logs. The VM workflow below exercises the Ubuntu GA guest kernel.

Upstream Node and distro PHP profile

Three-consumer acceptance has passed at both relocation paths with upstream Node 22.22.0 and Koffi 2.16.2 on these userspaces:

Userspace	glibc	Distro Python	Distro PHP CLI/FFI
Ubuntu 24.04	2.39	3.12.3	8.3.6
Debian 13	2.41	3.13.5	8.4.27
Fedora 44	2.43	3.14.8	8.5.12

These observations share Linux kernel 6.18.42 and establish CLI consumer acceptance for the recorded runtime versions. The image lock and reports record exact image identities, transfer/test-kit hashes and independent expected results.

Use --bindings --node-version VERSION during preparation to add CLI binding conformance. VERSION is an exact upstream Node release of at least 22; for example:

nix run .#tricu-distro-test -- prepare --engine podman \
  --directory /tmp/tricu-bindings-images --bindings --node-version 22.22.0 \
  --distro ubuntu24 --distro debian13 --distro fedora
nix run .#tricu-distro-test -- run --engine podman \
  --lock /tmp/tricu-bindings-images/images.json --output /tmp/tricu-bindings-results

Preparation pins the official node:VERSION-bookworm-slim image by registry digest and copies its /usr/local Node/npm installation into each distro image. Node's native support libraries and PHP/FFI come from distro packages. Koffi is installed by npm ci against the kit's committed package lock during preparation; acceptance uses those installed bytes offline. The image lock records the Node source image and npm input hashes. Changing those npm inputs requires preparing new images. Existing Python-only image locks retain their original profile.

On each compatible distro, both relocation paths run the shared Python/JS/PHP conformance fixtures against the same portable host: exact uint64 and UTF-8/byte codecs, bundle exports, ownership and error controls, and the compiled copier's success/cancellation/error and descriptor-cleanup cases. PHP uses process-local ffi.enable=true. JS/PHP checks also reject Nix mappings and require the selected host in their process mappings. Reports identify upstream Node, npm Koffi and distro PHP versions separately; a bindings-profile success requires all three consumers. The outer bindings-profile timeout is 1,200 seconds per distro.

Ubuntu 22.04 runs the ctypes GLIBC rejection control when selected with this profile. The kit build runs all three staged CLI consumers against the portable host using Nix-provided runtimes, including the copier helper imports, and writes bindings-smoke.json. The staged smoke and engine-free harness controls validate kit assembly; container runs produce the cross-distribution acceptance reports.

The test kit can also be copied to a Linux machine without Nix:

nix build .#tricu-distro-test-kit --no-link
cp -R "$(nix path-info .#tricu-distro-test-kit)" /tmp/tricu-distro-kit
# Transfer the copied directory, then use host Python 3.10+ and the container engine:
python3 /tmp/tricu-distro-kit/container_matrix.py --help

Fast classifier, archive-safety and image-lock controls run in nix flake check. Run tricu-distro-test with a working container engine to produce matrix evidence.

Ubuntu GA kernel VM acceptance

VM acceptance has passed on Ubuntu 24.04.5 x86-64 with kernel 6.8.0-147-generic, Python 3.12.3, upstream Node 22.22.0, Koffi 2.16.2 and PHP CLI/FFI 8.3.6. QEMU 10.2.2 with KVM ran both relocation paths with networking disabled. The recorded checks cover Nix-free execution, three-consumer conformance, compiled copier IO, repeated success/error calls, descriptor stability and filecheck mutation detection. The image lock preserves the kernel/config hashes, package inventory and frozen input identities for replay.

tricu-vm-test prepares an Ubuntu 24.04 amd64 cloud guest and runs the three consumer workloads on its GA 6.8 kernel. QEMU uses 2 virtual CPUs, 4 GiB RAM and a 16-GiB virtual disk. KVM is the default accelerator; --accel tcg selects software emulation. Run the commands on a Linux host with the selected accelerator available.

Obtain the Ubuntu 24.04 amd64 qcow2 cloud image from Ubuntu cloud images and its published SHA256 checksum. Preparation verifies the supplied checksum, copies the image, and records its identity. Keep the accepted Ubuntu bindings container image and its image-lock directory available: preparation exports that image's pinned Node/npm and Koffi installations using Podman or Docker.

nix run .#tricu-vm-test -- prepare \
  --image /tmp/noble-server-cloudimg-amd64.img \
  --image-sha256 PASTE_THE_PUBLISHED_SHA256 \
  --container-lock /tmp/tricu-bindings-images-v2/images.json \
  --directory /tmp/tricu-vm-images --engine podman

nix run .#tricu-vm-test -- run \
  --lock /tmp/tricu-vm-images/images.json \
  --output /tmp/tricu-vm-results

Choose new output directories for each preparation or run. Preparation boots with outbound networking, installs Ubuntu Python/PHP/FFI support packages, and records the kernel release, kernel/config hashes, package inventory and consumer inputs. The default kernel prefix is 6.8.. The prepared disk and its base image are hashed after guest shutdown; images.json binds them to the frozen transfer kit and consumer payloads.

Acceptance verifies those inputs and boots a fresh copy-on-write overlay with networking disabled. The guest runs the two relocation paths as nobody, checks the independent Python/Node/PHP expectations and records Nix-free execution. Host-side validation requires the prepared kernel, package inventory and complete consumer results. Serial output goes to serial.log, QEMU diagnostics to qemu.log, and the guest writes guest-report.json through a dedicated virtio serial port. Successful acceptance produces the combined report.json with host/guest identities, QEMU version, accelerator and artifact hashes.

Keep the preparation directory intact for replay. Each run retains its own overlay, seed ISO and diagnostics. The transferable tricu-vm-test-kit includes the runner and guest scripts; the host runner uses Python 3.11+, QEMU and genisoimage. The Nix harness gate exercises seed-ISO construction, VM network/drive arguments, image integrity and rejection of incomplete or changed-kernel reports.

Python example

The library artifact installs share/tricu/tricu.py. The tricu-python Nix package also provides the installable tricu-runtime Python module; select it in a Python environment through python3.withPackages. Its installed module finds the packaged native library automatically. With checkout/PyPI-style module installation, set TRICU_LIB to an explicitly built library path. Publication to PyPI is separate from the tracked packaging recipe.

Library(path) selects explicitly; Library() checks TRICU_LIB and installed library locations. One library is kept loaded per process. Context.number and to_number use exact Python integers in the uint64 range. to_bool checks boolean shapes, and child returns an owned handle. Strings and bytes retain embedded NULs. Context managers close contexts deterministically.

PYTHONPATH="$EMBED/share/tricu" python3 - "$EMBED/lib/libtricu-haskell.so" <<'PY'
import sys
from pathlib import Path
from tricu import Library

library = Library(sys.argv[1])
with library.context() as ctx:
    identity = ctx.load_bundle(Path('test/fixtures/id.arboricx').read_bytes(), 'id')
    argument = ctx.string('hello from Haskell')
    result = ctx.apply(identity, argument)
    print(ctx.to_string(result))
PY

For a complete installed consumer, see the Python file-integrity application. Its tricu worker is compiled at build time and packaged with the Python application.

Scoped handles and curried application

For a reusable context, Context.scope() owns temporary roots created through that scope. Constructors, retain, reduction, child inspection, bundle loading and application return tracked roots; run_io tracks both result roots. Borrowed inputs, including a worker loaded directly through the context, retain their original owners. Observations use the context's ordinary decoding methods.

with ctx.scope() as roots:
    argument = roots.string('hello')
    result = roots.apply_many(worker, argument)
    text = ctx.to_string(result)
# argument and result are released; text is an ordinary Python string.

apply_many(function, *arguments) borrows its inputs and returns one owned curried application, releasing private intermediate handles. With no arguments it returns an independently retained function handle. scope.own(handle) adopts an already owned root without retaining it; detach transfers ownership to the caller, and release explicitly retires a scoped root early. Duplicate adoption is rejected. Scope exit releases remaining roots in reverse order, also on exceptions. A context closed first has already released all its roots. Cleanup attempts every remaining release and preserves an existing body exception if cleanup also fails. Handles created directly through the context remain caller-owned.

Context.run_io(program) returns (outcome, value_handle, cleanup_handle). Outcome 0 means completed, 1 failed, and 2 cancelled. A completed value may itself be an ordinary error Result. Failure/cancellation reasons remain opaque lazy trees; cleanup is a list of IO problem values. Both returned handles are owned, including an empty cleanup list. Python boundary failures raise TricuError with the native status and diagnostic text.

Lifecycle and ownership

Call tricu_init before any other library entrypoint. Initialization is process-wide and idempotent, owns GHC runtime startup, and uses one default GHC capability. It ignores GHCRTS and host command-line RTS arguments and disables GHC application signal-handler installation, preserving the host's SIGINT/SIGPIPE handlers. This entrypoint is for processes that have not already initialized GHC. Keep one copy of the library loaded until process exit. Linux builds mark the library NODELETE so request-scoped loaders cannot unmap an active GHC runtime; repeated dlopen/dlclose and persistent PHP-request controls validate that lifetime. Host applications that already embed GHC or need several Haskell libraries require coordinated RTS integration. Runtime shutdown, unloading, and using inherited contexts after fork() are outside this interface; start a fresh process instead.

Contexts and handles are validated unsigned 64-bit IDs in a process-wide, nonreusing namespace. Handles belong to one context. Each successful handle output owns a root, including child inspection, retain and reduction. Inputs are borrowed. retain returns a different handle to the same Haskell value. Release removes that root; context destruction releases all remaining roots. Garbage collection and resident-memory reclamation are managed by GHC. Long-lived clients should release intermediate handles instead of accumulating them until context teardown.

Input buffers are copied during the call. UTF-8 input is strict scalar text; embedded NULs and U+FEFF are ordinary string contents. Outputs are length-delimited, independent native allocations. Free each exactly once with tricu_buffer_free, even empty buffers. Output buffers may outlive their originating context. Python's binding copies these into owned Python bytes and frees the native buffer.

Calls on a context serialize, including evaluation and IO. Different contexts can make concurrent calls through the threaded RTS. Context destruction waits for an active call; subsequent calls return invalid-context. Calls provide no callbacks or reentrant entry. Native IO uses OS process authority and shares process standard streams, environment and other process resources; contexts are not security isolation. See the IO contract, especially standard-stream ownership and per-run lease exclusion.

Demand and errors

Application creates a lazy suspension. reduce demands weak-head normal form; constructor inspection and decoding demand the parts they observe. Constructors and retain preserve lazy children. Handles represent values rather than explicit application nodes. Decoding uses the existing Haskell value codecs. Bundle loading verifies the Arboricx container and requires an explicit UTF-8 export name.

The additive tricu_to_bool entrypoint decodes Leaf as false and Stem Leaf as true, returning a type failure for other shapes. Bindings require the selected library to provide this entrypoint along with the rest of the header.

Boundary statuses distinguish invalid context, invalid handle, invalid input, codec/type failure, Haskell exception, and identity exhaustion. Context-local last_error returns diagnostic UTF-8 bytes, initially empty and retained across successful calls. Task outcomes use the separate IO result outputs above. Concurrent callers needing the matching last-error text should coordinate the failed call and diagnostic retrieval.

Outputs publish after fallible preparation and remain unchanged on failure. Pointers must refer to valid storage, and output regions must not overlap. Exceptions are contained at the foreign boundary; completed IO effects and progress of shared lazy evaluation remain observable after a failed call.

Reduction, observation and IO can run indefinitely. Use supervised subprocesses and OS resource limits to enforce deadlines, cancellation and memory budgets for untrusted or bounded work, and to isolate fatal RTS failures, process signals and heap exhaustion. Context destruction waits for active computation to finish.

Validation scope

Basic checks cover constructors, portable identity/append bundles, full byte and Unicode results, ownership, stale/foreign handles, malformed inputs, output publication, concurrent entry, repeated context teardown, contained exceptions, lazy application/discard, and opaque task diagnostics. Installed-library tests exercise the packaged runtime paths in a new Python process.

The integration target compiles the existing IO copier and checks repeated 64-KiB binary copies, missing-source and missing-parent Results, transferred-resource cancellation, and descriptor counts on Linux. Its compiler probe runs a cold two-module build through the shared library and through the CLI. It compares every library-published object against the CLI store, authenticates the receipt, requires byte-identical program bundles, and executes the result as 41 through the library.

Reports include small-call and bulk-codec timings and fresh-process compiler-job timings. The supervised compiler comparison sets MALLOC_ARENA_MAX=2 for both subprocesses so glibc's per-thread arena reservations fit alongside GHC/Python under the FFI child's 1-GiB address-space limit. The CPU/time limits remain 120 seconds. The allocator setting is process-local to these test jobs. The reports measure complete correctness workloads: the CLI imports packaged compiler data and packages its program bundle. Nix may reuse reports; rerun the tracked Python scripts against built artifacts for fresh observations. The compiler probe is an application build, separate from compiler-generation/fixed-point acceptance.