commit 44353a5b2d2705689f45e18d4182fb84a5e3e480
parent 9227922e4d67f6df3c79f783d5b3bd5f94acd5cf
Author: James Eversole <james@eversole.co>
Date: Sun, 25 Oct 2026 11:41:43 -0500
Consolidate native embedding and ship Nix-free Linux bundles
Use one full Haskell host ABI with dynamic and opt-in archive linkage. Package relocatable Linux x86-64 runtime dependencies with paired source materials, upstream notices, link-input provenance and integrity receipts.
Add pinned distro-container and Ubuntu GA kernel VM workflows for Python, Node and PHP CLI embedding. Exercise relocation, codecs, ownership, compiled IO, error handling and compatible shared-library replacement. Document deployment, acceptance configurations and redistribution requirements.
Validated with Nix checks, both linkage integrations, native bindings, linkage smoke, portable isolation and library replacement tests. External acceptance covers Ubuntu 24.04, Debian 13 and Fedora 44 userspaces plus Ubuntu kernel 6.8.0-147-generic under KVM.
Diffstat:
41 files changed, 2667 insertions(+), 403 deletions(-)
diff --git a/docs/build-and-run.md b/docs/build-and-run.md
@@ -4,11 +4,14 @@ Run these commands from the extracted tricu directory. You need [Nix](https://ni
with flakes enabled. Native development uses the Haskell host; builds and tests
run through Nix. The [native Haskell C interface](../ext/haskell/README.md)
provides embedding through C, Python, [native JavaScript](../ext/js/README.md),
-and [PHP](../ext/php/README.md). A [core-only C prototype](../ext/haskell/README.md#core-only-prototype)
-provides lazy tree reduction and value codecs for applications with their own IO:
-`nix build .#tricu-core-archive --no-link`. This x86-64 Linux variant links PIC
-Haskell archives into the shared object; `tricu-core` retains the dynamic-library
-baseline. The [compiler rebuilding guide](compiler-rebuilding.md.html#execution-interface-selection)
+and [PHP](../ext/php/README.md). An opt-in
+[archive-linked full host](../ext/haskell/README.md#full-host-linkage-variants)
+incorporates PIC Haskell archives into one shared object on x86-64 Linux:
+`nix build .#tricu-haskell-ffi-archive --no-link`. Both linkage variants expose
+reduction, bundle loading and IO and require their native Nix runtime closures.
+An opt-in [Nix-free transfer prototype](../ext/haskell/README.md#nix-free-deployment-prototype)
+packages native dependencies with relative loader paths and tests execution with
+`/nix/store` absent. The [compiler rebuilding guide](compiler-rebuilding.md.html#execution-interface-selection)
covers CLI/shared-library execution checks and compiler acceptance.
## Build the host
@@ -140,7 +143,7 @@ uses whole-value reads. Each active snapshot uses at most the configured artifac
byte limit in temporary disk space under `store/tmp`. Provision that filesystem
for the admission window, including slow clients holding verified snapshots.
Storage initialization and socket bind/listen failures exit nonzero. Monitor the
-HTTP health endpoint externally; the minimal image has no curl-based healthcheck.
+HTTP health endpoint externally.
Endpoints include the landing page, `GET /_arboricx/health`,
`POST /_arboricx/bundle` (201 with a SHA-256 hash),
@@ -237,7 +240,7 @@ multiline blocks, imports, loading files, and session operations; see
The v1 handler, chunked-upload and body-probe runner outputs are retired from
current Nix packages. Their source and executable evidence belong to the frozen
-rollback toolchains; no current runner silently selects or emulates those hosts.
+rollback toolchains.
Use the default IO runner and its acceptance gate above. The explicit
`arboricxStreamingRunner` and `arboricxStreamingTest` outputs exercise the same
supported composition and gates under their descriptive names.
diff --git a/ext/haskell/BoundaryTests.hs b/ext/haskell/BoundaryTests.hs
@@ -1,18 +1,13 @@
-{-# LANGUAGE CPP #-}
{-# LANGUAGE ScopedTypeVariables #-}
--- Inject bottoms through the Haskell-side registry, never a production C hook.
+-- Inject bottoms through the Haskell-side registry for boundary fault tests.
module Main where
import qualified EmbeddingCore as E
-#ifndef CORE_ONLY
import qualified Embedding as Host
-#endif
import Control.Monad (unless)
import Control.Monad.IO.Class (liftIO)
import Control.Concurrent
import Runtime.Tree (T(..))
-#ifndef CORE_ONLY
import Runtime.Value (ofString, ofNumber)
-#endif
import Foreign
import Data.Word
import Control.Exception (evaluate)
@@ -43,7 +38,6 @@ main = alloca $ \ctxOut -> alloca $ \out -> do
E.toU64 ctx value out >>= assert . (== 0)
peek out >>= assert . (== (0 :: Word64))
-#ifndef CORE_ONLY
-- Terminal task diagnostics remain opaque across the native boundary.
E.withContext ctx (E.publish out
(Fork (ofString "tricuIO") (Fork (ofNumber 2)
@@ -54,7 +48,6 @@ main = alloca $ \ctxOut -> alloca $ \out -> do
peek tag >>= assert . (== 1)
failed <- peek reason
E.reduce ctx failed out >>= assert . (== 5)
-#endif
-- An interrupted call restores the context lock and contains its exception.
entered <- newEmptyMVar
done <- newEmptyMVar
diff --git a/ext/haskell/README.md b/ext/haskell/README.md
@@ -65,111 +65,285 @@ plus scoped `native_math` generations and request-tampering rejection. These are
application/library workflows; compiler-pair acceptance and retention use the
[compiler rebuilding](../../docs/compiler-rebuilding.md) procedure.
-## Core-only prototype
-
-`tricu-core` provides the common C surface in
-[`include/tricu-core.h`](include/tricu-core.h): contexts, owned handles, tree
-constructors, lazy application/reduction, constructor inspection, number/boolean/
-byte/UTF-8 codecs, and boundary diagnostics. Application and observation have the
-same demand behavior as the full host. The host application supplies its own IO;
-this artifact has neither bundle loading nor a tricu IO entrypoint.
+## Full-host linkage variants
+
+Both native artifacts expose the complete ABI in [`include/tricu.h`](include/tricu.h):
+reduction, value codecs, authenticated bundle loading and native IO.
+`tricu-haskell-ffi` links Haskell dependencies dynamically;
+`tricu-haskell-ffi-archive` incorporates PIC Haskell archives and the threaded RTS
+in one shared object, with an ELF export map and unused-section collection.
+The archive variant is opt-in on x86-64 Linux. Its PIC compiler, boot libraries
+and dependency archives share one package set. The first build may compile that
+entire dependency graph. Native system libraries retain their Nix runtime paths;
+keep the complete runtime closure available for either variant.
```sh
-nix build .#tricu-core .#tricu-core-archive .#tricu-haskell-ffi --no-link
-CORE=$(nix path-info .#tricu-core)
-ARCHIVE=$(nix path-info .#tricu-core-archive)
-EMBED=$(nix path-info .#tricu-haskell-ffi)
-python3 ext/haskell/footprint.py --core "$CORE" --host "$EMBED" \
- --archive "$ARCHIVE" --output /tmp/tricu-runtime-footprint.json
+nix build .#tricu-haskell-ffi-archive .#tricu-haskell-ffi-archive-integration --no-link
+CHECK=$(nix path-info .#tricu-haskell-ffi-archive-integration)
+cat "$CHECK/integration.json" "$CHECK/compiler.json"
```
-Both core artifacts install `lib/libtricu-core.so` and `include/tricu-core.h`.
-`tricu-core` includes its dynamic Haskell dependencies alongside it.
-`tricu-core-archive` incorporates the required Haskell code and threaded RTS into
-one shared object, exporting the documented C API. It uses PIC archives from a
-GHC build configured with `enableRelocatedStaticLibs`, an ELF export map and
-unused-section collection. Native system libraries remain dynamic. Its first
-build requires a source build of GHC and its bundled libraries; profiling-library
-and documentation variants are omitted from that build. Normal compilation
-remains `-O2`, using the same reducer and complete core interface.
-
-These are C-only prototypes; the existing Python/JS/PHP bindings target the full
-host. `tricu.h` includes the common header and adds bundle/IO declarations.
-Choose **one artifact per process**: all variants provide the same common symbol
-names and own the same process-wide GHC initialization contract. Handles are not transferable between library instances.
-All lifecycle, pointer, ownership, serialization and resource-limit rules below
-apply to both variants.
-
-The split is a source/build dependency boundary. `EmbeddingCore` imports
-`Runtime.Tree` and `Runtime.Value` directly; the full `Embedding` module extends
-that same boundary with the bundle verifier and IO driver. Both use the exact
-same reducer, registry, codecs and initialization code. The core build compiles
-only these three Haskell modules. `src/core.exports` keeps the archive-linked
-artifact's public surface at the common C API. GHC 9.14 archive linkage explicitly
-roots `init_ghc_hs_iface`, matching GHC's executable-link startup interface rule.
-The linker checks both position independence and complete symbol resolution.
-
-The footprint script reports main-library bytes, installed package file/NAR
-sizes, full runtime closure NAR size, package tar.gz size, and gzip-compressed
-Nix closure export size. Compression uses level 6. The package archive requires
-the external runtime paths to remain available; it is not a standalone portable
-archive. The closure export includes entire transitive native packages and is a
-Nix distribution, not a universal native installer. Shared dependencies can
-already exist on a consumer's system, so closure totals are not necessarily
-incremental installation costs. None of these measurements describes resident
-memory. Sizes depend on the pinned platform/toolchain and dependency graph.
-
-All three artifacts run the same linked C ownership, codec and reduction-rule checks,
-Haskell bottom/discard/exception/concurrency controls, and installed-header/client
-checks. Linux builds also test repeated dlopen/dlclose. Fresh-process symbol checks
-require bundle/IO exports only in the host; the trimmed archive artifact also
-checks its complete exported symbol set. `nix flake check` includes the dynamic
-core and full host. The archive build and its comparison smoke are opt-in; the
-full host additionally retains its installed Python and IO tests.
+Both variants run C ownership/codec/reduction checks, Haskell demand/exception/
+concurrency controls, Linux loader-lifetime tests, Python ownership/IO checks
+and installed-header/client checks. The archive variant additionally checks the
+exact public symbol set and absence of Haskell shared dependencies. Its opt-in
+integration runs the compiled IO copier and cold two-module compiler comparison
+against the CLI. Default bindings select the dynamic full host.
+
+Select **one artifact per process**: both own process-wide GHC initialization
+and expose the same symbols. Handles belong to the selected library instance.
+Internally, `EmbeddingCore` isolates reducer/value operations and the registry;
+`Embedding` adds bundle verification and IO using that same boundary.
+
+### Footprint and linkage measurements
```sh
-nix build .#tricu-core-linkage-smoke --no-link
-SMOKE=$(nix path-info .#tricu-core-linkage-smoke)
+nix build .#tricu-haskell-ffi .#tricu-haskell-ffi-archive --no-link
+python3 ext/haskell/footprint.py \
+ --host "$(nix path-info .#tricu-haskell-ffi)" \
+ --host-archive "$(nix path-info .#tricu-haskell-ffi-archive)" \
+ --output /tmp/tricu-runtime-footprint.json
+nix build .#tricu-haskell-linkage-smoke --no-link
+SMOKE=$(nix path-info .#tricu-haskell-linkage-smoke)
cat "$SMOKE/summary.json"
```
-The comparison calls the same public C API in separate processes. Each workload
-creates fresh constant/distribution application chains, releases temporary roots,
-demands the result, and checks the complete result is `Leaf`. Graph-construction
-and weak-head-demand times are separate; one warmup precedes seven measured
-rounds at each size. Timings and peak RSS are observations without speed
-thresholds. These two finite workloads are performance smokes, not general
-application benchmarks. Installed `share/tricu/build.json` records GHC version
-and linkage for each artifact.
+The footprint report distinguishes library bytes, installed package file/NAR
+sizes, runtime closure NAR size, package tar.gz size and gzip-compressed Nix
+closure export size (compression level 6). Package archives require external
+runtime paths; closure exports are Nix distributions. Closure totals include
+whole transitive native packages.
+
+The linkage smoke calls the same public C API in separate processes. Each
+workload constructs fresh constant/distribution application chains, releases
+temporary roots, demands the result and checks that it is `Leaf`. Graph-building
+and weak-head-demand timings are separate, with one warmup and seven measured
+rounds per size. The report records peak RSS and timings for these workloads. Installed `share/tricu/build.json` identifies
+the compiler and linkage.
+
+## Nix-free deployment prototype
+
+`tricu-portable` packages the archive-linked full host and non-glibc native
+libraries into a relocatable Linux x86-64 tarball. ELF dependencies use `$ORIGIN`;
+system glibc and the consumer's language runtime remain external. See the
+[included transfer instructions](portable-README.md) for the layout, compatibility
+scope and paired source-delivery requirements.
+
+```sh
+nix build .#tricu-portable .#tricu-portable-test --no-link
+PORTABLE=$(nix path-info .#tricu-portable)
+CHECK=$(nix path-info .#tricu-portable-test)
+cat "$CHECK/report.json"
+# Distribute the runtime and matching source materials together:
+ls "$PORTABLE/tricu-linux-x86_64.tar.gz" "$PORTABLE/tricu-source-materials.tar.gz"
+cat "$PORTABLE/release.json"
+```
+
+The opt-in Linux acceptance requires working unprivileged user/mount namespaces.
+It extracts the actual archive at two locations, including spaces, and runs
+Python ABI/scoped-handle controls, compiled IO copying and the installed
+filecheck worker with independent digest oracles. Repeated successful/error
+calls check descriptor stability; CLI controls check record/verify and mutation.
+An empty mount root exposes only a relocated Python/glibc test OS, the release,
+fixtures and writable scratch. The test asserts `/nix/store` is absent, checks
+loaded mappings, and rejects a missing bundled GMP dependency. The test OS's
+`LD_LIBRARY_PATH` names only its synthetic system-library directory; release
+libraries resolve through their own relative paths. This isolated test checks
+Nix-free execution. `portable.json` records the required GLIBC symbol floor.
+
+### Notices, source materials and replacement
+
+The [redistribution contract](REDISTRIBUTION.md) describes the paired runtime
+and source archives, license choices and modification/replacement rights.
+`tricu-redistribution` resolves the link map's archive/object inputs and bundled
+ELF dependencies to actual source owners. The versioned component policy rejects
+unreviewed source distributions and missing required notices. It preserves named
+license texts and relevant leading C-family copyright/permission blocks, including
+vendored crypto implementations. The source archive includes upstream inputs,
+patches, Cabal revisions, complete tricu/Nixpkgs build recipes, native flake helper
+sources, and file/symlink hash inventories. Source-only notice texts are retained
+conservatively and identified as such.
+
+`tricu-portable` verifies source ownership and original library hashes before
+relocation. It installs the notice corpus, pairs the runtime with the source
+manifests, and verifies both archive contents before emitting `release.json`.
+Deliver both archives with that receipt.
+
+```sh
+nix build .#tricu-portable .#tricu-portable-replacement-test --no-link
+CHECK=$(nix path-info .#tricu-portable-replacement-test)
+cat "$CHECK/replacement.json" "$CHECK/report.json"
+```
+
+The replacement check adds a non-allocated ELF metadata section to each bundled
+LGPL library (`libgmp`, `libelf`, `libdw`, `libnuma`), verifies changed hashes and
+runs the same Nix-free workload at both relocation paths. This checks loading and
+execution with metadata-modified, ABI-compatible shared objects. The fast Nix gate
+covers owner ambiguity/missing owners,
+embedded notices, changed source/notice files, symlink manifests and archive
+pairing.
-### Archive-link readiness probe (x86-64 Linux)
+### Cross-distribution container acceptance
-A shared library linked from Haskell archives requires position-independent
-objects throughout its Haskell dependency graph, including the threaded RTS.
-The diagnostic target exercises the ordinary non-PIC compiler's prerequisite
-failure with the same core sources,
-C interface and ordinary `-O2` settings:
+The `tricu-distro-test` runner separates networked image preparation from offline
+acceptance. It uses distro-provided Python, system glibc and the actual transfer
+archive. Target userspaces are Ubuntu 24.04, Debian 13 and current Fedora; Ubuntu
+22.04 is an expected GLIBC-version rejection control. Python/userspace acceptance
+has covered Ubuntu 24.04 with glibc 2.39/Python 3.12, Debian 13 with glibc
+2.41/Python 3.13, and Fedora 44 with glibc 2.43/Python 3.14. The matrix records
+image, OS, glibc, Python and host-kernel identities for reproduction.
+
+Use a working local rootless Podman or Docker engine on Linux x86-64. First test
+the proposed baseline and older-glibc control, choosing new local directories:
+
+```sh
+nix run .#tricu-distro-test -- prepare --engine podman \
+ --directory /tmp/tricu-distro-images --distro ubuntu24 --distro ubuntu22
+nix run .#tricu-distro-test -- run --engine podman \
+ --lock /tmp/tricu-distro-images/images.json --output /tmp/tricu-distro-results
+cat /tmp/tricu-distro-results/report.json
+```
+
+Select `--engine docker` for Docker. Omit both `--distro` arguments during
+preparation to select all four targets. Preparation pulls linux/amd64 images,
+resolves registry digests, writes digest-pinned Dockerfiles and installs Python
+inside derived images. The local `images.json` pins the resulting image content
+IDs and hashes the Dockerfiles/package inventories. Package repositories can
+change, so rebuilding a prepared image is a new observation. Keep the prepared
+images and evidence for replay; engine save/load can transfer those exact images.
+The runner preserves prepared images and evidence for replay.
+
+Acceptance verifies local image identities, uses `--pull never`, disables network
+access and runs as an unprivileged container user with a read-only root, dropped
+capabilities, no-new-privileges, 1-GiB memory and bounded process/time/scratch limits.
+The container mount contains a copied test kit with compiled
+identity/append/copier/filecheck workers and the shared workload.
+The suite covers two archive extraction locations, complete file hashes, repeated
+success/error calls, CLI mutation detection and fresh-process ABI/IO/scope checks.
+Ubuntu 22.04 requires a ctypes loader error identifying a specifically unavailable
+newer GLIBC symbol version to pass its rejection control.
+
+Each distro produces a log and JSON report. A combined `report.json` is written
+only after every selected case passes, and includes image identities, image-lock
+and transfer hashes and the complete test-kit file hashes. Partial failures retain
+per-distro logs. The VM workflow below exercises the Ubuntu GA guest kernel.
+
+#### Upstream Node and distro PHP profile
+
+Three-consumer acceptance has passed at both relocation paths with upstream
+Node 22.22.0 and Koffi 2.16.2 on these userspaces:
+
+| Userspace | glibc | Distro Python | Distro PHP CLI/FFI |
+| --- | --- | --- | --- |
+| Ubuntu 24.04 | 2.39 | 3.12.3 | 8.3.6 |
+| Debian 13 | 2.41 | 3.13.5 | 8.4.27 |
+| Fedora 44 | 2.43 | 3.14.8 | 8.5.12 |
+
+These observations share Linux kernel 6.18.42 and establish CLI consumer
+acceptance for the recorded runtime versions. The image lock and reports record
+exact image identities, transfer/test-kit hashes and independent expected results.
+
+Use `--bindings --node-version VERSION` during preparation to add CLI binding
+conformance. `VERSION` is an exact upstream Node release of at least 22; for example:
+
+```sh
+nix run .#tricu-distro-test -- prepare --engine podman \
+ --directory /tmp/tricu-bindings-images --bindings --node-version 22.22.0 \
+ --distro ubuntu24 --distro debian13 --distro fedora
+nix run .#tricu-distro-test -- run --engine podman \
+ --lock /tmp/tricu-bindings-images/images.json --output /tmp/tricu-bindings-results
+```
+
+Preparation pins the official `node:VERSION-bookworm-slim` image by registry
+digest and copies its `/usr/local` Node/npm installation into each distro image.
+Node's native support libraries and PHP/FFI come from distro packages. Koffi is
+installed by `npm ci` against the kit's committed package lock during preparation;
+acceptance uses those installed bytes offline. The image lock records the Node
+source image and npm input hashes. Changing those npm inputs requires preparing
+new images. Existing Python-only image locks retain their original profile.
+
+On each compatible distro, both relocation paths run the shared Python/JS/PHP
+conformance fixtures against the same portable host: exact uint64 and UTF-8/byte
+codecs, bundle exports, ownership and error controls, and the compiled copier's
+success/cancellation/error and descriptor-cleanup cases. PHP uses process-local
+`ffi.enable=true`. JS/PHP checks also reject Nix mappings and require the selected
+host in their process mappings. Reports identify upstream Node, npm Koffi and
+distro PHP versions separately; a bindings-profile success requires all three
+consumers. The outer bindings-profile timeout is 1,200 seconds per distro.
+
+Ubuntu 22.04 runs the ctypes GLIBC rejection control when selected with this
+profile. The kit build runs all three staged CLI consumers against the portable
+host using Nix-provided runtimes, including the copier helper imports, and writes
+`bindings-smoke.json`. The staged smoke and engine-free harness controls validate
+kit assembly; container runs produce the cross-distribution acceptance reports.
+
+The test kit can also be copied to a Linux machine without Nix:
```sh
-nix build .#tricu-core-archive-probe --no-link
-PROBE=$(nix path-info .#tricu-core-archive-probe)
-cat "$PROBE/report.json"
-gzip -dc "$PROBE/link.log.gz" | head -40
+nix build .#tricu-distro-test-kit --no-link
+cp -R "$(nix path-info .#tricu-distro-test-kit)" /tmp/tricu-distro-kit
+# Transfer the copied directory, then use host Python 3.10+ and the container engine:
+python3 /tmp/tricu-distro-kit/container_matrix.py --help
```
-The report distinguishes a successful link from recognized non-PIC archive
-failures; unrelated compiler/linker errors fail the diagnostic build. It records
-the exact command and affected archive names. Its output is diagnostic evidence,
-not an installable or accepted runtime. A link success additionally runs the C
-client and export checks; installed packaging, lifecycle, and performance
-acceptance are separate requirements. A blocked link has no size or performance
-measurement.
+Fast classifier, archive-safety and image-lock controls run in `nix flake check`.
+Run `tricu-distro-test` with a working container engine to produce matrix evidence.
+
+### Ubuntu GA kernel VM acceptance
+
+VM acceptance has passed on Ubuntu 24.04.5 x86-64 with kernel
+`6.8.0-147-generic`, Python 3.12.3, upstream Node 22.22.0, Koffi 2.16.2 and
+PHP CLI/FFI 8.3.6. QEMU 10.2.2 with KVM ran both relocation paths with networking
+disabled. The recorded checks cover Nix-free execution, three-consumer conformance,
+compiled copier IO, repeated success/error calls, descriptor stability and
+filecheck mutation detection. The image lock preserves the kernel/config hashes,
+package inventory and frozen input identities for replay.
+
+`tricu-vm-test` prepares an Ubuntu 24.04 amd64 cloud guest and runs the three
+consumer workloads on its GA 6.8 kernel. QEMU uses 2 virtual CPUs, 4 GiB RAM and a
+16-GiB virtual disk. KVM is the default accelerator; `--accel tcg` selects software
+emulation. Run the commands on a Linux host with the selected accelerator available.
+
+Obtain the Ubuntu 24.04 amd64 qcow2 cloud image from
+[Ubuntu cloud images](https://cloud-images.ubuntu.com/noble/) and its published
+SHA256 checksum. Preparation verifies the supplied checksum, copies the image,
+and records its identity. Keep the accepted Ubuntu bindings container image and
+its image-lock directory available: preparation exports that image's pinned
+Node/npm and Koffi installations using Podman or Docker.
+
+```sh
+nix run .#tricu-vm-test -- prepare \
+ --image /tmp/noble-server-cloudimg-amd64.img \
+ --image-sha256 PASTE_THE_PUBLISHED_SHA256 \
+ --container-lock /tmp/tricu-bindings-images-v2/images.json \
+ --directory /tmp/tricu-vm-images --engine podman
+
+nix run .#tricu-vm-test -- run \
+ --lock /tmp/tricu-vm-images/images.json \
+ --output /tmp/tricu-vm-results
+```
-Nixpkgs' `enableRelocatedStaticLibs` GHC build option supplies PIC compilation for
-static libraries. Using it requires rebuilding the compiler's bundled libraries
-and RTS, plus PIC-compatible archives for any separately built dependencies.
-Passing `-fPIC` when compiling tricu itself changes only tricu's objects; it does
-not change prebuilt package archives.
+Choose new output directories for each preparation or run. Preparation boots with
+outbound networking, installs Ubuntu Python/PHP/FFI support packages, and records
+the kernel release, kernel/config hashes, package inventory and consumer inputs.
+The default kernel prefix is `6.8.`. The prepared disk and its base image are
+hashed after guest shutdown; `images.json` binds them to the frozen transfer kit
+and consumer payloads.
+
+Acceptance verifies those inputs and boots a fresh copy-on-write overlay with
+networking disabled. The guest runs the two relocation paths as `nobody`, checks
+the independent Python/Node/PHP expectations and records Nix-free execution.
+Host-side validation requires the prepared kernel, package inventory and complete
+consumer results. Serial output goes to `serial.log`, QEMU diagnostics to
+`qemu.log`, and the guest writes `guest-report.json` through a dedicated virtio
+serial port. Successful acceptance produces the combined `report.json` with
+host/guest identities, QEMU version, accelerator and artifact hashes.
+
+Keep the preparation directory intact for replay. Each run retains its own overlay,
+seed ISO and diagnostics. The transferable `tricu-vm-test-kit` includes the runner
+and guest scripts; the host runner uses Python 3.11+, QEMU and `genisoimage`.
+The Nix harness gate exercises seed-ISO construction, VM network/drive arguments,
+image integrity and rejection of incomplete or changed-kernel reports.
## Python example
@@ -298,11 +472,10 @@ Pointers must refer to valid storage, and output regions must not overlap.
Exceptions are contained at the foreign boundary; completed IO effects and
progress of shared lazy evaluation remain observable after a failed call.
-Reduction, observation and IO can run indefinitely. The interface supplies no
-fuel, hard deadlines, caller cancellation or per-context memory quotas. Fatal RTS
-failures, process signals and arbitrary heap exhaustion are not recoverable status
-guarantees. Use supervised subprocesses and OS resource limits for untrusted or
-bounded work. Context destruction waits for computation; it cannot interrupt it.
+Reduction, observation and IO can run indefinitely. Use supervised subprocesses
+and OS resource limits to enforce deadlines, cancellation and memory budgets for
+untrusted or bounded work, and to isolate fatal RTS failures, process signals and
+heap exhaustion. Context destruction waits for active computation to finish.
## Validation scope
@@ -324,9 +497,8 @@ Reports include small-call and bulk-codec timings and fresh-process compiler-job
timings. The supervised compiler comparison sets `MALLOC_ARENA_MAX=2` for both
subprocesses so glibc's per-thread arena reservations fit alongside GHC/Python
under the FFI child's 1-GiB address-space limit. The CPU/time limits remain 120
-seconds. The allocator setting is process-local to these test jobs. These are
-correctness smokes with measurements, not speed thresholds or
-isolated reducer comparisons: the CLI also imports packaged compiler data and
-packages its program bundle. Nix may reuse reports; rerun the tracked Python
+seconds. The allocator setting is process-local to these test jobs. The reports
+measure complete correctness workloads: the CLI imports packaged compiler data
+and packages its program bundle. Nix may reuse reports; rerun the tracked Python
scripts against built artifacts for fresh observations. The compiler probe is an
application build, separate from compiler-generation/fixed-point acceptance.
diff --git a/ext/haskell/REDISTRIBUTION.md b/ext/haskell/REDISTRIBUTION.md
@@ -0,0 +1,91 @@
+# Redistribution and replacement of native libraries
+
+tricu's own code is licensed under ISC; see `LICENSE`. The native embedding also
+uses third-party code. `THIRD-PARTY-NOTICES.md`, the `notices/` directory and
+`license-components.json` identify the components and preserve upstream license,
+copyright and permission texts. The notices conservatively include source-only
+parts of the upstream distributions and identify the components to which the
+terms apply. The upstream texts govern licensing; package-manager metadata
+provides an advisory index.
+
+## Distribute the paired artifacts
+
+The `tricu-portable` Nix output supplies:
+
+- `tricu-linux-x86_64.tar.gz`: the runtime, header, binding and notices;
+- `tricu-source-materials.tar.gz`: matching upstream sources, patches, Cabal
+ revisions, tricu source, build recipes, lock file and link-input evidence;
+- `release.json`: hashes pairing those two archives and the source manifests.
+
+Provide the source archive with the runtime. For downloads, provide equivalent
+access to both from the same distribution location, clearly linked and without
+an additional source-access charge. Preserve the license texts and source rights
+when copying or modifying either archive. Publish or deliver both archives
+together using this accompanying-source arrangement.
+
+Inside the source archive, `components.json` maps archive/object inputs and
+bundled shared libraries to source owners and records original input hashes.
+`sources/COMPONENT/` holds the unmodified upstream input, separately numbered
+Nix patches, a revised Cabal file where applicable, and evaluated recipe details.
+`recipe.json` uses `@OUTPUT_name@` for build-output placeholders. Full build logic
+is in `sources/nixpkgs/` and `sources/tricu/`, including the embedding build,
+PIC toolchain configuration and native-library relocation script. `SHA256.json`
+and `SYMLINKS.json` authenticate file contents and symbolic-link targets.
+The source materials include the full GHC distribution and its boot libraries,
+and full GCC sources for runtime components. The build command below obtains
+the tools and pinned inputs through Nix.
+
+## Copyleft runtime components
+
+The bundled `libgmp` and elfutils `libelf`/`libdw` use their LGPL-3.0-or-later
+licensing option. `libnuma` uses LGPL 2.1. GCC runtime components, including
+`libatomic`, carry GPLv3 terms with the GCC Runtime Library Exception 3.1.
+The source and notice sets retain the applicable GPL/LGPL and exception texts.
+System glibc remains external; source/notices for the glibc startup objects
+selected by the linker accompany this package as well.
+
+The bundled LGPL libraries remain separately loadable shared objects. You may
+modify and replace compatible libraries and debug those modifications, including
+by reverse engineering for that purpose, under their applicable licenses. The
+runtime loads ABI-compatible replacement libraries through ELF resolution.
+`portable.json` hashes describe the shipped package. Identify modified libraries
+as modified and retain the applicable modification notices.
+
+To use an ABI-compatible replacement, stop the embedding process, replace the
+corresponding file in `tricu/lib/`, and start a fresh process. Preserve the ELF
+SONAME, architecture and required exported ABI. `$ORIGIN` lookup permits the
+replacement to live beside `libtricu-haskell.so`; other libraries already loaded
+by an embedding application can influence ELF resolution. Test with the actual
+host application. The complete tricu source is also supplied for rebuilding or
+relinking against a changed interface. Keep each loaded library in place for
+the lifetime of its embedding process.
+
+## Build and modification record
+
+The 2026.10.24 packaging recipe links PIC Haskell archives and the threaded RTS
+into the full host. Native inputs incorporate the patches and build-time edits
+recorded in their recipe/source directories. The portable packager copies the
+native shared objects and changes ELF RUNPATH to `$ORIGIN`; their upstream source
+archives remain unmodified and their build patches are supplied separately.
+License headers extracted from source files retain their complete leading
+copyright/permission comment blocks. Refer to the original source for context.
+
+From an extracted source-materials directory, the native build uses the tracked
+recipes and selected local build-script snapshots:
+
+```sh
+nix build "path:$PWD/sources/tricu#tricu-haskell-ffi-archive" --no-link --no-write-lock-file \
+ --override-input nixpkgs "path:$PWD/sources/nixpkgs" \
+ --override-input flake-utils "path:$PWD/sources/flake-utils" \
+ --override-input flake-utils/systems "path:$PWD/sources/systems"
+```
+
+Nix may fetch build tools and pinned inputs. The upstream archives and patches
+are provided independently of that fetch mechanism for inspection, modification
+and rebuilding. ABI-compatible dynamic replacements use the existing tricu
+host. If you change the linking model or add dependencies, reassess their
+license requirements and regenerate the source/notice inventory.
+
+The build gates verify source ownership, required notice presence, hashes and
+archive pairing. Review the applicable license terms for your distribution
+and modifications.
diff --git a/ext/haskell/archive-probe.nix b/ext/haskell/archive-probe.nix
@@ -1,16 +0,0 @@
-# Diagnostic output, not an accepted embedding artifact. Run through Nix only.
-{ pkgs, source, haskellPackages ? pkgs.haskellPackages }:
-let
- ghc = haskellPackages.ghcWithPackages (p: [ p.mtl ]);
-in pkgs.runCommand "tricu-core-archive-probe" {
- nativeBuildInputs = [ ghc pkgs.python3 pkgs.stdenv.cc ];
- meta.platforms = [ "x86_64-linux" ];
-} ''
- mkdir -p ext/haskell objects $out
- cp -r ${source}/src .
- cp -r ${source}/ext/haskell/src ${source}/ext/haskell/include ext/haskell/
- cp ${source}/ext/haskell/tests.c ${source}/ext/haskell/test_exports.py ext/haskell/
- chmod -R u+w src ext
- export PYTHONDONTWRITEBYTECODE=1
- timeout 120s python3 ${./archive_probe.py} --output $out
-''
diff --git a/ext/haskell/archive_probe.py b/ext/haskell/archive_probe.py
@@ -1,106 +0,0 @@
-#!/usr/bin/env python3
-"""Probe the pinned archives' readiness for a C-loadable core shared library.
-
-Invoked by archive-probe.nix with the same reducer/API sources and ordinary -O2
-settings as build.nix. A recognized non-PIC link failure is diagnostic data;
-unrelated build errors fail the derivation. A successful link is not installed
-or presented as an accepted runtime: packaging/lifecycle/performance acceptance
-remains separate.
-"""
-import argparse
-from collections import Counter
-import gzip
-import json
-from pathlib import Path
-import re
-import subprocess
-import sys
-
-
-def classify(returncode, log):
- if returncode == 0:
- return 'link-succeeded'
- errors = [line for line in log.splitlines() if 'error:' in line]
- def known_error(line):
- return bool(re.search(r'error: .*\.a\(.*(?:recompile with -fPIC|unsupported reloc \d+ against global symbol)', line)
- or re.search(r'collect2: error: ld returned \d+ exit status', line))
- if (re.search(r'error: .*\.a\([^\n]+recompile with -fPIC', log)
- and all(known_error(line) for line in errors)):
- return 'blocked-non-pic-archives'
- raise RuntimeError('Archive probe failed for an unclassified reason; inspect its build log')
-
-
-def main():
- # Reject ordinary compile failures and mixed-cause link failures rather than
- # misreporting every unsuccessful build as a known toolchain limitation.
- assert classify(0, '') == 'link-succeeded'
- example = 'ld.gold: error: /tmp/libHSx.a(X.o): recompile with -fPIC'
- assert classify(1, example) == 'blocked-non-pic-archives'
- for bad in ('Main.hs: error: missing name', example + '\nld: error: undefined symbol'):
- try:
- classify(1, bad)
- except RuntimeError:
- pass
- else:
- raise AssertionError('unrelated failure accepted')
- parser = argparse.ArgumentParser(description=__doc__)
- parser.add_argument('--output', required=True, type=Path)
- args = parser.parse_args()
- args.output.mkdir(parents=True, exist_ok=True)
- command = [
- 'ghc', '-O2', '-static', '-shared', '-fPIC', '-flink-rts', '-threaded',
- '-optl-Wl,-z,nodelete',
- '-XOverloadedStrings', '-XScopedTypeVariables', '-XMultiWayIf', '-XLambdaCase',
- '-Wall', '-isrc', '-iext/haskell/src', '-outputdir', 'objects',
- '-optc-Iext/haskell/include',
- 'ext/haskell/src/EmbeddingCore.hs', 'ext/haskell/src/init.c',
- '-o', 'libtricu-core.so',
- ]
- ghc_version = subprocess.check_output(['ghc', '--numeric-version'], text=True).strip()
- if tuple(map(int, ghc_version.split('.')[:2])) >= (9, 14):
- command.insert(1, '-optl-Wl,-u,init_ghc_hs_iface')
- result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
- log = result.stdout.decode('utf-8', errors='replace')
- # Keep full diagnostics compressed rather than filling the console with tens
- # of thousands of repeated relocation errors. They are not runtime data.
- with gzip.GzipFile(args.output / 'link.log.gz', 'wb', mtime=1) as stream:
- stream.write(result.stdout)
- try:
- status = classify(result.returncode, log)
- except RuntimeError:
- print(log[-16000:], file=sys.stderr)
- raise
- errors = [line for line in log.splitlines() if 'error:' in line and 'recompile with -fPIC' in line]
- archives = Counter(name for line in errors for name in re.findall(r'/([^/ ]+\.a)\(', line))
- report = {
- 'schema': 'tricu.core-archive-probe.v1',
- 'ghc_version': ghc_version,
- 'link_status': status,
- 'link_exit_code': result.returncode,
- 'command': command,
- 'non_pic_archives': dict(sorted(archives.items())),
- 'relocation_examples': [re.sub(r'/nix/store/[^ ]+?/(libHS[^ ]+\.a|bin/ld.gold)', r'\1', line)
- for line in errors[:3]],
- 'runtime_artifact_accepted': False,
- 'size_and_performance_measured': False,
- }
- if status == 'link-succeeded':
- # Immediate ABI sanity checks; full installed-runtime and timing checks
- # are deliberately not inferred from a successful linker invocation.
- subprocess.run([
- 'cc', '-Wall', '-Wextra', '-Werror', 'ext/haskell/tests.c',
- '-Iext/haskell/include', '-L.', '-ltricu-core',
- '-Wl,-rpath,' + str(Path.cwd()), '-o', 'core-client',
- ], check=True)
- subprocess.run(['./core-client'], check=True)
- subprocess.run([sys.executable, 'ext/haskell/test_exports.py',
- './libtricu-core.so', 'core'], check=True)
- report['c_client_and_exports'] = 'passed'
- else:
- report['c_client_and_exports'] = 'not-run-link-blocked'
- (args.output / 'report.json').write_text(json.dumps(report, indent=2) + '\n')
- print(json.dumps(report, indent=2))
-
-
-if __name__ == '__main__':
- main()
diff --git a/ext/haskell/bindings/check.py b/ext/haskell/bindings/check.py
@@ -33,6 +33,6 @@ for language,command in commands.items():
assert actual==expected,(language,actual,expected)
reports[language]=actual
# Each consumer independently asserts IO files/outcomes/resource cleanup before
-# reporting. Timeouts supervise the foreign process, not the reducer itself.
+# reporting. Timeouts supervise the complete foreign process.
a.report.write_text(json.dumps({'passed':list(reports),'results':reports},indent=2,ensure_ascii=False)+'\n')
print('Native binding conformance passed: '+', '.join(reports))
diff --git a/ext/haskell/bindings/python.py b/ext/haskell/bindings/python.py
@@ -61,7 +61,7 @@ with Library(library).context() as ctx, Library(library).context() as other:
else: raise AssertionError('invalid number accepted')
if copier:
# Existing tracked IO acceptance also checks scope cleanup and descriptor
- # counts, using the public binding rather than raw ctypes declarations.
+ # counts through the public binding.
sys.path.insert(0,str(Path(__file__).resolve().parents[1]))
from test_embedding import copier as check_copier
check_copier(ctx.library, Path(copier[0]), bytes.fromhex(f['copyHex'])*f['copyRepeats'])
diff --git a/ext/haskell/build.nix b/ext/haskell/build.nix
@@ -1,17 +1,23 @@
-{ pkgs, source, coreOnly ? false, haskellPackages ? pkgs.haskellPackages, archiveHaskell ? false, trimArchive ? archiveHaskell }:
-assert !archiveHaskell || coreOnly;
+{ pkgs, source, haskellPackages ? pkgs.haskellPackages, archiveHaskell ? false, trimArchive ? archiveHaskell }:
assert !trimArchive || archiveHaskell;
let
- libraryName = if coreOnly then "tricu-core" else "tricu-haskell";
- entryModule = if coreOnly then "EmbeddingCore" else "Embedding";
- ghc = haskellPackages.ghcWithPackages (p: if coreOnly then [ p.mtl ] else [
+ libraryName = "tricu-haskell";
+ entryModule = "Embedding";
+ dependencies = p: [
p.aeson p.base16-bytestring p.cryptonite p.memory p.vector
p.network p.stm p.temporary p.mtl
- ]);
+ ];
+ ghc = haskellPackages.ghcWithPackages dependencies;
in pkgs.stdenv.mkDerivation {
pname = "${libraryName}${pkgs.lib.optionalString archiveHaskell "-archive"}-embedding";
version = "2026.10.24";
src = source;
+ outputs = [ "out" "linkEvidence" ];
+ passthru = {
+ compiler = haskellPackages.ghc;
+ haskellLibraries = pkgs.lib.filter (p: p ? isHaskellLibrary && p ? src)
+ (pkgs.lib.closePropagation (dependencies haskellPackages));
+ };
nativeBuildInputs = [ ghc pkgs.python3 ]
++ pkgs.lib.optionals pkgs.stdenv.hostPlatform.isLinux [ pkgs.patchelf ];
buildPhase = ''
@@ -20,6 +26,7 @@ in pkgs.stdenv.mkDerivation {
# GHC 9.14's executable linker roots the RTS-to-Haskell startup interface;
# shared archive linkage needs the same root to resolve the archive cycle.
ghc -O2 ${if archiveHaskell then "-static" else "-dynamic"} -shared -fPIC -flink-rts -threaded \
+ ${pkgs.lib.optionalString pkgs.stdenv.hostPlatform.isLinux "-optl-Wl,-Map,embedding-link.map"} \
${pkgs.lib.optionalString pkgs.stdenv.hostPlatform.isLinux "-optl-Wl,-z,nodelete"} \
${pkgs.lib.optionalString archiveHaskell "-optl-Wl,-z,text -optl-Wl,-z,defs"} \
${pkgs.lib.optionalString trimArchive "-optl-Wl,--version-script=ext/haskell/src/core.exports -optl-Wl,--gc-sections"} \
@@ -35,7 +42,7 @@ in pkgs.stdenv.mkDerivation {
doCheck = true;
checkPhase = ''
runHook preCheck
- ghc -O2 ${if archiveHaskell then "-static" else "-dynamic"} -fPIC -threaded ${pkgs.lib.optionalString coreOnly "-DCORE_ONLY"} \
+ ghc -O2 ${if archiveHaskell then "-static" else "-dynamic"} -fPIC -threaded \
-XOverloadedStrings -XScopedTypeVariables -XMultiWayIf -XLambdaCase \
-isrc -iext/haskell/src -outputdir boundary-objects \
ext/haskell/BoundaryTests.hs -o boundary-tests
@@ -47,28 +54,25 @@ in pkgs.stdenv.mkDerivation {
$CC -Wall -Wextra -Werror ext/haskell/reload.c -Iext/haskell/include -ldl -o reload-tests
timeout 30s ./reload-tests "$PWD/lib${libraryName}.so"
''}
- python3 ext/haskell/test_exports.py ./lib${libraryName}${pkgs.stdenv.hostPlatform.extensions.sharedLibrary} ${if coreOnly then "core" else "host"} ${pkgs.lib.optionalString trimArchive "public-only"}
- ${pkgs.lib.optionalString (!coreOnly) ''
+ python3 ext/haskell/test_exports.py ./lib${libraryName}${pkgs.stdenv.hostPlatform.extensions.sharedLibrary} ${pkgs.lib.optionalString trimArchive "public-only"}
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=ext/haskell/python timeout 60s python3 ext/haskell/test_embedding.py \
./lib${libraryName}${pkgs.stdenv.hostPlatform.extensions.sharedLibrary} test/fixtures > embedding-report.json
cat embedding-report.json
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=ext/haskell/python timeout 30s python3 \
ext/haskell/test_scopes.py ./lib${libraryName}${pkgs.stdenv.hostPlatform.extensions.sharedLibrary} test/fixtures
- ''}
runHook postCheck
'';
installPhase = ''
runHook preInstall
- mkdir -p $out/lib $out/include $out/share/tricu
+ mkdir -p $out/lib $out/include $out/share/tricu $linkEvidence
+ ${pkgs.lib.optionalString pkgs.stdenv.hostPlatform.isLinux "cp embedding-link.map $linkEvidence/"}
+ ghc-pkg dump > $linkEvidence/ghc-packages.txt
cp lib${libraryName}* $out/lib/
- cp ext/haskell/include/tricu-core.h $out/include/
- printf '{"ghc_version":"%s","haskell_linkage":"${if archiveHaskell then "archive" else "dynamic"}","core_only":${if coreOnly then "true" else "false"},"public_symbols_only":${if trimArchive then "true" else "false"}}\n' \
+ printf '{"ghc_version":"%s","haskell_linkage":"${if archiveHaskell then "archive" else "dynamic"}","public_symbols_only":${if trimArchive then "true" else "false"}}\n' \
"$(ghc --numeric-version)" > $out/share/tricu/build.json
- ${pkgs.lib.optionalString (!coreOnly) ''
- cp ext/haskell/include/tricu.h $out/include/''}
- ${pkgs.lib.optionalString (!coreOnly) ''
+ cp ext/haskell/include/tricu.h $out/include/
cp ext/haskell/python/tricu.py $out/share/tricu/
- cp embedding-report.json $out/share/tricu/''}
+ cp embedding-report.json $out/share/tricu/
${pkgs.lib.optionalString pkgs.stdenv.hostPlatform.isLinux ''
python3 ext/haskell/package_runtime.py $out/lib/lib${libraryName}.so
''}
@@ -84,13 +88,11 @@ in pkgs.stdenv.mkDerivation {
fi
''}
$CC -Wall -Wextra -Werror ext/haskell/tests.c -I$out/include \
- -L$out/lib -l${libraryName} -Wl,-rpath,$out/lib -o installed-core-tests
- GHCRTS=--tricu-test-invalid timeout 30s ./installed-core-tests
- python3 ext/haskell/test_exports.py $out/lib/lib${libraryName}${pkgs.stdenv.hostPlatform.extensions.sharedLibrary} ${if coreOnly then "core" else "host"} ${pkgs.lib.optionalString trimArchive "public-only"}
- ${pkgs.lib.optionalString (!coreOnly) ''
+ -L$out/lib -l${libraryName} -Wl,-rpath,$out/lib -o installed-embedding-tests
+ GHCRTS=--tricu-test-invalid timeout 30s ./installed-embedding-tests
+ python3 ext/haskell/test_exports.py $out/lib/lib${libraryName}${pkgs.stdenv.hostPlatform.extensions.sharedLibrary} ${pkgs.lib.optionalString trimArchive "public-only"}
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=$out/share/tricu timeout 60s python3 \
ext/haskell/test_embedding.py \
$out/lib/lib${libraryName}${pkgs.stdenv.hostPlatform.extensions.sharedLibrary} test/fixtures
- ''}
'';
}
diff --git a/ext/haskell/container_bindings.py b/ext/haskell/container_bindings.py
@@ -0,0 +1,70 @@
+"""Distro Python/PHP and upstream Node conformance against one relocated host."""
+import json
+import os
+from pathlib import Path
+import shutil
+import subprocess
+import sys
+
+
+def checked(command, env=None):
+ result = subprocess.run(command, text=True, capture_output=True, timeout=90, env=env)
+ if result.returncode:
+ raise RuntimeError(f'{command[0]} failed ({result.returncode}): {result.stdout}\n{result.stderr}')
+ return result.stdout.strip()
+
+
+def run(release, node_version):
+ source = Path('/work/bindings')
+ if not source.exists():
+ shutil.copytree('/kit/bindings', source)
+ (source/'ext/js/node_modules').symlink_to('/opt/tricu-js/node_modules')
+ # PHP consumes the transferred header.
+ shutil.copyfile(release/'include/tricu.h', source/'ext/php/tricu.h')
+ node = '/opt/node/bin/node'
+ actual_node = checked([node, '--version'])
+ if actual_node != 'v'+node_version:
+ raise ValueError('upstream Node version differs from image lock')
+ php = ['/usr/bin/php', '-d', 'ffi.enable=true']
+ php_info = json.loads(checked(php+['-r', 'echo json_encode(["version"=>PHP_VERSION,"ffi"=>extension_loaded("FFI"),"ffi_enable"=>ini_get("ffi.enable"),"integer_bytes"=>PHP_INT_SIZE]);']))
+ if not php_info['ffi'] or php_info['integer_bytes'] != 8:
+ raise ValueError('64-bit distro PHP with FFI required')
+ koffi = json.loads(Path('/opt/tricu-js/node_modules/koffi/package.json').read_text())['version']
+ locked = json.loads((source/'ext/js/package-lock.json').read_text())['packages']['node_modules/koffi']['version']
+ if koffi != locked:
+ raise ValueError('installed Koffi differs from package lock')
+ specification = source/'ext/haskell/bindings/fixtures.json'
+ expected = json.loads(specification.read_text())['expected']
+ arguments = [str(release/'lib/libtricu-haskell.so'), '/tests/fixtures', str(specification), '/tests/copy.arboricx']
+ js = '''import {readFileSync} from 'node:fs';
+import {pathToFileURL} from 'node:url';
+import {strict as assert} from 'node:assert';
+const {conformance} = await import(pathToFileURL(process.argv[1]));
+const report = conformance(...process.argv.slice(2));
+const maps = readFileSync('/proc/self/maps','utf8');
+assert(!maps.includes('/nix/store/'));
+assert(maps.includes(process.argv[2]));
+assert(maps.includes('/opt/tricu-js/node_modules/koffi/'));
+console.log(JSON.stringify(report));'''
+ php_script = '''array_shift($argv); require $argv[0];
+$maps = file_get_contents('/proc/self/maps');
+if (str_contains($maps, '/nix/store/') || !str_contains($maps, $argv[1])) {
+ throw new RuntimeException('unexpected native mappings');
+}'''
+ commands = {
+ 'python': [sys.executable, str(source/'ext/haskell/bindings/python.py')],
+ 'js': [node, '--input-type=module', '-e', js, str(source/'ext/js/test/conformance.mjs')],
+ 'php': php+['-r', php_script, str(source/'ext/php/tests/conformance.php')],
+ }
+ env = dict(os.environ, PYTHONPATH=str(release/'share/tricu'), PYTHONDONTWRITEBYTECODE='1')
+ results = {}
+ for language, command in commands.items():
+ actual = json.loads(checked(command+arguments, env))
+ if actual != expected:
+ raise ValueError(f'{language}: independent conformance expectations differ')
+ results[language] = actual
+ return dict(passed=list(results), results=results, node=actual_node, koffi=koffi, php=php_info,
+ provenance=dict(node='upstream official Node image, exact version and registry digest',
+ php='distro package with process-local ffi.enable=true',
+ koffi='npm ci from committed package lock'),
+ scope='CLI bindings, codecs, ownership, error controls and compiled copier IO')
diff --git a/ext/haskell/container_matrix.py b/ext/haskell/container_matrix.py
@@ -0,0 +1,256 @@
+#!/usr/bin/env python3
+"""Prepare pinned local distro test images, then run an offline acceptance matrix."""
+import argparse
+import hashlib
+import json
+import os
+from pathlib import Path
+import re
+import shutil
+import subprocess
+import sys
+import tempfile
+import uuid
+
+MATRIX = {
+ 'ubuntu24': ('docker.io/library/ubuntu:24.04', 'apt', 'pass'),
+ 'debian13': ('docker.io/library/debian:13', 'apt', 'pass'),
+ 'fedora': ('registry.fedoraproject.org/fedora:latest', 'dnf', 'pass'),
+ 'ubuntu22': ('docker.io/library/ubuntu:22.04', 'apt', 'glibc-rejection'),
+}
+
+
+def checked(command, **kwargs):
+ return subprocess.run(command, check=True, text=True, capture_output=True, **kwargs).stdout
+
+
+def inspect(engine, image):
+ row = json.loads(checked([engine, 'image', 'inspect', image], timeout=60))[0]
+ if row['Architecture'] != 'amd64' or row['Os'] != 'linux':
+ raise ValueError('matrix requires native Linux amd64 images')
+ if re.fullmatch(r'[0-9a-f]{64}', row['Id']):
+ row['Id'] = 'sha256:'+row['Id']
+ if not re.fullmatch(r'sha256:[0-9a-f]{64}', row['Id']):
+ raise ValueError('invalid image content identity')
+ return row
+
+
+BINDING_FILES = (
+ 'ext/js/package.json', 'ext/js/package-lock.json', 'ext/js/src/lib.js',
+ 'ext/js/test/conformance.mjs', 'ext/php/src/ffi.php', 'ext/php/tests/conformance.php',
+ 'ext/haskell/bindings/python.py', 'ext/haskell/bindings/fixtures.json',
+ 'ext/haskell/test_embedding.py', 'ext/js/test/runner.mjs',
+)
+
+
+def validate_bindings_kit(kit):
+ missing = [name for name in BINDING_FILES if not (kit/'bindings'/name).is_file()]
+ if missing:
+ raise ValueError(f'incomplete bindings test kit: {missing}')
+
+
+def prepare(args):
+ if args.distro and len(args.distro) != len(set(args.distro)):
+ raise ValueError('duplicate distro selection')
+ if args.bindings != bool(args.node_version):
+ raise ValueError('--bindings and --node-version must be selected together')
+ if args.node_version and (not re.fullmatch(r'\d+\.\d+\.\d+', args.node_version)
+ or int(args.node_version.split('.')[0]) < 22):
+ raise ValueError('select an exact upstream Node version >=22')
+ kit = Path(__file__).resolve().parent
+ if args.bindings:
+ validate_bindings_kit(kit)
+ args.directory.mkdir(parents=True, exist_ok=False)
+ node = None
+ if args.bindings:
+ image = f'docker.io/library/node:{args.node_version}-bookworm-slim'
+ checked([args.engine, 'pull', '--platform', 'linux/amd64', image], timeout=900)
+ upstream = inspect(args.engine, image)
+ digest = next((d for d in upstream.get('RepoDigests', [])
+ if re.fullmatch(r'[^\s]+@sha256:[0-9a-f]{64}', d)), None)
+ if digest is None:
+ raise ValueError('upstream Node image has no registry digest')
+ node = dict(version=args.node_version, image=image, digest=digest, image_id=upstream['Id'])
+ rows = []
+ for name in args.distro or MATRIX:
+ image, manager, expected = MATRIX[name]
+ checked([args.engine, 'pull', '--platform', 'linux/amd64', image], timeout=900)
+ base = inspect(args.engine, image)
+ digest = next((d for d in base.get('RepoDigests', [])
+ if re.fullmatch(r'[^\s]+@sha256:[0-9a-f]{64}', d)), None)
+ if digest is None:
+ raise ValueError(f'no registry digest for {image}')
+ # Record the exact base and execute installation only during preparation.
+ install = ('apt-get update && apt-get install -y --no-install-recommends python3 '
+ '&& rm -rf /var/lib/apt/lists/*') if manager == 'apt' else 'dnf -y install python3 && dnf clean all'
+ context = args.directory/name
+ context.mkdir()
+ dockerfile = f'FROM {digest}\nRUN {install}\n'
+ if args.bindings:
+ packages = ('apt-get update && apt-get install -y --no-install-recommends php-cli php-common php-ffi ca-certificates libstdc++6 libatomic1 '
+ '&& rm -rf /var/lib/apt/lists/*') if manager == 'apt' else 'dnf -y install php-cli php-common php-ffi ca-certificates libstdc++ libatomic && dnf clean all'
+ for filename in ('package.json', 'package-lock.json'):
+ shutil.copyfile(kit/'bindings/ext/js'/filename, context/filename)
+ dockerfile += (f'RUN {packages}\nCOPY --from={node["digest"]} /usr/local/ /opt/node/\n'
+ 'ENV PATH=/opt/node/bin:/usr/bin:/bin\n'
+ 'COPY package.json package-lock.json /opt/tricu-js/\n'
+ 'RUN cd /opt/tricu-js && npm ci --omit=dev && npm cache clean --force\n'
+ 'RUN php -d ffi.enable=true -r \'if (!extension_loaded("FFI")) { exit(1); }\'\n')
+ (context/'Dockerfile').write_text(dockerfile)
+ tag = 'tricu-distro-test:'+name+'-'+uuid.uuid4().hex
+ result = subprocess.run([args.engine, 'build', '--platform', 'linux/amd64',
+ '--tag', tag, str(context)],
+ text=True, capture_output=True, timeout=1800)
+ (context/'build.log').write_text(result.stdout+result.stderr)
+ result.check_returncode()
+ prepared = inspect(args.engine, tag)
+ packages = checked([args.engine, 'run', '--rm', '--network', 'none',
+ '--entrypoint', '/bin/sh', prepared['Id'], '-c',
+ "dpkg-query -W" if manager == 'apt' else "rpm -qa | sort"], timeout=60)
+ (context/'packages.txt').write_text(packages)
+ rows.append(dict(name=name, requested_image=image, base_digest=digest,
+ base_id=base['Id'], image_id=prepared['Id'], expectation=expected,
+ dockerfile_sha256=hashlib.sha256(dockerfile.encode()).hexdigest(),
+ packages_sha256=hashlib.sha256(packages.encode()).hexdigest()))
+ lock = dict(schema='tricu.distro-images.v1', engine=args.engine, images=rows)
+ if args.bindings:
+ lock.update(profile='bindings', node=node,
+ npm_lock_sha256=hashlib.sha256((kit/'bindings/ext/js/package-lock.json').read_bytes()).hexdigest(),
+ npm_package_sha256=hashlib.sha256((kit/'bindings/ext/js/package.json').read_bytes()).hexdigest())
+ (args.directory/'images.json').write_text(json.dumps(lock, indent=2)+'\n')
+ print(args.directory/'images.json')
+
+
+def load_lock(path):
+ lock = json.loads(path.read_text())
+ if lock.get('schema') != 'tricu.distro-images.v1' or not lock.get('images'):
+ raise ValueError('invalid/empty image lock')
+ if lock.get('profile', 'python') not in ('python', 'bindings'):
+ raise ValueError('unknown runtime profile')
+ if lock.get('profile') == 'bindings':
+ node = lock['node']
+ if not re.fullmatch(r'[^\s]+@sha256:[0-9a-f]{64}', node['digest']):
+ raise ValueError('upstream Node must be digest-pinned')
+ if not re.fullmatch(r'\d+\.\d+\.\d+', node['version']) or int(node['version'].split('.')[0]) < 22:
+ raise ValueError('exact upstream Node >=22 required')
+ names = set()
+ for row in lock['images']:
+ if row['name'] not in MATRIX or row['name'] in names:
+ raise ValueError('unknown/duplicate distro')
+ names.add(row['name'])
+ if row['expectation'] != MATRIX[row['name']][2]:
+ raise ValueError('changed acceptance expectation')
+ if not re.fullmatch(r'sha256:[0-9a-f]{64}', row['image_id']):
+ raise ValueError('image must be pinned by content ID')
+ evidence = [('Dockerfile', 'dockerfile_sha256'), ('packages.txt', 'packages_sha256')]
+ if lock.get('profile') == 'bindings':
+ dockerfile = (path.parent/row['name']/'Dockerfile').read_text()
+ if f'COPY --from={lock["node"]["digest"]} /usr/local/ /opt/node/' not in dockerfile:
+ raise ValueError('Node provenance differs from prepared recipe')
+ for file, key in [('package.json', 'npm_package_sha256'), ('package-lock.json', 'npm_lock_sha256')]:
+ if hashlib.sha256((path.parent/row['name']/file).read_bytes()).hexdigest() != lock[key]:
+ raise ValueError('changed Koffi preparation inputs')
+ for file, key in evidence:
+ content = (path.parent/row['name']/file).read_bytes()
+ if hashlib.sha256(content).hexdigest() != row[key]:
+ raise ValueError('changed prepared-image evidence')
+ return lock
+
+
+def run(args):
+ lock = load_lock(args.lock)
+ args.output.mkdir(parents=True, exist_ok=False)
+ kit = Path(__file__).resolve().parent
+ bindings = lock.get('profile') == 'bindings'
+ if bindings:
+ validate_bindings_kit(kit)
+ for filename, key in [('package-lock.json', 'npm_lock_sha256'), ('package.json', 'npm_package_sha256')]:
+ if hashlib.sha256((kit/'bindings/ext/js'/filename).read_bytes()).hexdigest() != lock[key]:
+ raise ValueError('prepared Koffi inputs differ from current kit; prepare new images')
+ results = []
+ with tempfile.TemporaryDirectory(prefix='tricu-distro-kit-') as directory:
+ # Copy the transfer kit into a standalone mount for rootless/SELinux engines.
+ copied = Path(directory)/'kit'
+ shutil.copytree(kit, copied)
+ for path in [copied, *copied.rglob('*')]:
+ path.chmod(0o755 if path.is_dir() else 0o644)
+ for row in lock['images']:
+ actual = inspect(args.engine, row['image_id'])
+ if actual['Id'] != row['image_id']:
+ raise ValueError('prepared image identity changed')
+ name = 'tricu-acceptance-'+uuid.uuid4().hex
+ command = [args.engine, 'run', '--rm', '--pull', 'never', '--name', name,
+ '--platform', 'linux/amd64', '--network', 'none', '--read-only',
+ '--cap-drop', 'ALL', '--security-opt', 'no-new-privileges',
+ '--memory', '1g', '--pids-limit', '128', '--user', '65534:65534',
+ '--tmpfs', '/work:rw,size=256m,mode=1777',
+ '--tmpfs', '/tmp:rw,size=64m,mode=1777',
+ '--volume', str(copied)+':/kit:ro,Z',
+ '--volume', str(copied/'tests')+':/tests:ro,Z',
+ '--workdir', '/work', '--entrypoint', '/usr/bin/env', row['image_id'],
+ '-i', 'PATH=/usr/bin:/bin', 'HOME=/tmp', 'PYTHONDONTWRITEBYTECODE=1',
+ '/usr/bin/python3', '/kit/container_probe.py', '--expect', row['expectation']]
+ if bindings:
+ command += ['--bindings', '--node-version', lock['node']['version']]
+ try:
+ result = subprocess.run(command, text=True, capture_output=True, timeout=1200 if bindings else 600)
+ log = args.output/(row['name']+'.log')
+ log.write_text(result.stdout+result.stderr)
+ if result.returncode:
+ raise RuntimeError(f'{row["name"]} failed (exit {result.returncode}); log: {log}\n'
+ +(result.stdout+result.stderr)[-4000:])
+ markers = [line.removeprefix('TRICU_REPORT=') for line in result.stdout.splitlines()
+ if line.startswith('TRICU_REPORT=')]
+ if len(markers) != 1:
+ raise ValueError('missing/duplicate container acceptance report')
+ report = json.loads(markers[0])
+ if report.get('expectation') != row['expectation'] or len(report.get('runs', [])) != 2:
+ raise ValueError('incomplete container report')
+ if bindings and row['expectation'] == 'pass':
+ if any(run.get('bindings', {}).get('passed') != ['python', 'js', 'php'] for run in report['runs']):
+ raise ValueError('incomplete binding acceptance report')
+ results.append(dict(image=row, report=report))
+ (args.output/(row['name']+'.json')).write_text(json.dumps(results[-1], indent=2)+'\n')
+ except subprocess.TimeoutExpired as error:
+ (args.output/(row['name']+'.log')).write_bytes((error.stdout or b'')+(error.stderr or b''))
+ raise
+ finally:
+ try:
+ subprocess.run([args.engine, 'rm', '--force', name], capture_output=True, timeout=30)
+ except subprocess.TimeoutExpired:
+ print(f'Container cleanup timed out: {name}', file=sys.stderr)
+ report = dict(schema='tricu.distro-acceptance.v1', results=results,
+ engine_version=checked([args.engine, '--version'], timeout=30).strip(),
+ image_lock_sha256=hashlib.sha256(args.lock.read_bytes()).hexdigest(),
+ kit_files={str(path.relative_to(kit)): hashlib.sha256(path.read_bytes()).hexdigest()
+ for path in sorted(kit.rglob('*')) if path.is_file()},
+ archive_sha256=hashlib.sha256((kit/'tricu-linux-x86_64.tar.gz').read_bytes()).hexdigest(),
+ profile=lock.get('profile', 'python'), node=lock.get('node'),
+ scope=('Distro Python/PHP and upstream Node userspace acceptance; shared host kernel' if bindings
+ else 'Distro Python/userspace acceptance; containers share the host kernel'))
+ (args.output/'report.json').write_text(json.dumps(report, indent=2)+'\n')
+ print(args.output/'report.json')
+
+
+def main():
+ p = argparse.ArgumentParser(description=__doc__)
+ sub = p.add_subparsers(dest='action', required=True)
+ prepare_parser = sub.add_parser('prepare')
+ prepare_parser.add_argument('--directory', type=Path, required=True)
+ prepare_parser.add_argument('--distro', choices=MATRIX, action='append')
+ prepare_parser.add_argument('--bindings', action='store_true')
+ prepare_parser.add_argument('--node-version', help='exact upstream Node release, e.g. 22.22.0')
+ run_parser = sub.add_parser('run')
+ run_parser.add_argument('--lock', type=Path, required=True)
+ run_parser.add_argument('--output', type=Path, required=True)
+ for parser in (prepare_parser, run_parser):
+ parser.add_argument('--engine', choices=['docker', 'podman'], default='podman')
+ args = p.parse_args()
+ if args.action == 'prepare':
+ prepare(args)
+ else:
+ run(args)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/ext/haskell/container_probe.py b/ext/haskell/container_probe.py
@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+"""Run Nix-free acceptance inside a container using distro Python."""
+import argparse
+import hashlib
+import json
+import os
+from pathlib import Path, PurePosixPath
+import platform
+import re
+import subprocess
+import sys
+import tarfile
+
+
+def incompatible_loader_error(message, actual_glibc):
+ """Recognize loader rejection of an unavailable, newer GLIBC symbol version."""
+ actual = tuple(map(int, actual_glibc.split('.')))
+ missing = re.findall(r"version [`']GLIBC_(\d+\.\d+(?:\.\d+)?)[`'] not found", message)
+ return bool(missing) and all(tuple(map(int, v.split('.'))) > actual for v in missing)
+
+
+def extract(archive, destination):
+ # Transfer members are directories and regular files only. This explicit
+ # reader works with the distro Python 3.10 incompatibility control as well.
+ with tarfile.open(archive, 'r:gz') as bundle:
+ for member in bundle:
+ path = PurePosixPath(member.name)
+ if path.is_absolute() or '..' in path.parts or not path.parts or path.parts[0] != 'tricu':
+ raise ValueError('invalid transfer member')
+ target = destination.joinpath(*path.parts)
+ if member.isdir():
+ target.mkdir(parents=True, exist_ok=True)
+ elif member.isfile():
+ target.parent.mkdir(parents=True, exist_ok=True)
+ with bundle.extractfile(member) as source, target.open('wb') as output:
+ import shutil
+ shutil.copyfileobj(source, output)
+ target.chmod(0o755 if member.mode & 0o111 else 0o644)
+ else:
+ raise ValueError('transfer must contain only regular files/directories')
+ return destination/'tricu'
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument('--expect', choices=['pass', 'glibc-rejection'], required=True)
+ parser.add_argument('--bindings', action='store_true')
+ parser.add_argument('--node-version')
+ args = parser.parse_args()
+ if args.bindings != bool(args.node_version):
+ raise ValueError('bindings profile requires an exact Node version')
+ assert not Path('/nix/store').exists()
+ assert not os.environ.get('LD_LIBRARY_PATH') and not os.environ.get('TRICU_LIB')
+ assert platform.machine() == 'x86_64'
+ glibc = os.confstr('CS_GNU_LIBC_VERSION').split()[-1]
+ report = dict(schema='tricu.distro-probe.v1', expectation=args.expect,
+ os_release=Path('/etc/os-release').read_text(), glibc=glibc,
+ python=sys.version, executable=sys.executable,
+ kernel=platform.release(), machine=platform.machine(), runs=[])
+ for location in ('first', 'relocated path with spaces'):
+ release = extract('/kit/tricu-linux-x86_64.tar.gz', Path('/work')/location)
+ inventory = json.loads((release/'share/tricu/portable.json').read_text())
+ for name, row in inventory['libraries'].items():
+ assert Path(name).name == name
+ assert hashlib.sha256((release/'lib'/name).read_bytes()).hexdigest() == row['sha256']
+ library = str(release/'lib/libtricu-haskell.so')
+ if args.expect == 'glibc-rejection':
+ # Require a successful ctypes import and a version-specific loader error.
+ child = subprocess.run([sys.executable, '-c', '''import ctypes, json, sys
+try:
+ ctypes.CDLL(sys.argv[1])
+except OSError as error:
+ print(json.dumps({'loader_error': str(error)}))
+ sys.exit(42)
+sys.exit(0)
+''', library], text=True, capture_output=True, timeout=30)
+ assert child.returncode == 42 and not child.stderr, child
+ error = json.loads(child.stdout)['loader_error']
+ assert incompatible_loader_error(error, glibc), error
+ report['runs'].append(dict(location=location, loader_error=error))
+ else:
+ env = dict(os.environ, PYTHONPATH=str(release/'share/tricu')+':/tests',
+ PYTHONDONTWRITEBYTECODE='1')
+ # Each relocation gets a fresh process/runtime and fresh workload files.
+ for name in ('checks.json', 'report.json'):
+ (Path('/work')/name).unlink(missing_ok=True)
+ subprocess.run([sys.executable, '/tests/portable_workload.py', str(release)],
+ env=env, check=True, timeout=240)
+ run = dict(location=location, **json.loads(Path('/work/report.json').read_text()))
+ if args.bindings:
+ from container_bindings import run as bindings_run
+ run['bindings'] = bindings_run(release, args.node_version)
+ report['runs'].append(run)
+ print('TRICU_REPORT='+json.dumps(report, sort_keys=True), flush=True)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/ext/haskell/core_bench.c b/ext/haskell/core_bench.c
@@ -1,7 +1,7 @@
#define _POSIX_C_SOURCE 200809L
/* Same public C calls for both linkages. Fresh suspensions each round; timing
* separates ABI/graph construction from demanding the completed reduction. */
-#include "tricu-core.h"
+#include "tricu.h"
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
diff --git a/ext/haskell/core_bench.py b/ext/haskell/core_bench.py
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
-"""Fresh-process C ABI linkage smoke; timings are observations, not thresholds."""
+"""Fresh-process C ABI linkage smoke with timing observations."""
import argparse
import json
from pathlib import Path
@@ -30,7 +30,7 @@ for index, (kind, size) in enumerate((kind, size) for kind in ('constant', 'dist
reduce_median_seconds=statistics.median(row['reduce_seconds'] for row in observed),
peak_rss_kib=max(row['peak_rss_kib'] for row in observed)))
args.output.write_text(json.dumps(dict(
- schema='tricu.core-linkage-smoke.v1',
+ schema='tricu.embedding-linkage-smoke.v1',
scope='Same public C API, fresh graphs, separate graph-build and WHNF-demand timings; no speed threshold',
samples=rows, summary=summary), indent=2) + '\n')
print(json.dumps(summary, indent=2))
diff --git a/ext/haskell/footprint.py b/ext/haskell/footprint.py
@@ -51,20 +51,17 @@ def measure(path, name):
def main():
parser = argparse.ArgumentParser(description=__doc__)
- parser.add_argument('--core', required=True, type=Path)
parser.add_argument('--host', required=True, type=Path)
- parser.add_argument('--archive', type=Path)
+ parser.add_argument('--host-archive', required=True, type=Path)
parser.add_argument('--output', required=True, type=Path)
args = parser.parse_args()
report = {
'method': 'installed files; Nix NAR sizes; deterministic tar and Nix export gzip level 6',
- 'core': measure(args.core, 'libtricu-core.so'),
'host': measure(args.host, 'libtricu-haskell.so'),
}
- if args.archive:
- report['archive'] = measure(args.archive, 'libtricu-core.so')
+ report['host_archive'] = measure(args.host_archive, 'libtricu-haskell.so')
args.output.write_text(json.dumps(report, indent=2, sort_keys=True) + '\n')
- for role in ('core', 'host', *(['archive'] if args.archive else [])):
+ for role in ('host', 'host_archive'):
row = report[role]
print(role + ': ' + ', '.join(
f'{key}={row[key] / 1048576:.2f} MiB' for key in (
diff --git a/ext/haskell/include/tricu-core.h b/ext/haskell/include/tricu-core.h
@@ -1,58 +0,0 @@
-#ifndef TRICU_CORE_H
-#define TRICU_CORE_H
-#include <stddef.h>
-#include <stdint.h>
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-/* Common C surface for the core and full host artifacts; choose one per process.
- * Core Haskell embedding ABI 1. See ext/haskell/README.md for lifecycle,
- * demand, concurrency and resource limits. Every output is unchanged on failure.
- * All pointers must be valid; distinct output parameters must not overlap.
- * Input buffers are borrowed for the call; NULL is allowed only at length zero.
- * Every successful handle output owns one root; inputs are borrowed. */
-typedef uint64_t tricu_context;
-typedef uint64_t tricu_handle;
-typedef uint32_t tricu_status;
-enum {
- TRICU_OK = 0, TRICU_INVALID_CONTEXT = 1, TRICU_INVALID_HANDLE = 2,
- TRICU_INVALID_DATA = 3, TRICU_TYPE_ERROR = 4,
- TRICU_EXCEPTION = 5, TRICU_ID_EXHAUSTED = 6
-};
-enum { TRICU_LEAF = 0, TRICU_STEM = 1, TRICU_FORK = 2 };
-
-/* Call once before other entrypoints; repeated calls are harmless. Owns RTS
- * initialization: use in a process that has not already initialized GHC.
- * Keep the library loaded until process exit. No shutdown entrypoint. */
-void tricu_init(void);
-uint32_t tricu_abi_version(void);
-tricu_status tricu_context_new(tricu_context *out);
-tricu_status tricu_context_free(tricu_context ctx);
-tricu_status tricu_retain(tricu_context ctx, tricu_handle value, tricu_handle *out);
-tricu_status tricu_release(tricu_context ctx, tricu_handle value);
-tricu_status tricu_leaf(tricu_context ctx, tricu_handle *out);
-tricu_status tricu_stem(tricu_context ctx, tricu_handle child, tricu_handle *out);
-tricu_status tricu_fork(tricu_context ctx, tricu_handle left, tricu_handle right, tricu_handle *out);
-/* Application constructs a suspension; reduce demands only its outer constructor.
- * Inspection and decoding also demand the portions they observe. */
-tricu_status tricu_apply(tricu_context ctx, tricu_handle fn, tricu_handle arg, tricu_handle *out);
-tricu_status tricu_reduce(tricu_context ctx, tricu_handle value, tricu_handle *out);
-tricu_status tricu_kind(tricu_context ctx, tricu_handle value, uint32_t *out);
-tricu_status tricu_child(tricu_context ctx, tricu_handle value, uint32_t index, tricu_handle *out);
-tricu_status tricu_from_u64(tricu_context ctx, uint64_t value, tricu_handle *out);
-tricu_status tricu_to_bool(tricu_context ctx, tricu_handle value, uint32_t *out);
-tricu_status tricu_to_u64(tricu_context ctx, tricu_handle value, uint64_t *out);
-tricu_status tricu_from_bytes(tricu_context ctx, const uint8_t *bytes, size_t len, tricu_handle *out);
-tricu_status tricu_from_utf8(tricu_context ctx, const uint8_t *bytes, size_t len, tricu_handle *out);
-tricu_status tricu_to_bytes(tricu_context ctx, tricu_handle value, uint8_t **out, size_t *len);
-tricu_status tricu_to_utf8(tricu_context ctx, tricu_handle value, uint8_t **out, size_t *len);
-/* Last boundary error, retained across successes. UTF-8, length-delimited.
- * Empty initially. Returned buffers are independent of contexts and must be freed
- * exactly once with tricu_buffer_free, including empty buffers. */
-tricu_status tricu_last_error(tricu_context ctx, uint8_t **out, size_t *len);
-void tricu_buffer_free(uint8_t *buffer);
-#ifdef __cplusplus
-}
-#endif
-#endif
diff --git a/ext/haskell/include/tricu.h b/ext/haskell/include/tricu.h
@@ -1,13 +1,60 @@
#ifndef TRICU_H
#define TRICU_H
-#include "tricu-core.h"
+#include <stddef.h>
+#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
+/* Haskell embedding ABI 1. See ext/haskell/README.md for lifecycle,
+ * demand, concurrency and resource limits. Every output is unchanged on failure.
+ * All pointers must be valid; distinct output parameters must not overlap.
+ * Input buffers are borrowed for the call; NULL is allowed only at length zero.
+ * Every successful handle output owns one root; inputs are borrowed. */
+typedef uint64_t tricu_context;
+typedef uint64_t tricu_handle;
+typedef uint32_t tricu_status;
+enum {
+ TRICU_OK = 0, TRICU_INVALID_CONTEXT = 1, TRICU_INVALID_HANDLE = 2,
+ TRICU_INVALID_DATA = 3, TRICU_TYPE_ERROR = 4,
+ TRICU_EXCEPTION = 5, TRICU_ID_EXHAUSTED = 6
+};
+enum { TRICU_LEAF = 0, TRICU_STEM = 1, TRICU_FORK = 2 };
+
+/* Call once before other entrypoints; repeated calls are harmless. Owns RTS
+ * initialization: use in a process that has not already initialized GHC.
+ * Keep the library loaded until process exit. No shutdown entrypoint. */
+void tricu_init(void);
+uint32_t tricu_abi_version(void);
+tricu_status tricu_context_new(tricu_context *out);
+tricu_status tricu_context_free(tricu_context ctx);
+tricu_status tricu_retain(tricu_context ctx, tricu_handle value, tricu_handle *out);
+tricu_status tricu_release(tricu_context ctx, tricu_handle value);
+tricu_status tricu_leaf(tricu_context ctx, tricu_handle *out);
+tricu_status tricu_stem(tricu_context ctx, tricu_handle child, tricu_handle *out);
+tricu_status tricu_fork(tricu_context ctx, tricu_handle left, tricu_handle right, tricu_handle *out);
+/* Application constructs a suspension; reduce demands only its outer constructor.
+ * Inspection and decoding also demand the portions they observe. */
+tricu_status tricu_apply(tricu_context ctx, tricu_handle fn, tricu_handle arg, tricu_handle *out);
+tricu_status tricu_reduce(tricu_context ctx, tricu_handle value, tricu_handle *out);
+tricu_status tricu_kind(tricu_context ctx, tricu_handle value, uint32_t *out);
+tricu_status tricu_child(tricu_context ctx, tricu_handle value, uint32_t index, tricu_handle *out);
+tricu_status tricu_from_u64(tricu_context ctx, uint64_t value, tricu_handle *out);
+tricu_status tricu_to_bool(tricu_context ctx, tricu_handle value, uint32_t *out);
+tricu_status tricu_to_u64(tricu_context ctx, tricu_handle value, uint64_t *out);
+tricu_status tricu_from_bytes(tricu_context ctx, const uint8_t *bytes, size_t len, tricu_handle *out);
+tricu_status tricu_from_utf8(tricu_context ctx, const uint8_t *bytes, size_t len, tricu_handle *out);
+tricu_status tricu_to_bytes(tricu_context ctx, tricu_handle value, uint8_t **out, size_t *len);
+tricu_status tricu_to_utf8(tricu_context ctx, tricu_handle value, uint8_t **out, size_t *len);
+/* Last boundary error, retained across successes. UTF-8, length-delimited.
+ * Empty initially. Returned buffers are independent of contexts and must be freed
+ * exactly once with tricu_buffer_free, including empty buffers. */
+tricu_status tricu_last_error(tricu_context ctx, uint8_t **out, size_t *len);
+void tricu_buffer_free(uint8_t *buffer);
+
enum { TRICU_COMPLETED = 0, TRICU_FAILED = 1, TRICU_CANCELLED = 2 };
-/* Verified Arboricx bundle; explicit UTF-8 export name, no implicit selection. */
+/* Load a verified Arboricx bundle with an explicit UTF-8 export name. */
tricu_status tricu_load_bundle(tricu_context ctx, const uint8_t *bytes, size_t len,
const uint8_t *name, size_t name_len, tricu_handle *out);
/* Runs a canonical tricuIO v2 envelope with OS process authority. Task outcome
diff --git a/ext/haskell/portable-README.md b/ext/haskell/portable-README.md
@@ -0,0 +1,48 @@
+# Relocatable native embedding prototype
+
+This Linux x86-64 transfer artifact contains the archive-linked full Haskell
+host, non-glibc native dependencies, `include/tricu.h`, and the Python binding
+under `share/tricu/`. Keep this directory layout together. Each bundled ELF
+uses `$ORIGIN` to resolve its dependencies. Runtime needs system glibc and the
+embedding application's own language runtime; Nix and GHC are build-time tools.
+
+After extracting `tricu-linux-x86_64.tar.gz` on the destination:
+
+```sh
+PYTHONPATH="$PWD/tricu/share/tricu" python3 - <<'PY'
+from tricu import Library
+with Library().context() as ctx:
+ value = ctx.number(42)
+ print(ctx.to_number(value))
+PY
+```
+
+The binding finds `lib/libtricu-haskell.so` relative to its installed location.
+C consumers include `include/tricu.h` and load/link that same library. Use one
+copy per process and preserve the ordinary embedding lifecycle and ownership
+contract. Programs execute with OS process authority. External supervision is
+required for hard execution and memory limits.
+
+`share/tricu/portable.json` records the library inventory, SHA-256 hashes,
+DT_NEEDED names, system-provided sonames and maximum required GLIBC symbol
+version. The hashes identify the packaged library bytes.
+The current symbol floor is glibc 2.38. Python/userspace acceptance covers
+Ubuntu 24.04 (glibc 2.39), Debian 13 (2.41) and Fedora 44 (2.43), with no
+`/nix/store`; Ubuntu 22.04 (2.35) is the expected GLIBC rejection control.
+VM acceptance covers Ubuntu 24.04.5 with kernel `6.8.0-147-generic` and
+Python 3.12.3, Node 22.22.0/Koffi 2.16.2 and PHP CLI/FFI 8.3.6. Existing loaded
+libraries in an embedding process can affect ELF symbol resolution. Validate
+the target application's actual runtime.
+
+Applications supply their own compiled Arboricx bundles. The acceptance workload separately
+packages the filecheck worker and tests complete hashes, repeated use, ordinary
+errors, a 1-MiB input, CLI record/verify/mutation, ABI/scopes and compiled binary
+copying at two relocated paths. Removing bundled GMP must cause loading to fail.
+
+The included ISC license covers tricu. Third-party texts and the component index
+are under `share/tricu/notices/` and `share/tricu/THIRD-PARTY-NOTICES.md`.
+Distribute this runtime with `tricu-source-materials.tar.gz` and `release.json`
+from the same build. The source archive carries matching upstream inputs,
+patches, Cabal revisions and build recipes; the release receipt hashes both
+archives. Read `share/tricu/REDISTRIBUTION.md` for source delivery, license
+choices, modification notices and compatible-library replacement instructions.
diff --git a/ext/haskell/portable.py b/ext/haskell/portable.py
@@ -0,0 +1,93 @@
+#!/usr/bin/env python3
+"""Relocate trusted, Nix-built ELF dependencies; system glibc remains external."""
+import argparse
+import hashlib
+import json
+import re
+import shutil
+import subprocess
+from pathlib import Path
+
+
+def output(*command):
+ return subprocess.check_output(command, text=True).strip()
+
+
+def dependencies(path):
+ text = output('ldd', str(path))
+ if 'not found' in text:
+ raise RuntimeError(text)
+ found = {}
+ for line in text.splitlines():
+ fields = line.split()
+ if len(fields) >= 3 and fields[1] == '=>' and fields[2].startswith('/'):
+ name = Path(fields[0]).name
+ path = Path(fields[2]).resolve(strict=True)
+ if name in found and found[name] != path:
+ raise RuntimeError(f'ambiguous native dependency: {name}')
+ found[name] = path
+ return found
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument('--host', type=Path, required=True)
+ parser.add_argument('--glibc', type=Path, required=True)
+ parser.add_argument('--output', type=Path, required=True)
+ parser.add_argument('--materials', type=Path, required=True)
+ args = parser.parse_args()
+ root = args.output
+ root.mkdir(parents=True, exist_ok=False)
+ (root/'lib').mkdir()
+ (root/'include').mkdir()
+ (root/'share/tricu').mkdir(parents=True)
+ library = args.host/'lib/libtricu-haskell.so'
+ materials = json.loads((args.materials/'components.json').read_text())
+ if hashlib.sha256(library.read_bytes()).hexdigest() != materials['host_sha256']:
+ raise RuntimeError('source materials identify a different embedding library')
+ loaded = dependencies(library)
+ if any(name.startswith('libHS') for name in loaded):
+ raise RuntimeError('portable packaging requires archive-linked Haskell')
+ system = {name: path for name, path in loaded.items() if path.is_relative_to(args.glibc)}
+ native = {name: path for name, path in loaded.items() if name not in system}
+ if set(native) != set(materials['shared_owners']):
+ raise RuntimeError('native source/notice inventory differs from bundled libraries')
+ for name, path in native.items():
+ row = next(row for row in materials['inputs'] if row.get('soname') == name)
+ if hashlib.sha256(path.read_bytes()).hexdigest() != row['sha256']:
+ raise RuntimeError(f'native source materials identify different bytes: {name}')
+ native[library.name] = library.resolve(strict=True)
+ versions = set()
+ inventory = {}
+ for name, source in sorted(native.items()):
+ if Path(name).name != name:
+ raise RuntimeError(f'non-relative library identity: {name}')
+ target = root/'lib'/name
+ shutil.copyfile(source, target)
+ target.chmod(0o755)
+ subprocess.run(['patchelf', '--set-rpath', '$ORIGIN', str(target)], check=True)
+ needed = output('patchelf', '--print-needed', str(target)).splitlines()
+ if any('/' in item or item not in native and item not in system for item in needed):
+ raise RuntimeError(f'incomplete dependency closure: {name}: {needed}')
+ assert output('patchelf', '--print-rpath', str(target)) == '$ORIGIN'
+ versions.update(re.findall(r'\bGLIBC_(\d+\.\d+(?:\.\d+)?)\b',
+ output('readelf', '--version-info', str(target))))
+ inventory[name] = dict(sha256=hashlib.sha256(target.read_bytes()).hexdigest(),
+ bytes=target.stat().st_size, needed=needed)
+ shutil.copyfile(args.host/'include/tricu.h', root/'include/tricu.h')
+ shutil.copyfile(args.host/'share/tricu/tricu.py', root/'share/tricu/tricu.py')
+ shutil.copyfile(args.host/'share/tricu/build.json', root/'share/tricu/build.json')
+ for name in ('THIRD-PARTY-NOTICES.md', 'REDISTRIBUTION.md', 'license-components.json'):
+ shutil.copyfile(args.materials/name, root/'share/tricu'/name)
+ shutil.copytree(args.materials/'notices', root/'share/tricu/notices')
+ report = dict(source_manifest_sha256=hashlib.sha256((args.materials/'SHA256.json').read_bytes()).hexdigest(),
+ source_symlinks_sha256=hashlib.sha256((args.materials/'SYMLINKS.json').read_bytes()).hexdigest(),
+ schema='tricu.portable-elf.v1', architecture='x86_64-linux',
+ glibc_symbol_floor=max(versions, key=lambda s: tuple(map(int, s.split('.')))),
+ system_sonames=sorted(system), libraries=inventory,
+ scope='Relocatable native embedding; system glibc; same-platform acceptance required')
+ (root/'share/tricu/portable.json').write_text(json.dumps(report, indent=2, sort_keys=True)+'\n')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/ext/haskell/portable_fixture.py b/ext/haskell/portable_fixture.py
@@ -0,0 +1,52 @@
+#!/usr/bin/env python3
+"""Build a relocated Python/glibc test OS, separate from the distributed library."""
+import argparse
+import shutil
+import subprocess
+from pathlib import Path
+from portable import dependencies
+
+def is_elf(path):
+ if not path.is_file():
+ return False
+ with path.open('rb') as stream:
+ return stream.read(4) == b'\x7fELF'
+
+
+p = argparse.ArgumentParser(description=__doc__)
+p.add_argument('--python', type=Path, required=True)
+p.add_argument('--stdlib', type=Path, required=True)
+p.add_argument('--loader', type=Path, required=True)
+p.add_argument('--output', type=Path, required=True)
+a = p.parse_args()
+root = a.output
+(root/'python/bin').mkdir(parents=True)
+(root/'system/lib').mkdir(parents=True)
+shutil.copytree(a.stdlib, root/'python/lib'/a.stdlib.name,
+ ignore=shutil.ignore_patterns('__pycache__', 'site-packages', 'test', 'tests', 'config-*'))
+shutil.copyfile(a.python.resolve(), root/'python/bin/python3')
+elfs = [path for path in (root/'python').rglob('*')
+ if is_elf(path)]
+native = {}
+for path in elfs:
+ path.chmod(0o755)
+ for name, original in dependencies(path).items():
+ if name in native and native[name] != original:
+ raise RuntimeError(f'ambiguous test-system dependency: {name}')
+ native[name] = original
+# Supply the test OS's full glibc surface, including librt used by the RTS.
+for path in a.loader.parent.glob('*.so*'):
+ if is_elf(path):
+ native[path.name] = path.resolve()
+native[a.loader.name] = a.loader
+for name, original in native.items():
+ target = root/'system/lib'/name
+ shutil.copyfile(original, target)
+ elfs.append(target)
+for path in elfs:
+ path.chmod(0o755)
+ # Preserve the loader's own ELF layout; only loaded objects need RUNPATH.
+ if path.name != a.loader.name:
+ subprocess.run(['patchelf', '--set-rpath', '/system/lib', str(path)], check=True)
+subprocess.run(['patchelf', '--set-interpreter', '/system/lib/'+a.loader.name,
+ str(root/'python/bin/python3')], check=True)
diff --git a/ext/haskell/portable_workload.py b/ext/haskell/portable_workload.py
@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+"""Shared Nix-free Python/IO workload; caller supplies /tests and /work."""
+import hashlib, json, os, subprocess, sys
+from pathlib import Path
+release = Path(sys.argv[1]).resolve()
+assert not Path('/nix/store').exists()
+assert not Path('/proc/self/root/nix/store').exists()
+assert not os.environ.get('TRICU_LIB')
+from tricu import Library, find_library
+assert find_library() == release/'lib/libtricu-haskell.so'
+library = Library()
+from python_filecheck import reuse
+report = reuse('/work')
+# Installed application CLI: record, verify and same-length mutation.
+cli = [sys.executable, '-c', 'from tricu_filecheck.cli import main; raise SystemExit(main())']
+for arguments in [('record', '/work/checks.json', 'reused.bin'), ('verify', '/work/checks.json')]:
+ result = subprocess.run(cli+list(arguments), check=True, text=True, capture_output=True, timeout=30)
+ assert json.loads(result.stdout)['ok'] is True
+entry = json.loads(Path('/work/checks.json').read_text())['files'][0]
+assert entry['sha256'] == hashlib.sha256(Path('/work/reused.bin').read_bytes()).hexdigest()
+changed = Path('/work/reused.bin')
+original = changed.read_bytes()
+changed.write_bytes(b'x'*len(original))
+result = subprocess.run(cli+['verify', '/work/checks.json'], text=True, capture_output=True, timeout=30)
+assert result.returncode == 1 and json.loads(result.stdout)['files'][0]['status'] == 'changed'
+changed.write_bytes(original)
+report['cli_record_verify_mutation'] = True
+# Full installed ABI/IO controls execute in fresh foreign processes.
+for script, extra in [('test_embedding.py', ['--copy-bundle', '/tests/copy.arboricx']),
+ ('test_scopes.py', [])]:
+ subprocess.run([sys.executable, '/tests/'+script,
+ str(release/'lib/libtricu-haskell.so'), '/tests/fixtures', *extra],
+ check=True, timeout=90)
+# No hidden Nix mappings or library-resolution fallback.
+from tricu_filecheck import FileChecker
+with FileChecker() as checker:
+ assert checker.hash_file('/work/reused.bin').sha256 == hashlib.sha256(Path('/work/reused.bin').read_bytes()).hexdigest()
+ maps = Path('/proc/self/maps').read_text()
+ assert '/nix/store/' not in maps
+ assert str(release/'lib/libtricu-haskell.so') in maps
+ assert str(release/'lib/libgmp.so') in maps
+report['nix_store_absent'] = True
+Path('/work/report.json').write_text(json.dumps(report, indent=2)+'\n')
diff --git a/ext/haskell/redistribution-policy.json b/ext/haskell/redistribution-policy.json
@@ -0,0 +1,24 @@
+{
+ "schema": "tricu.redistribution-policy.v1",
+ "components": {
+ "base16-bytestring-1.0.2.0": {"terms": "BSD-3-Clause", "required_notices": ["LICENSE"]},
+ "basement-0.0.16": {"terms": "BSD-3-Clause and retained source-level notices", "required_notices": ["LICENSE"]},
+ "bzip2-1.0.8": {"terms": "bzip2 license", "required_notices": ["LICENSE"]},
+ "cryptonite-0.30": {"terms": "BSD-3-Clause package; source-level terms include BSD-2-Clause SHA-256", "required_notices": ["LICENSE", "cbits/cryptonite_sha256.c"]},
+ "elfutils-0.194": {"terms": "LGPL-3.0-or-later option for libelf and libdw", "required_notices": ["COPYING", "COPYING-LGPLV3", "libelf/libelf.h", "libdw/libdw.h"]},
+ "gcc-15.2.0": {"terms": "GCC runtime components: GPL-3.0-or-later WITH GCC-exception-3.1", "required_notices": ["COPYING3", "COPYING.RUNTIME", "libatomic/config/posix/lock.c"]},
+ "ghc-9.14.1": {"terms": "GHC BSD-style license plus boot-library and vendored-source notices", "required_notices": ["LICENSE", "libraries/ghc-internal/LICENSE", "libraries/text/LICENSE"]},
+ "glibc-2.42-61": {"terms": "LGPL-2.1-or-later and source-specific notices/exceptions for linked startup objects; system shared glibc is external", "required_notices": ["COPYING", "COPYING.LIB", "LICENSES"]},
+ "gmp-with-cxx-6.3.0": {"terms": "LGPL-3.0-or-later option", "required_notices": ["COPYING", "COPYING.LESSERv3"]},
+ "libffi-3.5.2": {"terms": "MIT and retained source-level notices", "required_notices": ["LICENSE"]},
+ "memory-0.18.0": {"terms": "BSD-3-Clause and retained source-level notices", "required_notices": ["LICENSE"]},
+ "network-3.2.8.0": {"terms": "BSD-3-Clause", "required_notices": ["LICENSE"]},
+ "numactl-2.0.18": {"terms": "LGPL-2.1 option for libnuma; GPL command-line tools are not shipped", "required_notices": ["LICENSE.LGPL2.1", "LICENSE.GPL2"]},
+ "primitive-0.9.1.0": {"terms": "BSD-3-Clause", "required_notices": ["LICENSE"]},
+ "vector-0.13.2.0": {"terms": "BSD-3-Clause", "required_notices": ["LICENSE"]},
+ "vector-stream-0.1.0.1": {"terms": "BSD-3-Clause", "required_notices": ["LICENSE"]},
+ "xz-5.8.3": {"terms": "0BSD for liblzma", "required_notices": ["COPYING", "COPYING.0BSD"]},
+ "zlib-1.3.2": {"terms": "Zlib", "required_notices": ["LICENSE"]},
+ "zstd-1.5.7": {"terms": "BSD-3-Clause option", "required_notices": ["LICENSE", "COPYING"]}
+ }
+}
diff --git a/ext/haskell/redistribution.nix b/ext/haskell/redistribution.nix
@@ -0,0 +1,45 @@
+{ pkgs, host, source, nixpkgsSource, flakeUtilsSource, systemsSource }:
+let
+ # Match linker inputs and installed ELF owners across the native closure.
+ native = [ pkgs.gmpxx pkgs.libffi pkgs.elfutils pkgs.numactl pkgs.zlib
+ pkgs.zstd pkgs.xz pkgs.bzip2 pkgs.stdenv.cc.cc pkgs.glibc ];
+ components = [ host.compiler ] ++ host.haskellLibraries ++ native;
+ describe = p: let
+ clean = value: if builtins.isString value then
+ builtins.replaceStrings (map builtins.placeholder p.outputs)
+ (map (name: "@OUTPUT_${name}@") p.outputs) value
+ else if builtins.isList value then map clean value else value;
+ in {
+ name = p.name;
+ version = p.version or "unknown";
+ roots = pkgs.lib.unique (map toString ([ p ] ++ map (name: p.${name})
+ (builtins.filter (name: builtins.elem name p.outputs) [ "out" "lib" "libgcc" "dev" ])));
+ source = toString p.src;
+ patches = map toString (p.patches or []);
+ cabalRevision = if p ? editedCabalFile && p.editedCabalFile != null
+ then toString p.editedCabalFile else null;
+ licenseMetadata = p.meta.license or [];
+ homepage = p.meta.homepage or "";
+ recipe = {
+ inherit (p) name;
+ configureFlags = clean (p.configureFlags or []);
+ prePatch = clean (p.prePatch or "");
+ postPatch = clean (p.postPatch or "");
+ preConfigure = clean (p.preConfigure or "");
+ postConfigure = clean (p.postConfigure or "");
+ };
+ };
+ metadata = pkgs.writeText "tricu-redistribution-inputs.json" (builtins.toJSON (map describe components));
+in pkgs.runCommand "tricu-redistribution-materials" {
+ nativeBuildInputs = [ pkgs.python3 pkgs.glibc.bin ];
+ meta.platforms = [ "x86_64-linux" ];
+} ''
+ mkdir tools
+ cp ${./redistribution.py} tools/redistribution.py
+ cp ${./portable.py} tools/portable.py
+ python3 tools/redistribution.py --components ${metadata} \
+ --host ${host} --link-evidence ${host.linkEvidence} \
+ --source ${source} --nixpkgs ${nixpkgsSource} \
+ --flake-utils ${flakeUtilsSource} --systems ${systemsSource} \
+ --policy ${./redistribution-policy.json} --output $out
+''
diff --git a/ext/haskell/redistribution.py b/ext/haskell/redistribution.py
@@ -0,0 +1,194 @@
+#!/usr/bin/env python3
+"""Collect source/notice evidence for actual ELF and archive inputs. Fail closed."""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import re
+import shutil
+import tarfile
+
+from portable import dependencies
+
+
+def digest(path):
+ with path.open('rb') as stream:
+ return hashlib.file_digest(stream, 'sha256').hexdigest()
+
+
+def owner(path, components):
+ path = path.resolve(strict=True)
+ matches = [row for row in components
+ if any(path.is_relative_to(Path(root)) for root in row['roots'])]
+ if len(matches) != 1:
+ raise ValueError(f'expected one source owner for {path}, found {[r["name"] for r in matches]}')
+ return matches[0]
+
+
+def license_name(name):
+ if Path(name).suffix.lower() in ('.hs', '.json', '.sh', '.py', '.c', '.h'):
+ return False
+ return bool(re.match(r'^(copying|licen[cs]es?|copyright|notices?|authors)([._-].*|\d.*)?$',
+ Path(name).name, re.I))
+
+
+def notices(source, component):
+ """Preserve named texts and leading C-family copyright/permission blocks.
+
+ Header notices cover vendored implementations whose terms live beside code
+ (for example cryptonite's SHA-256 implementation). Source-only notices are
+ included conservatively and indexed by source component.
+ """
+ def read_notice(name, stream):
+ if license_name(name):
+ return stream.read()
+ if Path(name).suffix.lower() not in ('.c', '.h', '.cc', '.cpp', '.hpp', '.s'):
+ return None
+ if component.startswith('glibc-') and Path(name).name not in ('crti.S', 'crtn.S', 'start.S', 'abi-note.c', 'static-reloc.c'):
+ return None
+ if component.startswith('gcc-') and not re.search(r'/(libgcc|libatomic|include)/|/gcc/crtstuff\.c$', name):
+ return None
+ if component.startswith('ghc-') and not re.search(r'(^|/)(libraries|rts|includes)/', name):
+ return None
+ data = stream.read(65536)
+ match = re.match(rb'\s*((?:/\*.*?\*/\s*)+)', data, re.S)
+ if match and re.search(rb'copyright|permission|redistribution|SPDX-License', match[1], re.I):
+ return match[1]
+ return None
+ if source.is_dir():
+ for path in sorted(source.rglob('*')):
+ if path.is_file() and not path.is_symlink():
+ with path.open('rb') as stream:
+ data = read_notice(str(path), stream)
+ if data:
+ yield str(path.relative_to(source)), data
+ else:
+ with tarfile.open(source, 'r:*') as archive:
+ for member in archive:
+ if member.isfile():
+ with archive.extractfile(member) as stream:
+ data = read_notice(member.name, stream)
+ if data:
+ yield member.name, data
+
+
+def copy_input(source, target):
+ if source.is_dir():
+ shutil.copytree(source, target, symlinks=True)
+ else:
+ shutil.copyfile(source, target)
+
+
+def main():
+ p = argparse.ArgumentParser(description=__doc__)
+ for name in ('components', 'host', 'link-evidence', 'source', 'nixpkgs', 'flake-utils', 'systems', 'policy', 'output'):
+ p.add_argument('--'+name, type=Path, required=True)
+ a = p.parse_args()
+ components = json.loads(a.components.read_text())
+ policy = json.loads(a.policy.read_text())['components']
+ link_map = (a.link_evidence/'embedding-link.map').read_text()
+ paths = sorted(set(re.findall(r'/nix/store/[^\s()]+?\.(?:a|o)(?=[(\s]|$)', link_map)))
+ if not any(path.endswith('.a') for path in paths):
+ raise ValueError('archive linkage evidence missing')
+ selected = {}
+ inputs = []
+ for raw in paths:
+ path = Path(raw).resolve(strict=True)
+ component = owner(path, components)
+ selected[component['name']] = component
+ inputs.append(dict(kind='link-input', path=str(path), sha256=digest(path), component=component['name']))
+ shared = {}
+ for name, path in dependencies(a.host/'lib/libtricu-haskell.so').items():
+ component = owner(path, components)
+ # System glibc is supplied by the destination OS. The link map selects
+ # the source materials for glibc startup objects.
+ if component['name'].startswith('glibc-'):
+ continue
+ selected[component['name']] = component
+ shared[name] = component['name']
+ inputs.append(dict(kind='shared-library', soname=name, path=str(path),
+ sha256=digest(path), component=component['name']))
+ unknown = set(selected) - set(policy)
+ if unknown:
+ raise ValueError(f'license review required for new components: {sorted(unknown)}')
+ a.output.mkdir(parents=True, exist_ok=False)
+ sources = a.output/'sources'
+ sources.mkdir()
+ notice_root = a.output/'notices'
+ notice_root.mkdir()
+ rows = []
+ for name, component in sorted(selected.items()):
+ if Path(name).name != name:
+ raise ValueError('unsafe component identity')
+ directory = sources/name
+ directory.mkdir()
+ upstream = Path(component['source'])
+ filename = 'upstream' if upstream.is_dir() else upstream.name
+ copy_input(upstream, directory/filename)
+ copied_patches = []
+ for index, patch in enumerate(component['patches']):
+ destination = f'patch-{index:03d}-'+Path(patch).name
+ copy_input(Path(patch), directory/destination)
+ copied_patches.append(destination)
+ if component['cabalRevision']:
+ shutil.copyfile(component['cabalRevision'], directory/'revised.cabal')
+ (directory/'recipe.json').write_text(json.dumps(component, indent=2, sort_keys=True)+'\n')
+ files = []
+ unique = {}
+ for index, (original, data) in enumerate(notices(upstream, name)):
+ # Preserve source path in the manifest; numbered names avoid collisions
+ # and prevent arbitrary paths in upstream archive names being written.
+ folder = notice_root/name
+ folder.mkdir(exist_ok=True)
+ checksum = hashlib.sha256(data).hexdigest()
+ destination = unique.setdefault(checksum, folder/f'{index:04d}-{Path(original).name}.txt')
+ if not destination.exists():
+ destination.write_bytes(data)
+ files.append(dict(source_path=original, file=str(destination.relative_to(a.output)),
+ sha256=checksum))
+ if not files:
+ raise ValueError(f'no license/notice texts found for {name}')
+ for required in policy[name]['required_notices']:
+ if not any(row['source_path'] == required or row['source_path'].endswith('/'+required) for row in files):
+ raise ValueError(f'missing required notice: {name}: {required}')
+ rows.append(dict(name=name, version=component['version'], homepage=component['homepage'],
+ terms=policy[name]['terms'],
+ license_metadata=component['licenseMetadata'], upstream_file=filename,
+ patches=copied_patches, notices=files))
+ # Complete authored build recipes, lock, overlays and pinned Nixpkgs recipes
+ # accompany upstream sources/patches for inspection and rebuilding.
+ copy_input(a.source, sources/'tricu')
+ copy_input(a.nixpkgs, sources/'nixpkgs')
+ copy_input(a.flake_utils, sources/'flake-utils')
+ copy_input(a.systems, sources/'systems')
+ shutil.copytree(a.link_evidence, a.output/'link-evidence')
+ shutil.copyfile(a.policy, a.output/'redistribution-policy.json')
+ shutil.copyfile(a.source/'LICENSE', a.output/'LICENSE')
+ shutil.copyfile(a.source/'ext/haskell/REDISTRIBUTION.md', a.output/'REDISTRIBUTION.md')
+ report = dict(schema='tricu.redistribution.v1', components=rows, inputs=inputs,
+ shared_owners=shared,
+ host_sha256=digest(a.host/'lib/libtricu-haskell.so'),
+ coverage='Conservative archive/object input coverage, including discarded members; shared runtime inputs')
+ (a.output/'components.json').write_text(json.dumps(report, indent=2, sort_keys=True)+'\n')
+ index = ['# Third-party notices', '',
+ 'tricu is licensed under ISC; see LICENSE. The following source distributions',
+ 'supply linked code or bundled native libraries. Complete upstream notice',
+ 'texts are retained conservatively, including source-only subcomponents.',
+ 'The upstream texts govern licensing; Nix metadata provides an index.', '',
+ 'Matching source materials and build recipes accompany this release.', '']
+ for row in rows:
+ index += [f'## {row["name"]}', row['homepage'], '', row['terms'], '',
+ f'License/notice texts: `notices/{row["name"]}/`.',
+ f'Source and patch inputs: `sources/{row["name"]}/`.', '']
+ (a.output/'THIRD-PARTY-NOTICES.md').write_text('\n'.join(index)+'\n')
+ (a.output/'license-components.json').write_text(json.dumps(rows, indent=2, sort_keys=True)+'\n')
+ hashes = {str(path.relative_to(a.output)): digest(path)
+ for path in sorted(a.output.rglob('*')) if path.is_file() and not path.is_symlink()}
+ (a.output/'SHA256.json').write_text(json.dumps(hashes, indent=2, sort_keys=True)+'\n')
+ links = {str(path.relative_to(a.output)): str(path.readlink())
+ for path in sorted(a.output.rglob('*')) if path.is_symlink()}
+ (a.output/'SYMLINKS.json').write_text(json.dumps(links, indent=2, sort_keys=True)+'\n')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/ext/haskell/reload.c b/ext/haskell/reload.c
@@ -1,4 +1,4 @@
-#include "tricu-core.h"
+#include "tricu.h"
#include <assert.h>
#include <dlfcn.h>
#include <stdint.h>
diff --git a/ext/haskell/src/Embedding.hs b/ext/haskell/src/Embedding.hs
@@ -1,6 +1,6 @@
{-# LANGUAGE ForeignFunctionInterface #-}
{-# LANGUAGE OverloadedStrings #-}
--- Host extension to the shared core boundary; both artifacts use one reducer.
+-- Bundle loading and IO extend the internal reducer and value boundary.
module Embedding (module Embedding, module EmbeddingCore) where
import EmbeddingCore
diff --git a/ext/haskell/src/EmbeddingCore.hs b/ext/haskell/src/EmbeddingCore.hs
@@ -30,7 +30,7 @@ identities = unsafePerformIO (newIORef 1)
contexts :: MVar (M.Map Word64 (MVar (Maybe Context)))
contexts = unsafePerformIO (newMVar M.empty)
-- Force map updates while keeping their tree values lazy: release removes the
--- root now, and application publication never forces the suspended computation.
+-- root now, and application publication preserves the suspended computation.
data Context = Context { roots :: !(L.Map Word64 T), lastError :: !BS.ByteString }
data Fault = Fault Word32 String deriving Show
instance Exception Fault
@@ -52,9 +52,9 @@ newIdentity = do
else pure (fromInteger n)
-- Context locking serializes lazy evaluation as well as registry changes.
--- Destroy marks the cell closed before removing its name; racing lookups cannot
--- resurrect it. Exceptions roll back root-table publication, not IO effects or
--- evaluation already performed on shared Haskell thunks.
+-- Destroy marks the cell closed before removing its name; concurrent lookups
+-- respect that closed state. Exceptions roll back root-table publication.
+-- IO effects and evaluation already performed on shared Haskell thunks persist.
withContext :: Word64 -> Call (IO ()) -> IO Word32
withContext identity action = handle boundaryStatus $ mask $ \restore -> do
found <- withMVar contexts (pure . M.lookup identity)
diff --git a/ext/haskell/src/init.c b/ext/haskell/src/init.c
@@ -1,7 +1,7 @@
#include <Rts.h>
#include <pthread.h>
#include <stdlib.h>
-#include "tricu-core.h"
+#include "tricu.h"
static pthread_once_t initialized = PTHREAD_ONCE_INIT;
static void initialize(void) {
diff --git a/ext/haskell/test_container_harness.py b/ext/haskell/test_container_harness.py
@@ -0,0 +1,165 @@
+"""Engine-free controls; real distro acceptance remains an explicit matrix run."""
+import argparse
+import hashlib
+import subprocess
+import io
+import json
+from pathlib import Path
+import tarfile
+import tempfile
+import unittest
+from unittest.mock import patch
+
+import container_matrix as matrix
+import container_bindings as bindings
+from container_probe import extract, incompatible_loader_error
+
+
+class HarnessTests(unittest.TestCase):
+ def test_glibc_control(self):
+ self.assertTrue(incompatible_loader_error("libc.so.6: version `GLIBC_2.38' not found (required by libtricu)", '2.35'))
+ for error in ('libgmp.so.10: cannot open shared object file', 'Segmentation fault',
+ "version `GLIBC_2.34' not found", "version `GLIBCXX_3.4.30' not found",
+ "ModuleNotFoundError: No module named 'ctypes'"):
+ self.assertFalse(incompatible_loader_error(error, '2.35'))
+
+ def test_image_identity(self):
+ base = dict(Architecture='amd64', Os='linux', Id='a'*64)
+ with patch.object(matrix, 'checked', return_value=json.dumps([base])):
+ self.assertEqual(matrix.inspect('podman', 'test')['Id'], 'sha256:'+'a'*64)
+ for change in (dict(Architecture='arm64'), dict(Os='windows'), dict(Id='latest')):
+ with patch.object(matrix, 'checked', return_value=json.dumps([dict(base, **change)])):
+ with self.assertRaises(ValueError):
+ matrix.inspect('docker', 'test')
+
+ def test_lock_requires_fixed_expectation_and_evidence(self):
+ with tempfile.TemporaryDirectory() as directory:
+ root = Path(directory)
+ (root/'ubuntu22').mkdir()
+ row = dict(name='ubuntu22', image_id='sha256:'+'a'*64, expectation='glibc-rejection')
+ for file, key in [('Dockerfile', 'dockerfile_sha256'), ('packages.txt', 'packages_sha256')]:
+ (root/'ubuntu22'/file).write_text(file)
+ row[key] = hashlib.sha256(file.encode()).hexdigest()
+ path = root/'images.json'
+ def save(rows):
+ path.write_text(json.dumps(dict(schema='tricu.distro-images.v1', images=rows)))
+ save([row])
+ self.assertEqual(matrix.load_lock(path)['images'], [row])
+ for rows in ([], [row, row], [dict(row, image_id='ubuntu:22.04')],
+ [dict(row, expectation='pass')], [dict(row, name='../escape')]):
+ save(rows)
+ with self.assertRaises(ValueError):
+ matrix.load_lock(path)
+ save([row])
+ (root/'ubuntu22/packages.txt').write_text('changed')
+ with self.assertRaises(ValueError):
+ matrix.load_lock(path)
+
+ def test_bindings_preparation_and_lock(self):
+ with tempfile.TemporaryDirectory() as directory:
+ root = Path(directory)
+ for name in matrix.BINDING_FILES:
+ path = root/'kit/bindings'/name
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text('{}')
+ args = argparse.Namespace(directory=root/'images', distro=['ubuntu24'],
+ engine='podman', bindings=True, node_version='22.22.0')
+ image = dict(Id='sha256:'+'a'*64, RepoDigests=['example/node@sha256:'+'b'*64])
+ with patch.object(matrix, '__file__', str(root/'kit/container_matrix.py')), \
+ patch.object(matrix, 'checked', return_value='packages'), \
+ patch.object(matrix, 'inspect', return_value=image), \
+ patch.object(matrix.subprocess, 'run', return_value=subprocess.CompletedProcess([], 0, '', '')):
+ matrix.prepare(args)
+ lock = matrix.load_lock(args.directory/'images.json')
+ self.assertEqual(lock['profile'], 'bindings')
+ self.assertEqual(lock['node']['version'], '22.22.0')
+ recipe = (args.directory/'ubuntu24/Dockerfile').read_text()
+ self.assertIn('npm ci --omit=dev', recipe)
+ self.assertIn('php-ffi', recipe)
+ self.assertIn('COPY --from='+image['RepoDigests'][0], recipe)
+ path = args.directory/'images.json'
+ for change in (dict(version='20.19.0'), dict(digest='latest'),
+ dict(digest='other/node@sha256:'+'c'*64)):
+ path.write_text(json.dumps(dict(lock, node=dict(lock['node'], **change))))
+ with self.assertRaises(ValueError):
+ matrix.load_lock(path)
+ path.write_text(json.dumps(lock))
+ (args.directory/'ubuntu24/package-lock.json').write_text('changed')
+ with self.assertRaises(ValueError):
+ matrix.load_lock(path)
+ for enabled, version in ((True, None), (False, '22.22.0'), (True, 'latest'), (True, '20.19.0')):
+ args.bindings, args.node_version = enabled, version
+ with self.assertRaises(ValueError):
+ matrix.prepare(args)
+
+ def test_packaged_bindings_layout(self):
+ with tempfile.TemporaryDirectory() as directory:
+ kit = Path(directory)/'kit'
+ for name in matrix.BINDING_FILES:
+ path = kit/'bindings'/name
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text('{}')
+ matrix.validate_bindings_kit(kit)
+ for name in matrix.BINDING_FILES:
+ path = kit/'bindings'/name
+ renamed = path.with_name('store-hash-'+path.name)
+ path.rename(renamed)
+ with self.assertRaises(ValueError):
+ matrix.validate_bindings_kit(kit)
+ renamed.rename(path)
+ (kit/'bindings/ext/js/package.json').unlink()
+ args = argparse.Namespace(directory=Path(directory)/'images', distro=['ubuntu24'],
+ engine='podman', bindings=True, node_version='22.22.0')
+ with patch.object(matrix, '__file__', str(kit/'container_matrix.py')), \
+ patch.object(matrix, 'checked') as engine:
+ with self.assertRaises(ValueError):
+ matrix.prepare(args)
+ engine.assert_not_called()
+ self.assertFalse(args.directory.exists())
+
+ def test_binding_results_and_runtime_versions(self):
+ with tempfile.TemporaryDirectory() as directory:
+ root = Path(directory)
+ source = root/'work/bindings'
+ for name, body in {
+ 'work/bindings/ext/js/package-lock.json': json.dumps({'packages': {'node_modules/koffi': {'version': '2.16.2'}}}),
+ 'opt/tricu-js/node_modules/koffi/package.json': '{"version":"2.16.2"}',
+ 'work/bindings/ext/haskell/bindings/fixtures.json': '{"expected":{"result":42}}',
+ 'release/include/tricu.h': 'header',
+ }.items():
+ path = root/name
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text(body)
+ (source/'ext/php').mkdir()
+ php = json.dumps(dict(ffi=True, integer_bytes=8, version='8.3', ffi_enable='1'))
+ good = ['v22.22.0', php, '{"result":42}', '{"result":42}', '{"result":42}']
+ with patch.object(bindings, 'Path', side_effect=lambda name: root/str(name).lstrip('/')):
+ with patch.object(bindings, 'checked', side_effect=good):
+ self.assertEqual(bindings.run(root/'release', '22.22.0')['passed'], ['python', 'js', 'php'])
+ for values in (['v20.19.0'], ['v22.22.0', '{"ffi":false,"integer_bytes":8}'],
+ good[:3]+['{"result":0}']):
+ with patch.object(bindings, 'checked', side_effect=values):
+ with self.assertRaises(ValueError):
+ bindings.run(root/'release', '22.22.0')
+
+ def test_archive_members(self):
+ with tempfile.TemporaryDirectory() as directory:
+ root = Path(directory)
+ archive = root/'test.tar.gz'
+ for name, kind in [('tricu/file', tarfile.REGTYPE), ('../escape', tarfile.REGTYPE),
+ ('/tricu/file', tarfile.REGTYPE), ('tricu/link', tarfile.SYMTYPE)]:
+ with tarfile.open(archive, 'w:gz') as stream:
+ entry = tarfile.TarInfo(name)
+ entry.type = kind
+ entry.size = 1 if kind == tarfile.REGTYPE else 0
+ entry.linkname = '/outside' if kind == tarfile.SYMTYPE else ''
+ stream.addfile(entry, io.BytesIO(b'x'))
+ if name == 'tricu/file':
+ self.assertEqual((extract(archive, root/'out')/'file').read_bytes(), b'x')
+ else:
+ with self.assertRaises(ValueError):
+ extract(archive, root/'out')
+
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/ext/haskell/test_embedding.py b/ext/haskell/test_embedding.py
@@ -48,7 +48,7 @@ def checks(lib, fixtures):
assert ctx.kind(alias) == 1
for value in (0, 1, 42, 2**64-1):
assert ctx.to_number(ctx.number(value)) == value
- # This raw tree is 2**64, not a representable uint64.
+ # This raw tree is 2**64, exceeding the uint64 upper bound.
huge = ctx.number(1)
for _ in range(64):
huge = ctx.fork(leaf, huge)
@@ -86,7 +86,7 @@ def checks(lib, fixtures):
assert ctx.kind(cleanup) == 0
expect(3, lambda: ctx.run_io(leaf))
expect(3, lambda: ctx.run_io(envelope(ctx, leaf, version=1)))
- # Malformed action is a task-level failure, not a native boundary error.
+ # A malformed action produces a task-level failure.
outcome, diagnostic, cleanup = ctx.run_io(envelope(ctx, leaf))
assert outcome == 1 and ctx.kind(diagnostic) == 2 and ctx.kind(cleanup) == 0
# IO outputs are checked before execution and all remain unchanged.
diff --git a/ext/haskell/test_exports.py b/ext/haskell/test_exports.py
@@ -1,4 +1,4 @@
-"""Fresh-process checks of the common C surface and host-only extension."""
+"""Fresh-process checks of the complete C surface."""
import ctypes
import sys
import subprocess
@@ -11,8 +11,9 @@ core_names = ('init', 'abi_version', 'context_new', 'context_free', 'retain',
for name in core_names:
assert getattr(library, 'tricu_' + name)
for name in ('load_bundle', 'run_io'):
- assert hasattr(library, 'tricu_' + name) == (sys.argv[2] == 'host')
+ assert getattr(library, 'tricu_' + name)
-if len(sys.argv) > 3 and sys.argv[3] == 'public-only':
+if len(sys.argv) > 2 and sys.argv[2] == 'public-only':
symbols = subprocess.check_output(['nm', '-D', '--defined-only', sys.argv[1]], text=True)
- assert {line.split()[-1] for line in symbols.splitlines()} == {'tricu_' + name for name in core_names}
+ expected = core_names + ('load_bundle', 'run_io')
+ assert {line.split()[-1] for line in symbols.splitlines()} == {'tricu_' + name for name in expected}
diff --git a/ext/haskell/test_portable.py b/ext/haskell/test_portable.py
@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""Exercise a relocated release under an empty mount root with no Nix store."""
+import argparse
+import json
+from pathlib import Path
+import shutil
+import subprocess
+import tempfile
+import tarfile
+
+
+def main():
+ p = argparse.ArgumentParser(description=__doc__)
+ p.add_argument('--release', type=Path, required=True)
+ p.add_argument('--system', type=Path, required=True)
+ p.add_argument('--source', type=Path, required=True)
+ p.add_argument('--worker', type=Path, required=True)
+ p.add_argument('--copier', type=Path, required=True)
+ p.add_argument('--output', type=Path, required=True)
+ a = p.parse_args()
+ a.output.mkdir(parents=True, exist_ok=False)
+ with tempfile.TemporaryDirectory() as tmp:
+ tmp = Path(tmp)
+ tests = tmp/'tests'
+ tests.mkdir()
+ for name in ('test_embedding.py', 'test_scopes.py'):
+ shutil.copyfile(a.source/'ext/haskell'/name, tests/name)
+ for name in ('python_filecheck.py', 'process_rss.py'):
+ shutil.copyfile(a.source/'test'/name, tests/name)
+ (tests/'fixtures').mkdir()
+ for name in ('id.arboricx', 'append.arboricx'):
+ shutil.copyfile(a.source/'test/fixtures'/name, tests/'fixtures'/name)
+ shutil.copyfile(a.copier, tests/'copy.arboricx')
+ shutil.copytree(a.source/'demos/python-filecheck/tricu_filecheck', tests/'tricu_filecheck')
+ for path in [tests/'tricu_filecheck', *(tests/'tricu_filecheck').rglob('*')]:
+ path.chmod(0o755 if path.is_dir() else 0o644)
+ shutil.copyfile(a.worker/'worker.arboricx', tests/'tricu_filecheck/worker.arboricx')
+ pin = (a.worker/'worker.sha256').read_text().strip()
+ (tests/'tricu_filecheck/_bundle_pin.py').write_text(f'BUNDLE_SHA256 = "{pin}"\n')
+ reports = []
+ for location in ('first', 'relocated path with spaces'):
+ release = tmp/location
+ with tarfile.open(a.release, 'r:gz') as archive:
+ archive.extractall(tmp/'unpacked', filter='data')
+ (tmp/'unpacked/tricu').rename(release)
+ mount = '/'+location
+ shutil.copyfile(a.source/'ext/haskell/portable_workload.py', tests/'run.py')
+ (release/'lib').chmod(0o755)
+ import hashlib
+ inventory = json.loads((release/'share/tricu/portable.json').read_text())
+ for name, row in inventory['libraries'].items():
+ assert hashlib.sha256((release/'lib'/name).read_bytes()).hexdigest() == row['sha256']
+ work = tmp/(location+'-work')
+ work.mkdir()
+ command = ['bwrap', '--unshare-all', '--die-with-parent', '--new-session',
+ '--ro-bind', str(a.system/'python'), '/python',
+ '--ro-bind', str(a.system/'system'), '/system',
+ '--ro-bind', str(release), mount,
+ '--ro-bind', str(tests), '/tests',
+ '--bind', str(work), '/work', '--tmpfs', '/tmp',
+ '--proc', '/proc', '--dev', '/dev', '--chdir', '/work',
+ '--clearenv', '--setenv', 'HOME', '/tmp',
+ '--setenv', 'PATH', '/no-runtime-tools',
+ '--setenv', 'PYTHONHOME', '/python',
+ '--setenv', 'LD_LIBRARY_PATH', '/system/lib',
+ '--setenv', 'PYTHONDONTWRITEBYTECODE', '1',
+ '--setenv', 'PYTHONPATH', mount+'/share/tricu:/tests',
+ '/python/bin/python3', '/tests/run.py', mount]
+ result = subprocess.run(command, text=True, capture_output=True, timeout=240)
+ (a.output/(location+'.log')).write_text(result.stdout+result.stderr)
+ if result.returncode:
+ raise RuntimeError(f'exit {result.returncode}: {result.stderr or result.stdout}')
+ reports.append(json.loads((work/'report.json').read_text()))
+ # Removing a required bundled dependency must fail before initialization.
+ (release/'lib/libgmp.so.10').unlink()
+ negative = subprocess.run(command[:-2]+['-c', 'from tricu import Library; Library()'],
+ text=True, capture_output=True, timeout=30)
+ assert negative.returncode and 'libgmp.so.10' in negative.stderr, negative
+ (a.output/'report.json').write_text(json.dumps(dict(
+ schema='tricu.portable-acceptance.v1', runs=reports,
+ missing_dependency_rejected=True,
+ scope='Nix-free mount namespace; pinned relocated Python/glibc test OS; not cross-distro acceptance'),
+ indent=2)+'\n')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/ext/haskell/test_redistribution.py b/ext/haskell/test_redistribution.py
@@ -0,0 +1,93 @@
+"""Fast source/notice ownership, integrity and archive-safety controls."""
+import hashlib
+import io
+import json
+from pathlib import Path
+import tarfile
+import tempfile
+import unittest
+
+from redistribution import license_name, notices, owner
+from verify_redistribution import relative, verify, verify_archive
+
+
+class RedistributionTests(unittest.TestCase):
+ def test_notice_names(self):
+ for name in ('LICENSES', 'COPYING.RUNTIME', 'COPYING3', 'COPYING.LESSERv3', 'LICENSE.LGPL2.1'):
+ self.assertTrue(license_name(name), name)
+ for name in ('License.hs', 'licenses.json', 'license-check.sh', 'other'):
+ self.assertFalse(license_name(name), name)
+
+ def test_owners(self):
+ with tempfile.TemporaryDirectory() as directory:
+ root = Path(directory)
+ (root/'input.a').write_bytes(b'archive')
+ row = dict(name='library', roots=[str(root)])
+ self.assertEqual(owner(root/'input.a', [row]), row)
+ for rows in ([], [row, row]):
+ with self.assertRaises(ValueError):
+ owner(root/'input.a', rows)
+
+ def test_embedded_notice(self):
+ with tempfile.TemporaryDirectory() as directory:
+ archive = Path(directory)/'source.tar.gz'
+ block = b'/* Copyright Example. Redistribution permitted. */\n'
+ with tarfile.open(archive, 'w:gz') as out:
+ for name, body in [('pkg/LICENSES', b'Complete license'),
+ ('pkg/cbits/hash.c', block+b'int value;')]:
+ item = tarfile.TarInfo(name)
+ item.size = len(body)
+ out.addfile(item, io.BytesIO(body))
+ self.assertEqual(dict(notices(archive, 'example')), {
+ 'pkg/LICENSES': b'Complete license', 'pkg/cbits/hash.c': block})
+
+ def test_archive_pairing(self):
+ with tempfile.TemporaryDirectory() as directory:
+ archive = Path(directory)/'release.tar.gz'
+ with tarfile.open(archive, 'w:gz') as out:
+ item = tarfile.TarInfo('release/file')
+ item.size = 4
+ out.addfile(item, io.BytesIO(b'data'))
+ expected = {'file': hashlib.sha256(b'data').hexdigest()}
+ verify_archive(archive, 'release', expected, {})
+ for prefix, hashes in [('wrong-prefix', expected), ('release', {}),
+ ('release', {'file': 'changed'})]:
+ with self.assertRaises(ValueError):
+ verify_archive(archive, prefix, hashes, {})
+
+ def test_verifier_controls(self):
+ with tempfile.TemporaryDirectory() as directory:
+ root = Path(directory)
+ (root/'sources/example').mkdir(parents=True)
+ (root/'notices').mkdir()
+ (root/'sources/example/upstream').write_bytes(b'original source')
+ (root/'notices/LICENSE').write_bytes(b'original license')
+ checksum = hashlib.sha256(b'original license').hexdigest()
+ (root/'components.json').write_text(json.dumps(dict(components=[dict(
+ name='example', upstream_file='upstream',
+ notices=[dict(file='notices/LICENSE', sha256=checksum)])])))
+ (root/'source-link').symlink_to('sources/example/upstream')
+ (root/'SYMLINKS.json').write_text(json.dumps({'source-link': 'sources/example/upstream'}))
+ hashes = {str(p.relative_to(root)): hashlib.sha256(p.read_bytes()).hexdigest()
+ for p in root.rglob('*') if p.is_file() and not p.is_symlink() and p.name != 'SYMLINKS.json'}
+ (root/'SHA256.json').write_text(json.dumps(hashes))
+ self.assertEqual(verify(root)['components'], 1)
+ (root/'notices/LICENSE').write_bytes(b'altered license')
+ with self.assertRaises(ValueError):
+ verify(root)
+ (root/'notices/LICENSE').write_bytes(b'original license')
+ (root/'extra').write_text('unexpected')
+ with self.assertRaises(ValueError):
+ verify(root)
+ (root/'extra').unlink()
+ (root/'source-link').unlink()
+ (root/'source-link').symlink_to('elsewhere')
+ with self.assertRaises(ValueError):
+ verify(root)
+ for path in ('../outside', '/outside'):
+ with self.assertRaises(ValueError):
+ relative(root, path)
+
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/ext/haskell/test_vm_harness.py b/ext/haskell/test_vm_harness.py
@@ -0,0 +1,143 @@
+"""Engine-free VM command, seed, image-lock and acceptance-report controls."""
+import copy
+import json
+from pathlib import Path
+import subprocess
+import tempfile
+import tarfile
+import unittest
+
+import vm_matrix as vm
+
+
+class VMTests(unittest.TestCase):
+ def identity(self):
+ return dict(kernel='6.8.0-100-generic', machine='x86_64',
+ os_release='ID=ubuntu\nVERSION_ID="24.04"\n', proc_version='Linux',
+ kernel_sha256='a'*64, kernel_config_sha256='b'*64)
+
+ def test_guest_syntax(self):
+ source = Path(vm.__file__).with_name('vm_guest.py')
+ compile(source.read_text(), str(source), 'exec')
+
+ def test_commands(self):
+ root = Path('/tmp/vm path')
+ command = vm.qemu_command(root/'run.qcow2', root/'seed.iso', root, 'kvm', False)
+ self.assertEqual(command[-2:], ['-nic', 'none'])
+ self.assertIn('q35,accel=kvm', command)
+ self.assertNotIn('-virtfs', command)
+ self.assertNotIn('-fsdev', command)
+ self.assertIn('virtserialport,chardev=report,name=tricu.report', command)
+ online = vm.qemu_command(root/'prepared.qcow2', root/'seed.iso', root, 'tcg', True)
+ self.assertIn('q35,accel=tcg', online)
+ self.assertEqual(online[-1], 'user,model=virtio-net-pci')
+ for path in ('/tmp/a,b', '/tmp/a\nb'):
+ with self.assertRaises(ValueError):
+ vm.safe_path(path)
+
+ def test_real_seed_iso(self):
+ with tempfile.TemporaryDirectory() as directory:
+ root = Path(directory)
+ (root/'kit.tar.gz').write_bytes(b'kit')
+ (root/'consumers.tar.gz').write_bytes(b'consumers')
+ request = dict(mode='prepare', kernel_prefix='6.8.')
+ iso = vm.seed(root, request, root)
+ files = subprocess.check_output(['isoinfo', '-R', '-i', str(iso), '-f'], text=True)
+ for name in ('user-data', 'meta-data', 'request.json', 'vm_guest.py', 'kit.tar.gz', 'consumers.tar.gz'):
+ self.assertIn('/'+name, files)
+ self.assertEqual(json.loads((root/'seed/request.json').read_text()), request)
+ userdata = (root/'seed/user-data').read_text()
+ self.assertTrue(userdata.startswith('#cloud-config\n'))
+ command = json.loads(userdata.split(' - ', 1)[1])
+ self.assertEqual(command[:2], ['/bin/sh', '-ec'])
+ self.assertIn('/mnt/tricu-seed/vm_guest.py', command[2])
+
+ def test_consumer_links_and_real_extraction(self):
+ with tempfile.TemporaryDirectory() as directory:
+ root = Path(directory)/'consumers'
+ for name in ('node/bin/node', 'node/lib/node_modules/npm/bin/npm-cli.js',
+ 'tricu-js/node_modules/koffi/package.json'):
+ path = root/name
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text('fixture')
+ (root/'node/bin/nodejs').symlink_to('/usr/local/bin/node')
+ (root/'node/bin/npm').symlink_to('../lib/node_modules/npm/bin/npm-cli.js')
+ for name in ('yarn', 'yarnpkg'):
+ (root/'node/bin'/name).symlink_to('/opt/yarn-v1.22.22/bin/'+name)
+ archive_path = Path(directory)/'consumers.tar.gz'
+ def pack():
+ with tarfile.open(archive_path, 'w:gz') as archive:
+ for name in ('node', 'tricu-js'):
+ archive.add(root/name, arcname=name)
+ pack()
+ with self.assertRaises(tarfile.AbsoluteLinkError):
+ vm.validate_consumer_archive(archive_path)
+ changes = vm.normalize_consumers(root)
+ self.assertEqual(len(changes), 3)
+ self.assertEqual(str((root/'node/bin/nodejs').readlink()), 'node')
+ self.assertFalse((root/'node/bin/yarn').is_symlink())
+ self.assertEqual((root/'node/bin/npm').read_text(), 'fixture')
+ pack()
+ vm.validate_consumer_archive(archive_path)
+ bad = root/'node/bin/escape'
+ bad.symlink_to('/etc/passwd')
+ with self.assertRaises(ValueError):
+ vm.normalize_consumers(root)
+ bad.unlink()
+ (root.parent/'outside').write_text('outside')
+ bad.symlink_to('../../../outside')
+ with self.assertRaises(ValueError):
+ vm.normalize_consumers(root)
+
+ def test_prepared_lock(self):
+ with tempfile.TemporaryDirectory() as directory:
+ root = Path(directory)
+ files = ('base.qcow2', 'prepared.qcow2', 'kit.tar.gz', 'consumers.tar.gz')
+ for name in files:
+ (root/name).write_bytes(name.encode())
+ hashes = {name: vm.digest(root/name) for name in files}
+ lock = dict(schema='tricu.vm-images.v1', files=hashes, identity=self.identity(),
+ request=dict(kernel_prefix='6.8.', kit_sha256=hashes['kit.tar.gz'],
+ consumers_sha256=hashes['consumers.tar.gz']))
+ path = root/'images.json'
+ vm.write_json(path, lock)
+ self.assertEqual(vm.load_prepared(path), lock)
+ (root/'base.qcow2').write_bytes(b'changed')
+ with self.assertRaises(ValueError):
+ vm.load_prepared(path)
+
+ def test_acceptance_controls(self):
+ identity = self.identity()
+ request = dict(mode='run', node_version='22.22.0')
+ expected = dict(result=42)
+ lock = dict(identity=identity, packages='packages', expected=expected)
+ binding = dict(passed=['python', 'js', 'php'], node='v22.22.0',
+ results={name: expected for name in ('python', 'js', 'php')})
+ runs = [dict(location=location, nix_store_absent=True, cli_record_verify_mutation=True,
+ bindings=copy.deepcopy(binding)) for location in ('first', 'relocated path with spaces')]
+ report = dict(status='passed', identity=identity, request=request, packages='packages',
+ workload=dict(kernel=identity['kernel'], expectation='pass', runs=runs))
+ vm.validate_result(report, lock, request)
+ mutations = [dict(status='failed'), dict(packages='changed'),
+ dict(identity=dict(identity, kernel='6.18.42')),
+ dict(workload=dict(report['workload'], runs=runs[:1]))]
+ for change in mutations:
+ with self.assertRaises(ValueError):
+ vm.validate_result(dict(report, **change), lock, request)
+ for mutate in ('node', 'missing-consumer', 'wrong-result', 'missing-isolation'):
+ altered = copy.deepcopy(report)
+ run = altered['workload']['runs'][0]
+ if mutate == 'node': run['bindings']['node'] = 'v20.19.0'
+ if mutate == 'missing-consumer': run['bindings']['passed'] = ['python']
+ if mutate == 'wrong-result': run['bindings']['results']['js'] = dict(result=0)
+ if mutate == 'missing-isolation': run['nix_store_absent'] = False
+ with self.assertRaises(ValueError):
+ vm.validate_result(altered, lock, request)
+ for change in (dict(kernel='6.18.42'), dict(machine='aarch64'), dict(os_release='ID=debian'),
+ dict(kernel_config_sha256='missing')):
+ with self.assertRaises(ValueError):
+ vm.validate_identity(dict(identity, **change), '6.8.')
+
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/ext/haskell/tests.c b/ext/haskell/tests.c
@@ -1,4 +1,4 @@
-#include "tricu-core.h"
+#include "tricu.h"
#include <assert.h>
#include <string.h>
#include <signal.h>
diff --git a/ext/haskell/verify_redistribution.py b/ext/haskell/verify_redistribution.py
@@ -0,0 +1,120 @@
+#!/usr/bin/env python3
+"""Verify complete source/notice manifests and their pairing with a runtime tree."""
+import argparse
+import hashlib
+import json
+import tarfile
+from pathlib import Path, PurePosixPath
+
+
+def digest(path):
+ with path.open('rb') as stream:
+ return hashlib.file_digest(stream, 'sha256').hexdigest()
+
+
+def relative(root, name):
+ path = PurePosixPath(name)
+ if path.is_absolute() or '..' in path.parts:
+ raise ValueError('unsafe manifest path')
+ result = root.joinpath(*path.parts)
+ if not result.resolve().is_relative_to(root.resolve()):
+ raise ValueError('manifest path escapes its root')
+ return result
+
+
+def verify(materials, runtime=None):
+ hashes = json.loads((materials/'SHA256.json').read_text())
+ links = json.loads((materials/'SYMLINKS.json').read_text())
+ paths = list(materials.rglob('*'))
+ actual = {str(p.relative_to(materials)) for p in paths if p.is_file() and not p.is_symlink()}
+ if actual != set(hashes) | {'SHA256.json', 'SYMLINKS.json'}:
+ raise ValueError('source file inventory differs')
+ actual_links = {str(p.relative_to(materials)): str(p.readlink()) for p in paths if p.is_symlink()}
+ if actual_links != links:
+ raise ValueError('source symlink inventory differs')
+ for name, expected in hashes.items():
+ path = relative(materials, name)
+ if path.is_symlink() or digest(path) != expected:
+ raise ValueError(f'changed source/notice file: {name}')
+ components = json.loads((materials/'components.json').read_text())
+ for component in components['components']:
+ directory = relative(materials/'sources', component['name'])
+ if not relative(directory, component['upstream_file']).exists():
+ raise ValueError('missing upstream source')
+ if not component['notices']:
+ raise ValueError('missing component notices')
+ for notice in component['notices']:
+ if digest(relative(materials, notice['file'])) != notice['sha256']:
+ raise ValueError('notice hash differs')
+ if runtime:
+ metadata = json.loads((runtime/'share/tricu/portable.json').read_text())
+ if metadata['source_manifest_sha256'] != digest(materials/'SHA256.json'):
+ raise ValueError('runtime/source pairing differs')
+ if metadata['source_symlinks_sha256'] != digest(materials/'SYMLINKS.json'):
+ raise ValueError('runtime/source symlink pairing differs')
+ for name, row in metadata['libraries'].items():
+ if digest(relative(runtime/'lib', name)) != row['sha256']:
+ raise ValueError('runtime library hash differs')
+ expected_notices = {str(path.relative_to(materials/'notices')): digest(path)
+ for path in (materials/'notices').rglob('*') if path.is_file()}
+ installed_notices = {str(path.relative_to(runtime/'share/tricu/notices')): digest(path)
+ for path in (runtime/'share/tricu/notices').rglob('*') if path.is_file()}
+ if expected_notices != installed_notices:
+ raise ValueError('runtime notice inventory differs')
+ for name in ('THIRD-PARTY-NOTICES.md', 'REDISTRIBUTION.md', 'license-components.json'):
+ if digest(materials/name) != digest(runtime/'share/tricu'/name):
+ raise ValueError('runtime license documentation differs')
+ return dict(components=len(components['components']), source_files=len(hashes),
+ symbolic_links=len(links), runtime_paired=runtime is not None)
+
+
+def verify_archive(archive, prefix, hashes, links):
+ observed, observed_links = {}, {}
+ with tarfile.open(archive, 'r|gz') as stream:
+ for member in stream:
+ path = PurePosixPath(member.name)
+ if path.is_absolute() or '..' in path.parts or not path.parts or path.parts[0] != prefix:
+ raise ValueError('unsafe release archive path')
+ name = str(PurePosixPath(*path.parts[1:]))
+ if member.isdir():
+ continue
+ if name in observed or name in observed_links:
+ raise ValueError('duplicate archive member')
+ if member.issym():
+ observed_links[name] = member.linkname
+ elif member.isfile():
+ with stream.extractfile(member) as contents:
+ observed[name] = hashlib.file_digest(contents, 'sha256').hexdigest()
+ else:
+ raise ValueError('unsupported release archive member')
+ if observed != hashes or observed_links != links:
+ raise ValueError('release archive contents differ from verified material')
+
+
+def main():
+ p = argparse.ArgumentParser(description=__doc__)
+ p.add_argument('--materials', type=Path, required=True)
+ p.add_argument('--runtime', type=Path)
+ p.add_argument('--archives', type=Path)
+ a = p.parse_args()
+ report = verify(a.materials, a.runtime)
+ if a.archives:
+ hashes = json.loads((a.materials/'SHA256.json').read_text())
+ hashes.update({name: digest(a.materials/name) for name in ('SHA256.json', 'SYMLINKS.json')})
+ verify_archive(a.archives/'tricu-source-materials.tar.gz', 'tricu-source-materials', hashes,
+ json.loads((a.materials/'SYMLINKS.json').read_text()))
+ runtime_hashes = {str(path.relative_to(a.runtime)): digest(path)
+ for path in a.runtime.rglob('*') if path.is_file() and not path.is_symlink()}
+ verify_archive(a.archives/'tricu-linux-x86_64.tar.gz', 'tricu', runtime_hashes, {})
+ report['archive_contents_verified'] = True
+ report['schema'] = 'tricu.redistribution-release.v1'
+ report['archives'] = {name: dict(sha256=digest(a.archives/name), bytes=(a.archives/name).stat().st_size)
+ for name in ('tricu-linux-x86_64.tar.gz', 'tricu-source-materials.tar.gz')}
+ report['source_manifest_sha256'] = digest(a.materials/'SHA256.json')
+ report['source_symlinks_sha256'] = digest(a.materials/'SYMLINKS.json')
+ (a.archives/'release.json').write_text(json.dumps(report, indent=2, sort_keys=True)+'\n')
+ print(json.dumps(report, sort_keys=True))
+
+
+if __name__ == '__main__':
+ main()
diff --git a/ext/haskell/vm_guest.py b/ext/haskell/vm_guest.py
@@ -0,0 +1,131 @@
+#!/usr/bin/env python3
+"""Guest-side preparation and offline acceptance for the Ubuntu GA VM."""
+import hashlib
+import json
+import os
+from pathlib import Path
+import platform
+import shutil
+import subprocess
+import tarfile
+import traceback
+import time
+
+
+def digest(path):
+ with path.open('rb') as stream:
+ return hashlib.file_digest(stream, 'sha256').hexdigest()
+
+
+def command(*args):
+ try:
+ return subprocess.check_output(args, text=True, stderr=subprocess.STDOUT, timeout=1800)
+ except subprocess.CalledProcessError as error:
+ raise RuntimeError(f'{args[0]} failed ({error.returncode}): {error.output[-6000:]}') from error
+
+
+def unpack(path, destination, expected):
+ if digest(path) != expected:
+ raise ValueError(f'input hash differs: {path.name}')
+ with tarfile.open(path, 'r:gz') as archive:
+ archive.extractall(destination, filter='data')
+
+
+def identity():
+ kernel = platform.release()
+ return dict(kernel=kernel, machine=platform.machine(),
+ os_release=Path('/etc/os-release').read_text(),
+ proc_version=Path('/proc/version').read_text(),
+ kernel_sha256=digest(Path('/boot')/('vmlinuz-'+kernel)),
+ kernel_config_sha256=digest(Path('/boot')/('config-'+kernel)))
+
+
+def main():
+ mount = Path('/mnt/tricu-input')
+ mount.mkdir(parents=True, exist_ok=True)
+ for _ in range(60):
+ if Path('/dev/disk/by-label/cidata').exists() and Path('/dev/virtio-ports/tricu.report').exists():
+ break
+ time.sleep(1)
+ command('mount', '-o', 'ro', '/dev/disk/by-label/cidata', str(mount))
+ request = json.loads((mount/'request.json').read_text())
+ observed = identity()
+ if observed['machine'] != 'x86_64' or not observed['kernel'].startswith(request['kernel_prefix']):
+ raise ValueError(f'unexpected guest kernel/architecture: {observed}')
+ release = dict(line.split('=', 1) for line in observed['os_release'].splitlines() if '=' in line)
+ if release.get('ID', '').strip('"') != 'ubuntu' or release.get('VERSION_ID', '').strip('"') != '24.04':
+ raise ValueError('Ubuntu 24.04 guest required')
+ if Path('/nix/store').exists():
+ raise ValueError('guest has a Nix store')
+ if request['mode'] == 'prepare':
+ os.environ['DEBIAN_FRONTEND'] = 'noninteractive'
+ for unit in ('apt-daily.timer', 'apt-daily-upgrade.timer', 'apt-daily.service',
+ 'apt-daily-upgrade.service', 'unattended-upgrades.service'):
+ subprocess.run(['systemctl', 'mask', '--now', unit], check=False)
+ command('apt-get', 'update')
+ command('apt-get', '-o', 'DPkg::Lock::Timeout=300', 'install', '-y', '--no-install-recommends',
+ 'python3', 'php-cli', 'php-common', 'php-ffi', 'libstdc++6', 'libatomic1', 'ca-certificates')
+ unpack(mount/'consumers.tar.gz', '/opt', request['consumers_sha256'])
+ if command('/opt/node/bin/node', '--version').strip() != 'v'+request['node_version']:
+ raise ValueError('Node version differs')
+ command('/usr/bin/php', '-d', 'ffi.enable=true', '-r', 'if (!extension_loaded("FFI")) { exit(1); }')
+ unpack(mount/'kit.tar.gz', '/kit', request['kit_sha256'])
+ Path('/tests').symlink_to('/kit/tests')
+ destination = Path('/opt/tricu-vm')
+ destination.mkdir()
+ shutil.copyfile(mount/'vm_guest.py', destination/'guest.py')
+ (destination/'prepared.json').write_text(json.dumps(request))
+ Path('/etc/systemd/system/tricu-accept.service').write_text('''[Unit]
+Description=tricu VM acceptance
+After=local-fs.target
+[Service]
+Type=oneshot
+TimeoutStartSec=1800
+ExecStart=/usr/bin/python3 /opt/tricu-vm/guest.py
+StandardOutput=journal+console
+StandardError=journal+console
+[Install]
+WantedBy=multi-user.target
+''')
+ # The prepared image starts the acceptance service on its next boot.
+ command('systemctl', 'enable', 'tricu-accept.service')
+ command('systemctl', 'mask', 'systemd-networkd-wait-online.service')
+ Path('/etc/cloud/cloud-init.disabled').touch()
+ return dict(status='prepared', identity=identity(), request=request,
+ packages=command('dpkg-query', '-W'),
+ node=command('/opt/node/bin/node', '--version').strip())
+ if request['mode'] != 'run':
+ raise ValueError('unknown guest operation')
+ prepared = json.loads(Path('/opt/tricu-vm/prepared.json').read_text())
+ if request['kit_sha256'] != prepared['kit_sha256'] or request['node_version'] != prepared['node_version']:
+ raise ValueError('prepared input identity differs')
+ work = Path('/work')
+ work.mkdir()
+ os.chown(work, 65534, 65534)
+ result = subprocess.run(['/usr/sbin/runuser', '-u', 'nobody', '--', '/usr/bin/env', '-i',
+ 'PATH=/usr/bin:/bin', 'HOME=/tmp', 'PYTHONDONTWRITEBYTECODE=1',
+ '/usr/bin/python3', '/kit/container_probe.py', '--expect', 'pass',
+ '--bindings', '--node-version', request['node_version']],
+ cwd=work, capture_output=True, text=True, timeout=1200)
+ print(result.stdout, result.stderr, flush=True)
+ if result.returncode:
+ raise RuntimeError(f'guest workload failed: {result.stdout}\n{result.stderr}')
+ markers = [line[len('TRICU_REPORT='):] for line in result.stdout.splitlines() if line.startswith('TRICU_REPORT=')]
+ if len(markers) != 1:
+ raise ValueError('missing/duplicate workload report')
+ return dict(status='passed', identity=observed, request=request, workload=json.loads(markers[0]),
+ packages=command('dpkg-query', '-W'))
+
+
+if __name__ == '__main__':
+ try:
+ report = main()
+ except BaseException:
+ report = dict(status='failed', traceback=traceback.format_exc())
+ print(report['traceback'], flush=True)
+ try:
+ with open('/dev/virtio-ports/tricu.report', 'w') as stream:
+ stream.write(json.dumps(report, sort_keys=True)+'\n')
+ finally:
+ os.sync()
+ subprocess.run(['systemctl', 'poweroff', '--no-block'], check=False)
diff --git a/ext/haskell/vm_matrix.py b/ext/haskell/vm_matrix.py
@@ -0,0 +1,290 @@
+#!/usr/bin/env python3
+"""Prepare a pinned Ubuntu 24.04 GA guest, then boot offline acceptance overlays."""
+import argparse
+import hashlib
+import json
+import os
+from pathlib import Path
+import platform
+import re
+import shutil
+import subprocess
+import tarfile
+import tempfile
+import uuid
+
+from container_matrix import inspect, load_lock, validate_bindings_kit
+
+
+def digest(path):
+ with path.open('rb') as stream:
+ return hashlib.file_digest(stream, 'sha256').hexdigest()
+
+
+def checked(*command, **kwargs):
+ try:
+ return subprocess.check_output(command, text=True, stderr=subprocess.STDOUT, timeout=300, **kwargs).strip()
+ except subprocess.CalledProcessError as error:
+ raise RuntimeError(f'{command[0]} failed ({error.returncode}): {error.output[-4000:]}') from error
+
+
+def write_json(path, value):
+ path.write_text(json.dumps(value, indent=2, sort_keys=True)+'\n')
+
+
+def safe_path(value):
+ path = Path(value).resolve()
+ # QEMU drive options use commas as field separators.
+ if ',' in str(path) or '\n' in str(path):
+ raise ValueError('VM paths must be free of commas and newlines')
+ return path
+
+
+def seed(directory, request, payload=None):
+ source = directory/'seed'
+ source.mkdir()
+ write_json(source/'request.json', request)
+ if request['mode'] == 'prepare':
+ shutil.copyfile(Path(__file__).with_name('vm_guest.py'), source/'vm_guest.py')
+ shutil.copyfile(payload/'kit.tar.gz', source/'kit.tar.gz')
+ shutil.copyfile(payload/'consumers.tar.gz', source/'consumers.tar.gz')
+ script = ('mkdir -p /mnt/tricu-seed; '
+ 'mount -o ro /dev/disk/by-label/cidata /mnt/tricu-seed; '
+ 'exec /usr/bin/python3 /mnt/tricu-seed/vm_guest.py')
+ (source/'user-data').write_text('#cloud-config\nssh_pwauth: false\n'
+ 'runcmd:\n - '+json.dumps(['/bin/sh', '-ec', script])+'\n')
+ else:
+ (source/'user-data').write_text('#cloud-config\n')
+ (source/'meta-data').write_text('instance-id: tricu-'+uuid.uuid4().hex+'\nlocal-hostname: tricu-vm\n')
+ target = directory/'seed.iso'
+ checked('genisoimage', '-quiet', '-output', str(target), '-volid', 'cidata', '-joliet', '-rock', str(source))
+ return target
+
+
+def qemu_command(disk, iso, directory, accel, network):
+ return ['qemu-system-x86_64', '-machine', 'q35,accel='+accel,
+ '-cpu', 'host' if accel == 'kvm' else 'max', '-m', '4096', '-smp', '2',
+ '-display', 'none', '-monitor', 'none', '-no-reboot',
+ '-serial', 'file:'+str(directory/'serial.log'),
+ '-drive', f'file={disk},if=virtio,format=qcow2',
+ '-drive', f'file={iso},media=cdrom,readonly=on,format=raw',
+ '-device', 'virtio-serial-pci', '-chardev', 'file,id=report,path='+str(directory/'guest-report.json'),
+ '-device', 'virtserialport,chardev=report,name=tricu.report',
+ '-nic', 'user,model=virtio-net-pci' if network else 'none']
+
+
+def boot(disk, iso, directory, accel, network, timeout):
+ command = qemu_command(disk, iso, directory, accel, network)
+ write_json(directory/'qemu-command.json', command)
+ try:
+ with (directory/'qemu.log').open('w') as log:
+ subprocess.run(command, stdout=log, stderr=subprocess.STDOUT, check=True, timeout=timeout)
+ except (subprocess.CalledProcessError, subprocess.TimeoutExpired) as error:
+ raise RuntimeError(f'VM execution failed; inspect {directory / "serial.log"} and {directory / "qemu.log"}\n'
+ +(directory/'qemu.log').read_text()[-2000:]) from error
+ path = directory/'guest-report.json'
+ if not path.exists() or not path.stat().st_size:
+ raise ValueError(f'guest produced no report; inspect {directory / "serial.log"}')
+ report = json.loads(path.read_text())
+ if report.get('status') == 'failed':
+ raise RuntimeError(report.get('traceback', 'guest failed'))
+ return report
+
+
+def validate_identity(identity, prefix):
+ if identity.get('machine') != 'x86_64' or not identity.get('kernel', '').startswith(prefix):
+ raise ValueError('unexpected guest kernel or architecture')
+ fields = dict(line.split('=', 1) for line in identity.get('os_release', '').splitlines() if '=' in line)
+ if fields.get('ID', '').strip('"') != 'ubuntu' or fields.get('VERSION_ID', '').strip('"') != '24.04':
+ raise ValueError('unexpected guest userspace')
+ for key in ('kernel_sha256', 'kernel_config_sha256'):
+ if not re.fullmatch(r'[0-9a-f]{64}', identity.get(key, '')):
+ raise ValueError('missing kernel evidence')
+
+
+def normalize_consumers(root):
+ """Relocate exported Node aliases and retain self-contained consumer links."""
+ root = root.resolve()
+ adjustments = []
+ for path in sorted(root.rglob('*')):
+ if not path.is_symlink():
+ continue
+ target = path.readlink()
+ name = path.relative_to(root).as_posix()
+ # The official Node image's Yarn installation lives outside /usr/local.
+ # This payload selects Node/npm and Koffi for the CLI acceptance workload.
+ if name in ('node/bin/yarn', 'node/bin/yarnpkg') and re.fullmatch(r'/opt/yarn-v\d+\.\d+\.\d+/bin/yarn(?:pkg)?', str(target)):
+ path.unlink()
+ adjustments.append(dict(path=name, original=str(target), action='omit unused Yarn alias'))
+ continue
+ if target.is_absolute():
+ prefixes = {'/usr/local': root/'node', '/opt/node': root/'node', '/opt/tricu-js': root/'tricu-js'}
+ mapped = next((destination/target.relative_to(prefix) for prefix, destination in prefixes.items()
+ if target.is_relative_to(prefix)), None)
+ if mapped is None:
+ raise ValueError(f'consumer link points outside exported roots: {name}: {target}')
+ relative = os.path.relpath(mapped, path.parent)
+ path.unlink()
+ path.symlink_to(relative)
+ adjustments.append(dict(path=name, original=str(target), action='relativize', target=relative))
+ for path in root.rglob('*'):
+ if path.is_symlink() and not path.resolve(strict=True).is_relative_to(root):
+ raise ValueError(f'consumer link escapes payload: {path.relative_to(root)}')
+ return adjustments
+
+
+def validate_consumer_archive(path):
+ # Exercise the same data filter as guest extraction before any VM boot.
+ with tempfile.TemporaryDirectory(prefix='tricu-consumer-extraction-') as directory:
+ with tarfile.open(path, 'r:gz') as archive:
+ archive.extractall(directory, filter='data')
+ root = Path(directory)
+ for name in ('node/bin/node', 'node/lib/node_modules/npm/bin/npm-cli.js',
+ 'tricu-js/node_modules/koffi/package.json'):
+ if not (root/name).is_file():
+ raise ValueError(f'consumer archive missing {name}')
+
+
+def prepare(args):
+ directory = safe_path(args.directory)
+ image = safe_path(args.image)
+ kit = Path(__file__).resolve().parent
+ validate_bindings_kit(kit)
+ if not re.fullmatch(r'[0-9a-f]{64}', args.image_sha256) or digest(image) != args.image_sha256:
+ raise ValueError('cloud image differs from supplied SHA256')
+ info = json.loads(checked('qemu-img', 'info', '--output=json', str(image)))
+ if info.get('format') != 'qcow2' or info.get('backing-filename'):
+ raise ValueError('select a self-contained qcow2 cloud image')
+ lock = load_lock(args.container_lock)
+ if lock.get('profile') != 'bindings':
+ raise ValueError('select the accepted bindings-profile container lock')
+ row = next(row for row in lock['images'] if row['name'] == 'ubuntu24')
+ if inspect(args.engine, row['image_id'])['Id'] != row['image_id']:
+ raise ValueError('consumer-source image identity differs')
+ for filename, key in [('package.json', 'npm_package_sha256'), ('package-lock.json', 'npm_lock_sha256')]:
+ if digest(kit/'bindings/ext/js'/filename) != lock[key]:
+ raise ValueError('consumer-source npm inputs differ from kit')
+ directory.mkdir(parents=True, exist_ok=False)
+ consumers = directory/'consumers'
+ consumers.mkdir()
+ name = 'tricu-vm-export-'+uuid.uuid4().hex
+ checked(args.engine, 'create', '--pull', 'never', '--name', name, row['image_id'])
+ try:
+ for component in ('node', 'tricu-js'):
+ checked(args.engine, 'cp', name+':/opt/'+component, str(consumers/component))
+ finally:
+ checked(args.engine, 'rm', name)
+ if not (consumers/'node/bin/node').is_file() or not (consumers/'tricu-js/node_modules/koffi/package.json').is_file():
+ raise ValueError('consumer export has unexpected layout')
+ adjustments = normalize_consumers(consumers)
+ with tarfile.open(directory/'consumers.tar.gz', 'w:gz') as archive:
+ for component in ('node', 'tricu-js'):
+ archive.add(consumers/component, arcname=component)
+ validate_consumer_archive(directory/'consumers.tar.gz')
+ shutil.rmtree(consumers) # Only this newly created intermediate export.
+ with tarfile.open(directory/'kit.tar.gz', 'w:gz') as archive:
+ for path in sorted(kit.iterdir()):
+ archive.add(path, arcname=path.name)
+ shutil.copyfile(image, directory/'base.qcow2')
+ checked('qemu-img', 'create', '-f', 'qcow2', '-F', 'qcow2', '-b', 'base.qcow2', 'prepared.qcow2', cwd=directory)
+ checked('qemu-img', 'resize', 'prepared.qcow2', '16G', cwd=directory)
+ request = dict(mode='prepare', kernel_prefix=args.kernel_prefix, node_version=lock['node']['version'],
+ kit_sha256=digest(directory/'kit.tar.gz'), consumers_sha256=digest(directory/'consumers.tar.gz'))
+ iso = seed(directory, request, directory)
+ report = boot(directory/'prepared.qcow2', iso, directory, args.accel, True, args.timeout)
+ if report.get('status') != 'prepared' or report.get('request') != request:
+ raise ValueError('incomplete preparation report')
+ validate_identity(report['identity'], args.kernel_prefix)
+ manifest = dict(schema='tricu.vm-images.v1', request=request, identity=report['identity'],
+ packages=report['packages'], node=lock['node'], consumer_image=row,
+ cloud_image=dict(filename=image.name, sha256=args.image_sha256),
+ container_lock_sha256=digest(args.container_lock), consumer_adjustments=adjustments,
+ expected=json.loads((kit/'bindings/ext/haskell/bindings/fixtures.json').read_text())['expected'],
+ archive_sha256=digest(kit/'tricu-linux-x86_64.tar.gz'),
+ files={name: digest(directory/name) for name in ('base.qcow2', 'prepared.qcow2', 'kit.tar.gz', 'consumers.tar.gz')})
+ write_json(directory/'images.json', manifest)
+ print(directory/'images.json')
+
+
+def load_prepared(path):
+ lock = json.loads(path.read_text())
+ required = {'base.qcow2', 'prepared.qcow2', 'kit.tar.gz', 'consumers.tar.gz'}
+ if lock.get('schema') != 'tricu.vm-images.v1' or set(lock.get('files', {})) != required:
+ raise ValueError('invalid VM image lock')
+ for name, checksum in lock['files'].items():
+ if digest(path.parent/name) != checksum:
+ raise ValueError(f'changed prepared VM input: {name}')
+ if lock['request']['kit_sha256'] != lock['files']['kit.tar.gz'] or lock['request']['consumers_sha256'] != lock['files']['consumers.tar.gz']:
+ raise ValueError('prepared payload identity differs')
+ validate_identity(lock['identity'], lock['request']['kernel_prefix'])
+ return lock
+
+
+def validate_result(report, lock, request):
+ if report.get('status') != 'passed' or report.get('request') != request:
+ raise ValueError('incomplete VM acceptance')
+ if report.get('identity') != lock['identity'] or report.get('packages') != lock['packages']:
+ raise ValueError('guest kernel/userspace differs from prepared image')
+ workload = report['workload']
+ if workload.get('kernel') != lock['identity']['kernel'] or workload.get('expectation') != 'pass':
+ raise ValueError('workload kernel/expectation differs')
+ if [run.get('location') for run in workload['runs']] != ['first', 'relocated path with spaces']:
+ raise ValueError('incomplete relocation acceptance')
+ for run in workload['runs']:
+ binding = run['bindings']
+ if not run.get('nix_store_absent') or not run.get('cli_record_verify_mutation'):
+ raise ValueError('missing Nix-free workload controls')
+ if binding.get('passed') != ['python', 'js', 'php'] or binding.get('node') != 'v'+request['node_version']:
+ raise ValueError('incomplete consumer acceptance')
+ if set(binding['results']) != {'python', 'js', 'php'} or any(value != lock['expected'] for value in binding['results'].values()):
+ raise ValueError('consumer results differ from independent expectations')
+
+
+def run(args):
+ path = safe_path(args.lock)
+ lock = load_prepared(path)
+ directory = safe_path(args.output)
+ directory.mkdir(parents=True, exist_ok=False)
+ disk = directory/'run.qcow2'
+ checked('qemu-img', 'create', '-f', 'qcow2', '-F', 'qcow2', '-b', str(path.parent/'prepared.qcow2'), str(disk))
+ request = dict(lock['request'], mode='run')
+ iso = seed(directory, request)
+ report = boot(disk, iso, directory, args.accel, False, args.timeout)
+ validate_result(report, lock, request)
+ # The immutable backing inputs must survive the run unchanged.
+ load_prepared(path)
+ write_json(directory/'report.json', dict(schema='tricu.vm-acceptance.v1', guest=report,
+ image_lock_sha256=digest(path), prepared_files=lock['files'], archive_sha256=lock['archive_sha256'],
+ node=lock['node'], consumer_image=lock['consumer_image'], accelerator=args.accel,
+ host_kernel=platform.release(), qemu_version=checked('qemu-system-x86_64', '--version'),
+ seed_sha256=digest(iso), network='disabled', scope='Ubuntu 24.04 guest kernel; Python, Node and PHP CLI'))
+ print(directory/'report.json')
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ sub = parser.add_subparsers(dest='action', required=True)
+ p = sub.add_parser('prepare')
+ p.add_argument('--image', type=Path, required=True)
+ p.add_argument('--image-sha256', required=True)
+ p.add_argument('--container-lock', type=Path, required=True)
+ p.add_argument('--directory', type=Path, required=True)
+ p.add_argument('--engine', choices=['podman', 'docker'], default='podman')
+ p.add_argument('--kernel-prefix', default='6.8.')
+ r = sub.add_parser('run')
+ r.add_argument('--lock', type=Path, required=True)
+ r.add_argument('--output', type=Path, required=True)
+ for command in (p, r):
+ command.add_argument('--accel', choices=['kvm', 'tcg'], default='kvm')
+ command.add_argument('--timeout', type=int, default=2400)
+ args = parser.parse_args()
+ if args.accel == 'kvm' and not os.access('/dev/kvm', os.R_OK | os.W_OK):
+ raise ValueError('KVM access required; grant device access or select --accel tcg')
+ if args.action == 'prepare':
+ prepare(args)
+ else:
+ run(args)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/ext/php/README.md b/ext/php/README.md
@@ -53,14 +53,13 @@ or supervised processes with resource limits.
The Linux library is pinned in memory across `dlclose`, matching GHC's process
lifetime even when PHP releases a request-scoped FFI object. Checks cover CLI
-execution and repeated uploads/errors in one PHP built-in server process. Other
-PHP server configurations, including FPM/Apache process lifecycles, require their
-own acceptance; execute-after-fork reuse of initialized GHC is outside the contract.
+execution and repeated uploads/errors in one PHP built-in server process.
+Initialize GHC in the process that executes the bundle; after a fork, use a fresh
+executable process for embedding.
The `public/` example runs pure uploaded bundles with an explicit export and result
-format. It supplies no request-controlled native IO execution. Arbitrary reduction
-can exhaust resources or fail to terminate: this example requires external
-supervision and is not an untrusted-computation sandbox.
+format. Requests select pure bundle execution. Run this example under external
+process supervision with execution and memory limits for uploaded computations.
The CLI requires `run BUNDLE EXPORT`, accepts text arguments, and supports `--io`
and `--format string|number|bool|bytes` (default string; bytes render as hex).
diff --git a/ext/php/src/ffi.php b/ext/php/src/ffi.php
@@ -41,10 +41,8 @@ final class Library
if (self::$native === null) {
$header = false;
foreach ([__DIR__.'/..', __DIR__.'/../../haskell/include'] as $directory) {
- if (is_file($directory.'/tricu.h') && is_file($directory.'/tricu-core.h')) {
- $core = file_get_contents($directory.'/tricu-core.h');
- $host = file_get_contents($directory.'/tricu.h');
- if ($core !== false && $host !== false) $header = $core."\n".$host;
+ if (is_file($directory.'/tricu.h')) {
+ $header = file_get_contents($directory.'/tricu.h');
break;
}
}
@@ -99,9 +97,8 @@ final class Context
}
private function out(string $name, mixed ...$arguments): \FFI\CData
{
- // FFI scalar calls take PHP scalar values, not CData boxes. For uint64,
- // PHP exposes the same 64 bits as a signed int; FFI converts those bits
- // back to uint64 without a floating-point intermediary.
+ // FFI scalar calls take PHP scalar values. For uint64, PHP exposes the
+ // same 64 bits as a signed int; FFI converts those bits back to uint64.
$arguments = array_map(fn($arg) => $arg instanceof \FFI\CData &&
\FFI::typeof($arg)->getKind() === \FFI\CType::TYPE_UINT64 ? $arg->cdata : $arg, $arguments);
$out = $this->api->new('uint64_t');
@@ -139,8 +136,8 @@ final class Context
(strlen($text) === 20 && strcmp($text, '18446744073709551615') > 0)) {
throw new \RangeException('number must fit uint64');
}
- // PHP integers are signed. Marshal full-range uint64 through bytes,
- // never through a floating-point or overflowing PHP integer conversion.
+ // PHP integers are signed. Byte-wise arithmetic preserves the full
+ // uint64 range when marshaling decimal input.
$bytes = array_fill(0, 8, 0);
foreach (str_split($text) as $digit) {
$carry = (int)$digit;
diff --git a/flake.nix b/flake.nix
@@ -16,7 +16,7 @@
packageName = "tricu";
containerPackageName = "${packageName}-container";
- # Archived experiments must not enter active build/test source hashes.
+ # Keep active build/test source hashes scoped to current sources.
tricuSource = pkgs.lib.cleanSourceWith {
name = "tricu-source";
src = self;
@@ -63,6 +63,15 @@
overrides = nativeHaskellOverrides;
};
picHaskellPackages = haskellPackages.override {
+ # Build tools and dependencies must use the same boot-library IDs.
+ buildHaskellPackages = picHaskellPackages;
+ overrides = pkgs.lib.composeExtensions nativeHaskellOverrides (final: previous: {
+ mkDerivation = args: previous.mkDerivation (args // {
+ configureFlags = (args.configureFlags or []) ++ [
+ "--ghc-option=-fPIC" "--ghc-option=-optc-fPIC"
+ ];
+ });
+ });
ghc = haskellPackages.ghc.override {
enableRelocatedStaticLibs = true;
# These build-only variants are unused by the embedding experiment.
@@ -163,11 +172,11 @@
cp dist/build/tricu-bench/tricu-bench $out/bin/
'';
})).overrideAttrs (_: {
- # No implicit timing run without a caller-selected worker bundle.
+ # Require a caller-selected worker bundle for the timing run.
benchmarkPhase = "true";
});
- # Opt-in application build, never part of the fast gate. Retain the exact
+ # Opt-in application build with its own acceptance gate. Retain the exact
# sources, compiler metadata and receipts beside the worker bundle.
benchmarkWorkers = pkgs.runCommand "tricu-benchmark-workers" {
nativeBuildInputs = [ tricuPackage ];
@@ -194,18 +203,33 @@
inherit pkgs haskellPackages;
source = tricuSource;
};
- tricuCore = import ./ext/haskell/build.nix {
- inherit pkgs haskellPackages;
- source = tricuSource;
- coreOnly = true;
- };
- tricuCoreArchive = import ./ext/haskell/build.nix {
+ tricuHaskellArchive = import ./ext/haskell/build.nix {
inherit pkgs;
source = tricuSource;
haskellPackages = picHaskellPackages;
- coreOnly = true;
archiveHaskell = true;
};
+ embeddingIntegration = libraryPackage: pkgs.runCommand "tricu-haskell-ffi-integration" {
+ nativeBuildInputs = [ pkgs.python3 tricuPackage ];
+ } ''
+ export PYTHONDONTWRITEBYTECODE=1
+ export PYTHONPATH=${tricuSource}/ext/haskell/python
+ mkdir -p $out workspace
+ cp ${tricuSource}/lib/base.tri workspace/base.tri
+ cp ${tricuSource}/lib/math/native.tri workspace/native_math.tri
+ cp ${tricuSource}/lib/io.tri workspace/io.tri
+ cp ${tricuSource}/lib/handlers.tri workspace/handlers.tri
+ cp ${tricuSource}/demos/io/copy.tri workspace/copy.tri
+ tricu arboricx compile --file workspace/copy.tri --store copy-store \
+ --output $out/copy.arboricx --compile-timeout 120 +RTS -N1 -M1024m -RTS
+ library=${libraryPackage}/lib/libtricu-haskell${pkgs.stdenv.hostPlatform.extensions.sharedLibrary}
+ python3 ${tricuSource}/ext/haskell/test_embedding.py "$library" \
+ ${tricuSource}/test/fixtures --copy-bundle $out/copy.arboricx > $out/integration.json
+ python3 ${tricuSource}/ext/haskell/compiler_probe.py "$library" \
+ ${tricuSource}/bootstrap/build-stable.arboricx --tricu ${tricuPackage}/bin/tricu \
+ --directory compiler-probe > $out/compiler.log
+ cp compiler-probe/report.json $out/compiler.json
+ '';
tricuPython = pkgs.python3Packages.buildPythonPackage {
pname = "tricu-runtime";
version = "2026.10.24";
@@ -346,7 +370,7 @@
} ''
export PYTHONDONTWRITEBYTECODE=1
mkdir -p $out
- # Exercise installed-module discovery with no TRICU_LIB override.
+ # Exercise installed-module discovery through its default library lookup.
python3 -c 'from tricu import Library; c = Library().context(); c.close()'
timeout 90s python3 ${tricuSource}/ext/haskell/bindings/python.py \
${tricuHaskell}/lib/libtricu-haskell${pkgs.stdenv.hostPlatform.extensions.sharedLibrary} \
@@ -366,6 +390,198 @@
packages.${packageName} = tricuPackage;
packages.default = tricuPackage;
packages.tricu-haskell-ffi = tricuHaskell;
+ packages.tricu-haskell-ffi-archive = tricuHaskellArchive;
+ packages.tricu-redistribution = import ./ext/haskell/redistribution.nix {
+ inherit pkgs;
+ host = tricuHaskellArchive;
+ source = self;
+ nixpkgsSource = nixpkgs;
+ flakeUtilsSource = flake-utils;
+ systemsSource = flake-utils.inputs.systems;
+ };
+ packages.tricu-portable = pkgs.runCommand "tricu-portable-linux-x86_64" {
+ nativeBuildInputs = [ pkgs.python3 pkgs.patchelf pkgs.binutils pkgs.glibc.bin ];
+ meta.platforms = [ "x86_64-linux" ];
+ } ''
+ mkdir -p $out
+ python3 ${./ext/haskell/portable.py} --host ${tricuHaskellArchive} \
+ --glibc ${pkgs.glibc} --output $out/tricu \
+ --materials ${self.packages.${system}.tricu-redistribution}
+ cp ${./LICENSE} $out/tricu/LICENSE
+ cp ${./ext/haskell/portable-README.md} $out/tricu/README.md
+ tar --hard-dereference --sort=name --mtime=@1 --owner=0 --group=0 --numeric-owner \
+ -C $out -czf $out/tricu-linux-x86_64.tar.gz tricu
+ tar --hard-dereference --sort=name --mtime=@1 --owner=0 --group=0 --numeric-owner \
+ --transform='flags=r;s,^\.,tricu-source-materials,' \
+ -C ${self.packages.${system}.tricu-redistribution} -cf - . \
+ | gzip -n -6 > $out/tricu-source-materials.tar.gz
+ python3 ${./ext/haskell/verify_redistribution.py} \
+ --materials ${self.packages.${system}.tricu-redistribution} \
+ --runtime $out/tricu --archives $out
+ '';
+ packages.tricu-portable-test-system = pkgs.runCommand "tricu-portable-test-system" {
+ nativeBuildInputs = [ pkgs.python3 pkgs.patchelf pkgs.glibc.bin ];
+ meta.platforms = [ "x86_64-linux" ];
+ } ''
+ mkdir tools
+ cp ${./ext/haskell/portable.py} tools/portable.py
+ cp ${./ext/haskell/portable_fixture.py} tools/fixture.py
+ python3 tools/fixture.py --python ${pkgs.python3}/bin/python3 \
+ --stdlib ${pkgs.python3}/lib/python${pkgs.python3.pythonVersion} \
+ --loader ${pkgs.glibc}/lib/ld-linux-x86-64.so.2 --output $out
+ '';
+ packages.tricu-portable-test = pkgs.runCommand "tricu-portable-test" {
+ nativeBuildInputs = [ pkgs.python3 pkgs.bubblewrap ];
+ meta.platforms = [ "x86_64-linux" ];
+ } ''
+ python3 ${./ext/haskell/test_portable.py} \
+ --release ${self.packages.${system}.tricu-portable}/tricu-linux-x86_64.tar.gz \
+ --system ${self.packages.${system}.tricu-portable-test-system} \
+ --source ${tricuSource} --worker ${tricuFilecheckBundle} \
+ --copier ${self.packages.${system}.tricu-haskell-ffi-archive-integration}/copy.arboricx \
+ --output $out
+ '';
+ packages.tricu-distro-test-kit = pkgs.runCommand "tricu-distro-test-kit" {
+ nativeBuildInputs = [ pkgs.python3 pkgs.nodejs phpWithFfi ];
+ } ''
+ mkdir -p $out/tests
+ cp ${self.packages.${system}.tricu-portable}/tricu-linux-x86_64.tar.gz $out/
+ cp ${./ext/haskell/container_matrix.py} $out/container_matrix.py
+ cp ${./ext/haskell/container_probe.py} $out/container_probe.py
+ cp ${./ext/haskell/container_bindings.py} $out/container_bindings.py
+ mkdir -p $out/bindings/ext/{js,php,haskell/bindings}
+ cp -r ${./ext/js/src} $out/bindings/ext/js/src
+ cp -r ${./ext/js/test} $out/bindings/ext/js/test
+ cp ${./ext/js/package.json} $out/bindings/ext/js/package.json
+ cp ${./ext/js/package-lock.json} $out/bindings/ext/js/package-lock.json
+ cp -r ${./ext/php/src} $out/bindings/ext/php/src
+ cp -r ${./ext/php/tests} $out/bindings/ext/php/tests
+ cp ${./ext/haskell/test_embedding.py} $out/bindings/ext/haskell/test_embedding.py
+ cp ${./ext/haskell/bindings/python.py} $out/bindings/ext/haskell/bindings/python.py
+ cp ${./ext/haskell/bindings/fixtures.json} $out/bindings/ext/haskell/bindings/fixtures.json
+ cp ${./ext/haskell/portable_workload.py} $out/tests/portable_workload.py
+ cp ${./ext/haskell/test_embedding.py} $out/tests/test_embedding.py
+ cp ${./ext/haskell/test_scopes.py} $out/tests/test_scopes.py
+ cp ${./test/python_filecheck.py} $out/tests/python_filecheck.py
+ cp ${./test/process_rss.py} $out/tests/process_rss.py
+ mkdir $out/tests/fixtures
+ for fixture in id append true false notQ; do
+ cp ${tricuSource}/test/fixtures/$fixture.arboricx $out/tests/fixtures/
+ done
+ cp -r ${tricuSource}/demos/python-filecheck/tricu_filecheck $out/tests/tricu_filecheck
+ chmod -R u+w $out/tests/tricu_filecheck
+ cp ${tricuFilecheckBundle}/worker.arboricx $out/tests/tricu_filecheck/worker.arboricx
+ printf 'BUNDLE_SHA256 = "%s"\n' "$(cat ${tricuFilecheckBundle}/worker.sha256)" \
+ > $out/tests/tricu_filecheck/_bundle_pin.py
+ cp ${self.packages.${system}.tricu-haskell-ffi-archive-integration}/copy.arboricx $out/tests/
+ PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=$out python3 -c \
+ 'import sys; from pathlib import Path; from container_matrix import validate_bindings_kit; validate_bindings_kit(Path(sys.argv[1]))' "$out"
+ # Exercise the actual staged consumers and their helper imports. Nix
+ # supplies these smoke runtimes; distro acceptance remains external.
+ cp -r $out/bindings source
+ chmod -R u+w source
+ mkdir -p source/test
+ cp -r $out/tests/fixtures source/test/fixtures
+ ln -s ${tricuJs}/lib/node_modules source/ext/js/node_modules
+ cp ${self.packages.${system}.tricu-portable}/tricu/include/tricu.h source/ext/php/tricu.h
+ PYTHONDONTWRITEBYTECODE=1 \
+ PYTHONPATH=${self.packages.${system}.tricu-portable}/tricu/share/tricu \
+ python3 ${./ext/haskell/bindings/check.py} --source "$PWD/source" \
+ --library ${self.packages.${system}.tricu-portable}/tricu/lib/libtricu-haskell.so \
+ --python ${pkgs.python3}/bin/python3 --node ${pkgs.nodejs}/bin/node \
+ --php ${phpWithFfi}/bin/php --copier $out/tests/copy.arboricx \
+ --report $out/bindings-smoke.json
+ '';
+ packages.tricu-distro-test = pkgs.writeShellApplication {
+ name = "tricu-distro-test";
+ runtimeInputs = [ pkgs.python3 ];
+ text = ''
+ exec python3 ${self.packages.${system}.tricu-distro-test-kit}/container_matrix.py "$@"
+ '';
+ };
+ packages.tricu-vm-test-kit = pkgs.runCommand "tricu-vm-test-kit" {} ''
+ mkdir -p $out
+ cp -r ${self.packages.${system}.tricu-distro-test-kit}/. $out/
+ chmod u+w $out
+ cp ${./ext/haskell/vm_matrix.py} $out/vm_matrix.py
+ cp ${./ext/haskell/vm_guest.py} $out/vm_guest.py
+ '';
+ packages.tricu-vm-test = pkgs.writeShellApplication {
+ name = "tricu-vm-test";
+ runtimeInputs = [ pkgs.python3 pkgs.qemu pkgs.cdrkit ];
+ text = ''
+ exec python3 ${self.packages.${system}.tricu-vm-test-kit}/vm_matrix.py "$@"
+ '';
+ };
+ checks.portableVMHarness = pkgs.runCommand "tricu-portable-vm-harness" {
+ nativeBuildInputs = [ pkgs.python3 pkgs.cdrkit ];
+ } ''
+ mkdir tools
+ cp ${./ext/haskell/vm_matrix.py} tools/vm_matrix.py
+ cp ${./ext/haskell/vm_guest.py} tools/vm_guest.py
+ cp ${./ext/haskell/container_matrix.py} tools/container_matrix.py
+ cp ${./ext/haskell/test_vm_harness.py} tools/test_vm_harness.py
+ PYTHONDONTWRITEBYTECODE=1 python3 tools/test_vm_harness.py
+ mkdir -p $out
+ '';
+ checks.redistribution = pkgs.runCommand "tricu-redistribution-controls" {
+ nativeBuildInputs = [ pkgs.python3 ];
+ } ''
+ mkdir tools
+ cp ${./ext/haskell/redistribution.py} tools/redistribution.py
+ cp ${./ext/haskell/portable.py} tools/portable.py
+ cp ${./ext/haskell/verify_redistribution.py} tools/verify_redistribution.py
+ cp ${./ext/haskell/test_redistribution.py} tools/test_redistribution.py
+ PYTHONDONTWRITEBYTECODE=1 python3 tools/test_redistribution.py
+ mkdir -p $out
+ '';
+ packages.tricu-portable-replacement-test = pkgs.runCommand "tricu-portable-replacement-test" {
+ nativeBuildInputs = [ pkgs.python3 pkgs.bubblewrap pkgs.binutils ];
+ meta.platforms = [ "x86_64-linux" ];
+ } ''
+ cp -r ${self.packages.${system}.tricu-portable}/tricu .
+ chmod -R u+w tricu
+ printf 'User-modified ABI-compatible ELF metadata\n' > modification.txt
+ for library in libgmp.so.10 libelf.so.1 libdw.so.1 libnuma.so.1; do
+ objcopy --add-section .tricu-user-modification=modification.txt tricu/lib/$library
+ done
+ python3 - <<'PY'
+ import hashlib, json
+ from pathlib import Path
+ manifest = Path('tricu/share/tricu/portable.json')
+ data = json.loads(manifest.read_text())
+ replacements = {}
+ for name in ('libgmp.so.10', 'libelf.so.1', 'libdw.so.1', 'libnuma.so.1'):
+ library = Path('tricu/lib')/name
+ checksum = hashlib.sha256(library.read_bytes()).hexdigest()
+ assert checksum != data['libraries'][name]['sha256']
+ replacements[name] = dict(original_sha256=data['libraries'][name]['sha256'], replacement_sha256=checksum)
+ data['libraries'][name].update(sha256=checksum, bytes=library.stat().st_size)
+ manifest.write_text(json.dumps(data))
+ Path('replacement.json').write_text(json.dumps(dict(
+ method='Added a non-allocated ELF metadata section; ABI preserved; not a source rebuild',
+ libraries=replacements), indent=2))
+ PY
+ tar -czf modified.tar.gz tricu
+ python3 ${./ext/haskell/test_portable.py} \
+ --release "$PWD/modified.tar.gz" \
+ --system ${self.packages.${system}.tricu-portable-test-system} \
+ --source ${tricuSource} --worker ${tricuFilecheckBundle} \
+ --copier ${self.packages.${system}.tricu-haskell-ffi-archive-integration}/copy.arboricx \
+ --output $out
+ cp replacement.json $out/
+ '';
+ checks.portableContainerHarness = pkgs.runCommand "tricu-portable-container-harness" {
+ nativeBuildInputs = [ pkgs.python3 ];
+ } ''
+ mkdir tools
+ cp ${./ext/haskell/container_matrix.py} tools/container_matrix.py
+ cp ${./ext/haskell/container_probe.py} tools/container_probe.py
+ cp ${./ext/haskell/test_container_harness.py} tools/test_container_harness.py
+ cp ${./ext/haskell/container_bindings.py} tools/container_bindings.py
+ PYTHONDONTWRITEBYTECODE=1 python3 tools/test_container_harness.py
+ mkdir -p $out
+ '';
packages.tricu-generation-ffi-test = pkgs.runCommand "tricu-generation-ffi-test" {
nativeBuildInputs = [ pkgs.python3 ];
} ''
@@ -385,27 +601,21 @@
--directory $out/scoped
'';
- packages.tricu-core = tricuCore;
- packages.tricu-core-archive = tricuCoreArchive;
- packages.tricu-core-linkage-smoke = pkgs.runCommand "tricu-core-linkage-smoke" {
+ packages.tricu-haskell-linkage-smoke = pkgs.runCommand "tricu-haskell-linkage-smoke" {
nativeBuildInputs = [ pkgs.python3 pkgs.stdenv.cc ];
meta.platforms = [ "x86_64-linux" ];
} ''
mkdir -p $out/bin
$CC -O2 -Wall -Wextra -Werror ${tricuSource}/ext/haskell/core_bench.c \
- -I${tricuCore}/include -L${tricuCore}/lib -ltricu-core \
- -Wl,-rpath,${tricuCore}/lib -o $out/bin/dynamic-core-bench
+ -I${tricuHaskell}/include -L${tricuHaskell}/lib -ltricu-haskell \
+ -Wl,-rpath,${tricuHaskell}/lib -o $out/bin/dynamic-core-bench
$CC -O2 -Wall -Wextra -Werror ${tricuSource}/ext/haskell/core_bench.c \
- -I${tricuCoreArchive}/include -L${tricuCoreArchive}/lib -ltricu-core \
- -Wl,-rpath,${tricuCoreArchive}/lib -o $out/bin/archive-core-bench
+ -I${tricuHaskellArchive}/include -L${tricuHaskellArchive}/lib -ltricu-haskell \
+ -Wl,-rpath,${tricuHaskellArchive}/lib -o $out/bin/archive-core-bench
PYTHONDONTWRITEBYTECODE=1 python3 ${tricuSource}/ext/haskell/core_bench.py \
--dynamic $out/bin/dynamic-core-bench --archive $out/bin/archive-core-bench \
--output $out/report.json > $out/summary.json
'';
- packages.tricu-core-archive-probe = import ./ext/haskell/archive-probe.nix {
- inherit pkgs haskellPackages;
- source = tricuSource;
- };
packages.tricu-python = tricuPython;
packages.tricu-python-tests = tricuPythonTests;
packages.tricu-workflow-demo = tricuWorkflow;
@@ -437,27 +647,8 @@
# Source-compiling embedding acceptance stays opt-in, like the other
# application compilation checks. Basic C/Python checks run in the library.
- packages.tricu-haskell-ffi-integration = pkgs.runCommand "tricu-haskell-ffi-integration" {
- nativeBuildInputs = [ pkgs.python3 tricuPackage ];
- } ''
- export PYTHONDONTWRITEBYTECODE=1
- export PYTHONPATH=${tricuSource}/ext/haskell/python
- mkdir -p $out workspace
- cp ${tricuSource}/lib/base.tri workspace/base.tri
- cp ${tricuSource}/lib/math/native.tri workspace/native_math.tri
- cp ${tricuSource}/lib/io.tri workspace/io.tri
- cp ${tricuSource}/lib/handlers.tri workspace/handlers.tri
- cp ${tricuSource}/demos/io/copy.tri workspace/copy.tri
- tricu arboricx compile --file workspace/copy.tri --store copy-store \
- --output $out/copy.arboricx --compile-timeout 120 +RTS -N1 -M1024m -RTS
- library=${self.packages.${system}.tricu-haskell-ffi}/lib/libtricu-haskell${pkgs.stdenv.hostPlatform.extensions.sharedLibrary}
- python3 ${tricuSource}/ext/haskell/test_embedding.py "$library" \
- ${tricuSource}/test/fixtures --copy-bundle $out/copy.arboricx > $out/integration.json
- python3 ${tricuSource}/ext/haskell/compiler_probe.py "$library" \
- ${tricuSource}/bootstrap/build-stable.arboricx --tricu ${tricuPackage}/bin/tricu \
- --directory compiler-probe > $out/compiler.log
- cp compiler-probe/report.json $out/compiler.json
- '';
+ packages.tricu-haskell-ffi-integration = embeddingIntegration tricuHaskell;
+ packages.tricu-haskell-ffi-archive-integration = embeddingIntegration tricuHaskellArchive;
packages.tricu-native-bindings-integration = pkgs.runCommand "tricu-native-bindings-integration" {
nativeBuildInputs = [ pythonWithTricu pkgs.nodejs phpWithFfi ];
@@ -606,11 +797,10 @@
checks.${packageName} = tricuPackageTests;
checks.default = tricuPackageTests;
checks.haskellEmbedding = tricuHaskell;
- checks.haskellCore = tricuCore;
checks.pythonBinding = tricuPythonTests;
checks.jsBinding = tricuJsTests;
checks.phpBinding = tricuPhpTests;
- # Fast packaging/filesystem checks only, no compiler or application runs.
+ # Fast packaging and filesystem integrity checks.
checks.arboricxSite = pkgs.runCommand "arboricx-site-release-check" {
nativeBuildInputs = [ pkgs.python3 ];
} ''
@@ -663,7 +853,7 @@
devShells.default = pkgs.mkShell {
buildInputs = with pkgs; [
- # These tools are separate executables, not libraries linked into tricu.
+ # Package these build tools as separate executables.
pkgs.cabal-install
pkgs.haskellPackages.ghc-events
pkgs.ghcid
