IO: scoped tasks, streams, files and publication

Current hosts execute canonical IO envelopes. Native execution uses OS process authority; retired grant arguments and unsupported envelope versions fail before effects. Haskell provides filesystem, TCP/resolver, process/context and clock facilities. The Haskell browser host provides structured core/task execution and terminal streams. Capability queries report the selected backend's facilities. TLS, UDP and hard deadlines require additional facilities. Historical v1 execution requires a retained runtime.

The operation layout is a research protocol subject to revision.

Reproduce

nix flake check
nix build .#
PYTHONDONTWRITEBYTECODE=1 python3 test/io_protocol.py result/bin/tricu \
  --directory /tmp/new-io-protocol-check
PYTHONDONTWRITEBYTECODE=1 python3 test/interaction_inspection.py result/bin/tricu
PYTHONDONTWRITEBYTECODE=1 python3 test/io_streams.py result/bin/tricu \
  --directory /tmp/new-io-stream-check
PYTHONDONTWRITEBYTECODE=1 python3 test/io_tcp.py result/bin/tricu \
  --directory /tmp/new-io-tcp-check
PYTHONDONTWRITEBYTECODE=1 python3 test/io_directory.py result/bin/tricu \
  --directory /tmp/new-io-directory-check
PYTHONDONTWRITEBYTECODE=1 python3 test/io_positioned.py result/bin/tricu \
  --directory /tmp/new-io-positioned-check
PYTHONDONTWRITEBYTECODE=1 python3 test/io_files.py result/bin/tricu \
  --directory /tmp/new-io-file-check
PYTHONDONTWRITEBYTECODE=1 python3 test/io_tasks.py result/bin/tricu \
  --directory /tmp/new-io-task-check
PYTHONDONTWRITEBYTECODE=1 python3 test/io_publication.py result/bin/tricu \
  --directory /tmp/new-io-publication-check

Use new output directories. The scripts compile current source through the packaged compiler, execute under OS process authority, and record source hashes, commands, full results and measurements. File checks cover complete binary output through 1 MiB, direct and handled borrowing, ownership transfer, implicit joining, acquisition failures, and cancellation after an acknowledged chunk. Copying uses direct truncating output. Inputs and outputs must be distinct even through hard links/symlinks; interruption can leave partial output.

Task source oracles independently check the three outcomes, consumed tickets, ordinary domain errors, partial-write progress/failure traces for the actual IO.writeAll helper (including full 4096-byte acknowledgements, partial suffixes, and failures after confirmed progress), child handler inheritance, contextual overrides, callback routing, lazy label payloads, and explicit state-boundary rejection. Retention workloads run 16/256/1024 children over fresh 4096-byte chunks in one process. They force each chunk's list spine before returning it, then join or discard the result; using only constant or unforced payloads would conceal important retention bugs.

The file/task/publication scripts enforce workload-specific 128-MiB sampled RSS and 16-MiB GC maximum live-heap guards. Allocation volume is recorded separately. Shared runner reports now also include rts_process: whole-process allocated/copied bytes, MUT and GC CPU/elapsed time, and CPU/elapsed productivity from GHC's -s output. Missing measurements are absent, not invented zeros. These include loading, rendering and profiling's forced collections; they are not the separately measured invocation deltas. MUT includes allocation, interpreter and native work, not just useful reductions. These are sequential fresh-process observations, not concurrent capacity guarantees or hard runtime resource bounds. RSS polling can miss short peaks; RTS residency is GC-sampled.

IOTests.hs exercises the resource registry and driver independently of the compiler. IOTaskTests.hs adds deterministic blocking/failure controls, lease cancellation, fail-fast supervision, lexical cleanup, completion races, join consumption/cycles, transfer publication, and registry reclamation. Fake backends supply controls without production fault-injection flags.

Binary standard streams and reusable copying

IO.stdin, IO.stdout and IO.stderr return ordinary Results containing borrowed stream references. IO.copyStream reader writer composes bounded readSome and writeAll, stopping at explicit EOF or the first error. It acquires/closes neither endpoint and adds no host copy primitive. It expects the native read/progress protocol; it does not validate arbitrary replacement interpreters. Error values preserve writeAll's confirmed prefix and original uncertain progress; copying is not a transaction, a whole-message atomic write, or a whole-file retry operation. These APIs use octet lists, not arbitrary Unicode strings.

Each native run starts with exactly three inherited root entries. Repeated lookup returns the same reference/lease for that slot, without adding registry history. Descendants borrow them, including a first lookup while their parent is joining; child scope exit does not revoke them. Explicit close returns borrowed-resource. Root-owned handoff also returns borrowed-resource; descendant attempts can fail not-owner first. No transfer occurs on rejection. Root settlement drains leases and removes the references but never closes, flushes or changes the mode of the process handles. Run-local references become stale afterwards.

Each successful output operation flushes the host handle before acknowledging its byte count. That is not durability, peer consumption or transactionality. Broken pipes are ordinary io-error operation failures with unknown progress. A failed or cancelled inherited write poisons that slot for the rest of the run: subsequent writes return invalid-state with unknown progress rather than implicitly retrying bytes left in a failed host buffer. The host still owns that buffer and may flush it independently, including during process shutdown. Embedders must perform their own recovery before reuse in a new run; teardown does not discard/rollback bytes.

Lease exclusion is per standard slot per run. It does not coordinate separate runs, arbitrary host IO, or stdout/stderr redirected to aliases of the same OS stream. Avoid unsynchronized mixing. Terminal modes/line discipline are unchanged; this is not raw terminal IO or a text/line codec. Existing cancellation and foreign call limits still apply. No deadlines or platform-wide descriptor isolation exist.

demos/io/streams.tri provides direct, handled and attached-child pipelines. Compile/import it as usual, then run its stored export with --io --no-result to reserve stdout for program bytes (no --repeat framing):

printf 'hello\n' | result/bin/tricu arboricx run --module "$APP_MANIFEST" \
  --store "$WORK/runtime" --name main --io --no-result > "$WORK/output"

The stream harness compares entire binary outputs and stderr, exercises actual pipes plus file redirects, empty/boundary inputs through 1 MiB, broken pipes, close/handoff rejection, handlers and attached children. It enforces 128-MiB RSS and 16-MiB GC residency guards. These runs have no final rendering or forced profile collections: their RTS totals are whole-process observations, not matched comparisons with the publication/profile workloads. Native tests also exercise shared blocked-read leases, read/write cancellation, poisoned writers, host buffering preservation and non-closing settlement. All seven inherited-stream native cases are included in the Nix suite.

Positioned files, updates and append

IO.openReader opens a readable file; IO.openWriter creates/truncates a writable file. IO.openUpdate opens a readable/writable file at position zero without truncating an existing file, creating it if absent. IO.openAppender creates if absent and writes using the host's append mode. None provides exclusive creation, atomic replacement, snapshot isolation or publication durability.

IO.seekFile file offset uses an absolute byte offset; IO.filePosition file queries the logical cursor, including read-ahead adjustment. IO.fileSize file queries the opened file, not its potentially replaced pathname. IO.resizeFile file size may shrink or grow a writable non-append file, preserving the logical cursor even beyond the new EOF. Regular-file growth reads as zero-filled space; seeking beyond EOF alone does not extend the file. These operations use canonical naturals through 9223372036854775807 (reported as file-offset-max). Smaller filesystem/backend constraints can still fail. Malformed/out-of-range wire arguments are rejected before that operation's backend IO; negative or excessive host query results cannot escape as invalid naturals.

All byte and position/size operations on a file share one lease. They reject concurrent use of the same registry entry with busy; handoff preserves the cursor and invalidates old aliases. A seek followed by a read is not an atomic compound operation. Coordinate borrowers, transfer ownership, or use independent read handles; no positioned read-at/write-at operation is promised here. Separate OS handles/processes can change file contents or size independently. Queries are point-in-time observations, not a race-free verification policy.

Append handles have no seek, position or resize method: those calls return unsupported. Reads on writers/appenders return wrong-kind; resize on a reader returns unsupported. Standard streams and stages are not positioned-file handles, even if their underlying storage happens to be a file. Non-seekable/special files can report unsupported or backend IO errors. Haskell's process-local Handle locking may reject conflicting file opens that POSIX alone would permit; this restriction is advertised. Do not infer filesystem confinement or external-writer serialization from that host behavior.

Append placement is at the backend's current EOF on writes, not an open-time seek. This is not a whole-record atomicity guarantee: partial writes, interleaving with other writers and failure remain possible. IO exceptions during seek/resize report unknown progress/state, not a claim that nothing changed. There is no automatic retry, rollback, restoration after an operation failure, or new durability promise. Ordinary Results and scope cleanup retain the existing rules.

demos/io/positioned.tri composes bounded patching, resizing, prefix inspection and append. It is deliberately in-place editing, not a safe publication replacement. The focused harness compares complete outputs/results, including Unicode paths, empty files, shrinking/growth, updates/appends against 1-MiB files, mode errors and malformed offsets. A wide-offset control writes a sparse byte above 4 GiB and shrinks before comparing its entire final output; it does not materialize a multi-GiB source value or assert multi-GiB streaming performance. Native tests cover cursor mode switches, read-ahead invalidation, current-inode size, external-fd growth before append, bounds, cancellation/lease exclusion, transfer, host locking and uncertain mutation errors.

Paths and directory cursors

IO.statPath follows symlinks; IO.lstatPath observes a final link itself. Both use normal host resolution of intermediate components and trailing slashes. They return a regular-file size only for a regular file; other tags deliberately do not invent a zero byte length. Missing paths and permission errors remain ordinary errors, not a false existence Boolean. These observations do not pin later opens or constitute race-free authorization, immutable verification or confinement.

IO.createDirectory creates one directory with host mode 0777 subject to umask; it fails with already-exists rather than silently accepting another object. IO.ensureDirectory is a source convenience: attempt creation, and on already-exists use stat to check for a directory. Existing symlinks to directories are accepted. It is nonrecursive; caller-ordered initialization fits store/cache layouts. The check is only a point-in-time observation. Direct files retain their host creation modes/umask, not the private-stage permission contract.

IO.removeFile unlinks a file/link, IO.removeDirectory removes an empty directory, and IO.renamePath performs the host rename/replacement operation. No recursive removal, cross-filesystem copy fallback, no-replace rename, sync, rollback, or transaction is supplied. Path mutations persist after scope exit: scopes own handles, not arbitrary pathname changes. Mutation IO errors conservatively report unknown progress/state. Failed close IO now also reports unknown (correcting its former none report); this does not authorize retrying raw release operations.

IO.openDirectory owner path creates a scoped cursor over the opened directory. IO.nextDirectory yields one name at a time, skipping . and .., with an explicit, sticky EOF. It never sorts/materializes a complete listing. Host internal buffering is not specified. Enumeration order and concurrent directory mutation are not snapshot guarantees. Renaming the directory does not retarget the cursor, but later path operations built from its original pathname can target another object; no dirfd-relative child operations are provided here.

The cursor uses one exclusive lease and the ordinary lifetime/transfer/close rules. Failed or cancelled advancement poisons it: subsequent next calls return invalid-state; close or explicitly reopen instead of implicitly replaying an uncertain advance. EOF does not implicitly close the cursor. Closing attempts closedir once, never retrying a possibly freed native directory pointer.

Paths use canonical Unicode-scalar text with no NUL, mapped through the host's filesystem encoding. Directory names must be Unicode-scalar single components, up to 4096 scalars (advertised as directory-name-scalars). Undecodable names are not silently replaced or skipped: they fail and poison enumeration. This is not an arbitrary raw-byte POSIX pathname interface. Empty path arguments are rejected by these path/directory operations; use . for the current directory. Native capability restrictions expose host encoding, unsorted/non-snapshot iteration and cursor poisoning. No Unicode normalization is performed.

demos/io/directory.tri provides caller-ordered initialization and an inventory that intentionally accumulates entries, alongside a count that discards names and forces its small numeric accumulator before each request. The latter avoids retaining a lazy arithmetic chain; bounded native enumeration alone would not make arbitrary source accumulation bounded. The focused test compares complete inventories without assuming order, checks symlink/dangling-link and mutation behavior, rejects a real invalid-encoding name, and consumes up to 16,384 entries under RSS/live-heap guards. Native controls exercise ownership transfer, renamed open directories, cancellation/busy exclusion, EOF, invalid names, poisoning and once-only close, including uncertain/async release failures.

TCP listeners and duplex streams

Endpoints are canonical tagged lists:

• ["ipv4" [a b c d] port]: four octets and a 16-bit port.
• ["ipv6" [a b c d e f g h] scope port]: eight 16-bit words, a 32-bit interface scope ID and a 16-bit port. Flow information is zero.

IO.ipv4 / IO.ipv6 construct these values; they do not resolve names. IO.listenTCP owner endpoint backlog acquires an already bound/listening socket. Backlog must be 1–4096; OS clamping/queue policy still applies. Port zero requests an ephemeral listening port; query IO.localAddress to learn it. IPv6 listeners are explicitly IPv6-only, not silently dual-stack. Reuse-address is enabled; there is no reuse-port, ownership or exclusive-bind security guarantee. IO.connectTCP owner endpoint connects exactly that numeric address; zero peer port is rejected. There is no hidden DNS, address fallback or reconnect policy. tcp-address-families reports implemented encodings, not OS/network availability.

IO.acceptTCP listener owner leases the listener and reserves acquisition in the selected live ancestor/current scope before accepting. It publishes the connection before acknowledging success; cancellation cannot leave an unregistered accepted socket. The caller can use the existing atomic spawn-with-transfer operation to move it to a worker, with fresh handles and stale old aliases. Listener acceptance is exclusive; multiple pending accepts return busy. Acquiring into an explicitly selected ancestor retains that ancestor's lifetime, just as file factories do.

A connected stream permits one read and one write concurrently. Another read or another write returns busy. Read EOF is sticky and does not close the write direction. Write acknowledgements are positive accepted prefixes, not complete messages, delivery, peer consumption or transaction boundaries. IO.writeAll retains its ordinary partial/uncertain-progress rules. Source multi-call writes are not whole-message atomic. TCP has no datagram/message framing here.

IO.shutdownWrite leases the write direction, sends the local write half-close and preserves reading; successful repeats are harmless. Writes afterwards fail with invalid-state and no progress. Failed/cancelled reads or writes poison only the affected direction; later use reports invalid-state/unknown progress rather than replaying ambiguous work. Failed shutdown poisons writing too. Socket IO errors include stable categories such as connection-refused, connection-reset, address-in-use, broken-pipe and timed-out when the host supplies those errno values; otherwise the ordinary generic categories remain. An OS timeout error is not a new language deadline facility.

Address queries currently lease the whole resource. Transfer and close require both directions idle. Close prevents new leases and waits for pending ones; it does not close/reuse the descriptor beneath a blocked read/accept to wake it. Cancel the owning IO task to interrupt supported pending work, then settle/close. Scope failure/cancellation does that through the structured-task machinery. There are no hard deadlines; OS/kernel queued output can still reach a peer after local acknowledgement/close. Cancellation does not undo network effects. Files, standard streams and directories retain their existing single-lease rules.

demos/io/tcp.tri composes finite concurrent echo servers, lexical child handler inheritance, and a file/socket client with simultaneous sending and receiving to avoid bidirectional backpressure deadlock. The handled server's binary banner is source policy, not a native operation. Default workers are fail-fast; an explicitly supervised variant tests containment and discards outcomes intentionally, not as a production logging recommendation. These are raw TCP demonstrations, not HTTP, TLS or hardened public services. Registry deployments remain unchanged.

The focused real-socket suite compares every echoed/output byte through 1 MiB, fragmented sends, IPv4/IPv6, concurrent clients, repeated handoff, EOF/half-close, refusal/malformed-address errors, and accepted-peer resets with both fail-fast and contained workers. Native controls additionally cover directional lease exclusion, close/transfer while only one direction remains busy, blocked receive/write cancellation, listener reuse after cancelled accept, scope admission, structured settlement, endpoint bounds and non-demand of excess components. RSS/live-heap guards and ordinary RTS observations are regression controls, not throughput or maximum-concurrency guarantees.

Explicit host name resolution

IO.resolveTCP host family port limit uses the host TCP name resolver. Family is "ipv4", "ipv6" or "any"; port is 0–65535 and result limit is 1–64. Host input is 1–253 ASCII letters/digits/dots/hyphens. This admits ordinary host names and IPv4 literals accepted by the host; it is not strict DNS-label validation, IDNA conversion, hostname authorization or a numeric-address parser. Use the numeric endpoint constructors for IPv6 literals. Host search rules, hosts/NSS configuration, ordering, duplicates and address availability remain host-defined.

The result is [endpoints truncated], with a canonical natural 0/1 truncation flag. Only the bounded prefix is published; excess endpoint values are not demanded, and the returned prefix does not retain the discarded host list. The host resolver may internally buffer more results and delay cancellation. There is no detached resolver worker, guaranteed prompt interruption, public DNS deadline, exhaustive-answer claim, implicit connection, deduplication or address preference. OS resolution failures return ordinary resolution-failed with bounded diagnostic text; cancellation/unexpected faults are not converted to that Result.

IO.connectFirstTCP owner endpoints is an explicit source policy that tries candidates sequentially, stops at the first success and returns the last ordinary failure unchanged if none succeeds. An empty list returns not-found. It does not race candidates, add deadlines or elevate the selected endpoint into authorization. The named client demo chooses the resolver's bounded prefix, then uses that policy. Applications needing other retry, selection or authentication policies should compose their own. TLS peer identity is not supplied by successful resolution.

Native controls cover local/numeric host lookup, bounds/non-demand, host ordering, truncation, ordinary errors and cancellation propagation without requiring public DNS. The real socket source suite also resolves numeric/localhost names, checks complete output, and exercises failed-first/successful-second source fallback, including non-execution of a malformed later candidate and scope-bound lifetime.

Authenticated native byte snapshots and transfer

The Haskell native host implements two revision-1 stream operations:

• IO.snapshotBytes reader directory limit digest consumes the reader through EOF under one read lease and returns [snapshot size] inside a Result. digest is exactly 64 lowercase hexadecimal SHA-256 digits over the raw bytes. limit is a canonical natural in [0,2^63-1]. The EOF probe can consume one excess byte on a limit failure. A digest mismatch returns integrity-failed.
• IO.transferBytes reader writer count transfers exactly count bytes from the current cursor with both directional leases held. Positive short writes are completed without replaying accepted prefixes. Early EOF is invalid-data; an error can follow partial output, so callers close rather than retrying an uncertain transfer. A zero count still validates and leases both handles.

Snapshot preparation registers an anonymous temporary file before consuming input. Only authenticated snapshots are returned; they are read-only file resources owned by the current scope, with read/seek/position/size operations. They may be closed or handed off under the ordinary ownership rules. Their private writable handles remain native. Transfer reads the verified snapshot, so changes to the original file after authentication cannot change its output. Concurrent source mutation is accepted only if the captured bytes match the expected digest. This is integrity checking, not OS-process confinement.

Both operations use native chunks of at most 4096 bytes and constant-size hash state, avoiding tree-encoded bulk byte lists. Temporary payload disk is at most limit per snapshot, plus filesystem overhead. Source readers can still block; operation cancellation is interruptible, with no hard deadline guarantee. Errors and cancellation retain scoped cleanup and diagnostic precedence. These APIs are native-host facilities; other backends report their own capabilities.

The registry's bundle GET/HEAD path verifies before headers, logs the selected status, then sends the known length and verified body (HEAD sends headers only). Missing/corrupt/oversized objects return 404; malformed digests return 400. Logger failure before headers returns 500. Transfer failure after headers closes the connection. Whole-value artifact APIs and package metadata keep their existing resident-value contracts.

IOByteStreamTests covers boundaries through 1 MiB, binary/partial transfers, read-only snapshots, corruption, size limits, source mutation, cancellation, cleanup faults, early EOF and disconnected writers. test/io_http.py exercises framing, configured limits and failed logging. The production-runner acceptance gate additionally runs test/registry_download.py on the actual compiler bundle and bounds both upload/download retention in test/body_retention.py.

Authenticated indexed-tree reads

IO.readIndexedTree reader directory limit size digest consumes exactly size bytes from the current reader position, under one read lease. limit is the caller's maximum encoded-frame size; both are canonical integers in [0,2^63-1] and size must not exceed limit. Digest is exactly 64 lowercase hexadecimal digits, using the existing arboricx.indexed-tree.v1 SHA-256 domain (UTF-8 domain, NUL, encoded bytes). The reply is an ordinary Result containing the decoded tree. No tree is evaluated, exposed, or decoded before authentication succeeds. The caller supplies the expected digest: this checks content integrity, not publisher identity or source provenance.

The native transport creates a temporary file in the explicitly supplied directory and unlinks its name before receiving input. It copies/hashes bounded chunks and then decodes that same private snapshot, not a second pass over the public source. Changing the original inode cannot change the snapshot. This is not a point-in-time filesystem snapshot: authentication checks the bytes actually received. A mutation during copying ordinarily makes the expected digest fail. The directory must exist; empty directory strings are rejected (use ".").

Temporary payload storage is at most size bytes, plus filesystem bookkeeping: no expanded-tree serialization. Storage is reclaimed before ordinary completion; on terminal fault/cancellation, its registered resource is released during lexical settlement, with the existing primary-error/cleanup-diagnostic precedence. The snapshot is private transport, not a source-visible handle, durable publication, or recovery archive. Guessed registry keys cannot read, explicitly close, or transfer private spool entries; active private leases still delay scope release. This is lifecycle protection, not confinement against OS process authority. No fsync or publication is performed. A tmpfs directory uses RAM-backed storage; ordinary filesystems may cache writes too. Choose the directory and frame budget accordingly. There is no aggregate disk quota across tasks.

Framing is explicit: no EOF probe consumes bytes of the following frame. The source reader is not closed or rewound. Digest mismatch, truncation, or invalid format may have consumed input; their unknown progress is not authorization to retry. Validation of bounds and digest spelling precedes operation IO. An unauthenticated malformed frame reports digest mismatch, never a parser error. Snapshot acquisition errors and resource errors remain ordinary Results; unexpected faults/cancellation remain terminal. Ordinary primary failures retain bounded cleanup detail if release also fails.

Decoded nodes preserve sharing and the existing reader's structural/manifest checks, including backward references and duplicate-node rejection. Additional metadata limits are 64 section entries and 65,536 manifest bytes. Noncritical opaque sections remain supported and are authenticated even though not decoded. These are reader semantics, not a claim that every accepted encoding equals the canonical writer's bytes. Offset arithmetic is checked without integer wraparound. Transfer buffers are bounded, but the indexed DAG and verification tables are resident and grow with node count; the caller's frame budget must allow for that amplification. There is no hard parser CPU deadline or universal safe maximum DAG size.

The facility works on readable files, borrowed input and duplex streams. File and borrowed-input leases stay exclusive; duplex connections permit concurrent writing but not another reader. Other operations cannot interleave reads between chunks. The reader alone implies no compiler caller migration or deployment/backend switch; canonical writing and comparison are described below.

python3 test/io_indexed.py result/bin/tricu --directory /tmp/new-indexed-check

demos/io/indexed-read.tri demonstrates source-selected framing and spool policy. Native SnapshotTests cover authentication ordering, immutable-source isolation, shared DAGs, malformed offsets/nodes, metadata bounds, cleanup faults, cancellation, lease exclusion and wire non-demand. The independent source suite compares complete results and framing through 64-MiB encoded inputs. Its large inputs grow an opaque section while keeping the decoded DAG tiny: they test bounded transport, not large-DAG capacity. RTS reports and sampled RSS are included; very short runs may exit before a representative RSS sample. The 64-MiB transfer has 48-MiB sampled-RSS and 16-MiB maximum-live regression guards, not service guarantees.

Canonical indexed writing and exact comparison

IO.hashIndexedTree limit tree returns Result [size digest] for the canonical, name-neutral indexed encoding. IO.writeIndexedTree writer limit digest tree prepares the same encoding and checks its size and expected digest before any writer call; success returns [size digest]. It holds one write lease for the whole operation and handles positive partial acknowledgements without replaying acknowledged bytes. Reads may proceed concurrently on duplex connections; files, stages and borrowed outputs remain exclusive. Cancellation and host faults retain the existing structured-settlement rules.

Transport buffers are at most 4096 bytes. The native encoder retains the DAG and exact interning tables, not an expanded tree or whole encoded ByteString. Hash and write passes regenerate chunks rather than retaining a shared lazy chunk history. Bytes and digest domains match the existing canonical indexed writer. The size budget bounds encoded output, not tree evaluation, graph construction memory, CPU time or an aggregate quota. Preparation can allocate before discovering that output exceeds the limit; divergent source trees have no hard deadline.

Preflight failures have none writer progress. Once sending starts, IO failures have unknown aggregate progress: no atomicity, rollback, automatic retry, or resumption promise. A caller opening a truncating writer has already truncated its file before invoking this operation. Use a stage for publication safety.

IO.verifyIndexedTree reader limit tree compares one canonical frame byte-for-byte against a known tree and returns [size digest] on success. It does not merely compare hashes, decode untrusted bytes, or probe EOF. Whole-file winner verification must additionally check EOF; demos/io/indexed-write.tri does so. Its publication composition leaves a corrupt/excess existing winner untouched. An exists result is exact verification at observation time, not existing-winner durability or protection from subsequent external mutation. Compiler publication requires additional synchronization and ordering; the source-owned object library below supplies that policy without changing this demo into a durability claim.

python3 test/io_indexed_write.py result/bin/tricu --directory /tmp/new-indexed-write

Native tests cover canonical byte equivalence at varying chunk boundaries, physical sharing, partial progress, preflight non-effects/non-demand, interrupted leases, uncertain failures and exact comparison. Independent source checks cover hashes, complete output bytes, staged publication and corrupt/excess winners, plus a DAG with 200 shared fork levels and chains through 100,000 nodes. Those graphs remain indexed during round-trip; their node-table memory is expected to grow. Reported RSS is sampled and tiny runs may finish before representative samples.

Synchronization and source-owned indexed objects

IO.syncFile handle flushes writable file buffering and requests filesystem synchronization of the opened inode. It shares the file's exclusive cursor lease, preserves its logical position, and supports read/update/write/append file modes. It is not a standard-stream, stage, socket or directory-cursor operation. IO.syncDirectory path opens and validates a directory, then requests synchronization of that directory itself (not recursively). Paths follow the existing host/symlink rules. Nonblocking acquisition avoids hanging on a FIFO substituted for a directory.

Both operations return ordinary Results. Success means the host synchronization request succeeded, not a universal crash/power-loss guarantee, peer consumption, atomicity with another operation, or protection against subsequent mutation. Filesystem/device behavior can be weaker; some filesystems reject synchronization. Failures during sync have unknown durability/progress and do not imply rollback. Transient descriptor resources are registered before use and have exclusive private leases. Fault/cancellation cleanup follows lexical settlement; failed raw closes are not retried against potentially reused descriptor numbers. Ordinary sync failures retain their primary detail and bounded release diagnostics if both fail.

io_objects supplies source-owned putTree root limit tree and getTree root limit digest computations. These are library policy, not native object-store services. The store root must already exist; its creation and parent namespace durability belong to the caller. Put creates/synchronizes objects and three-character shard directories, uses a scoped stage and no-replace publication, then verifies every winner byte-for-byte through EOF. It synchronizes the verified open inode and shard directory before returning the digest. Callers must publish manifests/locks/aliases only after that successful return.

A mismatch or synchronization failure never deletes/overwrites a winning name and can occur after a new object became visible. There is no crash pruning or rollback. Root paths are trusted: this is not symlink confinement or a transaction against external mutable writers. Get validates lowercase digest spelling before path IO, gets the opened-file size, and uses the authenticated snapshot reader with the caller's encoded-size budget. Object domains, paths, canonical bytes and sharding match the existing compiler store. Returning an authenticated tree does not execute it or establish publisher identity.

python3 test/io_objects.py result/bin/tricu --directory /tmp/new-object-check

demos/io/objects.tri exercises these policies. Independent controls cover complete object bytes/results, fresh/existing/corrupt/excess winners, malformed digests, absent/blocked roots, limits, exponentially shared cross-store copying, and actual concurrent publishers. Native synchronization tests cover opened-inode identity/cursor behavior, modes, non-directories/FIFOs, leases, cancellation, primary/cleanup faults and once-only descriptor release. Power-loss durability is outside these tests. The packaged compiler uses this source-owned storage policy; full reproduction is covered by the compiler rebuilding guide.

Strict UTF-8 and source-text conveniences

IO.encodeUTF8 text accepts at most 1024 canonical Unicode scalars and returns at most 4096 octets. IO.decodeUTF8 final bytes accepts at most 4096 canonical octets and returns [text carry]. Final is canonical 0/1. A nonfinal valid incomplete suffix is returned as at most three octets; supply it before the next input chunk. Final input rejects incomplete suffixes. Invalid prefixes fail immediately, even when incomplete. Overlong encodings, surrogate code points, stray continuation bytes and values above U+10FFFF are rejected; no replacement decoding occurs. Empty chunks, NUL, BOM and Unicode noncharacters are valid codec data. No BOM stripping, newline folding or Unicode normalization occurs. Errors report offsets within this supplied chunk and return no partial decoded output.

These bounded codecs are stateless. Carry is ordinary source data, not hidden per-stream state; callers own sequencing and can use any byte transport. At most 4093 fresh bytes plus a three-byte carry fit a decode request. Bounds and canonical scalar/octet validation precede conversion; excess elements are not demanded.

io_text composes them into:

• readSource reader limit: whole-source Result (rawOctets, scalars), preserving exactly the bytes used for decoding and rejecting an initial BOM. The reader is borrowed, not closed. The byte limit is canonical [0,2^63-1]; a one-byte EOF probe at the boundary may consume one excess byte before rejecting oversize input. Failure is ["read-source" confirmedInput cause]; a failing read's own uncertain progress remains in the unchanged cause. No automatic retry or cursor rewind.
• encodeText limit scalars: whole-value encoded octets under a byte budget, without source-specific BOM rejection. It does not write a resource.
• writeText writer scalars: bounded sequential encoding/writing with no aggregate output quota or ownership transfer. Failure is ["write-text" confirmedBefore cause]; writeAll's partial-progress diagnostic is preserved when applicable.

Read/encode conveniences retain whole values and are not constant-memory source compilation. Budgets bound bytes, not all graph/interpreter memory or CPU. Individual read/write operations are leased; these source conveniences do not acquire one whole-text lease, guarantee whole-record atomicity, or make concurrent cursor use safe. The low-level codecs remain usable in bounded streaming applications instead.

python3 test/io_text.py result/bin/tricu --directory /tmp/new-text-check

Native tests independently exercise all 1,112,064 Unicode scalars against standard UTF-8 encoding, boundary/invalid vectors, every split of mixed text, canonical wire bounds and non-demand. Source checks compare complete encoded bytes, raw source provenance and scalar values across chunk boundaries; retain CR/LF, NUL, combining characters and interior BOM; reject initial BOM, malformed/incomplete input, excess budgets and surrogates; preserve an acknowledged prefix on later encoding failure. Whole-source pipelines through 64 KiB record RTS and sampled RSS observations for their retained source/decoded values.

Internal protocol

Envelope: ("tricuIO", (2, action)). Action is a pair of canonical numeric tag and payload. Comma-separated payload fields below are right-nested pairs, except where a list is explicitly shown. Pure/bind do not add Result envelopes.

Tag	Payload	Reply
0	value	raw value
1	action, continuation	sequencing
2	diagnostic	terminal failure
3	callback	new scope supplied to callback; returns body value after children and release
4	ignored	current scope reference
5	supervised, keepResult, handles, callback	Result [task joinTicket]; callback receives fresh handle list
6	join ticket	Result terminal outcome; consumes ticket at claim
7	join ticket	Result unit; discards the result, not the child
8	task, reason	Result unit acknowledging cancellation request
9	reason	cancel this computation, without resuming its continuation
10	facility name, group revision, operation name, arguments	Result (named external operation)

Current core tags are listed above. Facility calls use named requests. Earlier v2 tags 20–42 reject; historical v1 actions require a retained runtime:

IO.request group revision operation arguments
-- (10, (group, (revision, (operation, arguments))))

Group/operation names are nonempty ASCII identifiers of at most 64 characters: an initial lowercase letter followed by lowercase letters, digits or hyphens. Revisions are positive canonical 16-bit naturals. The outer envelope version is also a canonical 16-bit natural, and its sentinel is exactly canonical tricuIO. Native execution and indexed inspection enforce these envelope checks for both versions; noncanonical envelope aliases are deliberately rejected.

Implemented group revisions are all 1:

Group	Operation	Arguments	Reply inside Result
runtime	capabilities	unit t	capability association rows
runtime	inspect	action	shallow header description
standard	input / output / error	unit t	borrowed stdin reader / stdout writer / stderr writer
file	open-reader	scope, path	reader
file	open-writer	scope, path	writer (create/truncate)
file	open-appender	scope, path	append-only writer (create if absent)
file	open-update	scope, path	readable/writable file (create if absent, no truncation)
file	seek	file, absolute byte offset	unit
file	position	file	current logical byte offset
file	size	file	observed byte length of the opened file
file	resize	file, byte length	unit
path	stat	path	["file" bytes], ["directory"], ["symlink"], or ["other"]
path	lstat	path	same tags, without following final symlink (host path resolution rules apply)
path	remove-file	path	unit; unlink, not recursive deletion
path	rename	source, destination	unit; host replacement semantics
directory	create	path	unit; strict nonrecursive creation
directory	remove	path	unit; empty-directory removal
directory	open	scope, path	directory cursor
directory	next	cursor	pair ("entry", name) or ("eof", t)
tcp	listen	scope, (endpoint, backlog)	listener
tcp	connect	scope, endpoint	duplex stream
tcp	accept	listener, scope	duplex stream owned by the selected scope
tcp	local-address	listener or duplex stream	endpoint
tcp	peer-address	duplex stream	endpoint
tcp	shutdown-write	duplex stream	unit
resolver	resolve	[host family port limit]	[endpoints truncated]
indexed-tree	read	[reader directory limit size digest]	decoded tree
file	sync	handle	unit
directory	sync	path	unit
utf8	encode	scalar list	octet list
utf8	decode	[final octets]	[scalars carryOctets]
indexed-tree	hash	limit, tree	[size digest]
indexed-tree	write	[writer limit digest tree]	[size digest]
indexed-tree	verify	[reader limit tree]	[size digest]
stream	read	reader, positive limit	(data, nonempty bytes) or (eof, t)
stream	write	writer, nonempty bytes	positive accepted count
stream	snapshot	[reader directory limit digest]	[snapshot size]
stream	transfer	[reader writer count]	unit
resource	close	resource	unit
stage	open	scope, directory	staged writer
stage	seal	stage	unit after flush, sync and close
stage	publish	stage, replacement mode, destination	"published" or "exists"
digest	open	scope, algorithm	digest context; currently "sha256"
digest	update	digest, nonempty bytes	unit
digest	finish	digest	lowercase hexadecimal digest; consumes state

IO.capabilities reports association rows [key value]: schema (currently 1), io-version, backend, authority, core constructor names, groups as [name revision operationNames], limits (chunk bytes, transfer handles and maximum file offset and directory-name scalar limit TCP backlog, resolver input/output bounds and indexed metadata bounds), digest-algorithms, and restrictions. It describes this backend's implementation, not OS permissions, resource availability or a guarantee that a future call works. For example, no-replace publication still needs filesystem hard-link support. The native report covers native Linux capabilities.

A well-formed named request with an unavailable group, revision or operation returns unsupported without demanding its arguments. Unknown names do not reserve or specify future argument schemas. Malformed headers and unknown core constructors are terminal protocol failures; bad arguments to implemented requests are ordinary operation errors. Protocol failures carry ["protocol-error" "core/decode" boundedDetail "none"] as their diagnostic.

IO.inspect action returns ["core" constructorName] or ["facility" group revision operation "available"/"unsupported"] inside Result. This inspects only the header: it does not validate arguments, inspect payloads, follow bind actions, apply callbacks, resolve handles or predict success. Even a malformed bind payload can have a recognizable bind header. An invalid queried header returns an ordinary invalid-argument error from runtime/inspect. This is distinct from the richer read-only indexed bundle inspector, which follows explicit bind action structure under depth/occurrence limits while keeping v2 request arguments and callbacks opaque. Its JSON schema is tricu.interaction-inspection.v2 with supportedVersions: [1,2]; offline inspection never infers a deployment backend's support from an artifact's named header.

Spawn flags are canonical 0/1. A non-result child has t in the ticket position; its task control handle is still available while active. Transfer lists contain at most 256 handles. Task and join references use distinct markers, run identity and monotonic keys; they are not portable resources or secret capabilities.

Joined outcomes are lists: ["completed" value], ["failed" diagnostic cleanup], or ["cancelled" reason cleanup]. Cancellation is not an ordinary operation error inside the cancelled child. A supervisor can observe its final outcome as data. The CLI reports terminal failure/cancellation without forcing arbitrary diagnostic payloads. runDetailed/runInRuntime retain structured outcomes for embedding; a supplied runtime belongs to that execution and closes on exit.

Ordinary errors are four-element lists: category, operation, bounded detail and progress. Wire operation errors now use the requested group/operation, e.g. file/open-reader, stage/publish, or core/join, rather than leaking incidental backend labels such as open or task. Category/detail/progress are preserved. Terminal user diagnostics remain arbitrary trees; lifecycle cleanup diagnostics retain their resource-release context and are not fabricated operation replies. Canonical byte chunks/read bounds are 4096 bytes; paths are canonical Unicode scalar lists without NUL. Empty read means EOF, never transient progress. The file backend accepts the whole chunk on successful writes; source writeAll also handles partial positive acknowledgements. Its failures are ["write-all" confirmedBefore originalProblem]: confirmed bytes from earlier successful calls, plus the failing operation's unchanged diagnostic. They do not claim the failing operation accepted zero bytes. This deliberately replaces raw error forwarding, which lost earlier progress. Native write errors can have unknown progress, so blindly retrying an entire chunk is unsafe. Acknowledgement is not durability. Unknown/malformed computation constructors fail the task; invalid arguments to recognized operations return ordinary errors.

Lifetimes, admission and handoff

Scopes own resources and child tasks. Descendants borrow ancestor resources; returning a handle does not extend its lifetime. Normal exit closes admission, joins attached children, then releases resources. Existing descendants keep their borrowing rights and may acquire into their own still-open scopes. Failure or cancellation requests child cancellation before waiting. There are no detached tasks. An endless child makes normal scope exit wait until explicitly stopped.

Operations hold leases. Closing rejects new operations and waits for an existing lease before releasing; file cursor overlap returns busy. Repeated explicit close reports stale/closed, without retained tombstones. Scope cleanup finishes inner scopes first and releases resources in reverse acquisition order.

Spawn creates a gated worker before commitment. Resource reservation, transfer, task registration and opening its gate occur in one STM transaction. Invalid handle sets start no child and move nothing. Only idle resources owned by the attaching scope move; borrowed ancestor resources remain borrowed. Old aliases become stale; fresh handles preserve input order, while release follows original acquisition order. Explicit close claims closing atomically with token validation, so an old alias cannot close a newly transferred resource. Once published, a worker startup/execution fault is a task failure, not a rollback of successful spawn. OS/thread allocation before publication must succeed before any transfer.

Outcomes, cancellation and cleanup

Ordinary Results remain values. IO.require explicitly promotes a selected operation error to terminal failure. Uncontained child failure records the group's failure and cancellation of siblings transactionally, then wakes the body. Supervision must be selected at spawn, before the child runs. Joining later does not retroactively contain a fault; discarding a ticket does not disable supervision.

The first recorded stop request wins. Completion linearizes only after child and resource settlement, so a cancellation during normal joining/release can still win over a successful body. Cancellation after final completion reports stale or completed-task. A task has one consumable ticket: claiming join removes it before waiting; concurrent/duplicate claims fail. An interrupted join does not restore the ticket. Explicit and implicit joins participate in wait-cycle detection; an explicit cycle returns an ordinary error, while an unavoidable implicit scope cycle fails the group. Non-result and discarded completions leave no result archive. Unconsumed tickets deliberately retain their results until scope exit.

Controllers do not run user computation. A body worker executes the interaction tree; sticky cancellation is checked before interpreting each action. Waking a worker cannot replace its recorded reason or turn cancellation into a recoverable Result. Even a backend that catches the wakeup cannot resume the next source continuation. Nested lexical controllers receive cancellation through group state, not an exception injected into their resource-release protocol.

Cancellation waits for pending work to settle; it cannot undo bytes written or other external effects. Cleanup attempts remaining releases after synchronous failures, preserves an existing failure/cancellation, and bounds diagnostics to sixteen plus an omitted-count marker. Successful execution with failed required cleanup becomes failure. A reported failed release is not proof the OS reclaimed a broken backend resource. Host interruption of the outer runner is rethrown only after subtree settlement; repeated external interruption/process death remains outside the cleanup guarantee.

Runtime.Tree.apply is a lazy recursive evaluator, not an explicit budgeted reduction machine. The driver supplies action-boundary safe points; native pure reduction relies on GHC scheduling/allocation safe points. No prompt cancellation or hard deadline is promised for arbitrary divergent pure calculations, foreign calls or OS cleanup. Pending-IO cancellation tests do not establish those claims.

Handler bridge

H.scope, H.spawn, and H.runIO are concurrency-aware source boundaries. Normalization snapshots the actual lexical service environment and labels for the child, with an empty child continuation stack. A handler callback's child inherits the callback's outer environment, not the original requester's inner overrides. Child-local handler changes do not mutate parent routing. Label list spines are settled at boundaries without forcing payloads or unselected continuations.

H.runIO aliases H.runIO, which returns successful values directly and maps handler failures/unhandled requests to terminal failures. Service-level Results remain ordinary values. H.liftIO (IO.spawn ...) alone does not capture handler routing: use H.spawn for handler computations.

H.collect is a state-threading transformer, not shared mutable state. It rejects scope/spawn boundaries rather than silently sharing or dropping its state; install child-local collection inside the child body explicitly. General shared cells/channels and implicit child-state snapshots are not implemented. Captured services that refer to resources still obey the real resource scopes.

The handled copier exercises the current host bridge and lexical service routing.

Staged publication and independent digests

IO.openStage creates a mode-private temporary file in an existing directory (nonempty path; use . explicitly for the working directory). It is not visible at the destination name, but its temporary name is discoverable by the OS owner; this is not a sandbox. IO.writeSome/writeAll work on stages. IO.sealStage flushes, synchronizes the actual file descriptor and closes it; subsequent writes fail. On sync failure/interruption, the descriptor remains registered for scope cleanup, so release failure cannot replace the original sync failure. Directory synchronization uses the same ownership discipline. Failed descriptor closes are not blindly retried against a possibly reused OS descriptor; other cleanup is still attempted. No length, hash, artifact name or upload-session policy is hidden in the native staged writer.

Publication may cross directories on the same filesystem. Destination directories must already exist. Atomic rename/hard-link syscalls enforce the filesystem boundary; there is no copy fallback or automatic retry on cross-filesystem failure. This replaces the earlier same-directory-only restriction. Basenames containing .tricu-stage- are reserved for temporary files, preventing publication over this or another stage's private name. publishReplace atomically renames over a destination; publishNew atomically creates a hard link without replacement, then removes the private name. The latter requires filesystem hard-link support. Both synchronize the destination directory after successful publication, then the staging directory when it is distinct. Same-directory publication synchronizes the directory once. A failure during either synchronization or descriptor release can follow visible publication; it is not rollback permission. Synchronization stops at failure, and owned descriptor/private-name cleanup still runs. These are POSIX filesystem mechanics, not race-resistant confinement against other actors changing managed directories. Already-open readers of a replaced destination can still see its old inode.

Mode 0 means no-replace; mode 1 means replacement. "exists" leaves the stage sealed and makes no claim about the winner's contents or durability. "published" means this backend's file/directory synchronization requests completed, not a universal crash-consistency guarantee across every OS/filesystem/storage device.

Write, seal and publication execution failures freeze the stage against further writes/publication. Publication errors classify visibility in the progress field: not-published before the namespace operation, unknown on an ambiguous namespace failure, or published after successful link/rename but failed removal/directory sync/descriptor release. A cancellation after namespace change may also leave output visible. Neither failure nor cancellation authorizes retrying publication or deleting the destination. Explicit close/scope cleanup only closes the owned handle and removes the private name. A failed release can leave private staging files and is reported; automatic crash recovery/pruning is not implemented. Invalid decoded arguments never reach the backend; phase errors and publication execution errors are distinct from those argument checks.

Digest contexts have the same ownership, borrowing, transfer, lease and close rules, independently of files. Updates accept bounded nonempty canonical chunks; finish consumes the context, including the valid empty-input digest. Updates or another finish after finish report invalid-state; close releases the registry entry. Unsupported algorithm names return unsupported, not protocol failure. The native implementation forces each updated context so it does not retain a chain of old chunks.

Source-owned artifact policy

io_artifacts (lib/io/artifacts.tri) composes stages and digests through handlers. Its producer submits write requests and finally returns Result unit. The boundary owns the remaining count: early success, excess bytes, a producer ignoring failed writes, or an inner override forging acknowledgements cannot advance that count or authorize publication. A failed step ends the boundary without resuming the producer. The adapter's declared-length limit is 64 MiB; it is application policy, not a native staging quota. An excess chunk may have been written privately before its excess is detected (at most one bounded chunk); it is never published and is removed on scope exit.

The adapter validates the finished digest before deriving a flat destination name. An existing winner is independently read and hashed, with exact-length checking, before returning success. This is point-in-time content verification, not certification of another writer's crash durability or protection against subsequent external modification. It returns an ordinary Result digest; handler protocol failures remain terminal failures. The directory must already exist. Producer code remains trusted process-authorized code: this boundary does not constrain direct native IO or external filesystem mutation. It checks submitted chunks and acknowledgements, not arbitrary malicious programs. The producer is sequential: other requests are forwarded with lexical labels, but nested producer scope/spawn views are rejected instead of duplicating count state. The whole publication computation can itself run in a child task.

demos/io/publish.tri is a bounded file producer. The default source registry uses the streaming composition described below for socket uploads and publication.

IOStorageTests.hs independently covers SHA-256 vectors, phase/kind/bound checks, atomic no-replace races, transfer, pre/post-publication faults, cancellation during write/seal/publication, descriptor settlement, release failures and repeated scope reclamation. io_publication.py checks complete files and independent Python SHA-256 expectations through 1 MiB, Unicode paths, private file modes, existing-winner verification, corrupt winners, counter/acknowledgement controls, cancellation, and digest/path validation. It records allocation separately from the same workload-specific RSS/live-heap guards used by the first slice. Separate write/digest calls deliberately decode each chunk twice; no fusion or routing-only performance improvement is claimed.

Explicit compiler invocation boundary

The native supervisor defaults to IO and accepts an explicit --toolchain-io-version 2. Selected compilers execute with OS process authority. Canonical envelope validation precedes action demand; v1 selection requires a retained host. Returned manifests and receipts are authenticated. Startup loads and verifies the worker inertly.

python3 test/compiler_io_version.py result/bin/tricu --directory /tmp/new-compiler-version-check

Independent native workers check framing, inert startup, retired-grant rejection, invalid selections and authenticated outputs. The eleven CLI controls cover packaged and explicit selection. Full generation acceptance uses the separate reproduction workflow.

Source storage and isolated compiler projection

io_storage composes native mechanisms without adding a native store service:

• readBytes path limit returns a whole byte list, with bounded reads and a one-byte EOF probe at the budget boundary. Excess input can consume that one byte before failure. It is not framed input or a constant-memory whole-file representation.
• hashBytes limit bytes validates canonical octets and the byte budget while feeding bounded chunks to a scoped SHA-256 context. Empty bytes are valid.
• publishBytes directory destination replace limit bytes preflights before stage creation. Canonical flag 0 publishes without replacement and verifies every winner byte-for-byte through EOF, then synchronizes its open inode and directory. Flag 1 atomically replaces a mutable name after sealing; it does not race-read a subsequent legitimate alias replacement. Both return Result Boolean: false only for a verified existing immutable winner. Publication errors do not authorize rollback or retry; a destination can already be visible when synchronization fails.
• publishTree directory destination limit digest tree preflights canonical identity, uses native indexed transport, and verifies/synchronizes every no-replace winner. readTree directory path limit digest authenticates a snapshot before decoding; the explicit snapshot directory must be writable.
• parentPath is a POSIX lexical parent helper, including trailing separators and leading root separators. It does not resolve symlinks or .., validate a path, or establish confinement. Directory creation/durability remains caller policy.

Limits are canonical [0,2^63-1] encoded-byte budgets, not graph/heap or CPU quotas. Resident caller byte lists remain resident values; these helpers bound transport, not all application memory. Failures preserve the underlying operation problem or writeAll diagnostic. They do not provide a transactional whole-operation progress count or safe implicit retry. Immutable-store policy assumes cooperating writers; there is no transaction against external mutation or universal crash guarantee.

lib/source/io.tri is the compiler storage facade, imported as compiler_io. It constructs IO actions for the operations used by the source compiler. It preserves Result continuation/remainder conventions, uses a declared 64-MiB source/metadata/encoded-tree budget, and requires parent-first directory creation. Missing ancestors fail. Directory entries are synchronized before dependent publication. File-existence queries return false for missing paths and non-files, but propagate other operation errors. Indexed reads spool in the object's parent directory, suitable for the writable compiler application store. Callers must supply a writable parent directory.

The compiler source map selects compiler_io and its IO dependencies. The general application module io reexports the complete io facade.

python3 test/io_compiler_store.py result/bin/tricu \
  --directory /tmp/new-compiler-store-check --compiler-smoke

The focused Nix gate compiles current storage source, exercises exact chunk and budget boundaries, rejects malformed tails and octets before publication, preserves failed replacement targets, and checks corrupt immutable winners through EOF:

nix build .#ioStorageTest --no-link

Its cost/report.json records three-run median invocation time and cumulative allocation for 1-KiB, 4-KiB and 64-KiB hashes with exact and roomy byte budgets. The 4-KiB roomy-budget hash probe has a 160-MB allocation ceiling. The report also records 4095/4096-byte file writes; the full-bound write has a 32-MB allocation ceiling. Native acknowledgement of all 4096 bytes exhausts the validated input chunk, allowing writeAll to finish directly. Short acknowledgements retain the ordinary suffix/progress path. Wall time is reported rather than gated. This measures resident byte-list processing, not a constant- memory stream or peak RSS. To compare an explicitly selected historical storage source under the same compiler and workload, use:

nix shell --inputs-from . nixpkgs#python3 -c python3 test/io_storage_cost.py \
  result/bin/tricu --directory /tmp/storage-reference-new \
  --reference-storage /path/to/retained-storage.tri

Reference-storage runs record the source identity and bypass the candidate hash allocation ceiling. Write probes still select the current IO library and guard. The public base.length contract remains unchanged; internal storage counting operates on validated proper chunks.

Without --compiler-smoke, the test exercises actual lib/source/object.tri callers and storage policy. With it, the retained compiler builds a candidate stable sourceBuildGraph; explicit IO-version 2 selection then compiles an actual application dependency graph with cold and warm caches. Controls compare complete bundle bytes with the retained compiler, evaluate independent numeric and combining- character expectations, inspect exact raw UTF-8 provenance digests, and preserve prior output on BOM/malformed-UTF-8 failures. CRLF source rejection is compared with the retained compiler: preserving bytes is not a promise that the parser accepts every Unicode scalar as source whitespace.

Reports record projected input hashes, candidate bundle identity and IO version, commands and routine RTS observations. This procedure does not produce or accept an entire self-rebuilt compiler generation pair. Generation/reproduction metadata, backend support, deployment/default selection and rolling retention acceptance remain separate gates. Source bundle encoding and resident source/metadata values are not claimed to have become bounded-memory operations.

Artifact handler filesystem adapter

handler_artifacts.withFilesystem root byteLimit action and withFilesystemFor service root byteLimit action interpret the existing artifact request interface using native. Run the computation through handlers.runIO. prepareFilesystem root explicitly initializes the registry layout; the root's parent must already exist. Lexical overrides, named captured services and unknown-operation forwarding retain the interface's ordinary handler behavior.

The caller selects a byte limit. Heap and execution-time limits require separate controls. These operations retain whole byte lists. Loads authenticate the returned bytes against the requested raw SHA-256 name. Publication verifies and synchronizes every immutable winner through EOF; corrupt existing objects remain intact and produce a failure. Synchronization follows host-filesystem guarantees; objects can remain visible after a post-publication failure. Callers provide bundle validation and publisher authentication when required. Paths follow OS process authority. Invalid digest paths and malformed raw publication requests are rejected at the adapter boundary too.

python3 test/io_artifact_service.py "$TRICU" --directory /tmp/new-artifact-service
python3 test/handler_library.py "$TRICU" --directory /tmp/new-handler-controls

Controls include exact binary data across chunk boundaries, corrupt/trailing winners, byte budgets, startup failures, raw request validation, lexical and captured routing, forwarding, concurrent publication, stage cleanup and rejection of v1 grants before effects. These controls exercise the storage service; the HTTP listener has separate socket-level checks below.

arboricx_handlers.router root byteLimit sink method target headers body reuses registry routing/response policy with those services. packageLookup captures its root independently of artifact overrides; stdoutLog writes status-only access logs to the borrowed standard output. Each request interpretation runs in a supervised child whose join ticket is consumed. Interpretation/logger failures yield a redacted 500 after child cleanup; cancellation remains cancellation. Publication can precede either outcome and is not undone. Byte budgets also apply to captured package reads. This router expects the HTTP transport to validate framing and own the connection; it neither accepts sockets nor makes request deadlines available. The source probe checks complete HTTP bytes, captured package roots, authentication failures, logging failure after publication, ordinary status logging and cancellation.

Digest-selected streaming destinations

io_artifacts.publishWith stagingDirectory expected destination producer uses the existing sequential producer interface. After exact-length completion, digest validation and sealing, destination digest returns a handler Result containing the final path. Destination policy may initialize a digest-derived directory; it remains source code, not native upload/store policy. Callback failure abandons the stage. The existing publish helper retains its flat-directory behavior. Existing winners are length/digest verified through EOF and their opened inode and parent directory are synchronized before success.

handler_artifacts.publishStream root expected producer uses this mechanism for the registry layout: stage under root/tmp, then publish without replacement under root/objects/<first-three-digest-characters>/<digest>.arboricx. Initialization is explicit via prepareFilesystem; the temporary and object directories must be on the same filesystem. There is no whole-body buffer or second file copy. The source probe exercises new/existing winners through 1 MiB, short/excess streams, corrupted winners and private-stage cleanup. The 64-KiB/1-MiB checks retain generous 128-MiB RSS and 16-MiB live-heap regression guards for these workloads.

Native HTTP registry

io_http reuses the existing pure HTTP grammar, validation precedence, Content-Length decoding, 65,536-byte accumulated header-buffer limit (including prefetch), and 1,048,576-byte body limit. Transfer coding and duplicate lengths remain rejected. HEAD suppresses bodies only after the historical trusted parsing boundary. Each connection serves one request; surplus/pipelined bytes are ignored on close. Response writes are explicitly split into chunks of at most 4096 bytes: Each IO.writeAll call completes one bounded chunk.

arboricx_streaming connects this transport to router and publishStream. POST bundle input is sequential and acknowledged before the next read. Other request bodies, artifact downloads and package downloads remain whole-value operations whose memory cost grows with the retained values. Upload interpretation/logging runs in a supervised child and is joined before the response. Cancellation remains cancellation, and post-publication logging failure cannot undo publication. Connection ownership transfers atomically into its worker scope, which closes the socket after success, failure, reset or cancellation.

serve and serveCount take a canonical artifact byte limit (also enforced on uploads, in addition to the HTTP body limit) and an explicit connection window in [1,256]. Admission consumes the oldest join ticket before exceeding that window. This bounds active connections and retained outcomes but deliberately permits head-of-line blocking: a slow oldest connection delays new admission even if later workers finish. No request deadline, idle timeout, completion-order scheduling or slow-client availability guarantee is implemented. Excess connections wait in the OS backlog, whose capacity and behavior follow host configuration. serveCount drains its finite selection before returning; explicit scope cancellation cancels/joins active workers and cleans incomplete stages.

Build/run the native-OS-authority streaming runner:

nix build .#arboricxStreamingRunner
./result/bin/arboricx-streaming
nix build .#arboricxStreamingTest
python3 test/io_http.py /path/to/nix-built/tricu --directory /new/output

The runner binds port 8080 and uses ./store with a 16-connection FIFO window. Its packaged runtime store needs no compiler cache in the working directory. The default arboricxServerRunner and image use the same composition. Symlinks follow OS authority; wrong-digest bytes fail authentication. Deployment tests use v2 and retain retired-grant rejection controls. Public deployment is a separate operator action.

Socket controls include complete HTTP-byte expectations, binary upload boundaries through 1 MiB, authenticated downloads, corrupt winners, header/body rejection, prefetch/surplus, half-close/reset, logger failure after publication, bounded concurrent admission, repeated join-ticket consumption, startup failure, v1 grant rejection before effects, and explicit cancellation of active uploads. Streaming, admission and lifecycle jobs enforce 128-MiB RSS/16-MiB live-heap regression guards; these are test guards, not process-wide service capacity promises. The separate packaged deployment test also exercises whole-value 1-MiB download/persistence.

Monotonic observations, relative waits and scheduler yield

Native clock revision 1 provides monotonic (unit argument) and sleep (canonical milliseconds in [0,9223372036854775807]); scheduler revision 1 provides yield (unit argument). Source helpers are IO.monotonicNanoseconds, IO.sleepMilliseconds and IO.yield. All return ordinary Results. Clock samples are the host's unsigned 64-bit monotonic nanoseconds, with an unspecified local origin. Use differences within that clock domain to measure elapsed time. The host counter's representation/wrap limitations remain applicable.

Relative waits split host delays into at most one second, avoiding conversion of an enormous millisecond value to a bounded host microsecond integer. They are interruptible, may finish late, and make no hard deadline promise. Zero wait is a no-op, not an implicit scheduling barrier. Yield offers a scheduling opportunity; it does not guarantee alternation, fairness, ordering or message atomicity. No wall-clock reads, timeout/race combinator or deadline API is implied by this group.

demos/interactionTrees.tri now uses scoped children and consumed join tickets; the parent alone writes stdout. demos/interactionTrees/yield.tri installs child- local log collection and prints the collected results from the parent. This explicitly retires the old demo's incorrect claim that yield guarantees alternating writes and avoids concurrent stdout lease conflicts. Published historical fixture bundles retain their original provenance alongside current publications.

python3 test/io_clock.py result/bin/tricu --directory /new/clock-controls
python3 test/io_protocol.py result/bin/tricu --directory /new/protocol-controls

Native controls independently check capability data, canonical arguments, inert inspection, monotonic ordering, minimum elapsed sleep, exact waiter chunking and maximum-duration interruption. Actual source probes check zero/short waits, normal scope joining, explicit cancellation of a maximum-duration child wait, invalid inputs, and both migrated demos' complete output. Test timeouts are harness guards, not runtime deadline guarantees.

Direct-child subprocesses and bounded pipes (native POSIX)

IO.spawnProcess executable arguments directory environment graceMilliseconds uses the process revision 1 spawn operation. Its payload is the five-element list of those arguments; its success value is [task joinTicket stdinWriter stdoutReader stderrReader]. Pipes belong to the launching scope and support ordinary stream operations, borrowing, handoff and close. A failed/cancelled pipe operation poisons that endpoint against replay. Other runtime descriptors are not deliberately inherited (close_fds is enabled).

Directory selection is ["inherit"] or ["at" path]; environment selection is ["inherit"] or ["replace" [[key value] ...]]. Empty replacement means an empty environment. Duplicate/empty keys and keys containing = are rejected. Executable must be nonempty. Arguments are passed as a vector, not parsed as a shell command; executable lookup, script loading, path resolution and encoding follow the host process library/OS. An explicitly named shell can of course interpret its arguments. Execution uses OS process authority.

Preflight limits are 256 arguments, 256 environment entries, 4096 Unicode scalars per text field and 65536 scalars across executable, arguments, explicit cwd, keys and values. Text must be canonical, NUL-free Unicode scalars. These are not encoded OS argument-size guarantees: host limits/errors still apply. Grace is canonical milliseconds in [0,9223372036854775807], explicitly selected by the caller. Individual field/list bounds are checked before constructing host values; the aggregate bound is then checked before spawning. Excess elements beyond individual bounds are inert.

The process is monitored as an unsupervised structured child task. Normal scope exit joins it, even without consuming its ticket. Explicit IO.join returns the usual Result outcome; completion contains ["exit" code] or ["signal" number]. Exit status is data, including nonzero exits. Callers must consume and check the join outcome when command success matters. Launch errors are ordinary Results, acknowledged after internal task settlement. Unexpected monitor/cleanup failures propagate through the existing task machinery.

IO.cancel task reason, owning-scope cancellation or failure terminates the direct child with TERM, allows the selected grace, escalates to KILL if necessary, and reaps. Normal waiting stays in cancellable task work, including after the parent's body has completed. It does not block protected resource release. Status observation uses nonblocking host polling with nominal 10-ms pauses, not a detached waiter. Termination/reaping also attempts subsequent cleanup after an earlier release error; errors are retained with bounded detail and an explicit additional-error count when needed. Very long selected grace periods can delay cancellation settlement. Scheduling, OS calls and cleanup provide no hard deadline guarantee. Host process death, repeated external interruption and power loss are not cleanup guarantees.

Only the direct child is owned: no process-group signalling, descendant confinement, job control or automatic grandchild reaping is claimed. A descendant can retain a pipe writer after the direct child exits. This can prevent EOF indefinitely.

Pipe policy is explicit: close stdin to deliver EOF; drain stdout and stderr concurrently when either may fill. Waiting before doing that can deadlock. No implicit whole-output buffering, automatic input close on normal scope joining, rollback of external effects, or automatic command-success checking is supplied. The source demo demos/io/process.tri pumps a file into stdin while two scoped children drain stdout/stderr into files, then checks task completion and returns command status for its caller to judge. Files can contain partial output on failure.

nix flake check
nix build .#
python3 test/io_process.py result/bin/tricu --directory /new/process-controls
python3 test/io_protocol.py result/bin/tricu --directory /new/process-protocol
python3 test/io_tasks.py result/bin/tricu --directory /new/task-controls

Native controls cover real exit/argv/env/cwd behavior, binary backpressure, missing executables, implicit joins, failure, cancellation after observing closed scope admission with a live child, TERM-ignoring escalation/reaping, repeated lifecycle registry counts, failed-pipe poisoning, bounded/non-demanding protocol validation, and injected termination/cleanup failures and clock wrap. Source controls compare complete binary outputs through 1 MiB, including simultaneously full stdout/stderr, normal joining without an explicit ticket wait, literal arguments and nonzero status, and malformed bounds. Runtime RSS/maximum-live guards exclude child-process memory and are regression guards, not capacity or throughput promises. This slice is validated on native Linux; browser process support and scheduling changes require separate validation; it does not change compiler fixtures, deployment defaults, or accepted compiler generations.

Bounded, incremental line reading

io_text.lineCursor reader starts a single-consumer source cursor. Text.readLine cursor byteLimit returns an ordinary Result containing [event nextCursor]. Events are ["line" text "lf"], ["line" text "eof"] (for a nonempty final unterminated line), or ["eof"]. An empty LF-terminated line is a line, not EOF; a trailing LF does not manufacture an extra EOF line.

The explicit canonical limit is0..9223372036854775807 encoded bytes per line, excluding LF. CR, NUL, BOM and all valid Unicode scalars remain content; no CRLF normalization, BOM stripping or Unicode normalization occurs. This is generic text, not readSource's stricter compiler-source policy. UTF-8 decoding is strict and incremental with at most three carry bytes. Framing splits at LF before decoding, so malformed text in a later buffered line does not invalidate an earlier line. Byte bounds are checked before decoding each framed piece.

Reads request at most4093 bytes and preserve bounded read-ahead in the next cursor. Each returned line is a whole value, occupying space proportional to its selected limit; processing many lines need not retain the entire stream. This is not a constant-space arbitrary-length-line facility. No aggregate input, CPU or wall-time limit is implied. demos/io/lines.tri includes both collecting and discarding consumers; only the latter is used for repeated-line retention controls.

Thread only the returned cursor forward. Serialize cursor use with all other reads on the same reader. The cursor is ordinary source data; native leases cover individual reads. Buffered bytes and observed EOF are cached data that can survive resource closure. A fresh read requires a live reader handle. Cursor representation is an internal library detail.

Read/UTF-8/line-size failures return ["read-line" observedLineBytes cause], without a next cursor. Retire that cursor on failure or cancellation; automatic draining, resynchronization and recovery are not provided. The count includes the current framed piece on a bound/decode failure, excludes LF/read-ahead belonging to later lines, and is not a file offset or safe retry position. Native decoder byte locations are relative to that decoder invocation. The underlying stream may already have consumed read-ahead; reconstructing a fresh cursor is not lossless recovery. Invalid limits reject before reading, including when the reader is stale.

The migrated getLineAsync demo starts a child read, writes prompts from the parent, then consumes the child's join ticket. EOF and an empty name differ, and write/read failures are checked. greet and forkAwait also use IO; the parent serializes forkAwait output rather than assuming concurrent writes are atomic. Historical published v1 fixtures remain frozen alongside the current releases.

python3 test/io_lines.py result/bin/tricu --directory /new/line-controls
python3 test/io_text.py result/bin/tricu --directory /new/text-controls

Independent controls cover LF/EOF/CR/BOM/NUL, multibyte carry at the4093-byte read boundary, malformed and incomplete UTF-8, line limits, a valid line before malformed buffered data, stale readers, preflight, cancellation with stdin held open, and full interactive output. Split pipe writes exercise fragmentation but may be coalesced by the OS; stateless UTF-8 controls independently cover every split. Prompts must arrive before the harness supplies input. Repeated-line RSS/maximum-live guards are regression bounds, not capacity guarantees; whole-line collection remains bounded by the caller's selected per-line budget.

Read-only process context

IO.workingDirectory (unit payload, process/working-directory) returns Result host cwd. IO.environmentVariable name (process/environment-variable) returns Result ["missing"] or ["value" text]; an empty value is ["value" ""]. Names are nonempty, NUL-free Unicode scalar text, exclude =, and fit4096 scalars. Returned cwd/values have the same scalar bound and strict text requirements; oversized or unrepresentable values fail rather than truncate or normalize. Host acquisition may materialize a value before checking its size; this is not an allocation quota.

These are read-only, point-in-time OS observations under OS process authority. Application arguments are explicit exported-function parameters. Subprocess cwd/env selection is supplied explicitly when spawning each process.

python3 test/io_context.py result/bin/tricu --directory /new/context-controls

Native controls check actual cwd/PATH observations, malformed names/payloads, strict scalar/size validation and inert inspection. Independent source controls run in a selected cwd and distinguish absent, empty, Unicode, boundary-size and oversized environment values. These facilities expose only a requested variable; value-validation diagnostics use fixed error messages.

Basic callers: lexical configuration, state, files and bundle bytes

Text.readText reader byteLimit is a bounded whole-text convenience using the strict incremental decoder without compiler-source BOM rejection. Initial BOM, NUL and CRLF are preserved. It shares the whole-source accumulation machinery and retains the complete decoded text. readSource retains its separate raw-byte and source policy contract unchanged.

The environment demo now uses lexical configuration handlers and demonstrates restoration of the outer lexical value. The state demo explicitly passes state through a sequential state machine rather than recreating a global host get/put slot. Neither implies shared mutable cells. safeRead and shout use scoped generic UTF-8 reads with an explicit1-MiB budget. writeThenRead checks write and close results before opening the reader. Its main requires TRICU_DEMO_OUTPUT instead of overwriting a shared fixed /tmp filename; its exported run path text entrypoint also accepts an explicit path.

demos/runArboricxBundle.tri reads bounded binary bytes and uses the existing portable reader to select/apply the default export. Validation is not authentication, sandboxing, or a computation-resource quota. The portable reader permits bytes outside declared sections; its Result remainder is an internal directory cursor, not an EOF indicator. Invalid/truncated sections and trailing bytes inside the manifest section still reject. This migration does not silently tighten the independently documented container profile.

python3 test/io_callers.py result/bin/tricu --directory /new/caller-controls

Controls compare complete output/results for generic text/BOM/CRLF/NUL, missing and malformed input, read budgets, failed writes, explicit output-path selection, lexical restoration, state transitions, binary bundle execution, permitted external padding and malformed section/container negatives. Frozen historical fixtures and published package releases remain separate from these live-source migrations.

Offline v2 package inspector

lib/packages/io.tri and demos/package-inspector/ are the canonical native-v2 package facade and consumer. Reads and hashes take explicit byte budgets; the inspector selects1MiB and hashes the same byte value it validates. Native failures remain structured diagnostics. The arboricx.inspect.v1 report format is unchanged. Inspection validates graphs and exports without executing them.

nix build .# .#inspectorOracle --no-link
HOST=$(nix path-info .#)/bin/tricu
ORACLE=$(nix path-info .#inspectorOracle)/bin/inspector-oracle
nix shell --inputs-from . nixpkgs#python3 -c python3 test/library_inspector.py "$HOST" \
  --oracle "$ORACLE" --directory /new/native-package-acceptance
nix shell --inputs-from . nixpkgs#python3 -c python3 test/provenance.py "$HOST" \
  --accepted /new/native-package-acceptance --directory /new/package-replay

The first command publishes into a new local registry under default label 0.5.0-io-v2, fetches and binds a clean consumer, stops the registry, then compiles and runs offline. Complete reports are compared with the independent Haskell oracle. Controls cover malformed and unselected exports, sharing, inert IO, immutable publication, isolated dependency updates, missing files, OS permission denial, byte budgets, and rejection of both read grants and --unsafe-io. Run as a non-root user for the OS permission check. Explicit --io-version 2 is accepted; current-source publication no longer offers v1 selection.

The second command captures four input packs, removes its disposable producer, and rebuilds the facade, foundation, inspector library and consumer from retained inputs. It checks exact modules/receipts, distinct source histories, and rejects an authenticated false output claim and producer-cache poisoning.

Existing published releases retain their bytes. Acceptance writes to the caller-selected new directory. The optional --stage-site operation adds accepted bundles/releases to an exclusively owned local static-site directory; network deployment remains a separate operator action.

The remaining demos/interactionTrees/{echoServer,httpServer,arboricxServer}.tri entrypoints now explicitly construct v2 programs. Echo and simple HTTP are sequential demonstrations: one scoped connection at a time, no idle timeout or availability guarantee. Echo pumps bounded chunks until EOF instead of truncating at the first receive. HTTP retains the existing framing/body limits and HEAD rules. The registry demo uses the already validated streaming service with FIFO16 admission and a source-selected1-MiB budget. It initializes ./store, not a shared fixed /tmp/store, and announces its port only after successful startup. Each exposes a counted server entrypoint for reproducible tests; main serves forever. The packaged registry application and image also default to v2. Historical images and their authenticated inputs remain available for explicit rollback.

python3 test/io_demo_servers.py result/bin/tricu --directory /new/demo-servers checks real binary echo, HTTP method/framing/HEAD responses, binary registry publication/download, and that listeners no longer accept after their scopes finish. The larger native HTTP/streaming controls remain the service's independent coverage.

Validated program/backend boundary

IO envelope versions, compiler entrypoints and native embedding ABI versions are separate contracts. Native embedding uses Haskell ABI 1 and IO protocol 2.

Program/operation	Haskell native (validated Linux)	Haskell browser
Pure tree evaluation / stored definitions	Yes	Yes
Pure stable module-compiler entrypoint	Yes	Independently pinned browser compiler
Filesystem compiler workflow	IO	Pure compilation over supplied sources
Historical IO v1	Retained runtime required	Inert inspection only
IO envelope/core	Structured core	Structured core and terminal profile
Named facilities	Query IO.capabilities	Runtime, standard streams, strict UTF-8
Applications	Files, pipelines, tasks, processes and registry	Structured tasks and terminal pipelines

Browser terminal input uses UTF-8 line/EOF messages. Stop requests structured cancellation and waits for admitted operations, tasks and leases to settle. Emergency disposal terminates the Worker without promising resource settlement. A non-returning reduction can delay structured cancellation; external supervision bounds native work. Canonical v2 framing is checked before action demand. The JS reference graph supplies independent pure conformance checks.

Browser compilation invokes source_unit.compileStableModuleDefinitions from its independently authenticated compiler library. Published assets retain their own source and compiler provenance. Reproduce native and browser boundaries with:

nix build .#tricu-native-bindings-integration .#arboricxSite --no-link
nix shell --inputs-from . nixpkgs#nodejs -c \
  node --test ext/demo-site/test/io-version-boundary.test.mjs
SITE=$(nix path-info .#arboricxSite)
TEST_IO_VERSION=1 TEST_BACKEND=haskell-wasm ARBORICX_SITE_ROOT="$SITE" \
  nix shell --inputs-from . nixpkgs#nodejs nixpkgs#chromium -c \
    node ext/demo-site/test/backend-browser.mjs "$SITE/runtime/haskell" chromium

Independent Haskell task/resource/file/indexed/text controls retain malformed framing, non-demand, cancellation/cleanup, exact byte and publication oracles. Shared-library generation tests compare complete inventories with the CLI; full compiler-pair acceptance and retention remain separate. See the browser IO contract and compiler rebuilding.