Portable finite-stream parser protocol

lib/parser.tri parses a finite tricu list of arbitrary tokens. It supplies shared machinery for binary, HTTP, and source consumers. lib/binary.tri retains its legacy result-shaped, unconditionally backtracking combinators; their names and behavior differ from this API. Its readU8, expectU8, and peekU8 now use the shared byte-stream primitives internally, crossing an explicit reply-to-Result boundary. binaryReadU8Reply and binaryExpectU8Reply expose consumption-aware replies for new callers.

Representation

A parser is input -> reply. Constructors and accessors in lib/parser.tri are the interface to the following encodings; consumers should not depend on nested fst/snd paths:

reply      = pair success? (pair payload (pair rest (pair consumed? committed?)))
diagnostic = pair code (pair expected (pair found (pair location contexts)))

payload is the parsed value on success and a diagnostic on failure. A diagnostic field of t means unspecified; contexts are a list ordered from outermost to innermost. The generic parser supplies no positions; adapters may put a source span or byte position in location. Error codes currently used by generic primitives: 1 unexpected EOF, 2 unexpected token, 3 non-consuming repetition. parserFailure takes a structured diagnostic. parserExpect puts the expected token into the diagnostic; parserSatisfy leaves its expected field unspecified.

The flags describe the whole parser invocation. Consumption means actual stream advancement; commitment is a separate flag. A parser that claims consumption must have advanced the stream. An unconsumed failure normally retains the original input as its remainder. parserAttempt restores that input on failure, but retains the diagnostic (including its location).

Choice and sequencing

• parserBind combines consumption and commitment of both stages with OR, including when the second stage fails.
• parserCommit succeeds without consuming and marks the invocation committed. A subsequent failure within a sequence inherits that commitment.
• parserOr runs its right alternative on the original input only when the left fails without consumption or commitment. Otherwise it returns the left reply. An unconsumed failure from the right is returned as-is.
• parserAttempt leaves successes intact; on failure, it restores the input and clears both flags. It cannot undo commitment established outside its scope.
• parserLabel replaces the expected field on failure, and parserContext prepends an enclosing context. Neither changes flags or remainder.
• parserOptional is choice with a pure nothing alternative: committed or consumed failures are not silently converted to absence.

Repetition

parserMany and parserSome reject every zero-width successful iteration with code 3, including an iteration that committed without consuming. An unconsumed, uncommitted failure ends repetition successfully; all other failures propagate, preserving prior consumption and commitment. parserSepBy and parserSepBy1 do not permit a trailing separator: consuming the separator and then failing to parse the element propagates a consumed failure. parserSepBy accepts an empty sequence only if its first element fails without consuming or committing; a later repetition failure (including zero-width success) is never converted into an empty sequence.

parserMergeFlags is the common reply adapter used by bind and lexer composition; it preserves the selected payload/remainder while combining progress flags.

parserMatchReply matches only the status projection. It does not eagerly destructure the entire reply: a handler may ignore payload, remainder or flags, even when those fields would diverge. Non-Leaf statuses retain the library's existing truthiness. Handlers can therefore preserve opaque or divergent unused fields.

Binary migration is one-way at this boundary: binaryLegacyReply codeOf reply projects a generic reply into the legacy Result (value, remainder) and retains the generic remainder; binaryLegacyRollback codeOf input reply restores the original input on failure, matching the historical expectation parsers. Both map structured diagnostics to legacy numeric codes explicitly. They do not translate legacy Result failures into generic replies: legacy parsers may already have restored input after consuming bytes, so their Result alone cannot justify a consumed flag. Legacy orParser still retries after every failure; new callers use parserOr, adding parserAttempt only at chosen rollback boundaries. Fixed-width byte reads and existing Arboricx consumers remain on the legacy side until separately migrated.

lib/http.tri is another non-compiler consumer. httpLineReply requires CRLF: truncation and bare CR/LF fail rather than acting as successful terminators. httpThreeWordsReply retains a request-line context; httpHeaderLineReply validates token names, the colon, and field-value bytes, rejecting folded headers and preserving opaque obs-text/body suffixes. The HTTP boundary translates parser failures to its Result/400 protocol.

Content-Length uses a separate bounded native-natural decoder: malformed digits are 400, valid oversized values are 413, including overflow followed by a scan for malformed trailing bytes. Leading zeroes are permitted. Duplicate lengths and Transfer-Encoding are rejected before decoding. Generic parser modernization does not change socket receive/framing or body-limit policies.

Generic choice returns the selected diagnostic as supplied. Position-aware consumers provide their own location and context adapters.

HTTP framing and receive costs

The HTTP receiver caps the entire accumulated receive buffer, including any body bytes prefetched in the final chunk. Delimiter detection preserves the exact suffix. The current accumulation/search path can repeat work as chunks arrive. See HTTP pipeline checks for coverage and cost measurements when changing this path.