# Module System and Content Store Design

Status: design draft.

This document records the intended direction for reworking `tricu` modules,
imports, Arboricx storage/transport, and the content store. It is not an
implementation plan yet; it is a shared design target.

## 1. Problem Statement

The current module/import/content-store system is useful as a prototype, but it
is not coherent enough to build on indefinitely.

Current behavior combines several partially-overlapping systems:

- `!import "path.tri" Namespace` and `!import "path.tri" !Local` perform
  filesystem-relative source preprocessing;
- imported definitions are flattened into one program;
- namespace qualification is implemented by string rewriting;
- evaluation uses a flat `Map String T` environment;
- the Haskell content store stores Tree Calculus Merkle nodes plus an ad hoc
  `terms` table with comma-separated names and tags;
- the REPL can resolve names from the content store, including multiple versions;
- Arboricx bundles provide compact indexed transport objects;
- `lib/arboricx/server.tri` already sketches a filesystem-backed object store.

This works only when users and maintainers are mindful of sharp edges:

- names serve too many roles at once;
- modules are not first-class semantic objects;
- imports are closer to AST paste-and-prefix than resolution;
- `!Local` imports can create global collisions;
- content identity, human aliases, source files, and evaluated terms are not
  cleanly separated;
- the SQLite schema is convenient but not a principled content-addressed store;
- Arboricx transport and long-lived storage are not clearly distinguished.

## 2. Design Principles

### 2.1 Content addressability is foundational

Immutable content should be identified by hashes. Human names should be metadata
or workspace aliases over content, not semantic identity.

This follows the core lesson from systems such as Unison: separate stable
content identity from ergonomic naming and namespace organization.

### 2.2 The content store is language-neutral

The content store must not be married to `tricu` or Haskell.

It stores a small set of portable Arboricx artifacts: module manifests,
complete tree terms, and direct Contract terms. Lower-level Merkle/bundle
formats exist for transport and DAG tooling, but the store core should treat all
objects as content-addressed bytes with formats/media types.

`tricu` and Haskell are clients/tooling. They are not the semantic owners of the
store.

### 2.3 Contracts are portable enough to integrate

The store may integrate with Contracts because a contract is itself an ordinary
Tree Calculus term. Contracts are not a Haskell-private or `tricu`-private
semantic layer.

A module manifest may reference a contract object, but the contract is evaluated
by ordinary Tree Calculus reduction, not by a special checker.

### 2.4 Modules should reflect definitions as they actually exist

The module system should conform to the reality of content-addressed immutable
artifacts and mutable human aliases. We should not contort definitions to fit a
traditional text-file module system if that fights the storage model.

### 2.5 Transport and storage are different jobs

Indexed Arboricx bundles are excellent transport/execution objects. Merkle DAGs
are better long-lived persistence objects. These should remain separate but
interoperable representations.

## 3. Conceptual Architecture

```text
Content Store
  neutral content-addressed object store

Arboricx CAS / Merkle Store
  Tree Calculus node/object formats suitable for persistence and dedupe

Arboricx Bundle
  compact indexed transport/execution format

Contract Term
  ordinary tree-valued contract function, applied by reduction

Module Manifest
  immutable export map from names to content objects and optional contract terms

Workspace
  mutable aliases, selected versions, package pins, and user-facing names

tricu
  one frontend/toolchain that emits/consumes these portable artifacts
```

The content store stores objects. Arboricx defines important object formats.
Contracts are ordinary tree-valued functions; `tricu` produces and consumes those
formats.

### 3.1 Execution imports versus contract checking

Import resolution has two intentionally different performance profiles.

For normal execution/evaluation, resolving a module import should hydrate only
the executable exports directly demanded by the importing source. Exported Tree
Calculus values are complete normal forms: importing `foo` does not require
hydrating separate `bar` or `baz` exports that may have helped build it. This is
the fast path for `!import`, including `!Local` imports.

Contract checking is a runtime boundary check. It may load exported contract term
objects for the symbols that participate in a boundary. That slower path remains
separate from execution hydration:

```text
Haskell loads contract terms.
tricu applies them by reduction.
```

Reusable contract catalogs are ordinary `tricu` libraries/tree terms, not a separate
core CAS artifact kind.

For locally built workspace modules, advertised direct export contracts may be
checked before the manifest alias is written. Producer checking includes
advertised contracts from any imported modules used by that source, so a module
cannot publish a local annotated export that contradicts a dependency's exported
contract. If producer checking fails, the module alias is not written.

Consumer checking then resolves selected module exports and loads their exported
contract term objects. Those contracts are applied at the import boundary using
the standard contract helpers. For external or prebuilt manifests, the advertised
contract is a trusted boundary declaration; the consumer may still re-apply it at
the boundary.

## 4. Content Store Direction

### 4.1 Store core

The store core should be a content-addressed object store:

```text
hash -> object bytes
hash -> object kind / media type
hash -> optional metadata/index entries
```

The hash should be over canonical bytes with domain separation. The object kind
or media type determines how a client interprets those bytes.

Current module/check object kinds:

```text
arboricx.module-manifest.v1
arboricx.tree-term.v1
arboricx.tree-term.v1
```

Merkle nodes and indexed bundles remain lower-level Arboricx transport/DAG
formats, but they are not the module/eval storage model. typed programs and contract
catalogs are ordinary tree terms unless a future external tooling use case proves
that they need their own object kind.

The store core should not need to know what a `tricu` definition means.

### 4.2 Filesystem-backed layout

The long-term store should converge with the direction already sketched in
`lib/arboricx/server.tri`:

```text
store/
  objects/
    abc/
      abc123...object
  aliases/
    names/
    modules/
    packages/
  manifests/
  tmp/
```

SQLite may remain useful as an optional index/cache, but it should not be the
canonical store model.

### 4.3 Structural references, not language dependencies

The store may understand structural content references when they are part of an
object format. For example, a Merkle node naturally references child hashes:

```text
Leaf
Stem childHash
Fork leftHash rightHash
```

This is not a `tricu` dependency graph. It is content structure.

Language/tool-level relationships such as "compiled from source", "exported by
module", or "checked with contract" can live in manifests or indexes. They
should not be required by the store core.

## 5. Arboricx Role

Arboricx should be understood as a family of portable Tree Calculus artifact
formats, not as a single storage mechanism.

### 5.1 Arboricx Bundle

The existing indexed `.arboricx` format remains the preferred transport and
execution object:

- compact;
- self-contained;
- deterministic;
- easy to parse in constrained runtimes;
- suitable for deployment and HTTP serving;
- structurally verifiable without hash recomputation per node.

It says:

```text
Here is everything you need, densely packed.
```

### 5.2 Arboricx CAS / Merkle Store

The persistent store should use content-addressed structural objects:

```text
Leaf
Stem childHash
Fork leftHash rightHash
```

This enables dedupe across definitions, modules, packages, and versions. A large
program that shares subtrees with other programs should not store those subtrees
multiple times.

It says:

```text
Here are immutable objects, addressable independently.
```

### 5.3 Pack and unpack

Transport and storage should interoperate explicitly:

```text
CAS root(s) -> pack   -> indexed Arboricx bundle
Arboricx bundle -> unpack -> CAS root(s)
```

The bundle can be treated as an opaque content-addressed blob by the store, and
it can also be unpacked into Merkle nodes for dedupe and partial reuse.

## 6. Modules

### 6.1 Module identity

A module should be an immutable manifest object. Its identity is the hash of its
canonical manifest bytes.

A module name is not identity. It is a workspace alias or package-level alias to
a module hash.

### 6.2 Module contents

A module manifest should primarily be an export map:

```text
module hash
exports:
  name -> content reference
metadata:
  package
  version
  description
  license
  createdBy
optional:
  contract artifact refs
  ABI/media type info
  source/provenance refs
```

The manifest should be portable and mostly format-oriented. It should not depend
on Haskell data structures or `tricu`-specific internal semantics.

### 6.3 Export entries

An export entry may eventually look conceptually like:

```text
name: "map"
object: sha256:...
kind: arboricx.tree-term.v1
abi: arboricx.abi.tree.v1
contract: sha256:...       -- optional contract term
source: sha256:...     -- optional source/provenance object
```

Executable module exports are complete normalized tree terms stored as one
`arboricx.tree-term.v1` object per named export. Merkle-node storage remains
available for DAG-oriented tooling, but module/eval imports should not store or
hydrate every subtree as a separate filesystem object.

### 6.4 Import behavior

Imports should resolve module aliases or content references to module manifests,
then bind selected exports into the local source scope.

Export selection has one intentional aggregator special case:

```text
module with local top-level definitions -> exports only those local definitions
module with only imports                -> reexports the evaluated import env
```

This lets files such as `prelude.tri` act as explicit barrel modules without
making every ordinary module reexport its imports. A module that defines even one
local top-level name does not implicitly reexport imported names.

The future pipeline should be:

```text
parse source
resolve imports/names to module exports and content refs
lower source using resolved refs
emit a contract artifact
check evidence when requested
store/export artifacts
```

It should not be:

```text
paste imported ASTs into one file and rewrite strings
```

## 7. Workspace Layer

Mutable human-facing state belongs in a workspace layer.

Examples:

```text
List        -> module hash
Http        -> module hash
map         -> definition/tree hash
selected List version -> module hash
package pin prelude -> package/module hash
```

The workspace is where names, selections, pins, and aliases live. Renaming should
usually mutate workspace aliases, not immutable content objects.

This gives humans stable ergonomic names without making names semantic identity.

## 8. Definition Identity

There are two useful identities and we should support both.

### 8.1 Tree identity

A Tree Calculus value has a Merkle root hash. This identifies the executable tree
itself.

This is the right identity for:

- execution;
- dedupe;
- bundle roots;
- low-level artifact sharing.

### 8.2 Module/export identity

The module manifest is the higher-level artifact boundary. It pairs each export
name with its compiled tree term and an optional contract term reference.

The content store should not require extra definition/source/provenance objects,
and fully untyped Tree Calculus code must remain valid.

## 9. Contract Integration

Contracts attach to modules/exports as ordinary tree-term objects. A contract is
a `tricu` function `Tree -> Result Tree Tree`; it is not a special artifact
kind and it does not require a separate checker binary.

A module manifest pairs each export name with its compiled tree term and an
optional contract term reference. The importer loads the contract object and
applies it at the boundary.

An imported definition can be assigned a local name while lowering source. Its
global identity remains a content hash or module export ref. The intended split
is:

```text
Local source name: "List.map"
Resolved object:    sha256:...
Exported contract:  sha256:...
```

There is no typed-program evidence graph and no local numeric checker symbols.
The contract term itself is the authority.

Untyped code remains valid with no contract artifact. If a boundary has no
contract information, it simply performs no runtime check. We should not pretend
that untyped functions have an implicit `Any -> Any -> ...` contract.

## 10. Import Syntax Direction

Exact syntax is future work, but the current `!import` form should be considered
a transitional mechanism.

Future imports should distinguish:

- path-based source imports for local development;
- workspace/module alias imports;
- explicit content-addressed imports;
- selected/exposed names;
- qualified versus unqualified binding.

Possible directions:

```tri
import "./list.tri" as List
import List exposing (map foldl)
import #abc123... as List
```

The syntax should be designed after the object/module model is clearer.

## 11. Migration Strategy

A plausible migration path:

1. Define the neutral object store model and filesystem layout.
2. Implement Merkle node persistence against that layout.
3. Add pack/unpack between CAS roots and indexed Arboricx bundles.
4. Replace ad hoc SQLite `terms` names/tags with workspace aliases or a clearer
   index layer.
5. Define module manifest objects.
6. Teach source imports to resolve manifests/exports instead of rewriting ASTs.
7. Attach contract terms to module exports.
8. Gradually migrate existing `lib/` and `demos/` imports.

Compatibility shims may keep existing `!import` working during migration.

## 12. Open Questions

- What exact canonical byte format should store objects use?
- Should module manifests be binary, tree-encoded, or both?
- What media type/kind registry do we need first?
- How should object references be represented in source syntax?
- How should workspaces be stored and shared?
- What is the minimum useful module manifest?
- Should source files compile directly to module manifests, or should manifests
  be produced by explicit package commands?
- How much Arboricx bundle metadata should reference CAS roots?
- What GC/reachability model should the store eventually use?

## 13. Summary

The desired design is:

```text
Content store:
  portable CAS for immutable objects and structural references

Arboricx bundle:
  compact indexed transport/execution object

Arboricx CAS:
  persistent Merkle DAG/object representation for dedupe and partial reuse

Modules:
  immutable manifests mapping export names to content objects and optional
  contracts

Workspace:
  mutable human aliases, version selections, and package/module pins

Contracts:
  ordinary tree-valued functions attached to exports and applied by pure Tree
  Calculus reduction at boundaries
```

The key architectural rule is that hashes provide stable identity, while names
provide human usability. The module system should be built on that separation.