semif-api-rocm

SemIf HTTP API and rocm flake
Log | Files | Refs | README | LICENSE

README.md (3635B)

semif-api

HTTP API wrapper around TheoLeeCJ/SemIf, consumed as a library (semif_phase1) — no scorer code is copied. See ../README.md for the quickstart and ../docs/llama-backend.md for the GGUF backend (SEMIF_BACKEND=llama) and its NixOS options; the notes below cover the torch path.

Run

nix develop <repo root>
semif-serve                      # 127.0.0.1:8321, or SEMIF_HOST/SEMIF_PORT

The torch model is loaded once at startup (env overrides: SEMIF_MODEL, SEMIF_REVISION, SEMIF_MAX_TOKENS).

API

The web UI rides in the wheel as package data (src/semif_api/web/) and is served same-origin at /ui/ — no CORS.

GET /healthz

Model metadata, backend, backend_status, and model_state (pending = selected GGUF not yet confirmed by a score, loaded).

GET /models / POST /models — llama backend only

List the llama-server's model aliases, or switch with {"model": "<alias>"}. The alias is validated before the current model is unloaded; unloading is best-effort (failure returns a warning, and the old model may stay resident until evicted). The new GGUF loads lazily on the first scored request; a restart restores the configured default. Unknown alias → 422, server failure → 502. Torch returns switching: false.

POST /decide — one decision

{
  "id": "support-1",
  "state": "The deployment completed at 14:02 UTC...",
  "question": "Is there evidence that the deployment succeeded?",
  "options": [
    {"id": "yes", "description": "The deployment succeeded."},
    {"id": "no", "description": "The deployment did not succeed."},
    {"id": "insufficient", "description": "The evidence is insufficient to decide."}
  ]
}

→ scorer result: option_ids, probabilities (conditional option scores — uncalibrated as decision confidence), option_logits, prompt_sha256, model/revision metadata.

POST /decide-batch — one shared state, many decisions

{
  "state": "...shared evidence...",
  "decisions": [{"id": "...", "question": "...", "options": [...]}, ...]
}

→ {"results": [...], "timing": {...}}, each result tagged shared_timing. Every decision in a batch shares the exact same state. On llama, rows run sequentially (timing.mode: llama-sequential) with server prefix caching.

POST /plan — llama backend only

Rule generation from an action transcript; torch refuses with 422 (it only reads logits). Game-agnostic: the caller composes prompt from facts it owns (goal, what triggered planning, rules currently in effect) plus transcript turns (system|user|assistant). One chat completion under a universal system prompt with thinking enabled; sampling is pinned to the model card's thinking-mode settings.

{
  "id": "self-plan-1",
  "prompt": "Objective: ...\nTrigger: ...\nPrevious rules (the actor failed despite them — amend or replace):\n...",
  "transcript": [{"role": "user", "content": "Observation: …\nChosen action: run\nOutcome: ..."}]
}

→ {"id", "rules", "reasoning", "truncated", "usage", "model", "total_seconds"}. rules is injected into later decision states; reasoning is the thinking trace when the server surfaces one. See ../docs/games.md for the demos built on this.

Notes