CLI Reference

Run model-aware commands through the serve helper from Get Started:

serve <command> [arguments]

The underlying entry point is uv run --project models/fmtts --extra serve python -m fmtts.serving. Every command that touches the store verifies RND_SERVING_INSTANCE against the store’s instance.json before reading or writing — except init-instance, which creates that identity. implementations reads only the local implementation table, so it neither needs credentials nor proves you have them; serve status is the real access check.

Discovery and instance

implementations

List the FMTTS implementation table, artifact policy, and GPU-identity policy.

serve implementations

status

Read-only instance report, including remote tag checks:

serve status
serve status --history <release-or-model-id>

init-instance

Stamp an empty store with write-once identity. This is the only command that does not pre-verify an instance:

serve init-instance --name <name> --purpose {production,dev} [--notes <text>]

Run it exactly once on a genuinely new store. It does not create a bucket, Modal environment, secrets, or router.

Releases

mint

serve mint <release> \
  --implementation <name> \
  [--gpu H100] \
  [--snapshot off|cpu|gpu]

Defaults are GPU H100 and snapshot gpu. Mint always pins the current HEAD — to release a historical commit, check it out first. It creates and pushes runtime/<release>, writes the immutable descriptor, and creates candidate lifecycle state. Release names are globally unique and never reused.

deploy

serve deploy <release> \
  [--no-activate] \
  [--repair] \
  [--gpu <arch>] \
  [--scaledown <seconds>] \
  [--max-inputs <positive-int>] \
  [--target-inputs <positive-int>] \
  [--max-containers <positive-int>] \
  [--batch-wait-ms <non-negative-int>] \
  [--no-warmup]

• --no-activate leaves a candidate after successful verification.
• --repair explicitly replaces a mismatched app from the pin and audits the repair.
• --gpu is an operational override only for implementations where GPU is not identity, currently Torch.
• --target-inputs is validated against --max-inputs. Values above 1 — and --max-containers / --batch-wait-ms — require a release minted from serving runtime v2 or newer, whose process-local batcher coalesces concurrent requests across callers; a release pinning runtime v1 (which serializes GPU passes) refuses them.
• --no-warmup skips the runtime’s setup synthesis.

The internal deploy-pinned command is executed only by deploy inside a materialized checkout. Do not call it directly.

Scratch

scratch start

serve scratch start <selection> \
  [--name <display-label>] \
  [--implementation torch|tensorrt] \
  [--gpu H100] \
  [--recipe key=value,key=value] \
  [--watch]

selection is <run>/<step-ref> or a full model-manifest key. Display labels are non-unique metadata; the returned UUID is canonical.

scratch watch

serve scratch watch <session-uuid> \
  [--implementation <name>] [--gpu H100]

Reconnect to the recorded implementation and deployed GPU. Reconnect logic intentionally takes the implementation from the session record rather than changing it from this invocation; use a new session to change implementations.

scratch list

serve scratch list

Lists active sessions, current generation, and expiry.

scratch stop

serve scratch stop <session-uuid>

Closes the session terminally and attempts to stop every revision app. The router reaper retries cleanup failures.

Publication

publish

Publish one or more training selections or existing model IDs:

serve publish <selection> [<selection> ...] \
  --runtime <release> \
  [--recipe key=value,key=value] \
  [--named NAME/LABEL]

--named applies to exactly one publication and works with any selection form. It is required when republishing an existing named model because the new model needs a fresh create-only label.

--recipe applies only to training selections. Republishing an existing model ID refuses it — a republication carries the source model’s composition with its blobs, so there is nothing left for a recipe to select. Publish from the run and step instead.

Promote a scratch revision instead of passing selections:

serve publish \
  --from-scratch <session-uuid> \
  --runtime <release> \
  [--revision <revision-uuid>] \
  [--named NAME/LABEL]

Selections and --from-scratch are mutually exclusive and exactly one form is required. --from-scratch also refuses --recipe: promotion recomposes the session’s own recorded recipe, fixed when the session started.

The internal pin-identity --implementation <name> command is used by promotion to derive identity digests from a materialized release pin. Do not use it as a deployment or publication command.

TensorRT engines

engines build

serve engines build <model-id> --release <release> [--force]

Build from the release’s pin on the descriptor’s exact GPU, upload plans, and create the immutable engine-set manifest. --force bypasses the builder Volume cache.

engines import

serve engines import <directory> <model-id> \
  --release <release> \
  --source-commit <40-hex-sha> \
  [--gpu-arch H100]

Import a runtime-loadable local engine set after verifying its build identity. The source commit must equal the release pin.

Replay drills

Both commands talk to the live router and need SERVING_ROUTER_URL and SERVING_API_TOKENS (the serve helper supplies them).

replay bless

serve replay bless <model-id> --request <request.json>

Sends the request through the router, verifies the attestation, and stores the request verbatim with digests of the returned audio as a write-once pair under replays/. One pair per (model, request digest).

replay run

serve replay run [--model <model-id>]

Re-sends every blessed pair (or one model’s) and classifies each outcome: exact match, platform_drift (same identity, different bytes), model_change, identity_failure, or missing_artifact. Exits non-zero on any finding, including expected platform_drift — account for that before wiring it into a scheduled job. See Replay drills.

Lifecycle

deprecate

serve deprecate <target> [<target> ...] \
  [--successor <same-kind-target>] [--why <reason>]

Targets are release names or full model IDs. Deprecation hides but does not stop existing requests.

disable

serve disable <target> \
  [--successor <same-kind-target>] [--why <reason>]

Makes a model or release unroutable. A release disable also invalidates its deployment record.

reactivate

serve reactivate <target>

Releases require a current verified healthy deployment. After disabling a release, run deploy first and reactivate second.

unpublish

serve unpublish <model-id> [<model-id> ...] [--why <reason>]

Requires disabled models and creates irreversible write-once removal markers.

Aliases

alias set

serve alias set <name> --level <0..8> <target> [--why <reason>]

Level 0 targets a model ID. Level N targets alias:<name> at exactly N−1. set creates an alias or moves an existing alias only when the supplied level matches its immutable level.

alias move

serve alias move <name> <target> [--why <reason>]

Moves an existing alias without restating its level.

Generic model-store commands

Run these under storage credentials from the rnd checkout:

./with_storage_env.sh uv run --project models/fmtts \
  python -m serving.model_store [--store <url>] <command>

Command	Effect
ls [run]	List manifests without downloading blobs.
show <key>	Print one manifest.
verify <key>	Download and SHA-256 every referenced blob.
trim <run> [--keep N] [--step N] [--kind KIND]	Delete selected non-servable manifests; defaults to old train-full manifests.
delete-step <run> <step>	Delete all deletable manifests at one training step.

Ad-hoc checkpoint transfer and legacy catalog commands live at python -m serving.checkpoint_store:

Command	Effect
ls [prefix]	List raw keys.
put <local-path> <key>	Upload a file.
get <key> <local-path>	Download a file.
catalog [--run RUN] [--json]	List legacy training-checkpoint metadata.

These raw transfer commands do not create hosted model lifecycle or release validation. Use serve publish for hosted models.