Preview and Release

This is the normal path from inference work in a branch to a callable hosted model. The examples use the serve shell helper from Get Started.

1. Resolve weights and start scratch

Choose an existing training step or model and start with Torch unless the change itself is TensorRT-specific — a TensorRT scratch session rebuilds its full engine set (~25 H100-minutes) on every source revision:

serve scratch start <run>/120000 \
  --implementation torch \
  --gpu H100 \
  --name "$USER/duration-fix" \
  --watch

The selection is resolved once. Record the printed session UUID; it is the only stable scratch target.

If you need a non-default stored composition, add a recipe:

serve scratch start <run>/120000 \
  --recipe averaging=ema_h1pct,dtype=bf16 \
  --name "$USER/ema-check" \
  --watch

Recipes apply to training-step sources. A published model’s composition is already fixed; start from its underlying step to compose it differently.

2. Invoke and iterate

Fetch the current draft contract:

curl --fail-with-body --silent --show-error \
  -H "Authorization: Bearer $SERVING_API_TOKEN" \
  "$SERVING_ROUTER_URL/scratch-sessions/<session-uuid>" | jq

Invoke /scratch-sessions/<session-uuid>/invoke using the returned spec. Each successful watched source update creates and verifies a new generation before the session pointer moves.

Ctrl-C detaches the watcher without closing the session. Use serve scratch watch <session-uuid> to reconnect, or serve scratch stop <session-uuid> when the work is no longer needed.

3. Commit the exact source you tested

Promotion compares the selected scratch revision’s source and dependency digests with the release pin. Commit the source that produced the result you intend to keep:

git status --short
git add <serving-source-paths>
git commit -m 'fmtts: describe the inference change'

Do not include unrelated work merely to make the tree globally clean. Minting HEAD checks the implementation’s serving-relevant source, builder inputs, and installed requirements lock; dirt elsewhere does not block it.

4. Mint a release

Choose a globally unique, lowercase dash-separated release name:

serve mint fmtts-r7 --implementation torch --gpu H100 --snapshot gpu

Minting:

1. validates the implementation and release app name;
2. checks serving-relevant files are committed when pinning HEAD;
3. creates and pushes runtime/fmtts-r7, which makes the commit reachable;
4. conditionally writes releases/fmtts-r7.json;
5. creates its candidate lifecycle record.

Mint always pins the current HEAD; releasing a historical commit means checking it out first, so the cleanliness check always applies to what you are actually releasing. --snapshot accepts off, cpu, or gpu; it is identity-relevant, so changing it later means another release.

5. Deploy and activate

serve deploy fmtts-r7

Deploy materializes the descriptor’s pin in a temporary checkout, executes the pinned model-owned entry point, builds the Modal image, verifies the app’s baked identity, writes a healthy deployment record, and moves a candidate to active.

Use --no-activate to leave a verified candidate for direct inspection. Publication refuses candidates; rerun ordinary deploy when it is ready to activate.

Operational settings can change on a redeploy without a new release:

serve deploy fmtts-r7 --scaledown 300 --max-inputs 8

Torch also accepts --gpu as an operational override. TensorRT does not: its GPU is part of artifact identity. Releases minted from serving runtime v2 batch concurrent requests across callers on one GPU: --target-inputs above 1 sets the autoscaler target, --batch-wait-ms bounds the coalescing wait, and --max-containers caps fan-out. A release pinning runtime v1 serializes GPU passes and refuses those controls; max_inputs above 1 still overlaps CPU work and absorbs bursts there.

6. Publish the model

If the exact implementation, source, dependencies, weights, and recipe match the scratch revision, promote it:

serve publish --from-scratch <session-uuid> --runtime fmtts-r7

Promotion publishes an ordinary model, records the selected revision mapping, and closes the session. Use --revision <revision-uuid> if the session’s current revision is not the one you selected.

Otherwise publish the source normally:

serve publish <run>/120000 --runtime fmtts-r7

serve publish <run>/120000 \
  --runtime fmtts-r7 \
  --recipe averaging=ema_h1pct,dtype=bf16

You can also republish existing immutable models under new code, reusing all of their blobs:

serve publish <model-id-1> <model-id-2> --runtime fmtts-r7

Named-model sources require a new create-only label:

serve publish named/voice/v1.json \
  --runtime fmtts-r7 \
  --named voice/v2

Publication is usually seconds with no tensor transfer when the selected recipe is already prepared. A derivable cast may need a one-time download and conversion.

7. Build TensorRT artifacts when required

For a TensorRT release, a published model exists before its engines do and reports not_ready: engine_set(H100) until the exact artifact is present:

serve engines build <new-model-id> --release fmtts-r7-trt

The builder runs from the release pin on its exact GPU, uploads plans as content-addressed blobs, and creates an immutable engine-set manifest bound to the model manifest digest, release descriptor digest, builder commit, and architecture.

8. Create or move an alias

Create a level-0 caller-facing alias:

serve alias set fmtts/my-model --level 0 <new-model-id> \
  --why 'initial release'

Or move an existing alias without restating its immutable level:

serve alias move fmtts/my-model <new-model-id> \
  --why 'validated fmtts-r7 publication'

9. Verify the finished path

serve status

curl --fail-with-body --silent --show-error \
  -H "Authorization: Bearer $SERVING_API_TOKEN" \
  "$SERVING_ROUTER_URL/models/fmtts/my-model" | jq

The model should report servable: true. Invoke the alias and retain the returned identity block with the output; that is the proof of what ran.