Models and Releases

A hosted model has two immutable halves:

• The model says what to run: weights, assets, resolved configuration, contract, lineage, and a release binding.
• The runtime release says how to run it: implementation, exact code commit, image identity, GPU policy, and artifact requirements.

They are separate so the same stored weights can be republished under improved inference code without copying tensors or rewriting an existing model.

From a training step to a model

Modern training saves content-addressed blobs and four small manifests for each durable step:

Manifest	Purpose
train-full	Exact resume state, including optimizer state.
train-noopt	Model state without optimizer state.
serving-inputs	Resolved inference config, assets, prepared casts, and publishable recipes. It is an input to publication, not a model.
inference	An auto-written legacy manifest, still produced by every save today (slated for removal at cutover). It is not a published model.

The auto-written inference manifest explains two things you will observe. Run-derived publications routinely get a release-slug suffix (…inference-fmtts-r2.json): the plain candidate key is already occupied by the training-written manifest, and write-once keys never change meaning. And serve status reports orphans on freshly trained runs — those are these manifests, which the router refuses with model_not_found and reason: "unpublished" rather than treating as models.

Publishing reads serving-inputs, composes a model body, asks the target release’s live CPU functions to generate the contract and validate the manifest, then conditionally creates an immutable model manifest. For older steps saved before serving-inputs existed, FMTTS bootstraps through the same auto-written inference manifest.

Publication validation is structural and identity-focused. It validates schemas, generated spec coherence and examples, all blob:// references, and blob existence. It does not download and materialize every multi-gigabyte weight during publication; full loading and digest verification happen in runtime setup, first use, and replay drills.

Model IDs

The object-store key is the canonical model ID, including .json:

runs/<scope>/<run>/checkpoints/step-<9+ digits>.inference[-variant].json
named/<name>/<create-only-label>.json

Run-derived models stay next to their training manifests. Variants may add composition, release, or digest suffixes when a shorter candidate key is already occupied. The suffix prevents collisions; the manifest contents—not the suffix—define behavior.

Use named/ for externally trained models, compositions that do not belong to one run, or a source model already under named/. Labels are create-only:

serve publish <selection> --runtime fmtts-r7 --named my-model/v1

A model manifest contains:

• A generic envelope: schema, kind, model_type, creation time, and a content-addressed blob table.
• serving.runtime_release: release name and a denormalized source commit.
• serving.spec: the model’s generated, versioned request/result contract.
• model: operative model-specific configuration and weight overlay instructions.
• lineage: descriptive provenance such as training step, composition recipe, or republished source.
• Optional meta: display and bookkeeping data.

Runtime releases

A release descriptor is stored at releases/<name>.json. It contains:

• globally unique release name and model_type;
• implementation name, currently torch or tensorrt for FMTTS;
• repository, exact 40-character commit, and runtime/<name> reachability tag;
• identity-relevant deployment settings: snapshot mode, Python version, image-recipe version, and exact GPU architecture when artifacts require it;
• whether the implementation requires engine sets.

Release names are lowercase dash-separated words and are never reused. A descriptor is the authority; the git tag only keeps its commit reachable.

torch treats GPU choice as operational: a deploy may override it, and responses observe the actual device. tensorrt treats GPU architecture as identity because compiled engines are architecture-specific. Inside the system, H100 means exactly H100; Modal’s selector syntax is introduced only at the dispatch boundary.

Aliases

Aliases are mutable, audited names over immutable model IDs:

fmtts/dubbing-best  level 0 → <model_id>
fmtts/prod          level 1 → alias:fmtts/dubbing-best

Level 0 targets a model ID. Level N targets an existing alias at exactly N−1, up to level 8. Levels are immutable after creation. This descending structure prevents cycles even when different aliases move concurrently.

Callers should normally request an alias. Every response records the requested alias, the full resolution chain, and the resolved model ID. Durable experiment and result records should store that resolved ID, never only the alias.

The runs/<scope>/<run>/{last,best} namespace is reserved for training-step pointers and cannot enter ordinary model-alias chains.

Lifecycle

Models and releases share three durable states; releases begin in a fourth:

State	Listed by default	Existing requests	New publication against release
candidate	No	No	No; releases only.
active	Yes	Yes	Yes.
deprecated	No	Yes	No.
disabled	No	No	No.

The expected path is candidate → active → deprecated → disabled for releases and active → deprecated → disabled for models. Explicit audited reversals exist for mistakes and recovery. Every lifecycle record retains the full adjacent transition history; use serve status --history <target> to inspect it.

Deprecation is a discovery change, not a traffic cut. Disable is the fail-closed control. Unpublish is a later, irreversible identity-removal marker and requires the model to be disabled first.

Routability and readiness

A model request is allowed only when all of the following hold:

1. no removal marker exists;
2. model state is active or deprecated;
3. bound release state is active or deprecated;
4. the instance has a healthy deployment record verified against the current release descriptor;
5. every required artifact exists and is correctly bound—for TensorRT, the engine set for that model, release, and architecture.

Readiness is derived on every cache refresh; it is not a mutable flag someone can accidentally set. Missing deployment or artifacts returns not_ready, while an intentional lifecycle withdrawal returns model_disabled or release_disabled.

What does not mutate a model

• Deploying or redeploying its release.
• Moving an alias to or away from it.
• Running its weights in a scratch session.
• Republishing the same weights against another release. That creates a new model ID linked by lineage.
• Building a TensorRT engine set. The artifact gets its own immutable identity bound to the existing model and release.