Add a Model Type

The generic serving project knows storage, identity, lifecycle, deployment records, aliases, scratch orchestration, publication, readiness, routing, and attestation. It must not import model code. Each model project owns the wrappers and manifest interpretation that require model knowledge.

FMTTS under models/fmtts/fmtts/serving/ is the working reference. Add a new model type inside its own project and pass its implementation records into the public generic APIs; do not create a dependency from top-level serving/ into models/.

1. Define an implementation table

Expose a plain mapping from stable implementation names to records. Each record declares:

Field	Contract
name	Stable string stored in the release descriptor.
engine_sets	"none" or "required"; minting and readiness enforce agreement.
gpu_is_identity	Whether deploy-time GPU overrides are forbidden. Engine-bearing implementations normally set this true.
load_session(store, context, manifest, label, cache_dir)	Verify and load the model, returning (session, engine_set_sha256_or_none). A successful return must be safe to attest.

Do not infer artifact policy from implementation names, GPU flags, or files on a cache volume. The record is the single declaration.

2. Define one image recipe

Release and scratch packaging must be two modes over one recipe:

• an immutable base-image digest;
• reproducible OS packages, currently resolved from a dated Debian snapshot;
• a complete pinned Python requirements lock;
• a declared exact source set added last.

Release deployment bakes source into the final layer. Scratch attaches the same source set outside the dependency image. Dependency-recipe and source digests are separate so promotion can prove both.

Include every non-Python file needed when Modal re-imports the module inside the container: requirements locks, store schemas, model configs, or compiled-extension metadata. Identity values must be baked into the image environment. Deploy-shell environment variables are absent during container re-import and must never become hidden identity inputs.

3. Generate the contract from code

Implement a pure function equivalent to FMTTS’s generate_spec(manifest_body). It returns the model-specific request and result contract at serving.spec.

The release app must expose a CPU function that generates this spec from a draft body and another that validates a complete model manifest. Publication calls the live pinned release functions before it writes anything.

Validation should check, without allocating a GPU:

• model-manifest and model-owned schema;
• model type and release binding;
• every config reference against the blob table;
• blob existence (HEAD checks only — publication verifies no digests and moves no tensor bytes);
• regenerated spec equality, static contract coherence, and all worked examples.

Do not promise full weight materialization at publication time. The runtime loader must repeat content and identity verification when it fetches and loads bytes.

4. Implement the session seam

The loaded session must provide the model behavior used by the wrapper. FMTTS exposes:

• synthesize(...) for GPU/model work;
• to_wav_bytes(...) and sr for CPU result formatting.

Runtime apps are @modal.concurrent, but sessions are not assumed thread-safe. Put a single engine object between every Modal input and the underlying session. The current FMTTS GpuEngine serializes GPU passes while allowing request validation, audio decode/encode, and uploads to overlap.

target_inputs may exceed 1 only when the engine truly batches across requests — serving runtime v2’s process-local batcher is the FMTTS example, and deploy tooling refuses the batching controls for releases pinning runtime v1, where a raised autoscaler target would park long calls behind one another. The batcher lives inside this engine seam; the router and app protocol stay per-call request/response.

5. Implement release and scratch apps

The release app must provide:

• a parameterized runtime class, currently one pool per (model_id, manifest_sha256);
• synthesize(request) and synthesize_to_urls(request, output_targets) methods returning typed envelopes;
• a CPU runtime_metadata() identity probe;
• CPU spec-generation and publication-validation functions;
• no public URL—the router is the only HTTP front door.

The scratch app uses one app per revision, carries no model parameters, and exposes the same two runtime methods plus its own runtime_metadata() probe. It must never report release identity.

Caller-correctable model validation failures belong in the typed runtime envelope. Unexpected setup and infrastructure failures should raise so the router maps them to upstream_failure; do not hide them with a model fallback.

6. Attest the right identity

Deployment probes and per-response attestations are related but distinct.

For a release, the deploy probe verifies baked release name, model type, implementation, source commit, descriptor digest, snapshot mode, Python version, image-recipe version, and identity GPU where required. The per-response attestation carries release, commit, descriptor digest, loaded model-manifest digest, engine-set digest or null, and optional observed GPU.

For scratch, both boundaries use session/revision identity rather than release identity. The response attestation includes:

• session and revision UUIDs plus generation;
• immutable revision-snapshot digest;
• source, dependency-recipe, spec, and draft-manifest digests;
• engine-set digest or null;
• deploy-time app version;
• optional observed GPU.

Echo baked identity values; do not recompute them from mutable ambient state at request time. Observe only facts whose purpose is observation, such as Python version or actual Torch device name.

7. Supply a scratch adapter

The generic ScratchAdapter needs model-owned callables:

Callable	Responsibility
resolve_source(selection)	Resolve mutable input once to an immutable manifest key and digest.
build_draft(resolved_source, recipe)	Re-read and digest-check the source, construct model body and spec, and omit runtime_release.
deploy_revision(context, draft_bytes)	Deploy a fresh app name and report app ID/version.
fetch_metadata(app_name)	Read the deployed app’s identity probe.
build_artifacts(context, draft)	Build and return a revision-bound artifact document, or None when no artifacts are required.

Draft construction must be deterministic for a resolved source and recipe. Promotion recomposes and compares the operative blobs and model sections byte-for-byte with the selected snapshot.

8. Add the model-owned CLI

The entry point should import the implementation table and call generic serving APIs for mint, deploy, scratch, publish, lifecycle, aliases, artifacts, and status. Release deployment must re-execute that same entry point from a temporary checkout of the descriptor pin; otherwise the caller’s worktree could silently deploy different code.

Keep internal pin/deploy helper commands out of normal user workflows. Validate names and arguments before creating immutable descriptors, tags, or artifact manifests.

9. Verify the integration

At minimum, exercise these boundaries with integration-style tests:

• descriptor/table agreement and exact GPU vocabulary;
• deploy from pin without changing the caller’s branch, index, or worktree;
• metadata mismatch refusal and explicit repair;
• publication through live pinned spec/validation functions;
• request defaults, constraints, examples, binary wire fields, and upload delivery;
• cold load with blob and artifact digest checks;
• response-attestation match and tamper refusal;
• scratch start, source sync, failed revision isolation, lease renewal/expiry, and promotion equality;
• readiness before and after required artifact creation;
• lifecycle, alias-chain, removal-marker, and concurrent compare-and-swap behavior.

Run a real Modal cold start and at least one real model request before treating a new implementation as ready. Schema tests alone cannot validate image imports, GPU loading, engine compatibility, or the returned audio.