Store and Security

One serving instance is one object-store prefix, one Modal environment, one router, and one credential set. Nothing references objects across instances.

Instance identity

A new store is stamped once at instance.json:

{
  "schema": 1,
  "name": "prod",
  "purpose": "production",
  "layout_version": 1,
  "created": "<RFC3339 time>"
}

Every normal operator command and the router require RND_SERVING_INSTANCE and compare it with this document. The check prevents an immutable publish or lifecycle operation from silently landing in the wrong bucket or prefix.

Object layout

instance.json
blobs/sha256/<64-hex>

runs/<scope>/<run>/checkpoints/step-<step>.<kind>.json
named/<name>/<label>.json
releases/<release>.json
artifacts/<release>/<gpu-arch>/<model-id>
scratch-revisions/<session>/<revision>.json
scratch-artifacts/<session>/<revision>/<draft-sha>.<gpu-arch>.json
removals/<model-id>
events/<YYYY-MM-DD>/<timestamp>-<operation-id>.json

mutable/lifecycle/<model-id-or-release-key>
mutable/deployments/<release>.json
mutable/aliases/<name>.json
mutable/scratch-sessions/<session>.json
mutable/webhooks.json

Model type is a field, not another path segment. Release names are globally unique. A model ID is its full runs/...inference*.json or named/...json key.

Immutable and mutable regimes

Objects outside mutable/ are write-once under normal tooling:

• blob content is addressed by SHA-256;
• model manifests, descriptors, engine sets, scratch snapshots, events, and removal markers are conditionally created;
• a retry with the same identity is accepted where defined;
• different contents at an occupied identity are a conflict, never an overwrite.

Objects under mutable/ use strongly consistent reads and compare-and-swap updates. Lifecycle records preserve complete audit history; aliases preserve target history; deployment records are replaceable observations; scratch sessions are leases over immutable revisions.

Tigris conditional creates and ETag compare-and-swap were verified against the live backend. Deleting an object allows its key to be created again, which is why unpublication writes a separate permanent removal marker instead of relying on deletion.

Credentials

Credentials are split by responsibility:

Principal	Required access
Released and scratch runtime apps	Read model manifests, blobs, descriptors, revision snapshots, and artifacts.
Router	Read serving state plus write scratch lease updates and webhook records.
Operator CLI	Read/write store, Modal deploy access, and git tag-push access for minting.
HTTP caller	One shared router bearer token; no direct store or Modal access.

In the current repository layout:

• rnd/.env.storage holds the store profile, RND_CHECKPOINT_STORE, and RND_SERVING_INSTANCE.
• rnd/.env.infra holds the shared router API-token set.
• Simpcat’s .env.infra currently owns the Modal service-user credentials used by operator deploy commands.
• .env.keys is the gitignored private decryption key. Never copy decrypted values into loose files, command arguments, documentation, or logs.

The runtime secret is conventionally <model_type>-inference; the router secret is serving-router. Secrets and apps are Modal-environment scoped.

HTTP security boundary

SERVING_API_TOKENS is a comma-separated set to support rotation. The router applies the same bearer-token check to model discovery, invocation, scratch, alias, and webhook endpoints.

This boundary keeps the open internet out; it is not an authorization system. Anyone with one token can reach every model and scratch session and can manage webhooks. Scratch owner and display names are metadata, not access control.

Generated API docs are disabled. Webhook posts are Ed25519-signed (see HTTP API); consumers verify before applying anything. Presigned upload URLs are separately scoped bearer capabilities; the router and runtime validate their hosts and never include them in error messages.

Do not expose this router to external customers without first adding a real account, entitlement, and audit design.

Caching

Successfully read immutable objects can be cached for the router process lifetime. Missing immutable objects cannot: an absent engine set or removal marker may appear later, so negative lookups use a short freshness TTL, currently 15 seconds by default.

Mutable records use the same short TTL and event invalidation. A lifecycle or alias update therefore has a bounded seconds-scale propagation window. Events can invalidate early but never extend the TTL.

Warm released-model requests commonly make zero object-store reads. Scratch invocation intentionally reads and compare-and-swaps its session record to enforce and renew the lease.

Modal Volumes are caches only. Content-addressed materializations may be shared; mutable compiled caches are namespaced by release or scratch source. Deleting a Volume must not destroy authoritative model or engine data.

Events and reconciliation

Every successful durable state change attempts to append a deterministic event after the authoritative write. Event append failure produces a warning but does not turn the already successful operation into a failure.

Events are a bounded notification feed, not a transaction log:

• object update and event create are separate writes;
• retries collapse through a deterministic operation ID;
• clients may miss events or webhooks;
• periodic scans of authoritative objects are required;
• a complete mirror must be rebuildable from the store.

Current operational boundaries

• Simpcat model mirroring and scratch/recompute UI are not yet integrated on its main branch.
• Cross-request GPU batching requires a release minted from serving runtime v2 or newer; releases pinning runtime v1 serialize GPU passes and deploy tooling refuses target_inputs > 1 for them.
• Store blob GC runs as a daily Modal cron (serving.gc_app): mark, condemned-set publication, fence, sweep, coordinated with writers through pins. It is a dry run unless the serving-blob-gc secret sets BLOB_GC_DELETE=1. unpublish alone still frees nothing: every stored manifest — an unpublished model’s included — remains a GC root that pins its blobs.
• Release disable enforces router refusal and invalidates deployment health; automatic provider-app stopping and retention cleanup are separate operational work.
• The shared token set has no per-user or per-model authorization.
• Webhook delivery is ordered and at-least-once from durable per-subscription cursors, and every post is Ed25519-signed; consumers verify against the router’s published key and deduplicate on operation_id. See HTTP API.
• Git runtime/* tags are reachability insurance and should be protected against update and delete. Only mint ever creates them, so no create restriction — and therefore no blessed automation identity — is needed. The descriptor remains authoritative if a tag is missing, but future redeployability is at risk.