Modal: Serverless GPUs for Inference and Agents

Modal is a serverless compute platform popular with ML teams. You write Python functions, attach GPU types, and Modal packages, scales, and bills for the runtime. Common uses: model serving, batch inference, fine-tunes, and agent side-workers.

• You need GPUs without owning cluster operations
• Burst workloads that should scale to zero
• Python-native ML and agent code rather than Kubernetes YAML first

• Fast path from local function to remote GPU
• Image caching and dependency packaging
• Autoscaling and concurrency controls for inference endpoints
• Good fit for experimental agent tools (OCR, embeddings, batch jobs)

• Cold starts still matter for interactive UX
• Costs can surprise you if long GPU jobs are misconfigured
• Not a full substitute for a multi-tenant production Kubernetes platform with custom networking policies

Billed on compute and GPU time plus storage. Always set timeouts, concurrency caps, and budget alerts before giving agents permission to spawn jobs.