RunPod: GPU Cloud

RunPod rents GPU compute by the hour or by the second. You can spin up dedicated "pods" with preconfigured templates or use serverless endpoints that scale to zero. It is a practical middle ground between self-hosting and full serverless abstraction.

• You want dedicated GPU machines without a long-term cloud contract.
• You need predictable per-hour pricing for development or batch jobs.
• You want to self-host models but not buy hardware.
• Your workload needs full root access, custom drivers, or specific CUDA versions.

• Wide GPU selection. RTX A4000/4090, A100, H100, and community-contributed templates.
• Templates. Prebuilt PyTorch, Stable Diffusion, Ollama, and vLLM images save setup time.
• Serverless endpoints. Autoscaling GPU inference with per-second billing.
• Persistent storage. Network volumes keep models and data between pod restarts.
• Community marketplace. Find templates and community-contributed configurations.

• Operational overhead. Unlike Modal, you still manage the machine: drivers, environment, container restarts.
• Spot availability. Popular GPUs can be hard to get during peak demand.
• Networking costs. Egress and persistent storage add up separately.
• Cold starts. Serverless endpoints can take 10–60 seconds to warm up depending on image size.

• Pods: from ~$0.25/GPU-hour for consumer GPUs, up to $2+/hour for H100s.
• Serverless: per-second billing with network volume and compute charges.
• A development agent using a mid-range GPU 8 hours per day costs roughly $50–$150/month plus storage.

Developers and small teams who want the flexibility of rented GPUs without managed platform abstraction. Good for running Ollama, vLLM, or custom inference servers. See our RunPod GPU deployment recipe for a tested setup.

• Ollama / vLLM for local model serving in the cloud.
• OpenClaw / Hermes agents needing remote model backends.
• Pinecone / pgvector for retrieval-augmented agent memory.