Providers
The --provider flag tells APIPod where to run the container. The PROVIDER enum accepts five values: auto, localhost, runpod, scaleway, and azure. Today, APIPod implements localhost and runpod; scaleway and azure raise NotImplementedError at backend resolution. There is no --provider socaity value: socaity is an orchestrator, not a provider.
Routes the container to RunPod. Combined with --compute serverless this produces a RunPod serverless endpoint; combined with --compute dedicated under --orchestrator local it falls back to FastAPI.
Runs the FastAPI router on the local machine. Used for dev, integration tests, and any deploy that does not leave the host. This is the default when no --provider flag is passed.
APIPod picks the backend based on orchestrator and compute. Under serverless this dispatches to RunPod; under dedicated it stays on FastAPI. Use this when you do not want the provider hard-coded in apipod.json.
| Capability | runpod | localhost | scaleway / azure |
|---|---|---|---|
| Status | Working | Working | NotImplementedError today |
| Serverless backend | RunPod serverless router | FastAPI router + in-memory job queue | Planned (RunPod-only today) |
| Dedicated backend | FastAPI under local orchestrator; NotImplementedError under socaity (Celery backend planned) | FastAPI router | Planned |
| Default when no --provider flag | No | Yes (PROVIDER.LOCALHOST is the hard default) | No |
| Required extras | pip install apipod[runpod] (adds runpod>=1.7.7) | None beyond pip install apipod | N/A |
| Bind host / port | Managed by RunPod | 0.0.0.0:8000 (override with --host / --port) | N/A |
| Auto-scaling | Delegated to RunPod (scale-to-zero, cold start) | None. Single in-process worker thread. | N/A |
| Suitable for | Cloud serverless deploys | Dev, integration tests, local FastAPI runs | Not usable today |
APIPod itself does not maintain a GPU SKU catalog. GPU selection happens at the provider level. For RunPod, you pick the SKU when you create the serverless endpoint or pod from the image APIPod built. Current per-GPU rates are published at socaity.ai/Pricing.
| Provider | GPU selection | Where to pick | Pricing source | Status |
|---|---|---|---|---|
| runpod | RunPod-managed (NVIDIA T4 through H100 SXM) | In the RunPod dashboard when creating the serverless endpoint or pod that consumes the APIPod-built image. | socaity.ai/Pricing for per-GPU rates routed through Socaity; RunPod console for direct rates. | Working |
| localhost | Host machine GPU (if any) | Whatever CUDA device is visible to the host process. No selection step. | N/A. No metered pricing for localhost runs. | Working |
| scaleway | Planned | N/A. Backend resolution raises NotImplementedError. | N/A | Planned |
| azure | Planned | N/A. Backend resolution raises NotImplementedError. | N/A | Planned |
Today, the platform sets the region (RunPod EU). The --region flag is accepted by socaity build and written to the deploy config, but it is not yet wired to provider routing, so per-workload selection is on the roadmap. APIPod does not maintain a region whitelist.
| Provider | Where region is set | --region behavior | Data residency |
|---|---|---|---|
runpod | Selected by the platform (RunPod EU) today; per-workload selection is on the roadmap. | Accepted by socaity build and written to the deploy config, but not yet read. Region is platform-selected today. | EU (platform-selected) |
localhost | N/A (runs on the host) | Ignored. The router binds to --host (default 0.0.0.0) and --port (default 8000). | Wherever the host runs |
scaleway / azure | Planned | Resolution raises NotImplementedError today. | Planned |
| If you need… | Choose | Why |
|---|---|---|
| Run the container on your laptop or CI box | localhost | Default value. The FastAPI router binds to 0.0.0.0:8000 and starts an in-process worker thread. |
| Ship a serverless GPU endpoint that scales to zero | runpod | socaity build --provider runpod --compute serverless generates a RunPod-compatible image. RunPod handles cold start and autoscaling. |
| Let APIPod choose based on --compute | auto | Resolves to RunPod under serverless and to FastAPI under dedicated. Useful when one image targets both modes. |
| Run a dedicated long-lived FastAPI service | localhost | With --orchestrator local --compute dedicated, the FastAPI router runs as a plain uvicorn process on the chosen host and port. |
| Pin the deploy to Scaleway or Azure | scaleway / azure | Not available today. Both raise NotImplementedError at backend resolution (apipod/api.py:90-99, 104-107). |
socaity build --provider runpod --compute serverless produces a RunPod-ready image today. --provider scaleway and --provider azure are reserved enum values; both raise NotImplementedError at runtime and are planned. There is no --provider socaity value: Socaity is an orchestrator (--orchestrator socaity), not a provider. Three places, applied in this order: an explicit --provider flag wins; then the APIPOD_PROVIDER env var; then the hard default, localhost. The same precedence applies to --orchestrator (env: APIPOD_ORCHESTRATOR, default local) and --compute (env: APIPOD_COMPUTE, default dedicated).
# Per-command flag (wins over env and defaults)
socaity build --provider runpod --compute serverless
# Environment variables (read by apipod/common/settings.py)
export APIPOD_PROVIDER=runpod
export APIPOD_COMPUTE=serverless
export APIPOD_ORCHESTRATOR=local
socaity start
# apipod-deploy/apipod.json (written by socaity scan)
# {
# "provider": "runpod",
# "compute": "serverless",
# "orchestrator": "local"
# }- Build. How socaity scan and socaity build resolve a base image and generate the Dockerfile.
- Deploy serverless. End-to-end flow for shipping an APIPod image to RunPod serverless.
- Deploy dedicated. Run the FastAPI router as an always-on uvicorn process.
- Endpoints. How @app.endpoint registers routes on the FastAPI and RunPod routers.