Getting started with APIPod

Ship a Python function as a cloud GPU API. You install APIPod, decorate one function, serve it locally on port 8000, and call it from a client. By the end of this page you have a running endpoint you can deploy to RunPod.

socaity build runs docker build. You need a working Docker install before you deploy, not just the Python package.

• Docker. Install Docker Desktop (or Docker Engine) and make sure the daemon is running (docker info succeeds). Without it, socaity build fails immediately.
• Disk space. The RunPod CUDA base image is large (a devel base is ~18 to 22 GB); with model layers, budget ≥ 30 GB free. Docker stores images on its own disk, which on Docker Desktop defaults to the system drive. For big ML images, move it to a roomier volume under Settings → Resources → Advanced → Disk image location.
• Architecture (Apple Silicon). RunPod runs linux/amd64 GPUs. On an Apple-Silicon (arm64) Mac you must cross-build: docker build --platform linux/amd64 …. A native arm64 image will not run on RunPod. The cross-build uses emulation and is slow.
• Hugging Face access (for open models). If your service downloads weights from Hugging Face, gated models (e.g. Gemma) require accepting the license on the model page and setting an HF_TOKEN environment variable at deploy time. See Secrets.

pip install apipod

APIPod requires Python ≥ 3.10 and defaults to Python 3.10 when generating Docker images. The package pulls in fastapi, uvicorn, and media-toolkit. The RunPod extra is optional: pip install apipod[runpod] adds runpod>=1.7.7 for serverless deploys.

Create a file called service.py. Decorate your function with @app.endpoint(path): APIPod registers the route, validates inputs, and serialises the return value.

from apipod import APIPod

app = APIPod()

@app.endpoint("/generate")
def generate(prompt: str, steps: int = 20) -> dict:
    """Generate an image from a text prompt."""
    # Swap this stub for your real model call.
    image = my_model.run(prompt=prompt, num_inference_steps=steps)
    # media-toolkit types (ImageFile, AudioFile, VideoFile) serialise themselves;
    # plain dicts pass through as JSON.
    return {"image": image}

if __name__ == "__main__":
    app.start()

APIPod boots a uvicorn server on http://0.0.0.0:8000 by default and exposes Swagger UI at /docs. The default factory call APIPod() uses the local orchestrator, dedicated compute, and the localhost provider, so requests run synchronously inside the same process.

socaity start

Starting APIPod service...
Loaded endpoints:
  POST /generate

Listening on http://0.0.0.0:8000
Swagger UI     at http://localhost:8000/docs

Call the endpoint with requests, curl, or the Swagger UI. With the default dedicated configuration, the response is the function's return value, not a job handle.

import requests

resp = requests.post(
    "http://localhost:8000/generate",
    json={"prompt": "a lone astronaut on a neon planet", "steps": 20},
)
# Default APIPod() runs requests synchronously, so resp.json() is
# the function's return value (the image payload), not a job handle.
print(resp.json())

curl -X POST http://localhost:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "a lone astronaut on a neon planet", "steps": 20}'

socaity build generates a Dockerfile from the project scan and (optionally) runs docker build. APIPod recommends a base image based on whether it detects CUDA, PyTorch, TensorFlow, or ONNX in your requirements. RunPod is the production provider supported today; Scaleway and Azure are listed in the enum but raise NotImplementedError on every code path.

# Generate apipod-deploy/apipod.json from a project scan.
socaity scan

# Build the Docker image. APIPod picks a base image based on
# detected frameworks (PyTorch, TensorFlow, ONNX) and CUDA.
socaity build

# Start the service against a target provider. RunPod is supported
# in production today; scaleway and azure raise NotImplementedError.
socaity start --orchestrator socaity --compute serverless --provider runpod

# Push the resulting image to a registry and create a RunPod endpoint,
# or trigger the deploy from the Studio dashboard at socaity.ai.

A minimal APIPod project only needs service.py. socaity scan writes a project manifest to apipod-deploy/apipod.json; add a requirements.txt or a custom Dockerfile only when you need to override the defaults.