Job System

A Job is a single asynchronous invocation of a model on Socaity. When you call a hosted model, Socaity creates a Job, queues it on a worker that matches the model's hardware profile, and returns a handle you can poll, await, or cancel. Jobs are the unit you bill against, the unit you observe, and the unit you cancel.

Streaming is alpha. The STREAMING state appears in the canonical enum, and Socaity flags it on responses with Content-Type: text/event-stream. A public iterator on the Job handle is not shipped yet. For now, treat streaming as an internal signal that the worker is producing incremental output.

Job States

The Socaity SDK defines nine job states on the APIJobStatus enum. Four are terminal (FINISHED, FAILED, TIMEOUT, CANCELLED); once a job lands in a terminal state, its status will not change again. Provider-specific status strings are normalised on read: RunPod's IN_QUEUE becomes QUEUED, IN_PROGRESS becomes PROCESSING, COMPLETED becomes FINISHED; Replicate's STARTING becomes QUEUED, BOOTING becomes PROCESSING, SUCCEEDED becomes FINISHED.

PENDING

QUEUED

PROCESSING

STREAMING

FINISHED

FAILED

TIMEOUT

CANCELLED

UNKNOWN

State	Meaning	Terminal?	Result available?
PENDING	Job submitted, waiting to be validated and queued.	No	No
QUEUED	Validated and waiting for a GPU worker to pick it up.	No	No
PROCESSING	A GPU worker is actively processing the job.	No	No
STREAMING	Job is producing incremental results (LLM tokens, partial video).	No	No
FINISHED	Inference completed successfully. Result is available.	Yes	Yes
FAILED	An error occurred during inference. Check the error field.	Yes	No
TIMEOUT	Job exceeded the configured execution time limit.	Yes	No
CANCELLED	Job was cancelled by the client before or during processing.	Yes	No
UNKNOWN	Status string could not be mapped to a known state. Polling continues.	No	No

Lifecycle Diagram

PENDING

queued

PROCESSING

GPU active

FINISHED

result ready

Submitting a Job

Calling a hosted model returns a Job handle immediately. The handle exposes .response (the latest BaseJobResponse), .is_terminal, .get_result(), and .cancel(). The handle inherits from MrMeseex (in the meseex package), which runs the upload, submission, and polling tasks in the background.

import os
from socaity.sdk.replicate.black_forest_labs import flux_schnell

flux = flux_schnell(api_key=os.getenv("SOCAITY_API_KEY"))

# The call returns immediately. The handle is a Job object.
job = flux(prompt="a neon city at night")

# meseex_id is assigned immediately (format: "meseex_<uuid>").
print(f"Job id: {job.meseex_id}")
# job.response is None until the SDK posts the job to the provider; it fills
# in as the background poller runs, so read it after get_result() (below).

# get_result() blocks until the job is terminal. The SDK polls internally
# at a 1 s cadence and tolerates 4 consecutive transport errors.
# On FAILED -> raises TaskException. On CANCELLED -> raises TaskCancelledException.
# On local timeout -> returns None (does not raise).
result = job.get_result(timeout_s=120)
print(result)  # parsed MediaFile, str, or whatever the model returns

Polling Intervals

The SDK polls the backend for you. The polling loop runs inside the Job handle on a fixed cadence; you do not need to write your own loop unless you want to log progress between checks. The defaults below are baked into the @polling_task decorator on ApiJobManager._poll_status.

Setting	Default	Behavior
Poll interval	1 s	Fixed cadence on ApiJobManager._poll_status. Not exponential.
Total timeout	3600 s	Hard ceiling on the internal polling task before it gives up.
Error tolerance	4 consecutive errors	The poller absorbs up to 4 transport errors in a row before raising (raises on the 5th).
get_result() check loop	10 ms	Once polling has updated state, get_result() wakes via short 10 ms sleeps until the handle is terminal.

Block on a result with .get_result(timeout_s=...). On FAILED, the SDK re-raises TaskException; on CANCELLED, it raises TaskCancelledException; on local timeout, it returns None rather than raising. Pass default_value_on_error=<sentinel> if you want a failed job to return that sentinel instead of raising.

from meseex import TaskException, TaskCancelledException

# The SDK polls in the background. You rarely need a manual loop.
# Use get_result() and let the internal poller do the work.

job = flux(prompt="a neon city")

try:
    result = job.get_result(timeout_s=120)
except TaskException as err:
    # Job ended in FAILED. Read the underlying message off the handle.
    print(f"Job failed: {job.response.error}")
    raise
except TaskCancelledException:
    print("Job was cancelled before completion.")
    raise

if result is None:
    # get_result() returns None on the local timeout_s, not on backend failure.
    print("Local timeout reached, job still running on the backend.")
else:
    print(result)

Progress Reporting

The progress field on BaseJobResponse is a float from 0.0 to 1.0. Worker code sets it via JobProgress.set_status(progress=...) inside the handler. Not every model reports granular progress, so check for None before formatting. Once the job is terminal, .runtime_info on the handle returns a (delay_seconds, execution_seconds) tuple for RunPod and Replicate workloads.

import time

# Read .response.progress and .is_terminal while the internal poller
# updates them. No .refresh() call is needed; the poller pushes new state
# onto the handle from a background task.

job = flux(prompt="a cinematic landscape")

while not job.is_terminal:
    progress = job.response.progress
    if progress is not None:
        filled = int(progress * 40)
        bar = "#" * filled + "-" * (40 - filled)
        print(f"\r[{bar}] {progress:.0%}", end="", flush=True)
    time.sleep(1)

print()  # newline after the progress bar
print(job.get_result())  # safe: handle is already terminal, returns instantly

Webhook vs Polling

Polling is what the SDK does today. Webhook delivery from the Socaity backend is planned but not yet exposed on the public Job handle. The comparison below describes both delivery models so you can decide which one to design for.

Aspect	Polling (today)	Webhooks (planned)
Complexity	Built into the SDK. No infrastructure to provision.	You run a public HTTPS endpoint and verify signatures.
Latency to result	Bounded by the 1 s poll interval.	Push delivery from the Socaity backend, no poll wait.
Backend traffic	One status check per second per active job.	One outbound POST per status change.
Best fit	Scripts, notebooks, jobs you await inline.	Web apps and pipelines that fan out and resume later.
Error recovery	The poller tolerates 4 transport errors before raising.	Your endpoint handles retries and idempotency.

Cancellation

Call .cancel() on the Job handle to request cancellation while the job is in PENDING, QUEUED, or PROCESSING. The SDK hits the cancel_job_url stamped on the response when the backend provides one, and falls back to local cancellation when it does not. Defaults: wait=False, timeout_s=30.0, poll_interval_s=0.5. After cancellation, calling .get_result() on the handle raises TaskCancelledException. Socaity does not bill for cancelled jobs.

Known issue. In the current SDK build, .cancel() raises an internal error on Replicate-backed jobs (a fix is in progress). The signature below is the intended interface; until the fix ships, treat in-flight jobs as non-cancellable and rely on timeout_s on .get_result() instead.

import os
from socaity.sdk.replicate.black_forest_labs import flux_schnell

flux = flux_schnell(api_key=os.getenv("SOCAITY_API_KEY"))
job = flux(prompt="a neon city")

# .cancel() returns once the request is acknowledged.
# Defaults: wait=False, timeout_s=30.0, poll_interval_s=0.5.
if not job.is_terminal:
    job.cancel()

# Read .response after the internal poller settles.
# If the worker had already produced a result, the handle is FINISHED,
# not CANCELLED. Socaity does not bill cancelled jobs.
final = job.response.status
print(f"Final status: {final}")  # CANCELLED or FINISHED

Cancellation is best-effort. If the worker has already started the forward pass, the job may finish before the cancel signal reaches it; in that case the handle settles on FINISHED and you still receive the result. Socaity does not bill cancelled jobs, and the active-only billing model means you only pay for GPU-seconds spent in PROCESSING.

Serverless vs Dedicated GPU

Service Catalog