Runpod Python SDK

The runpod Python package is the single SDK for building on Runpod. It includes the decorator-based app SDK that was previously shipped as Flash, the rp command-line interface, the client for calling existing Serverless endpoints, the Serverless worker SDK, and the API wrapper for managing Pods and other resources.

import runpod

app = runpod.App("hello")

@app.queue(gpu="H100", dependencies=["torch"])
def hello():
    import torch
    return {"gpu": torch.cuda.get_device_name(0)}

@runpod.local_entrypoint
def main():
    print(hello.remote())

Run the file with rp flash dev main.py. The hello function runs on a Runpod GPU, and main runs on your machine.

Flash is now part of the Runpod SDK. If you have existing code that uses the runpod-flash package, it continues to work. To move an existing project, see Migrating from Flash.

Get started

Quickstart

Install the SDK and run a function on a GPU.

Migrating from Flash

Move an existing Flash project to the Runpod SDK.

CLI reference

Use the rp CLI to log in, develop, and deploy.

Install

The SDK requires Python 3.10 to 3.14, and the features on this page require runpod 2.0 or later.

pip install runpod

# If using uv:
uv add runpod

Until 2.0.0 is released, install the release candidate with pip install runpod==2.0.0rc1. Installing the package also installs the rp CLI. The same CLI is available under the longer name runpod.

Authenticate

rp login

This opens your browser to approve access. Credentials are saved to ~/.runpod/config.toml and are shared by the SDK and the CLI. See rp login for other options, including named profiles.

You can also set the RUNPOD_API_KEY environment variable. This is the only environment variable the SDK reads for authentication.

What’s in the SDK

The runpod package covers four areas. Each one is independent, so you can use any of them on its own.

Area	Use it to	Main entry points
Apps	Define functions and classes in Python that run on Runpod GPUs or CPUs, then deploy them as endpoints.	`runpod.App`, `@app.queue`, `@app.task`, `@app.api`, `rp flash`
Endpoint client	Call a Serverless endpoint that already exists, by ID.	`runpod.Endpoint`
Serverless worker	Write the handler that runs inside a custom Serverless worker container.	`runpod.serverless.start`
API wrapper	Create and manage Pods, endpoints, templates, and other resources.	`runpod.create_pod`, `runpod.get_pods`, and related functions

Apps

An app is a collection of Python functions and classes that run on Runpod. Create one with runpod.App, then attach resources to it with decorators. Each decorator defines a different kind of remote compute.

Resource types

Decorator	What it creates	Use it for
`@app.queue`	An autoscaling, queue-based Serverless endpoint.	Inference, batch jobs, and other request/response work.
`@app.task`	One ephemeral Pod per call. The Pod provisions, runs to completion, and terminates.	Training, fine-tuning, and other long-running jobs.
`@app.api`	A load-balanced Serverless endpoint that serves a Python class over HTTP.	Web services and APIs with multiple routes.

The following example defines one of each:

import runpod
from runpod import App, Model, NetworkVolume, Secret

app = App("inference")

models = NetworkVolume("models", size=100)
llama = Model("meta-llama/Llama-3.1-8B-Instruct")

# Autoscaling job queue on H100s, scales to zero when idle
@app.queue(
    gpu="H100",
    workers=(0, 3),
    dependencies=["vllm"],
    mounts={"/runpod-volume": models},
    model=llama,
    env={"HF_TOKEN": Secret("hf-token")},
)
def chat(prompt: str):
    import vllm

    llm = vllm.LLM(model=str(llama.path))
    return llm.generate(prompt)

# One ephemeral Pod per call
@app.task(gpu="H100", gpu_count=2, mounts={"/models": models})
def finetune(steps: int = 1000):
    ...
    return {"loss": 0.1}

# Load-balanced HTTP service
@app.api(cpu="cpu3c-1-2")
class Counter:
    @runpod.init
    def setup(self):
        self.count = 0

    @runpod.post("/bump")
    async def bump(self, body: dict):
        self.count += body.get("by", 1)
        return {"count": self.count}

    @runpod.get("/value")
    async def value(self):
        return {"count": self.count}

In this example:

chat runs on an autoscaling queue endpoint with between 0 and 3 workers. Runpod stages the model weights on the host before the worker starts (model=llama), the network volume is mounted at /runpod-volume, and the HF_TOKEN environment variable is read from a Runpod secret.
finetune provisions a two-GPU Pod for each call and terminates it when the function returns.
Counter is served over HTTP. The method marked with @runpod.init runs once when each worker starts, and methods marked with @runpod.get and @runpod.post become routes. Workers keep state in memory between requests.

Specify hardware with gpu= or cpu=, which are mutually exclusive. For GPUs, use a short name such as "4090", "H100", or "B200", a runpod.GpuGroup pool such as runpod.GpuGroup.ANY, or a runpod.GpuType device. Pass a list to allow several options. If you don’t set workers, resources scale between 0 and 3 workers. For every option, see Resource parameters, GPU types, and CPU types.

Call remote functions

Decorated functions keep their Python identity. You call them through methods that control where and how they run:

Call	Behavior
`fn.remote(...)`	Runs the function on Runpod and blocks until the result is ready.
`await fn.remote.aio(...)`	The async form of `.remote()`.
`fn.spawn(...)`	Submits the work and returns a `Job` handle immediately, without waiting for the result.
`fn.stream(...)`	Streams results back as the function produces them.
`fn.local(...)`	Runs the function in your local Python process. Remote storage isn’t mounted.

For HTTP services defined with @app.api, call routes through the class, for example Counter.post("/bump", {"by": 3}) or Counter.get("/value"). .remote() and job.wait() don’t impose an execution timeout by default. For streaming examples and how spawned tasks are cleaned up, see Dev sessions.

Local entrypoints

Mark the function that drives your app with @runpod.local_entrypoint. It runs on your machine and is the entry point for rp flash dev:

@runpod.local_entrypoint
def main():
    print(chat.remote("Why is the sky blue?"))  # Blocks for the result
    job = finetune.spawn(steps=500)  # Returns a Job immediately

App discovery imports your modules synchronously. Keep blocking work inside an entrypoint or a remote function, not at the top level of a module.

Storage

Attach storage with the mounts parameter, which maps a path on the worker to a volume:

NetworkVolume(name_or_id, size=50, datacenter=None, create=True) references a network volume in a single datacenter. The app chooses a datacenter that works for every resource sharing the volume.
GlobalVolume(name_or_id, create=True) references global storage that isn’t tied to a datacenter.

Both types look up a volume by name or ID and create it if it doesn’t exist. Set create=False to require an existing volume. To choose between network and global volumes, see Storage overview.

Mount rules

Mount rules depend on the resource type:

@app.task supports one network volume and one global volume, mounted at different paths that don’t overlap.
@app.queue and @app.api support one volume, mounted at /runpod-volume.
Global volumes require GPU compute, for both tasks and endpoints.

Inside a remote function, volume.path returns the path where the volume is mounted. Read .path inside remote functions, not at the top level of a module. See Storage for volume placement and billing.

Develop and deploy

Use the rp flash commands to iterate on an app and ship it:

rp flash dev main.py  # Live dev session: edit, re-run, logs stream back
rp flash deploy       # Deploy production endpoints

A dev session creates temporary endpoints and deletes them when you press Ctrl-C. To learn more, see Build an app, Dev sessions, and Deploy an app. See the CLI reference for all commands.

Endpoint client

To call a Serverless endpoint that already exists, such as one you deployed from the console or with rp flash deploy, use runpod.Endpoint with the endpoint ID.

import runpod

endpoint = runpod.Endpoint("ENDPOINT_ID")

# Wait for the result
result = endpoint.run_sync({"prompt": "Hello"})
print(result)

# Or submit a job and check on it later
run_request = endpoint.run({"prompt": "Hello"})
print(run_request.status())
print(run_request.output())

By default, the client uses the API key from rp login, RUNPOD_API_KEY, or runpod.api_key. To use a different key for one endpoint, pass it to the constructor: runpod.Endpoint("ENDPOINT_ID", api_key="API_KEY"). A key passed to the constructor takes precedence over the global key. See Send API requests for more examples.

Serverless worker

If you build your own worker container, use runpod.serverless.start to register the handler that processes jobs:

import runpod

def handler(job):
    job_input = job["input"]
    return {"output": job_input}

runpod.serverless.start({"handler": handler})

To learn how to write handlers, including streaming, progress updates, and error handling, see Handler functions.

API wrapper

The SDK includes functions for managing Runpod resources through the REST API:

import runpod

pods = runpod.get_pods()
pod = runpod.create_pod("test", "runpod/stack", "NVIDIA GeForce RTX 4090")
runpod.stop_pod(pod["id"])
runpod.terminate_pod(pod["id"])

Limitations

5th-generation CPU instances (cpu5c) are only available in EU-RO-1. 3rd-generation instances are available in more data centers. See CPU types.
Apps can scale workers quickly across several endpoints, so you may reach your account’s worker limit. Contact Runpod support to raise it.

Next steps

Run your first function with the SDK quickstart.
Build an app with several resource types.
Configure resources such as GPUs, workers, and storage.
Migrate from Flash if you have an existing Flash project.
Learn about the runtime that runs your code on workers.
Browse the examples in the runpod-python repository.

Get started

Runpod SDK (Beta)

Flash

Serverless

Pods

Storage

Public Endpoints

Instant Clusters

Accounts and billing

Organizations (Beta)

Integrations

Hub

Reference

Get started

Quickstart

Migrating from Flash

CLI reference

Install

Authenticate

What’s in the SDK

Apps

Resource types

Call remote functions

Local entrypoints

Storage

Mount rules

Develop and deploy

Endpoint client

Serverless worker

API wrapper

Limitations

Next steps

​Get started

Quickstart

Migrating from Flash

CLI reference

​Install

​Authenticate

​What’s in the SDK

​Apps

​Resource types

​Call remote functions

​Local entrypoints

​Storage

​Mount rules

​Develop and deploy

​Endpoint client

​Serverless worker

​API wrapper

​Limitations

​Next steps

Get started

Install

Authenticate

What’s in the SDK

Apps

Resource types

Call remote functions

Local entrypoints

Storage

Mount rules

Develop and deploy

Endpoint client

Serverless worker

API wrapper

Limitations

Next steps