Skip to main content
This guide is for teams that call models on Replicate or host their own Cog models there. Calls to official models map to Runpod Public Endpoints where Runpod hosts the same model. Your own Cog models and deployments map to Runpod Serverless queue-based endpoints, and trainings map to Pods or @app.task runs in the Runpod SDK. At the end you’ll have each workload running on Runpod, your client code calling it, and Replicate scaled down.
You can hand most of this migration to a coding agent. See Migrate with a coding agent.

How Replicate maps to Runpod

What’s different

What works differently

You run your own copy of community models. On Replicate, any public model can be called by name on shared hardware. On Runpod, a model is either a Public Endpoint that Runpod hosts, or an endpoint in your account that you deploy from a Hub repo, a container image, or a runpod.App. Billing follows workers, not predictions. For Replicate public models you pay for the time a prediction runs. A Runpod Serverless endpoint bills per second while a worker is running, which includes cold start, execution, and the idle timeout after a job. With active workers set to 0, you pay nothing while no worker is running. This is closest to how Replicate bills private models and deployments. Inputs are validated by your code. Cog generates an input schema from the type annotations on predict() and rejects bad input before your code runs. A Runpod handler receives job["input"] as a plain dictionary. Validate it yourself, for example with the validator utility. With @app.queue in the Runpod SDK, the keys of input are passed to your function as keyword arguments. File outputs are yours to store. Replicate uploads any Path your predictor returns and gives the client a replicate.delivery URL that expires after one hour. A Runpod handler returns JSON. Upload files to your own S3-compatible bucket and return the URL, or return base64 for small files. Response payloads are limited to 10 MB for /run and 20 MB for /runsync. File inputs arrive as strings. Cog downloads a file input and hands your code a local Path. On Runpod, your handler receives whatever the client sent (usually a URL) and downloads it itself. The Replicate client libraries also upload local files for you. With Runpod, host the file somewhere your worker can reach and send its URL. Results are kept for a shorter time. Replicate keeps API prediction data for one hour. Runpod keeps a job’s result for 30 minutes after an async (/run) job completes and 1 minute after a sync (/runsync) job completes. Fetch results promptly, or use a webhook. Sync waits are longer. Prefer: wait holds a Replicate request open for up to 60 seconds. /runsync waits 90 seconds by default, and you can set ?wait= from 1000 to 300000 milliseconds. Timeouts are set per endpoint and per job. Replicate predictions time out after 30 minutes unless you arrange otherwise, and Cancel-After sets a per-prediction deadline. Runpod has an execution timeout per endpoint (600 seconds by default in the console, up to 7 days) that you can override per request with policy.executionTimeout, plus a job TTL. Streaming is polled. Replicate streams over server-sent events from urls.stream. On Runpod, a handler that yields exposes chunks at GET /stream/JOB_ID, which you call repeatedly until the job finishes. For LLMs served through an OpenAI-compatible route, stream=True works as it does with any OpenAI-compatible server. Webhooks fire on completion. Replicate can send start, output, logs, and completed events and signs each request. Runpod calls your webhook when the job finishes and retries up to 2 more times, 10 seconds apart, if your server doesn’t return 200.

What has no direct equivalent

Requirements

  • Access to your Replicate account or organization, and an API token (REPLICATE_API_TOKEN).
  • A Runpod account with credits, and a Runpod API key.
  • Docker if you plan to build worker images, and an account with a container registry such as Docker Hub.
  • An S3-compatible bucket if your models return files.
  • Python 3.10 to 3.14 for the Runpod SDK.
  • The source code for each Cog model you host (cog.yaml, predict.py, and weights).

Step 1: Take inventory of what you run on Replicate

List what you have on Replicate before you create anything on Runpod. Replicate’s HTTP API covers most of what you need, and the commands below use curl and jq. Confirm which account or organization the token belongs to:
List your deployments with their hardware and scaling settings:
Each result includes current_release.configuration with hardware, min_instances, and max_instances. These become the GPU pool, active workers, and max workers on Runpod. Results are paginated, so follow the next URL until it’s null. List recent predictions to see which models and versions your code calls in practice:
This returns 100 predictions per page, most recent first. Use the predict_time values to size timeouts and to pick a GPU later. List the trainings you’ve run, and the hardware SKUs Replicate offers so you can read the hardware values:
The models API lists public models, so find your own models (including private ones) on your Replicate dashboard. For each model you own, fetch its details and its input schema:
The Input schema lists every input with its type, default, and constraints. Keep it: it’s the contract your Runpod handler has to honor. Then find every place your code touches Replicate:
Sort each hit into client calls (replicate.run, predictions.create, webhooks handlers), model code (cog.yaml, predict.py), and CI (cog push in pipelines). Also note anywhere you store or allow-list replicate.delivery URLs. For each workload, write down the model identifier, whether it’s an official model, a community model, or your own, the hardware SKU, min and max instances, typical and worst-case run time, input and output types (especially files), and whether callers use sync mode, polling, streaming, or webhooks.

Step 2: Set up Runpod

Create a Runpod account, add credits, and create an API key. Then install the tools you need:
runpodctl doctor walks you through API key and SSH key setup. rp login stores credentials in ~/.runpod/config.toml, and the SDK also reads RUNPOD_API_KEY. On macOS you can install the CLI with brew install runpod/runpodctl/runpodctl. See the runpodctl overview and the SDK overview. If you use a coding agent, also connect the Runpod MCP server.

Step 3: Choose where each workload lands

Public Endpoint model map

These are the model families Runpod hosts as Public Endpoints, with the endpoint ID you put in the URL. Check the model reference for the full list, since models are added and retired.
The same model family can ship in different variants on each platform, and input field names differ (for example, a Replicate model may take aspect_ratio where the Runpod endpoint takes width and height). Compare the input schema on the Replicate model page with the parameters on the Runpod model page before you switch.
If the model you call isn’t in the reference (Whisper and other speech-to-text models, for example), search the Hub for a worker you can deploy:
If there’s no hosted model and no Hub repo, and the model has open weights, deploy it yourself using the handler or runpod.App path in Step 4. Closed-weights models are only available on Runpod if they appear in the model reference.

GPU mapping

Serverless endpoints select hardware by GPU pool. Map each Replicate hardware SKU to a pool with the same or more VRAM, then test: the fit depends on your model’s memory use, not on the SKU name. Run GET https://api.replicate.com/v1/hardware to see the SKUs your account can use. On Runpod, list available GPU types with runpodctl gpu list and availability by data center with runpodctl datacenter list, and see GPU types for every device. Stock varies by data center and over time, so give an endpoint two or three pools in priority order when your model fits on more than one.

Step 4: Move your images and code

Pick the path that matches each workload from Step 3.

Path 1: Call a Public Endpoint instead of a Replicate model

There’s no image or code to move. Skip to Step 7 for the client changes.

Path 2a: Run your existing Cog image with cog-worker

cog-worker wraps a Cog model in a Runpod Serverless worker. You build an image that names the Replicate model and version, push it to your registry, and create an endpoint from it:
COG_REPO, COG_MODEL, and COG_VERSION are the owner, model name, and 64-character version ID from the model’s Versions tab on Replicate. The full walkthrough is in Migrate a Cog model.
This path keeps the FastAPI server that Cog uses to run the model inside the worker, so each job passes through an extra layer before it reaches your model, which can add a delay. Use it to move quickly, then port to a native handler. Send one test request to confirm how the worker maps input to your predictor’s arguments and how it returns file outputs before you move traffic.

Path 2b: Port predict.py to a Runpod handler

This is the long-term path. The translation is mechanical: Here’s a complete Cog model before the port. It loads a text-to-image pipeline in setup() and returns a PNG:
cog.yaml
predict.py
Current Cog documentation uses run: "run.py:Runner", BaseRunner, and a run() method. BasePredictor, predict(), and cog predict are deprecated there (cog run -i replaces cog predict -i) but still work, and Replicate’s own guides still show them. The port below is the same for both: setup() becomes module-level code, and predict() or run() becomes the handler.
Here’s the same model as a Runpod worker. The handler validates input, generates the image, and returns a URL if a bucket is configured or base64 if not:
handler.py
What matters in this file:
  • The model loads at module level, so it loads once per worker and stays in memory between jobs. Anything inside handler() runs on every job.
  • validate() returns either errors or validated_input with defaults filled in. Returning {"error": ...} sends your message back to the caller instead of running the model. See Validate inputs.
  • rp_upload.upload_image() reads BUCKET_ENDPOINT_URL, BUCKET_ACCESS_KEY_ID, and BUCKET_SECRET_ACCESS_KEY from the worker’s environment. BUCKET_ENDPOINT_URL must include your bucket name. See S3 bucket configuration.
  • The returned dictionary becomes the job’s output. There’s no fixed output shape, so decide on one and keep it stable for your clients.
The Dockerfile replaces the build section of cog.yaml:
Dockerfile
requirements.txt
The base image already includes PyTorch and CUDA, so requirements.txt only lists what the handler adds. Pin the versions you run on Replicate so outputs match. If your model is on Hugging Face, you can leave the weights out of the image and use cached models instead. Test the handler on a machine with a GPU before you build:
Then build for linux/amd64 and push:
The --platform linux/amd64 flag is required, including on Apple Silicon. Use a version tag or digest, not :latest, so you know which image each worker runs. See Create a Dockerfile and Deploy workers from Docker Hub.

File inputs

If your predictor takes a Path input, the handler receives a string. Download it at the start of the handler:
Clients that passed a local file handle to replicate.run() relied on the Replicate client to upload it. Change those callers to upload the file to your bucket first and send the URL. Request payloads are limited to 10 MB for /run and 20 MB for /runsync, so base64 inputs only suit small files.

Streaming output

A Cog predictor that returns Iterator[str] becomes a generator handler:
handler.py
Each yield is available from /stream/JOB_ID. return_aggregate_stream also makes the collected output available from /status and /runsync. See Streaming handlers.

Concurrency

Cog’s @cog.concurrent(max=N) on an async run() (or concurrency.max in older cog.yaml files) maps to a concurrent handler, where one worker processes several jobs at once.

Path 3: Port to a runpod.App

If you’d rather not maintain a Dockerfile, define the model as a function in the Runpod SDK. The SDK packages your code and dependencies and runs them on Runpod’s runtime images:
main.py
How this lines up with Cog:
  • @app.queue(...) replaces cog.yaml: gpu is the hardware, workers is the scaling range, and dependencies is the package list. torch is already on the built-in GPU images.
  • The function’s parameters are the inputs. A request’s input keys are passed as keyword arguments, so {"input": {"prompt": "..."}} calls generate(prompt="...").
  • Imports go inside the function so your local machine doesn’t need them, and the pipeline is cached in a module-level variable so it loads once per worker.
  • model=sdxl_turbo stages the Hugging Face weights on the host before the worker starts, and sdxl_turbo.path points to them inside the worker.
  • This example returns base64. For larger files, upload to your bucket inside the function and return the URL. Pass bucket credentials with env and runpod.Secret.
Run it against real hardware with rp flash dev main.py. See the SDK quickstart and resource parameters.

Environment variables and secrets

Cog models take secrets as cog.Secret inputs on each prediction. On Runpod, set them once as environment variables on the endpoint and read them with os.environ. Store the value as a Runpod secret and reference it as {{ RUNPOD_SECRET_secret_name }} in the variable’s value, or use runpod.Secret("secret-name") in a runpod.App. Remove secret fields from your request inputs.

Step 5: Move your data and models

Cog models carry their weights in the image or fetch them in setup(), so there’s usually no separate storage to copy. What you need to move is model weights and any files your callers depend on. Weights in your Cog image. If your weights are in the model repository or downloaded during the Cog build, they’re already in your source. Bake them into the worker image as shown in Step 4, or put them on a network volume. Weights from Hugging Face. Use cached models. Runpod tries to start workers on hosts that already have the model and makes it available under /runpod-volume/huggingface-cache/hub/. Set it in the console’s Model field, with --model-reference in runpodctl, or with model=runpod.Model(...) in the SDK. Fine-tuned weights created on Replicate. A Replicate training adds a new version to its destination model. Replicate’s fine-tuning guide describes a Download weights option for trained models, and the training object (GET /v1/trainings/TRAINING_ID) has an output field. Download the weights while you still have access, then put them where your Runpod worker can read them. Large or shared weights. Create a network volume and copy files to it through the S3-compatible API:
The S3-compatible API uses a separate S3 API key and is available in the data centers listed on that page. On Serverless workers the volume is mounted at /runpod-volume, so this file is at /runpod-volume/loras/my-lora.safetensors. A network volume lives in one data center, and an endpoint that mounts it runs its workers there. Choose a data center that has the GPU pool you need (runpodctl datacenter list). Output files your users still reference. Files on replicate.delivery from API predictions are deleted after an hour, so anything you need should already be in your own storage. Update allow-lists and any code that expects replicate.delivery hosts.

Step 6: Deploy on Runpod

Public Endpoints

There’s nothing to deploy. Test the model in the Hub playground, then go to Step 7.

Hub repo

Deploy a Hub repo to a Serverless endpoint in your account:
GPU settings, container disk, and default environment variables come from the Hub release. Override the GPU with --gpu-id and environment variables with --env KEY=VALUE. In the console, open the Hub, select the repo, and click Deploy.

Serverless endpoint from your image

A Replicate deployment is one API call with a model, version, hardware, and instance range:
On Runpod, create a Serverless template that holds the image and environment, then an endpoint that holds the hardware and scaling:
min_instances becomes --workers-min (active workers) and max_instances becomes --workers-max. --gpu-id takes a pool ID or a GPU type ID. Use the template ID that template create returns. For a private registry, create credentials with runpodctl registry create and pass --registry-auth-id to template create. Check runpodctl serverless create --help for the current flags, or see the runpodctl serverless reference. The same endpoint through REST API v2, without a separate template:
type and scaling are required. REST v2 defaults differ from the console: flashboot defaults to OFF, timeout to 300000 ms, and workers.idleTimeout to 10 seconds, so set them explicitly. The response includes the endpoint ID you use in request URLs. In the console, go to Serverless, click New Endpoint, and import from a Docker registry or a GitHub repository. The GitHub integration builds the image for you each time you create a GitHub release, which replaces cog push in CI.

Scaling and cold starts

To reduce cold starts without paying for an always-on worker, enable FlashBoot, use cached models or bake weights into the image, keep the image small, and raise the idle timeout so a worker stays warm between bursts. To remove cold starts for latency-sensitive traffic, set active workers to 1 or more.

runpod.App

Deploy the app from Step 4:
rp flash deploy builds your code and dependencies into an artifact, creates the endpoints, and prints a link to each one. Scaling lives in the decorator (workers=(0, 3), idle_timeout=60), so changing it is a code change followed by another rp flash deploy. See Deploy an app.

Trainings

Replicate trainings call a trainer model’s train() function and push the result to a destination model. On Runpod, run your own training script and write the weights to a network volume that your serving endpoint also mounts. With a Pod:
The volume is mounted at /workspace on the Pod and at /runpod-volume on Serverless workers, so a file the Pod writes to /workspace/outputs/lora is at /runpod-volume/outputs/lora in your handler. Delete the Pod when training finishes. The volume keeps the weights. With the Runpod SDK, @app.task provisions a Pod for each call and terminates it when the function returns:
main.py
Both functions mount the same volume, so generate reads what train wrote, and the SDK places both in the volume’s data center. See Storage.

Step 7: Update the code that calls Replicate

Run a model and wait for the result

Before, with the Replicate client:
After, against a Runpod Public Endpoint:
After, against your own endpoint with the Runpod SDK:
What changes in every caller:
  • The model is in the URL path (/v2/ENDPOINT_ID/...), not in a version field or a /models/OWNER/NAME/ path.
  • The Python runpod.Endpoint client returns the job’s output directly. Over HTTP, the response is a job object and your data is under output.
  • There’s no FileOutput object. You get the URL or base64 string your endpoint returns. Public Endpoint output URLs expire after 7 days, so download what you need to keep.
  • A model error doesn’t raise ModelError. Check for status of FAILED and read error in the job object.

Create a job and poll

Before, with predictions.create and polling:
After, with /run and /status:
Or with the SDK, where endpoint.run() returns a job handle:
Fetch the result within 30 minutes of completion. To cancel, send POST /cancel/JOB_ID in place of prediction.cancel(). If /run times out on the network, look the job up with /status before you resubmit, because a retry creates a second job that’s billed separately.

Webhooks

Before:
After, webhook is a top-level field next to input:
Runpod posts the job result to your URL when the job finishes. Return 200. If the call fails, Runpod retries up to 2 more times with a 10-second delay. Update your receiver for the new payload: status is COMPLETED or FAILED instead of succeeded or failed, and there are no start, output, or logs events. Replace Replicate’s signature verification (validateWebhook, or the webhook-signature HMAC check) with your own check, such as a token in the URL path plus a GET /status/JOB_ID call to confirm the job.

Streaming

Before:
After, for a Public Endpoint LLM or a vLLM worker, use the OpenAI-compatible route:
After, for your own generator handler, submit with /run and read /stream:
For your own vLLM endpoint, the base URL is https://api.runpod.ai/v2/ENDPOINT_ID/openai/v1. See OpenAI compatibility.

Deployments

Calls to POST /v1/deployments/OWNER/NAME/predictions (or replicate.deployments.predictions.create) become calls to your endpoint’s /run or /runsync. The request body keeps its input object, and webhook stays a top-level field.

Calls from a runpod.App

If the caller is Python and the target is a deployed runpod.App, you can call the function by name instead of by endpoint ID:

Control-plane automation

Scripts that manage Replicate resources map to runpodctl or REST v2. See the API map below. The most common change is CI: replace cog login and cog push r8.im/OWNER/MODEL with docker build, docker push, and an update of the template’s image (runpodctl template update TEMPLATE_ID --image NEW_IMAGE), or connect the repository through the GitHub integration.

Step 8: Verify, cut over, and shut down

1

Send a real request from outside

Call the Runpod endpoint from the same environment your application runs in, with a production-shaped input:
The first request includes a cold start. Check that status is COMPLETED, and note delayTime (queue time, including cold start) and executionTime, both in milliseconds.
2

Compare outputs and timing

Run the same inputs with fixed seeds on both platforms. Compare outputs, compare executionTime with Replicate’s metrics.predict_time, and measure cold and warm latency separately. Differences in GPU, library versions, and model variant all change results, so pin versions before you conclude anything.
3

Check failure paths

Send an invalid input and confirm you get the error shape your client expects. Send a job that exceeds your execution timeout and confirm it ends in TIMED_OUT. Trigger a webhook and confirm your receiver handles the Runpod payload.
4

Run both in parallel and shift traffic

Put the provider behind a flag in your client and send a small share of traffic to Runpod. Watch the endpoint’s Metrics tab for delay time, execution time, and failures, and worker logs for errors. Increase the share as the numbers hold, and raise max workers as traffic moves.
5

Stop billing on Replicate

Once Replicate receives no traffic, scale each deployment to zero, then delete it. Replicate only deletes deployments that have been offline and unused for at least 15 minutes:
If Replicate rejects max_instances of 0 for your deployment, set the instance counts in the deployment’s Settings tab instead. Private models bill while their instances are online, so confirm on your Replicate dashboard that nothing is still running. Download any fine-tuned weights you want to keep before you delete models, then disable the API tokens your application used at replicate.com/account/api-tokens.
6

Set cost guards on Runpod

Keep active workers at 0 unless you need warm capacity, and set max workers to the concurrency you intend to pay for. Set an execution timeout so a stuck job can’t run indefinitely. Runpod also reduces max workers on endpoints that receive no requests for several days. Review your account’s spending limit, and delete network volumes you no longer need, since they bill for storage until deleted.

Reference

API map

CLI command map

Configuration field map

Environment variable map

Status map

Migrate with a coding agent

A coding agent with the Runpod skills and MCP server can do the inventory, write the handler, deploy the endpoint, and update your client code. Install both first.
The MCP server signs in with OAuth (“Sign in with Runpod”), so you don’t need an API key to start. See Agent skills and MCP servers for agent and client IDs. Then give your agent this prompt from the root of the repository that uses Replicate:

Troubleshooting

The request returns 400 or the handler reports missing input. The body must be {"input": {...}}. Remove Replicate-only fields such as version and stream from the top level. With a runpod.App, the keys inside input must match the function’s parameter names. The job stays IN_QUEUE. No worker is available. Check that max workers is above 0, that the GPU pool is available in the allowed data centers (runpodctl datacenter list), and that the image pulls. Add a second GPU pool, or remove a data center restriction. An endpoint that received no requests for several days may have had its max workers reduced, so raise it again. The worker starts and then exits, or never becomes ready. The image was built for the wrong architecture, the handler isn’t started, or the model takes too long to load. Build with --platform linux/amd64, make sure the container runs python -u /handler.py and that the file calls runpod.serverless.start, and raise RUNPOD_INIT_TIMEOUT for slow model loads. Read the worker logs in the endpoint’s Logs tab. Every request is slow. The model is loading inside handler(). Move it to module level so it loads once per worker, as setup() did in Cog. The job ends in TIMED_OUT although the model works. The job ran past the endpoint’s execution timeout, or it expired in the queue. Raise the execution timeout on the endpoint or set policy.executionTimeout on the request. Replicate allowed 30 minutes by default, and the Runpod console default is 10 minutes. /status returns 404, or the output is missing. The result expired. Results are kept for 30 minutes after an async job completes and 1 minute after a sync job. Fetch sooner, use a webhook, or write outputs to your own bucket. The response fails with a payload size error, or the client receives a truncated result. The output is over the 10 MB (/run) or 20 MB (/runsync) limit, usually because of base64 files. Upload to a bucket and return the URL. rp_upload fails or returns an unusable URL. Check that BUCKET_ENDPOINT_URL, BUCKET_ACCESS_KEY_ID, and BUCKET_SECRET_ACCESS_KEY are set on the endpoint, and that BUCKET_ENDPOINT_URL includes the bucket name. Workers read environment variables at start, so new values apply to new workers. CUDA or driver errors on some workers only. Workers in a pool can land on hosts with different driver versions. Set a minimum CUDA version on the endpoint (--min-cuda-version 12.8, or gpu.minCudaVersion in REST v2) that matches your image. Client code breaks on status checks. Runpod statuses are uppercase and spelled differently: COMPLETED, not succeeded, and CANCELLED with two Ls. Update comparisons using the status map.

Next steps

Handler functions

Write standard, streaming, and concurrent handlers.

Endpoint settings

Tune workers, timeouts, FlashBoot, and GPU priority.

Public Endpoints model reference

See every hosted model, its parameters, and its pricing.

Runpod SDK

Build apps with @app.queue, @app.task, and @app.api.
Last modified on November 14, 2026