Skip to main content
This guide is for teams that call Fireworks AI from production code and want to run the same workloads on Runpod. Fireworks Serverless inference maps to either a Runpod Public Endpoint (when Runpod hosts the model) or your own OpenAI-compatible endpoint on Runpod Serverless. Fireworks on-demand deployments, custom models, and LoRA adapters map to a Serverless endpoint running the vLLM worker, and training maps to a Pod. When you finish, your application sends the same OpenAI-format requests to a Runpod base URL, and nothing is left billing on Fireworks.
You can hand most of this migration to a coding agent. See Migrate with a coding agent.

How Fireworks AI maps to Runpod

Fireworks is a model platform: you pick a model ID and Fireworks runs the inference stack. Runpod is a compute platform with a hosted model catalog on top. On Runpod, you either call a model Runpod already hosts, or you run an open-source inference server (vLLM) on GPUs you configure.

What’s different

What works differently

You choose the model weights and the serving flags. A Fireworks model ID hides the checkpoint, precision, chat template, and tool-call parser. On a vLLM endpoint, you pick the Hugging Face repo and set those yourself with environment variables. Output can differ from Fireworks for the same model family if the quantization or chat template differs, so compare outputs before you cut over. Billing moves from tokens to time for your own endpoints. Fireworks Serverless bills input, cached input, and generated tokens. A Runpod Serverless endpoint bills per second while a worker runs, which includes the cold start, request processing, and the idle timeout. Public Endpoints bill per use (per token for text models). A workload with steady traffic and high GPU utilization behaves differently from a spiky, low-volume one, so estimate from your own request logs. A cold endpoint queues requests instead of rejecting them. A Fireworks deployment scaled to zero returns 503 with DEPLOYMENT_SCALING_UP and doesn’t queue the request, so clients retry. A Runpod queue-based endpoint holds the request in its queue while a worker starts. The trap is the synchronous path: /runsync and the /openai/v1 route sit behind an edge timeout of about 100 seconds, so a first request that waits on a large model load can return a 524. Use /run plus /status for the first request, raise the idle timeout, or keep an active worker. Scale-down happens much sooner by default. Fireworks’ default scale-to-zero window is 1 hour (minimum 5 minutes). Runpod’s console default idle timeout is 5 seconds, and the maximum is 3,600 seconds. Set it deliberately. Workers are billed while they wait. Autoscaling signals are different. Fireworks scales on load targets such as concurrent_requests or tokens_generated_per_second. Runpod scales on queue delay (seconds a request waits) or request count (queued plus in-progress requests divided by a target). The vLLM worker’s MAX_CONCURRENCY (default 30) sets how many requests one worker takes at once. Idle resources are cleaned up differently. Fireworks deletes a deployment with min replicas of 0 after 7 days without traffic. Runpod reduces an idle endpoint’s max workers to 2 after 3 days and to 0 after 7 days, and keeps the endpoint. Raise max workers to use it again. Model names change. accounts/fireworks/models/llama-v3p1-8b-instruct becomes the Hugging Face repo the worker serves, for example meta-llama/Llama-3.1-8B-Instruct. Set OPENAI_SERVED_MODEL_NAME_OVERRIDE to serve it under a name of your choice. Reasoning output uses a different field. Fireworks returns reasoning in reasoning_content. vLLM v0.32.0 (the version the current vLLM worker release is built on) returns it in reasoning. Read both fields in your client. Reasoning also needs REASONING_PARSER set on the worker. Fireworks request extensions don’t carry over. Remove service_tier, context_length_exceeded_behavior, perf_metrics_in_response, and the x-session-affinity header unless you’ve confirmed the target accepts them. Fireworks lowers max_tokens when a request would exceed the context window. On Runpod, set max_tokens explicitly. The Runpod vLLM docs list a default of 16. Token usage in streams. Fireworks includes usage in the last streamed chunk by default. If your code reads it, test that path against your Runpod endpoint. Rate limits are per endpoint, not per model. Fireworks Serverless enforces adaptive tokens-per-minute limits per model. Runpod enforces request-rate limits per endpoint and operation (see rate limits). Throughput on your own endpoint is bounded by your workers.

What has no direct equivalent

Going the other way, Runpod gives you things Fireworks deployments don’t: any container image and any inference engine, SSH access to Pods, network volumes shared between training and serving, a job queue with webhooks and /status polling, and a batch jobs API while Fireworks’ Batch API is paused.

Requirements

  • A Runpod account with credits and a Runpod API key.
  • firectl signed in to the Fireworks account you’re migrating (firectl whoami shows the account).
  • A Hugging Face access token if any model is gated or private.
  • Your Fireworks request logs or billing export, to size GPUs and estimate cost.

Step 1: Take inventory of what you run on Fireworks AI

List every resource in the account. Add --output json (or -o json) to capture the full configuration.
For each deployment, export the settings you’ll translate in Step 3 and Step 6.
Record the base model, deployment shape, accelerator type and count, precision, min and max replica counts, the three scaling windows, load targets, region, max context length, and whether addons are enabled. replicaStats.readyReplicaCount shows how many replicas are serving. Serverless usage has no resource to list, so find it in code. This command finds SDK imports, base URLs, model IDs, and environment variables.
For each call site, write down the model ID, the route (/chat/completions, /completions, /embeddings, /rerank, /responses, or the Anthropic-compatible /v1/messages), and which features it uses: streaming, tools, response_format, vision input, reasoning_effort, or service_tier. Gateways such as LiteLLM reference Fireworks as fireworks_ai/MODEL, so check their config files too.
Fireworks’ docs say the Batch API is paused and that stored Batch input datasets and job outputs will be permanently deleted. If you have Batch data there, download it now with firectl dataset download DATASET_ID --output-dir ./data.

Step 2: Set up Runpod

Create an API key on the API keys page, then install the CLI and export the key.
runpodctl doctor stores the key and verifies the setup. You need runpodctl 2.4.0 or later for --model-reference. Check with runpodctl version. Inference clients keep using the OpenAI SDK, so there’s nothing else to install for them. If you also want to build custom workers in Python, install the Runpod SDK with pip install runpod (see the SDK overview). To let a coding agent manage resources, connect the Runpod MCP server.

Step 3: Choose where each workload lands

Use this table to place each item from your inventory.

Models that Runpod hosts

Build your model map from the Public Endpoints model reference, which lists image, video, audio (text-to-speech), and text models with their prices. These are the text models in the model reference today. Confirm that a model is live in the Hub before you map a call site to it. The OpenAI-compatible base URL for a Public Endpoint is https://api.runpod.ai/v2/SLUG/openai/v1. Every Public Endpoint also accepts the native routes: https://api.runpod.ai/v2/SLUG/runsync and /run. For Fireworks model IDs that match these, such as accounts/fireworks/models/kimi-k3, compare the model card on each side before you assume they’re the same checkpoint and precision. For everything else, the general answer is your own vLLM endpoint: Fireworks’ serverless catalog is larger than Runpod’s Public Endpoints catalog.

GPU mapping

Fireworks accelerator types translate to Runpod GPU pools (for Serverless) or GPU type IDs (for Pods) as follows. IDs come from the GPU types reference. You aren’t limited to the hardware Fireworks used. A model that Fireworks served on an 80 GB card at FP8 may fit a 48 GB pool (AMPERE_48, ADA_48_PRO) or a 24 GB pool (AMPERE_24, ADA_24) depending on its size and quantization. Pick the GPU that fits the model and your latency target, and list more than one pool so workers have a fallback. Stock varies by data center, so check it before you commit to a pool.

Step 4: Move your images and code

Most Fireworks users have no container images to move, because Fireworks runs the inference server. On Runpod, the prebuilt vLLM worker fills that role, so this step is about translating serving configuration into environment variables. The vLLM worker turns any vllm serve flag into an environment variable in upper snake case (--max-model-len becomes MAX_MODEL_LEN). VLLM_EXTRA_ARGS passes flags that have no variable. Check the model’s Hugging Face README and the model-specific settings for the parser and tokenizer values your model family needs. If the model needs a vLLM version newer than the Hub worker ships, use the upstream vllm/vllm-openai image on a load balancing endpoint instead (see Step 6). Any image you build yourself must be linux/amd64. If you run workloads vLLM doesn’t serve (embeddings with a different server, rerankers, image models), you write a worker. A queue worker is a Python handler started with runpod.serverless.start({"handler": handler}). See handler functions and deploy a worker.

Step 5: Move your data and models

Base models from the Fireworks library

If you deploy a Fireworks-hosted model (accounts/fireworks/models/NAME), don’t move weights. Find the matching Hugging Face repo and reference it. Runpod caches the model on worker hosts.

Custom models and fine-tuned weights

Download each custom model and adapter from Fireworks.
Fireworks’ docs note that an adapter is tied to the exact base checkpoint it was trained against, which can differ from the public Hugging Face weights. Download the base with firectl model download too if you aren’t sure, or ask Fireworks which base was used. To serve a fine-tune as a single model, merge the adapter into the base. This follows the PEFT recipe in Fireworks’ own docs.
merge.py
The merged directory is a standard Hugging Face checkpoint in BF16. Then put the weights where a Runpod worker can load them. You have three options.

LoRA adapters without merging

To keep adapters separate (the equivalent of Fireworks multi-LoRA), push each adapter to Hugging Face and load it by name. LORA_MODULES takes a JSON list.
Clients then set model to the adapter name (support-adapter) instead of Fireworks’ MODEL#DEPLOYMENT form. Set MAX_LORA_RANK to at least your adapter’s rank (the default is 16), and MAX_LORAS to the number of adapters used in one batch.

Datasets and training

Fireworks training datasets use the OpenAI chat format (messages with role and content), which most open-source trainers read directly. Download them, then train on a Pod that writes to a network volume.
The Pod and the volume must be in the same data center, so confirm the GPU has stock there before you create the volume. Run your trainer over SSH (for example PEFT with the Transformers Trainer, or Axolotl), write checkpoints under /workspace, then delete the Pod with runpodctl pod delete POD_ID. The volume keeps the adapter. See connect to a Pod and the fine-tuning overview. Runpod has no managed counterpart to Fireworks’ Serverless Training, reinforcement fine-tuning, or DPO jobs. You bring the training loop, and a Pod bills per second until you stop or delete it.

Step 6: Deploy on Runpod

Public Endpoint

There’s nothing to deploy. Confirm the model works for your prompts in the Hub playground, then go to Step 7.

vLLM queue endpoint from the Hub

This is the default target for on-demand deployments and for serverless models Runpod doesn’t host. The before and after below deploy Llama 3.1 8B Instruct with scale to zero and up to three replicas.
What matters in the Runpod command:
  • --hub-id takes the listing ID that runpodctl hub search vllm returns. Replace HUB_ID with the ID from your own search output.
  • --gpu-id accepts a pool ID or a GPU type ID. Always pass it. Without it, the endpoint uses the Hub release’s default pools, which may be larger GPUs than you need.
  • --model-reference attaches the cached model. It takes the full Hugging Face URL with a ref (:main, a tag, or a commit), and it’s GPU only. Each endpoint supports one cached model. Keep MODEL_NAME pointed at the same repo.
  • --idle-timeout 600 mirrors the 10-minute scale-to-zero window. Lower it to pay for less idle time, or raise it (up to 3,600) to avoid cold starts.
  • --scale-by delay --scale-threshold 4 adds workers when requests wait longer than 4 seconds. Request count scaling (--scale-by requests) is the closer match to a concurrent_requests load target and is the recommended scaler for LLM workloads, but REST API v2 rejects an idle timeout on a queue-based endpoint that scales on request count. This example keeps the long idle timeout, so it scales on queue delay. If you prefer request count scaling, drop the idle timeout and use active workers to stay warm.
  • --workers-min is the number of active workers. Set it to 1 or more to mirror --min-replica-count above 0. Active workers bill continuously.
  • HF_TOKEN references a Runpod secret named hf_token. Llama models are gated, so create the secret first. Public models don’t need a token.
  • --env applies with --hub-id. With --template-id, environment variables come from the template.
The command prints the endpoint ID. In the console, the same deployment is Hub > vLLM > Deploy, with the model in the Model field. See the vLLM quickstart. A new endpoint isn’t ready when the create call returns. Poll its health until a worker is ready.
The first deployment can take several minutes while hosts pull the worker image and the model. You aren’t billed for a cached model download.

vLLM load balancing endpoint (alternative)

Use this when you need a specific vllm/vllm-openai tag or want requests to reach vLLM directly with no queue. Create it with REST API v2. The args string holds the vllm serve arguments, and the three environment variables tell Runpod where the health check lives.
The OpenAI base URL for this endpoint type is https://ENDPOINT_ID.api.runpod.ai/v1, and the gateway checks your Runpod API key, so don’t also set a vLLM API key. This image is a CUDA 13 build, which is why the request sets minCudaVersion. Weights download to the container disk, so size disk for the model plus headroom. Load balancing endpoints don’t have a queue, /run, webhooks, or batch jobs.

Translate the rest of your deployment settings

Step 7: Update the code that calls Fireworks AI

For chat and text completions, three values change: the base URL, the API key, and the model name. Put the base URL and model name behind configuration so you can switch back during cutover.

Python (OpenAI SDK)

Always set max_tokens. Qwen3 is a reasoning model, so a small max_tokens can end the response inside its thinking block.

Python (Fireworks SDK)

The Fireworks SDK (from fireworks import Fireworks) exposes the same chat.completions.create call shape, so inference code moves to the OpenAI SDK with a changed import and constructor.
Replace fireworks-ai with openai in your dependencies once no code imports it. Code that uses the Fireworks SDK for platform automation (client.deployments.create(...), datasets, training jobs) moves to runpodctl or REST API v2. See the API map.

TypeScript

If you use the Vercel AI SDK, the @runpod/ai-sdk-provider package works with Public Endpoints and with your own endpoint IDs.

curl

Tool calling

The request and response format stays the OpenAI tools format. On your own endpoint, it works only when the worker has ENABLE_AUTO_TOOL_CHOICE=true and a TOOL_CALL_PARSER that matches the model, as set in Step 6.
If tool_calls comes back empty and the call appears as text in content, the parser doesn’t match the model’s tool format.

Feature parity for the OpenAI-compatible surface

Each cell reflects what the platform’s docs state. “Check the model” means the docs don’t settle it, so test before you rely on it.

Batch and offline inference

Fireworks’ Batch API used JSONL files with custom_id and body per line. On a Runpod queue endpoint, each request is a job. The vLLM worker’s native input takes messages and a sampling_params object. This script submits a Fireworks-format batch file as Runpod jobs and records which job belongs to which custom_id.
submit_batch.py
Fetch each result with GET https://api.runpod.ai/v2/ENDPOINT_ID/status/JOB_ID, or receive it at the webhook URL. Async results are kept for 30 minutes after completion, so collect them promptly or use webhooks. /run accepts 1,000 requests per 10 seconds per endpoint, so pace large files. For very large offline workloads, batch jobs run on dedicated batch workers and take a JSON array of the same {"input": ...} objects.

Image and audio models

Public Endpoints for image, video, and text-to-speech models use the native job format, not the OpenAI format. A request is POST https://api.runpod.ai/v2/SLUG/runsync (or /run) with a body of {"input": {...}}, and the result is under output (for example output.image_url).
Output URLs expire after 7 days, so download results you need to keep. Each model’s page lists its parameters.

Control-plane automation

Scripts and CI jobs that call firectl or the Fireworks REST API move to runpodctl or REST API v2. See the CLI command map and the API map.

Step 8: Verify, cut over, and shut down

1

Send a real request from outside

Use the same prompts your application sends, from the same network. For a cold endpoint, submit the first request asynchronously so the model load doesn’t hit the synchronous timeout.
When the job is COMPLETED, a worker is warm and the OpenAI route responds normally.
2

Compare outputs and latency

Replay a sample of production prompts against both platforms. Compare output quality, tool-call correctness, JSON validity, time to first token, and tokens per second. Differences usually trace to quantization, chat template, or parser settings.
3

Measure cost on real traffic

Run representative load, then read worker time from the endpoint’s Metrics tab or runpodctl billing serverless. Tune the idle timeout, active workers, and GPU pool from that data.
4

Run both in parallel and shift traffic

Switch the base URL and model name through configuration for a small share of traffic, watch error rates and latency, then increase the share. Keep the Fireworks path callable until you’re done.
5

Shut down Fireworks AI

After traffic has moved and you’ve downloaded any weights and datasets you need, stop the resources that bill.
Fireworks bills reserved capacity until the reservation ends whether or not you use it, so talk to your Fireworks account manager about any reservation. Serverless usage stops billing when you stop sending requests.
On the Runpod side, guard cost with three settings: max workers caps concurrency and spend, active workers of 0 lets the endpoint scale to zero, and the execution timeout stops runaway jobs. Delete test endpoints with runpodctl serverless delete ENDPOINT_ID, and check for leftover network volumes with runpodctl network-volume list.

Reference

CLI command map

API map

Note the two Runpod hosts: api.runpod.ai serves requests to endpoints, and api.runpod.io serves the management API. Both take Authorization: Bearer RUNPOD_API_KEY.

Environment variable map

These are client-side variables, since Fireworks doesn’t run your containers.

Status map

Request-level states on a Runpod queue endpoint are IN_QUEUE, IN_PROGRESS, COMPLETED, FAILED, CANCELLED, and TIMED_OUT. Fireworks chat completion requests are synchronous and have no job state.

Migrate with a coding agent

Install the Runpod skills and MCP server in your agent. MCP sign-in uses OAuth (“Sign in with Runpod”), so you don’t need an API key to start.
See agent skills and MCP servers for agent and client IDs. Then give the agent this prompt from the root of your repository.

Troubleshooting

The first request returns a 524 or times out. The synchronous routes have an edge timeout of about 100 seconds, and the first request to a cold worker waits for the model to load. Send the first request with /run and poll /status, raise the idle timeout so workers stay warm between requests, or set one active worker. “Invalid model” or a 404 on the model name. You sent the Fireworks ID. The model value must be the Hugging Face repo the worker serves or your OPENAI_SERVED_MODEL_NAME_OVERRIDE value. List what the endpoint serves with client.models.list(). Responses are cut off after a few tokens. Fireworks adjusts max_tokens to the context window for you. The Runpod vLLM docs list a default max_tokens of 16. Set it on every request. Tool calls come back as plain text. The worker is missing ENABLE_AUTO_TOOL_CHOICE=true, or TOOL_CALL_PARSER doesn’t match the model’s tool format. See the model-specific settings. The reasoning trace disappeared. Your client reads reasoning_content. vLLM v0.32.0 returns reasoning. Read both, and set REASONING_PARSER on the worker. Without a parser, thinking text arrives inside content. Workers stay throttled or never start. The selected pool has no free GPUs in the allowed data centers, or the endpoint’s CUDA constraint excludes the available hosts. Add a second GPU pool, remove data center restrictions, and check stock with runpodctl gpu list. A network volume restricts the endpoint to one data center. The worker crashes while loading the model. Usually out of memory. vLLM reserves VRAM for the KV cache on top of the weights. Lower MAX_MODEL_LEN, use a quantized repo, or move to a larger pool. Gated models fail to download without a valid HF_TOKEN. Output quality differs from Fireworks. Check that the Hugging Face repo matches the checkpoint and precision Fireworks served, and that the chat template is the model’s own. For fine-tunes, confirm you merged the adapter into the exact base it was trained on. The bill is higher than expected at low traffic. A long idle timeout or an active worker bills for time with no requests. Lower the idle timeout, set active workers to 0, or move low-volume models to a Public Endpoint if one is listed. 429 responses. On Runpod, a 429 is the endpoint’s request-rate limit, not a sign that workers are saturated. Queue-based endpoints hold excess requests in the queue, so watch queue delay and raise max workers.

Next steps

vLLM on Serverless

Deploy and configure OpenAI-compatible LLM endpoints.

Public Endpoints

Browse the models Runpod hosts and their parameters.

Endpoint settings

Tune workers, idle timeout, and autoscaling.

Cached models

Cut cold starts by caching Hugging Face models on hosts.
Last modified on November 14, 2026