OpenAI Batch API: half price for anything that can wait a day

What the OpenAI Batch API is

The Batch API is an asynchronous way to send many API requests at once. Instead of calling the model and waiting for each answer, you upload a file of requests, OpenAI runs them when it has spare capacity, and you download a file of answers. In exchange for giving up the immediate answer, every token is billed at half price.

The requests themselves are ordinary: each line of the file is the same body you would send to /v1/responses, /v1/chat/completions, /v1/embeddings, /v1/completions, /v1/moderations, /v1/images/generations or /v1/images/edits. The model, the prompt and the output format do not change. Only the delivery does.

The rule of thumb: if nobody is waiting on the answer, it should not be billed at the interactive price. Nightly evaluations, labelling a dataset, summarizing yesterday's support tickets, embedding a document archive and re-running a prompt change across past inputs all qualify.

How to use the Batch API, step by step

A batch takes four calls: write a JSONL file, upload it, create the batch, then download the results when it finishes. The example classifies support tickets with the official Python SDK.

1. Write one request per line

{"custom_id": "ticket-1042", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-6.1-sol", "messages": [{"role": "system", "content": "Classify the ticket as billing, bug, how-to or other. Reply with the label only."}, {"role": "user", "content": "I was charged twice this month."}], "max_completion_tokens": 10}}
{"custom_id": "ticket-1043", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-6.1-sol", "messages": [{"role": "system", "content": "Classify the ticket as billing, bug, how-to or other. Reply with the label only."}, {"role": "user", "content": "The export button does nothing on Safari."}], "max_completion_tokens": 10}}

2 and 3. Upload the file and create the batch

import json
from openai import OpenAI

client = OpenAI()

with open("requests.jsonl", "w") as f:
    for t in tickets:
        f.write(json.dumps({
            "custom_id": f"ticket-{t['id']}",
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": {
                "model": "gpt-6.1-sol",
                "messages": [
                    {"role": "system", "content": INSTRUCTIONS},  # identical on every line: cacheable
                    {"role": "user", "content": t["text"]},
                ],
                "max_completion_tokens": 10,
            },
        }) + "\n")

batch_file = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
    input_file_id=batch_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",            # the only window available
    metadata={"job": "ticket-labels"},  # optional, shows up when you list batches
)
print(batch.id, batch.status)  # batch_abc123 validating

4. Download the results and match them by custom_id

batch = client.batches.retrieve("batch_abc123")

if batch.status in ("completed", "expired", "cancelled"):
    labels = {}
    if batch.output_file_id:
        for line in client.files.content(batch.output_file_id).text.splitlines():
            r = json.loads(line)
            labels[r["custom_id"]] = r["response"]["body"]["choices"][0]["message"]["content"]

    failed = []
    if batch.error_file_id:  # failed and expired requests land here, not in the output file
        for line in client.files.content(batch.error_file_id).text.splitlines():
            failed.append(json.loads(line)["custom_id"])

A batch moves through validating, in_progress and finalizing to completed. It ends in failed if the input file is invalid, expired if the 24 hours run out, or cancelled if you call client.batches.cancel(batch_id); cancelling takes up to 10 minutes while in-flight requests finish. The output file is deleted 30 days after the batch completes, so copy results somewhere you keep.

What the Batch API costs, and how it stacks with caching

Batch prices are exactly half of standard prices, and the cached input rate gets the same cut. Prompt caching is automatic on OpenAI models: a request that starts with the same long prefix as a recent one is billed at the cached rate for that prefix. In a batch where every request shares the same instructions, most of the input is that prefix.

Short-context prices (up to 272K input tokens) from OpenAI's pricing page on 2026-10-11. To see what your own prompt costs at these rates, paste it into the token counter.

A worked example: 100,000 tickets

Illustrative: classify 100,000 support tickets on gpt-6.1-sol. Each request carries 1,500 tokens of shared instructions and examples, 500 tokens of ticket, and returns 100 tokens. That is 200M input tokens and 10M output tokens.

The cached rows assume every request hits the cache; real hit rates are lower, so treat them as the ceiling. Two things stand out. Caching and batching together cut the bill by almost 80%, and both need the same discipline: the shared part goes first and is byte-identical on every line. And once the instructions are cached, the variable part of each request is most of the input bill, which neither discount touches.

Batch API or Flex processing?

Flex processing gives you Batch API prices on a normal synchronous request. Set service_tier to "flex" and the request is billed at batch rates, with prompt caching on top. The trade is slower responses and occasional 429 Resource Unavailable errors, which you are not charged for. OpenAI's examples raise the client timeout to 15 minutes.

response = client.with_options(timeout=900.0).chat.completions.create(
    model="gpt-6.1-sol",
    service_tier="flex",   # batch prices, synchronous call
    messages=[...],
)

Pick Flex when each step needs the previous answer, as in a background agent or a multi-step pipeline, or when the job is too small to be worth a file. Pick the Batch API when you have thousands of independent requests, want a separate rate limit pool, or need a model Flex does not support yet.

What breaks in production

The OpenAI forum threads that rank for this topic are mostly about the same handful of surprises. Each has a simple fix once you know it is there.

• Polling. You do not have to poll. Register a webhook endpoint in the dashboard and OpenAI sends batch.completed, batch.failed, batch.expired or batch.cancelled; the SDK's client.webhooks.unwrap() verifies the signature. Keep a slow poll (every 10 to 30 minutes) as a fallback for missed deliveries.
• Completed does not mean succeeded. Batch status is about the job, not the requests. A completed batch can have failed lines in the error file, and a request can return a 200 that is truncated or off-format. Validate each answer and requeue failures in a new batch.
• Expiry keeps what finished. If 24 hours pass, finished requests stay in the output file and are billed; unfinished ones go to the error file with code batch_expired. Read their custom_ids and submit them again.
• The queued-token limit. Each model has a cap on input tokens waiting across all your batches, shown in your organization's limits page. A batch that would exceed it fails validation, so split big jobs and submit the next part when one finishes. Count tokens before submitting rather than guessing.
• Results out of order. The output file is not in input order. Always map by custom_id, and make it something you can join on (a ticket ID, a row key), not a line number.
• One model per file, 200 MB per file. Mixing models fails validation. For images, a URL instead of base64 keeps the file small.
• "My batch results differ from my live results." The same prompt can give different answers on any two calls unless you pin the model snapshot and keep sampling settings identical. Batching does not change the model; check that the body really is the same.
• Conversations. Each line is independent: a batch cannot use the answer of one request in the next. Multi-turn work needs one batch per turn, or Flex.

Batch APIs at Anthropic and Google

The other two big providers sell the same deal: half price for work that can wait up to a day. The limits differ, which matters if you run the same job across providers.

Sources: Anthropic's batch processing docs and Gemini's Batch Mode docs. For Claude, batches often run longer than the 5-minute cache, so Anthropic suggests the 1-hour cache for shared context; the trade-offs are in our Anthropic prompt caching guide.

Batching halves the price; it does not shrink the prompt

The Batch API discounts every token you send, including the ones the model never needed. Batch jobs are where this adds up: they are large by definition, they often carry whole documents, retrieved passages or raw records per line, and nobody reads the prompts because nobody is waiting on them.

The order that keeps the most money: first remove what the model does not need from the variable part of each request (quoted email threads, signatures, boilerplate, irrelevant retrieved chunks, empty JSON fields), then put the shared instructions first so they cache, then batch. In the example above, trimming each ticket from 500 to 250 tokens saved another 23% after both discounts were already applied.

The full list of levers, from caching to model choice, is in the LLM cost optimization playbook. For retrieved context specifically, see RAG context compression.

Frequently asked questions