How LLM API pricing actually works

Updated August 20, 2026 · All prices verified against official provider pages.

Every major LLM provider bills the same way at the core: per token, with separate rates for input and output, quoted per million tokens. Everything else — caching, batch tiers, context surcharges — is a modifier on that core. This guide is the full mental model, using current, verified prices.

Input vs output: the 3–6x asymmetry#

Input tokens (your prompt) are cheap to process; output tokens (the model's response) are generated one at a time and cost more. The ratio is remarkably consistent — GPT-5.6 bills $5.00 in / $30.00 out per 1M (6x), Claude Sonnet 5 $2.00 / $10.00 (5x), Gemini 3.7 Flash $0.75 / $3.75 (5x).

The practical consequence: workload shape matters as much as model choice. A RAG app stuffing 20K tokens of context to produce a 200-token answer is an input-heavy workload — it cares about the input rate. A code generator emitting long files is output-heavy. Comparing models on input price alone regularly picks the wrong model.

Cached input: the biggest discount on the page#

When requests share a prompt prefix — a system prompt, few-shot examples, a long document — providers serve the repeated part from a cache and bill it at a cached-input rate, typically ~90% off:

Two catches: Anthropic and OpenAI's GPT-5.6 family bill cache writes at a premium, and Google bills cache storage by the token-hour. The details are in the prompt caching guide.

Reasoning tokens bill as output — including the ones you never see#

Reasoning models think before they answer, and that hidden thinking bills at the output rate. Google's price page says it outright: output prices "include thinking tokens." A 300-token visible answer can carry thousands of billed reasoning tokens behind it — the reasoning-token guide covers how to see them.

Service tiers: the same tokens at half or double price#

Most providers now sell the same model at multiple prices:

• Batch (asynchronous, typically within 24h): 50% off at OpenAI, Anthropic, Google, and Fireworks. xAI gives 20%, and only on some models.
• Flex / off-peak: OpenAI's Flex matches its Batch pricing for supported models. DeepSeek halves everything outside its peak UTC windows.
• Priority / Fast: OpenAI's Fast mode runs roughly 2x (varies by model), Google's Priority 1.8x, xAI's 2x, Anthropic's Fast-mode preview 2x.

If a nightly pipeline is running at the standard synchronous rate, you are donating a 50% margin back to your provider.

Long-context surcharges reprice the whole request#

On several flagships, exceeding a context threshold doesn't reprice the excess — it reprices every token in the request:

• OpenAI GPT-5.4+: above ~272K input → roughly 2x input, 1.5x output.
• Google Pro-class: above 200K prompt tokens → $2.00 → $4.00 input on Gemini 3.1 Pro.
• xAI: every Grok model, at 200K → roughly 2x everything.
• Anthropic: notably, no surcharge — Claude 4.6+ models take 1M tokens at standard rates.

One oversized request can cost double what the calculator said. Watch for it.

Dated snapshots: the same name, a different price#

gpt-4o costs $2.50/$10.00 — but the snapshot gpt-4o-2024-05-13 costs $5.00/$15.00, and API responses echo the dated snapshot even when you requested the alias. Whatever prices your cost tracking, it must price the response-reported model, not the model you asked for.

What a price table can't tell you#

List prices answer "what does a token cost." They can't answer "what did my product spend, on which customer, in which feature" — that takes per-call tracking with real token counts from real responses. That's the gap Marginal fills: one track() call per LLM response, priced server-side against a daily-synced catalog, sliced by any field you define.

Browse current prices for every major model or estimate a workload with the cost calculator.

List prices are the easy part.

Marginal tracks what you actually spend — every LLM call priced at that day's rates and sliced by customer, feature, or any field you define. Integration is one track() call.

Start trackingRead the docs