LLM API Pricing: How Per-Token Costs Actually Work
A capability of the Behest AI Token FinOps platform.
Last updated:
A practical, finance-toned guide for engineers and finance owners pricing AI work: what you actually pay for per token, why the headline rate rarely predicts the bill, and how to compare providers on real cost instead of a rate card.
The short version
How does LLM API pricing work?
LLM API pricing is per-token: you pay separately for input, output, and reasoning tokens, with output priced highest. List price rarely predicts your bill — retries, long context, and agent loops multiply calls. Behest's AI Token FinOps control center attributes and caps the real cost.
Why the rate card doesn't tell you the cost
Provider pricing pages advertise a per-token rate, and it's easy to assume your cost is just that rate times your usage. It almost never is. The published number is one input into a bill shaped by how many tokens each task really moves — across input, output, and reasoning — and by how many times a single user action turns into a billed model call.
That's why this guide focuses on mechanics rather than today's numbers: rates change often and vary by model, but the way per-token pricing behaves is stable. Understand the mechanics and you can read any provider's pricing page, estimate a realistic cost, and — more importantly — see why the real question isn't list price at all, but whether the cost is attributed and controlled.
How per-token pricing actually works
Six mechanics that sit between a provider's headline rate and the number on your invoice.
Input, output, and reasoning tokens are priced separately
Every provider meters your prompt (input tokens) and the model's reply (output tokens) as distinct line items, and reasoning models add a third: the hidden 'thinking' tokens spent before the visible answer. Your effective rate is a blend of all three, not the single headline number.
Output tokens cost more than input tokens
Generating text is the compute-heavy step, so providers price output well above input — often several times higher. A task that reads a little and writes a lot is priced very differently from one that reads a lot and writes a little, even at the same total token count.
Long context and prompts get re-billed every call
There's no free memory. A long system prompt, retrieved RAG context, and full conversation history are re-sent — and re-billed as input — on every single call. Context that feels 'set once' is actually paid for on each turn, which is why long-context workloads climb faster than call counts suggest.
Batch, off-peak, and cached-input discounts exist
Most providers offer levers that lower the effective rate: batch/async tiers for non-urgent work, reduced off-peak pricing, and cached-input discounts that charge less for a prompt prefix you reuse. They only help if your workload can actually use them — treat them as conditional, not automatic.
Agents and multi-step chains multiply the calls
One user action rarely equals one billed call. An agent plans, calls tools, reflects, and retries — fanning a single request into many model calls, each re-sending context. The per-token price barely moved; the number of billed tokens per task exploded.
List price × usage ≠ your bill
Naïve math (tokens per request × requests × list rate) almost always underestimates. Retries, timeouts, duplicate 'try again' calls, reasoning-token bloat, and agent loops all sit between the list price and the invoice — which is why teams are surprised by the number that arrives.
Frontier, mid-tier, and small models: relative economics
Rather than quote numbers that expire, here's how the model tiers relate to one another. Within any tier, remember that output tokens are billed above input, so the effective cost depends on your task's read-versus-write balance.
| Model tier | Best fit | Relative price |
|---|---|---|
| Frontier / flagship | Hardest reasoning, long-horizon agents, complex synthesis and code | $$$Highest per-token |
| Mid-tier / balanced | General chat, drafting, and most production traffic | $$Moderate per-token |
| Small / fast | Classification, routing, extraction, high-volume simple tasks | $Lowest per-token |
Official provider pricing
Provider list prices change often, so we don't reprint the numbers here — the links below go to each provider's own pricing page, verified as of July 2026. Always confirm the current rate for the exact model you plan to use.
How to compare LLM API prices in six steps
A checklist that compares providers on the cost you'll actually pay, not the rate on the page.
- 1
Compare blended cost per task, not the headline per-token rate
Estimate the input and output tokens a real task consumes, apply each provider's input and output rates, and compare the resulting cost per task. Two providers with similar headline numbers can differ sharply once the input/output split of your actual workload is applied.
- 2
Price your real context, not a toy prompt
Include the system prompt, retrieved context, and conversation history you actually send, because all of it is re-billed as input every call. A benchmark run on a one-line prompt tells you almost nothing about a production workload with long context.
- 3
Account for reasoning and agent multipliers
If the work uses a reasoning model or an agent, model the extra thinking tokens and the fan-out of tool calls and retries. The multiplier — not the base rate — usually dominates the bill for agentic workloads.
- 4
Factor in only the discounts you'll actually use
Credit batch, off-peak, or cached-input pricing to a provider only if your workload can realistically use them. A discount you can't apply because your traffic is real-time and uncached shouldn't move your comparison.
- 5
Add the failure tax
Budget for retries, timeouts, and re-asks — the calls that produce no useful output but still bill tokens. A provider that fails or times out more often can cost more per successful task even at a lower list price.
- 6
Decide on measured, attributed spend
Run a representative sample of your real traffic through each candidate and measure the attributed cost per user, project, and task. A short measured trial settles a pricing debate that headline rates never will.
The real question isn't list price
It's whether every model call is attributed to an owner and capped by a budget. Get a directional read on your exposure before you pick a provider.
Run the AI cost exposure calculatorFrequently asked questions
- Why is my effective cost per request higher than the list price?
- Because the list price is a per-token rate, and a request bills many tokens across several hidden layers. Output tokens are priced above input; a reasoning model spends thinking tokens you never see in the answer; long system prompts, RAG context, and history are re-billed as input on every call; and retries, timeouts, and agent tool-loops add calls that produce no visible output but still charge tokens. Your effective cost per request is the sum of all of that, so it routinely runs well above what the headline rate implies.
- Do cheaper models actually reduce total AI costs?
- Sometimes — but not by default. A cheaper model lowers total AI costs only when it still meets your quality bar for that task; if it fails and work has to be retried on a larger model, the 'cheap' path can cost more. The reliable approach is to route each request to the cheapest model that clears the quality bar for that specific task rather than picking one model for everything. Behest's smart routing does exactly that and can lower AI costs by up to ~30%, depending on use case — the savings depend heavily on your task mix, not on a headline rate.
- Why do output tokens cost more than input tokens?
- Generating text is the expensive step. Reading your prompt (input) can be processed in parallel, but producing the reply (output) is generated one token at a time, which uses more compute per token — so providers price output above input, frequently several times higher. The practical consequence is that two requests with identical total token counts can cost very differently depending on how much the model had to write versus read.
- What's the most reliable way to compare LLM API prices?
- Compare blended cost per task on your own traffic, not headline per-token rates. Estimate the real input and output tokens a task uses, include the context you actually send, add reasoning and agent multipliers, credit only the discounts your workload can use, and then run a representative sample through each provider and measure the attributed cost. A short measured trial settles the comparison in a way published rate cards cannot.
Price the model. Then control the cost.
Comparing rate cards is the easy part. Put every model call under attributed budgets and request-path enforcement so the bill stops surprising you.