How to Reduce LLM API Costs Without Hurting Quality

Your AI feature launched, users like it, and then the invoice arrived. Costs grew faster than usage, and nobody can say exactly why. Was it the new document search? The longer system prompt? The agent that sometimes loops 15 times before answering?

LLM API bills are predictable once you understand what drives them, and most teams can cut them substantially without users noticing a difference. This guide covers the techniques that work, when to use each and how to make sure quality doesn’t slip while you save.

What actually drives your LLM bill

LLM APIs charge per token, typically quoted per million tokens. Your cost for each request comes from 4 things:

• Input tokens: your system prompt, instructions, examples, retrieved documents, tool definitions and the full conversation history
• Output tokens: the response, which usually costs several times more per token than input. On Anthropic’s published price list, output costs 5 times the input rate for its current models
• Reasoning or thinking tokens: on reasoning models, the model’s internal work, which is generally billed as output
• Which model you choose: prices vary widely between a provider’s smallest and largest models

Then multiply by volume. Agents and multi-step workflows make several calls per user request, so a small inefficiency can repeat many times. If tokens are new to you, read our explainer on AI tokens and context windows first.

Measure cost and quality before you cut

Track where the tokens go

You can’t cut what you can’t see. Every API response includes usage data with input and output token counts. Log it with each request, along with:

• The feature or endpoint that made the call
• The model used
• Cached vs uncached input tokens
• Number of calls per user action
• Latency

Then calculate cost per feature, per user action and per customer. Often, 1 or 2 features, or 1 runaway agent loop, drive much of the spend. Set spend limits and budget alerts in each provider’s console as a safety net. The same discipline applies to the rest of your cloud bill; see FinOps and cloud cost optimization.

Build an eval set so you can cut safely

Every cost-cutting change risks a quality drop. Before you switch models or trim prompts, build a test set of 50 to 100 real requests with expected answers or a scoring rubric. Run it before and after each change and compare quality and cost side by side.

This is what makes “without hurting quality” a measured claim instead of a hope. Our guide on how to evaluate LLM outputs shows how to build one in a day.

Key takeaway: The cheapest token is the one you never send. But never cut blind: pair every cost change with an eval run so you know exactly what you saved and what, if anything, you gave up.

The main cost-cutting techniques compared

Prompt caching: pay less for the parts that repeat

Many requests start with the same content: a long system prompt, tool definitions, a policy document or product catalog. Prompt caching lets the provider reuse the already-processed prefix instead of charging full price every time.

How the 2 largest providers handle it:

• OpenAI enables prompt caching by default on supported models. Its documentation describes cached input billed at a reduced rate of up to 95% off, with a minimum prefix length (1,024 tokens on its newer models).
• Anthropic uses a cache_control setting, either automatic or with explicit breakpoints. A cache read costs 10% of the base input price on most of its models, while writing to the cache costs 1.25 times base input for the default 5-minute lifetime or 2 times for a 1-hour lifetime. Anthropic notes caching pays off after a single cache read at the 5-minute duration.

To get the most from caching:

1. Put static content first: instructions, tool definitions and reference documents at the start; user-specific and changing content at the end.
2. Don’t put timestamps or IDs in the cached part. Any change to the prefix breaks the cache.
3. Append to conversations rather than rewriting earlier turns.
4. Check the usage fields in responses to confirm cache hits are actually happening.

Exact rates, minimums and lifetimes vary by model and change over time, so confirm them in your provider’s current docs.

Batching: half price for work that can wait

Not every request needs an answer in 2 seconds. Nightly reports, tagging support tickets, summarizing call transcripts, generating product descriptions and running evaluations can all wait hours.

OpenAI’s and Anthropic’s batch APIs both offer a 50% discount on input and output tokens for asynchronous processing. OpenAI’s batch jobs have a 24-hour completion window, though they often finish sooner. Anthropic notes that its batch discount can be combined with prompt caching.

Audit your workload: anything a person isn’t actively waiting on is a candidate for batching.

Model routing: use the smallest model that does the job

The price gap between model tiers is large. On Anthropic’s price list, for example, its smallest current model (Claude Haiku 5.5) costs $0.10 per million input tokens for prompts up to 100,000 tokens, while its most capable models cost $10. Prices change, but the pattern of large gaps between tiers holds across providers.

Many requests don’t need the biggest model. Classification, extraction, simple Q&A, formatting and short summaries often work well on smaller models. A routing approach looks like this:

1. Default to a small or mid-size model for each task, and only move up if evals show it’s needed.
2. Route by task type: send classification and extraction to small models, and complex reasoning or writing to larger ones.
3. Escalate on uncertainty: try the cheaper model first and send the request to a larger model if the answer fails a validation check.
4. Re-test when new models launch. Today’s small model can often match last year’s large one.

Reasoning models deserve special attention, because their thinking tokens can multiply output costs. Many let you set an effort level. Anthropic’s documentation describes lower effort as using fewer tokens, including thinking, for speed and cost, with some capability reduction. Use high effort only where your evals show it matters. Our guide to AI reasoning models explains when they’re worth it.

For some private or high-volume workloads, a model on your own hardware may be cheaper still; see how to run an LLM locally.

Trim prompts and control output length

Input trimming:

• Cut redundant instructions. Long system prompts often repeat the same rule 3 ways. Test a shorter version against your evals.
• Limit few-shot examples to the 2 or 3 that matter most.
• Retrieve less, but better. In RAG systems, sending the top few relevant chunks instead of whole documents saves tokens and often improves accuracy.
• Summarize long chat histories instead of resending every turn.
• Remove unused tool definitions from requests that don’t need them.

Output control:

• Set a sensible maximum output length for each feature.
• Ask for concise answers explicitly, with a word limit or a format.
• Use structured output (such as JSON) for machine-read responses, which removes filler text.
• Stop agent loops early with step limits and clear exit conditions.

Response caching is a final option: if many users ask the same question, store the answer in your own cache and skip the API entirely. Exact-match caching is safe; “semantic” caching of similar questions needs careful testing so users don’t get answers to a slightly different question.

Where to start: a 2-week cost-reduction plan

1. Days 1 to 2: Log tokens, model and feature for every call. Find your top 3 cost drivers.
2. Days 3 to 4: Build or update an eval set for those features.
3. Day 5: Restructure prompts for caching (static first) and confirm cache hits.
4. Week 2, day 1: Move any non-urgent jobs to batch processing.
5. Week 2, days 2 to 3: Test a smaller model or lower effort level on each top feature against your evals.
6. Week 2, day 4: Trim prompts, retrieval and output length, rerunning evals after each change.
7. Week 2, day 5: Set budget alerts and a monthly cost review.

If you’re budgeting a new project rather than trimming an existing one, see how much AI implementation costs, and our LLM API walkthrough for production basics.

Conclusion

High LLM bills usually come from a handful of fixable habits: resending the same long prompts, using the biggest model for everything, paying real-time prices for work that can wait and letting outputs ramble. Measure first, build an eval set, then apply caching, batching, routing and trimming one at a time. You’ll keep the quality your users expect and pay noticeably less for it.

Sources

• Claude API: Pricing — per-token prices by model, output vs input rates, 50% batch discount and cache pricing multipliers
• Claude API: Prompt caching — cache_control, cache lifetimes, prefix order and common pitfalls
• OpenAI API: Prompt caching — automatic caching, minimum prefix length, cached input discount and static-content-first structure
• OpenAI API: Batch — 50% discount and 24-hour completion window
• Claude API: Effort — trading token usage against capability with effort levels