Quick answer: LLM API costs come down to tokens times a per-token rate, so you cut costs by using a smaller model where you can, sending fewer tokens, caching what repeats, and batching what is not urgent. The biggest wins are usually right-sizing the model and cleaning up bloated context, not squeezing pennies on output length.
I have watched teams panic about their LLM bill and reach for the wrong lever first. Below are the levers that move the number, roughly in order of impact, with honest notes on the tradeoffs.
First, Understand the Bill
Every LLM API charges per token, split into input (what you send) and output (what the model generates). Output tokens almost always cost more than input tokens.
That formula tells you where to look. Your cost is: (input tokens times input rate) plus (output tokens times output rate), summed over every request. So there are exactly three things you can change: the rate (which model), the token counts, and the number of requests. Every lever below is one of those three. If tokens are unfamiliar territory, start with what is a token in LLMs, because the whole game is measured in tokens.
Lever 1: Right-Size the Model
The single biggest cost decision is which model you call. Providers offer tiers: small and cheap, mid-range, and frontier. The price gap between tiers is large, often several times per token.
The mistake is defaulting every task to the most capable (and most expensive) model. Most production workloads are a mix, and a lot of that mix is simple: classification, extraction, short summaries, routing. Those rarely need a frontier model.
| Task type | Suggested tier | Reasoning |
|---|---|---|
| Classification, tagging, routing | Small / fast | Well within a cheap model's ability |
| Extraction, short summaries | Small to mid | Rarely needs top-tier reasoning |
| General assistant, most chat | Mid to frontier | Balance of quality and cost |
| Hard reasoning, agentic, coding | Frontier | Worth the premium when correctness matters |
A common pattern is a router: a cheap model (or simple logic) classifies the request, then sends only the hard ones to the expensive model. Done well, this cuts cost dramatically while keeping quality where it matters. Test quality per route before you commit, since a task that is "simple" in theory sometimes needs more model than you expect.
Do not downgrade blindly. Measure output quality on real tasks, then move the tasks that hold up to a cheaper tier.
Lever 2: Trim the Context
After the model choice, the fastest win is sending fewer input tokens without losing information.
The worst offender in most apps is bloated context: giant system prompts sent on every request, full conversation histories that are never trimmed, and raw web pages pasted in as HTML. That HTML is the sneakiest, because a page that reads as a short article can tokenize into tens of thousands of tokens of markup, scripts, and navigation.
Cleaning inputs is pure upside: it lowers cost and improves answers, because the model spends its attention on real content instead of junk. Converting pages to clean markdown before they reach the model is the highest-leverage version of this. A single call handles it:
import requests
content = requests.post(
"https://api.link.sc/v1/fetch",
headers={"x-api-key": "lsc_your_key_here"},
json={"url": "https://example.com/article", "format": "markdown"},
).json()["content"]
# `content` is a fraction of the token count of the raw HTML
For the full treatment of trimming web data before it hits the model, see token optimization: feeding web data to LLMs.
Lever 3: Use Prompt Caching
If you send the same large chunk of context on many requests (a fixed system prompt, a set of few-shot examples, a big reference document), prompt caching can cut the cost of that repeated portion sharply.
The idea: the provider caches your stable prefix, and subsequent requests that reuse it pay a small fraction of the normal input price for the cached part. Cache reads are far cheaper than reprocessing the same tokens every time.
The catch is that caching is a prefix match. Anything that changes the cached prefix, even one byte, invalidates it. So keep the stable content (frozen system prompt, deterministic examples) at the front, and put the variable content (the user's actual question, timestamps, per-request IDs) at the end. A stray datetime.now() in your system prompt silently breaks caching on every request. Check your provider's docs for the exact syntax and minimum cacheable size.
Lever 4: Batch What Is Not Urgent
Many providers offer a batch mode: submit a large set of requests, get results back within some window (often up to a day), at a significant discount versus real-time calls.
If your workload is not latency-sensitive (overnight enrichment, bulk classification, offline analysis), batching is close to free money. You are already going to make the requests; doing them in batch mode costs meaningfully less for the same tokens.
The tradeoff is latency. Do not batch anything a user is waiting on. Do batch the pipeline work happening behind the scenes.
Lever 5: Cache Your Results
Separate from provider-side prompt caching, cache your own outputs. If the same question comes in repeatedly, or the same URL gets processed over and over, do not pay the model to answer it twice.
A simple result cache keyed on the input (a normalized question, a URL) can eliminate a surprising share of requests in real apps, because traffic tends to cluster. This applies to your data-fetching layer too. If ten users ask about the same page, fetch and process it once, then serve the cached clean content. The same caching discipline pays off when you give your AI agent internet access, where repeated fetches are a common and avoidable cost.
Lever 6: Retrieve Less, But Better
In retrieval-augmented apps, a common cost sink is stuffing the prompt with everything that might be relevant. More retrieved context means more input tokens on every single request, and past a point it also hurts accuracy by burying the answer.
The fix is better extraction, not more documents. Retrieve the few most relevant chunks, cleanly extracted, rather than a dozen noisy ones. A tight, well-chosen context beats a bloated one on both cost and quality.
This is where getting clean content matters again. When you retrieve web pages, extracting the real article as markdown (instead of dumping full HTML) means each retrieved chunk carries more signal per token. Fewer, cleaner chunks is the goal. The link.sc fetch and search endpoints are built for exactly this: search returns full page content, and fetch returns clean markdown, so your retrieval layer feeds the model signal instead of noise. See the link.sc docs for the endpoints.
Lever 7: Stream for UX, Not Cost
One clarification, because it trips people up: streaming does not reduce cost. You pay for the same output tokens whether you stream them or not.
What streaming does is improve perceived latency, since the user sees text appear immediately instead of waiting for the full response. That is a real UX win and worth doing, but do not add it expecting a smaller bill. The cost levers are the model, the tokens, and the request count, not the delivery mechanism.
Putting It Together
A rough priority order for most teams:
- Right-size the model per task. Biggest single lever.
- Clean and trim context. Fast win, also improves quality.
- Cache repeated context and repeated results. Cuts redundant spend.
- Batch non-urgent work. Free discount on background jobs.
- Retrieve less but better. Lower input tokens and better answers.
Notice that the top levers improve quality at the same time they cut cost. That is the tell of a good optimization. Anything that makes both the bill and the output worse (like blindly slashing model quality on tasks that need it) is a false economy. Measure quality as you cut, and keep the frontier model where correctness actually matters.
The Bottom Line
LLM cost optimization is not about one clever trick. It is about matching the model to the task, sending clean and minimal context, caching what repeats, and batching what can wait. Do those consistently and you will often cut your bill by a large margin while making your product faster and more accurate. The cleaner your data pipeline, the cheaper and better everything downstream gets.
Cutting LLM costs starts with clean, minimal input. link.sc turns any URL into lean markdown and returns full-content search in one API. Start free.