Deep Read, cost

What a context window really costs you

A big window is a bigger checkbook, slower answers, and thinner attention. Here is how to spend it well.

The Curio Team · 5 min readWritten with AI from the sources below and checked against them before publishing.

A context window feels free. It isn't. Every token you send gets priced, queued, and read, and the longer your prompt runs the more each of those three costs climbs. We think of the window as a budget with three separate ledgers: money, latency, and attention. Most teams watch only the first. The other two quietly decide whether our answers are fast and whether they are any good. A big window is a bigger checkbook, not a free one.

Tokens are the unit of the bill

A token is a chunk of text, usually a few characters or part of a word. Models read and write in tokens, not letters, and providers bill per token in both directions. As a rough rule, one token runs about four characters of English, so 1,000 tokens is roughly 750 words. Common words are a single token. Rare words, code, and other languages fragment into many, which is why a page of dense JSON costs more than a page of plain prose. You can preview this. Most providers ship a tokenizer or a token counter, so you never have to guess how big a prompt really is before you send it.²

Input tokens and output tokens are priced separately, and output is often several times more expensive than input. That asymmetry matters. A bloated prompt is a fixed tax on every single call. It never generates value on its own, yet you pay it every time the model runs. So the first question we ask of any prompt is simple. Does this token change the answer? If not, it is dead weight.

We do the arithmetic once and it sticks. Say you paste a 10,000 token reference document into every request. At a typical input price of a few dollars per million tokens, one call is cheap. Ship 100,000 calls a month and that one paste becomes a real line item, before the model has written a word. Now imagine you trimmed it to 2,000 tokens. Same answers, one fifth the input bill. That gap is money you were setting on fire for no reader benefit. Check the current per-model rates in the provider's own pricing docs, because they change and they differ by model.¹

Attention is finite, so signal gets diluted

Here is the part that surprises people. A bigger window does not mean the model reads everything with equal care. Attention is spread across the whole prompt, and studies of long-context retrieval show accuracy sags when the key fact sits in the middle of a long input. Researchers named this the lost-in-the-middle effect. Models tend to use the beginning and the end well, and skim the belly. Put the thing that matters at the top or the very end. Position is leverage you get for free.³

This changes how you write a prompt. Bury a rule in paragraph nine of a 40-page dump and the model may miss it. State it up front. Restate the hard constraint at the close. The signal-to-noise ratio of the window is a real quantity, and every irrelevant token lowers it. More context can make an answer worse, not better, because it hands the model more chances to latch onto the wrong detail.

Latency scales too. More input tokens mean more compute before the first word comes back, so a 30,000 token prompt feels slower than a 3,000 token one even when the answer is identical. Users feel that delay. We have watched response times double just from context nobody actually needed. Trim the prompt and the clock speeds up. Speed is a feature, and the window is where you win or lose it. A snappy answer that is roughly right often beats a slow answer that is exactly right, because people abandon the wait.

How to spend the window well

We treat the window like a carry-on bag. Pack what earns its place, leave the rest at home. Put instructions and the current question where the model looks hardest, which is the start and the end. Keep a large stable block, like a long system prompt or a reference file, in a fixed position so caching can help. Order is not cosmetic here. It decides what gets read and what gets paid for twice. A well-ordered prompt is faster, cheaper, and sharper at the same time, which is rare in engineering.

That fixed position unlocks prompt caching. When the leading chunk of your prompt is identical across calls, providers can cache it and charge far less to reread it, which cuts both cost and latency on the repeated part. The trick is order. Static content first, the changing question last, so the cache prefix stays stable. Move one character near the top and the cache misses. We have seen repeated-prompt costs fall sharply this way, and the effect grows with volume.⁴

A bigger window is more room to be wrong, not a license to dump everything in.

Retrieval beats stuffing. Instead of pasting a whole manual, fetch the three passages that answer the question and send those. Summarize old turns of a long chat rather than replaying them verbatim. A running summary of 300 tokens can replace 8,000 tokens of transcript and often reads clearer to the model. Measure with our real inputs, not toy ones. Count the tokens you actually send, watch the latency, and read the answers side by side. A trimmed prompt that scores the same is a pure win, and you booked it without touching the model.

The goal is never the largest prompt. It is the smallest prompt that still gets the job done. Tokens are money. Tokens are time. And past a point, more tokens are worse answers. Budget all three, and the window stops being a cost you tolerate and starts being a lever you pull.