Enter, output, cache life: the name on the AI token bill.

Let's start with the results:There are only two words in these terms that really cost you more: "Cacheft" and "output" token.The rest are their notes.

I've been billed three times in two months, and it's the same thing: I think I'm saving money, and I'm paying the full price for the same prefix. This is the one where you can answer at least one question...Why do I have 8,000 tokens on the bill when I've only said one word?

I. Token. Why is Chinese more expensive?

Token is the smallest unit for model text processing. It's not a word. It's not a word.It's a piece cut by the semantic.I don't know. In the same sentence, the number of blocks cut can differ from model to code.

I'll take it. o200k_base The code actually measured several groups and the difference was greater than expected:

Text	Number of characters	Token Number	Character/token
The quick brown box jumps over the lazy dog.	44	10	4.4
The quick brown fox jumped over that lazy dog.	15	16	0.94
Hello, world.	5	4	1.25
CVE-2026-109618	15	8	1.88

See the problem.In the same sentence, 10 tokens in English and 16 in Chinese.About 4 characters in EnglishOne word, one tokenI don't know.

This is very unfriendly to Chinese users: the "1 token ≈4 character" is an English-language experience.You'll underestimate more than half of it in Chinese.

There's a counterintuitive:CVE-2026-109618 This number, 15 characters burn 8 tokens. Numbers and hyphens are cut very badly.So it's much more expensive than it looks.

Two, one request, four paragraphs.

The first time I looked at the bill, it wasn't expensive.It's only one word, but 8,000 token.I don't know. Then it became clear that more than you did.

The input of a request is usually made in four pieces:

Paragraph	English Name	What is it?	Will it change?
System Hint	System / inclusion	Who are you, the rules, the output format?	It's basically the same.
Tool Definition	I don't think so.	Function JSON Schema and description	Very rarely.
History of dialogue	Messages / history	Each round of questions and answers before you, take them all.	Add only
Questions for this round	I'm sorry, sir.	That thing you just called.	Every time.

The first time I was scared, it was.The tool defines the amount of 1,200 tokensI don't know. A function with parameter description, JSON Schema expands to take special place. You wrote 20 tools, you could eat 2,000 by definition.And you didn't type a word in this part, but it's always charged.

There's a key point behind it:The first three paragraphs are “additional and non-modified” and the fourth paragraph changes each time.It's not a coincidence. It determines whether the cache can be hit.

Three: input, output, and that one in the output.

The official caliber of these three relationships is as follows:

Enter token (input / policy tokens): Everything you send through contains the four paragraphs above.
Output: Modelled. Attention...Not just the text you saw.
Token (resoning tokens): The reasoning model is what it thinks before it gives the answer. You can't see, but it...At output pricesI don't know.

The third is the real reason for many people to double their bills. I run a sort of job with a theory model, and I see 20 words in the answer, and the bill is more than 900.That 880 invisible token, it's all in there.

So it's normal to have a contradiction:The shorter the answer, the cheaper the less.

The output price is usually about five times the input price (see the table below for each household). This explains a principle of integrity:It can be solved with input. Don't let it out.Put the answer template, the format example, in the system hint (lower), so that it does not organize itself in such a way that it is expensive.

Cacheline and unattractive: this is the section of the whole text that should be remembered

What are they?

Models are counted as intermediate when they are read. This part is the most expensive.

Prompt cache is a prefixed state.Next time we meet the same prefix, we'll use it directly and no longer recalculate.

So there were two words:

Cache hit: This prefix was found in the cache. This part was entered and pressed.The discount.Fees.
Cache miss: I can't find it. Just recalculate the price.

The terms of the hit are tougher than you think.

The key is the word "prefix."It's not about asking the same question.

I did an experiment: I took a 151 token system hint, changed it to different locations, and looked at how much it could be reused.

Changes	Reusable Prefix	Percentage
Add a sentence at the end	150 / 151	99%
Change the word in the middle.	50 / 151	33%
One more space at the beginning	0 / 151	0%
Start with a word.	0 / 151	0%

See that "one more space at the beginning"?One space, caches, scrapped, recalculated 151 tokens at original prices.

It's not a bug, it's the principle: Cache is the state of accumulation from the beginning, the first token changes, and the back is all dead.

So there is a rule that can be applied immediately:The constant content is in the front and the variable content in the last.

The most common way to flip is to insert the current time, requested ID, randomly sorted fields at the beginning of the system hint. That's like a new cache every time.These fields are either moved to the end or simply not put in.

There are a couple of easy-to-neglect thresholds.

There's a minimum length.Too short a prefix does not give the cache at all. OpenAI is GPT-5.6 and later 1,024 token; Anthropic is segregated by model, both in 512 / 1,024 / 2,048 / 4,096. If it's not long enough, it's not wrong.Silence., you can only see by returning the quantum field.
There's time for survival.OpenAI default 30 minutes (up to 12 hours); Anthropic default 5 minutes, up to 1 hour. And this time...Start at request, not counting from the end of the answer -- if the answer runs in four minutes, the 5-minute slot has only one minute window.
We'll wait for each other.Cache Entry Waits for First RequestStart returningThen it's available. Ten requests were sent in the same second, and all 10 were missed.
Changed models, changed tools, changed reasoning slots.The prefix changes and the hit rate is zero.

What's the hit rate?

Don't guess. Both families gave them a volume field:

OpenAI  : usage.prompt_tokens_details.cached_tokens
Anthropic: usage.cache_read_input_tokens / cache_creation_input_tokens

Anthropic, the special intuitive:Both returned 0, which means they didn't even reach the cache.- Usually because prefixes are not minimal.

V. Price structure: writing more money to read less

Cache is not free, and many have missed it.The first time you write in the cache, you have to raise the price; then read it before you make a discount.

Operation	OpenAI	Anthropic
Normal Input	1x (baseline)	1x (baseline)
Cache writing. Short slot	1.25 x (30 minutes)	1.25 x (5 minutes)
Cache writing long-term	2.5 x (12 hours)	2x (1 hour)
Cache Read (Category)	0.1 x(GPT-6.1 is 0.05 x)	0.1 x(Partial model 0.025 – 0.05 x)
Output	About 5x	About 5x

Take the normal 0.1 x read, 1.25 x write two books:

Write once + reuse onceCache 1.25 + 0.1 = 1.35 x;no cache is 1 + 1 = 2 x.Province 32 per cent.
Write once + repeat nine times: Cache 1.25 + 0.9 = 2.15 x;not cache is 10x.Province 78 per cent.

This means:In one case, the cache was barely or even slightly lost; it was only when it was repeated more than ten times that the savings were real.Those "one-time temporary conversations" are just a problem.

By the way:Cache writing is based on the token segment, not by the number.The breakpoint itself is free, and the money spent depends on how long you actually have.

VI. PUNISHMENTS, SYSTEM PUNISHMENTS, min

The two terms are often used intermingled, but they are different in terms of costing and authority.

Name	English	Who wrote it?	Priority	Fee attribution
System Hint	Stymprompt / institutions	Platformr or Application Developer	Highest	Enter token
Developer Message	I'm not sure I'm going to be able to do that.	Apply Developer	Lower (over users)	Enter token
User hint	It's our business.	End user	Less than the two above	Enter token
Assistant Message	There's no reason why you're here.	Models generate themselves	I don't know.	Output Token

Two easy points:

The word "advertisement" is a big word.It refers in general to all the text instructions you send to the model, and the "systemal hint" is the one that is the most forward and highest priority. When it comes to "advertisements are too long", the real problem is often that the system is too long to define them.
Role is also expensive.Every message is one layer {"role":...,"content":...}And the layer itself counts as token. When there's a lot of messages, it'll be thousands of things.

So the clean-up dialogue is working -- not genius, yes.Cut off that part of history.I don't know. But in turn, it is important to note that after clearing up, the historical prefixes of the previously held dialogue will not be needed.

Seven or seven of the pits I stepped on, copy the conclusions.

Use the word "character" to estimate the Chinese budget.Wrong. Chinese is close to one word, and you'll be twice as short. It's ready. tiktoken Defining, or looking back at usage.
I thought I would get hit if I asked a similar question.Can't. Must be completely consistent from the first token.
Inserts a time stamp at the beginning of the system hint.This is the most common cache killer. Move to the end.
It's cheap to see short answers.The reasoning model's thinking process is costed at output price, answer 20 words, charge 900 token is normal.
Cache for one-time dialogue.Re-use only saved 32 per cent, but added a layer of complexity and debugging costs. It's worth it at least three times.
You can guess the hit rate without looking at a measurement field.Two. cached_tokens / cache_read_input_tokens They all returned in clear code.
He also asked to use the cache to boost speed.The cache will not be available until the first response begins, and requests for the same second are largely missed.

VIII. A list of what has been done

We can change today.

Define system hints and toolsEverything that changes.Move to the end of the message.
Tool PressFixed OrderArrange. The order changes the prefix, the prefix changes.
First count: Type out the number of tokens in the system + tool definition. More than 2,000 is worth optimizing.

It's worth half a day.

Disassembly a long-system hint into a "consistent + variable segment " , with a cache break at the end of the constant segment.
Run round with real traffic. Record. cached_tokens Percentage of total inputs. Less than 30% indicates prefix design problems.
Press the output side as well: the "output only" is not allowed to explain, the output price is five times the input.

Don't do it.

Don't write a nine-thirty tip for the cache.Cache is money, not clarity.
Don't add to one-off scripts the logic of caches, which are more complicated than savings.
It is not assumed that caches can be used across different models.Switching models is changing caches, with zero hits.

Finally.

These words are taken to the end, and they say only one thing:The model charges "how much it reads in" and "how much it writes," and the cache is the only mechanism that can discount "read."

So it's not exactly how beautiful the hint is, is it?Put that pile of things that don't change every time. Let's go.I don't know.

Most of the money you can't save is not too expensive. Your prefix changes every time.

I'll leave you with the most practical action: next time I see an abnormal bill, don't doubt the model's price increase. cached_tokens That column.It's not zero, decides whether you're paying for math or double.