Why your AI agent costs 4× the estimate, and how to fix it

In this lesson you'll learn:

• how to decompose a surprise bill into four ratios from the per-call log;
• why the fixes go in order of multiplier removed per day, not by urgency;
• how to make a prompt cache pay, and the dashboard trap that gets it switched off.

Sound familiar?#

• The bill tripled and traffic did not move.
• The first response was a budget increase, and next quarter the bill grew again.
• Someone found hundreds of "suspiciously fast" responses and proposed turning the cache off.

A bill that leaves its estimate on flat traffic means behaviour changed, and behaviour is only visible per call.

Why does an AI agent cost more than estimated?#

Because cost is a product of habits, not one runaway. Each habit is a sensible decision made by a careful person, and each on its own looks modest. At SkyWays, the fictional airline this manual follows, the day-75 bill was 4.4 times its estimate with traffic flat. Four ratios explained it:

1.6 × 1.5 × 1.3 × 1.4 ≈ 4.4. Cost per case went from $0.60 to $2.64.

Fix the bill, step by step#

Step 1 · Read the per-call log, not the price list#

Take the log of individual model calls for a baseline week and for now, and compute the four ratios. If you do not have a per-call log (tokens in, tokens out, cache tokens read, model, attempts, per case), that is the first fix: route every call through one gateway that writes it.

Step 2 · Check that the factors multiply to the invoice#

Multiply the four ratios. If the product matches the ratio of this month's bill to the estimate, you have found the whole story. If it does not, a fifth cause exists, often idle infrastructure that bills for existing rather than for use.

Step 3 · Fix in order of multiplier removed per day#

Priority = (factor − 1) ÷ days to fix. It is usually not the order that feels most urgent:

The breaker comes last and still matters: the other fixes lower a rate, but a breaker makes a runaway impossible, however it is triggered.

Step 4 · Make the cache pay#

A cached prefix costs a little more to write and much less to read. Against the input price, a five-minute cache write costs about 1.25×, an hour-long write about 2×, and a read about 0.1×, so a prefix used twice has already paid for itself, and one used ten times costs about a fifth. But it only pays if it hits: put nothing volatile inside the cached block, keep the request after the marker, and use one model per task, because the cache is scoped to the model.

Step 5 · Avoid the dashboard trap#

Cache hits return fast. A latency dashboard that flags very fast responses as suspected failures will report hundreds of them, and someone will propose switching the cache off, raising the bill by about a third within a day, because the anomaly was the cache working. Put the cache tokens read in every trace row, and exclude cache hits from the alert.

Step 6 · Close the loop in the design#

The fix is not finished until the decision that allowed the habit changes: the model-tier decision record gets a new version, and cost per case becomes a monitored number with an owner and an alert. The cost loop

Where you'll use it#

• The first time a bill surprises anyone, before any budget conversation.
• In P1 design, where the context layout, routing and caching are decided.
• In P3 operation, with cost per case on the same dashboard as quality.

Why it matters#

Escalating cost is one of the three reasons Gartner gives for expecting over 40% of agentic AI projects to be cancelled by 2027. A bill handled as a budget question recurs next quarter with a different multiple; handled as a design question, it falls, and stays down.

Key takeaways#

1. A surprise bill is usually four habits multiplying (context, tier, cache, attempts), visible only per call.
2. Fix in order of (factor − 1) ÷ days, and fit a breaker so a runaway is impossible.
3. Make the cache hit, mark hits in the trace, and close the loop in the design with a monitored cost per case.

FAQ#

How do I reduce LLM costs in production?#

Measure per call first. Then trim the context you send each turn, route easy calls to a cheaper model, make your prompt cache actually hit, and cap loops and tokens per case. Most of the saving usually comes from the first two, in a day's work.

Does prompt caching reduce costs?#

Yes, when the cached prefix is reused. A cache write costs slightly more than normal input and a cache read costs a fraction of it, so a prefix reused even twice saves money. It stops saving if anything changing sits inside the cached block, or if the model changes mid-task.

What is a normal cost per task for an AI agent?#

There is no normal; there is your estimate, derived from tokens per call, calls per case and the price of each tier, and there is your measured cost per case against it. Track the ratio, and alert when it passes a threshold: this manual's default is 3×.

Why did our AI costs go up with no traffic change?#

Because behaviour changed: longer context, more calls on the expensive model, a cache that stopped hitting, or more retries, or infrastructure that bills while idle. The per-call log shows which; the invoice cannot.

Apply it in your role#

Across the enterprise. A central gateway with cost tags per feature makes every team's bill diagnosable, and charging back by feature makes each owner care about theirs.

The ten-minute workflow. Diagnose a bill from the per-call log:

Here is our per-call log for two periods: <CSV: timestamp, feature, model, input_tokens,
output_tokens, cached_tokens, attempt>. Per case, compute tokens per call, the share on each model
tier, the cache hit ratio and attempts. Show each factor's ratio between the periods, confirm they
multiply to the bill's ratio, and rank the fixes by (factor − 1) ÷ days to fix, using: <estimates>.

Sources and credits#