Denial of wallet is a security failure
Traditional denial of service exhausts capacity. Against a metered API there is no capacity to exhaust, because the provider will keep serving as long as the bill is paid — so the resource an attacker exhausts is money. OWASP names this unbounded consumption, and it is the failure mode agents are most exposed to, because an agent that can decide to retry can decide to retry forever.
It does not take an attacker. A loop that treats a malformed response as retryable, a planner that re-reads a large document on every step, or a prompt-injected instruction to summarise a site a thousand times all produce the same shape: a large number of individually legitimate calls. Anything that only inspects one call at a time will approve every one of them.
The question a budget answers is not “is this call allowed?” but “is this call allowed given everything already spent?”
Enforce before the call, reconcile after
A budget checked after a call has already been forwarded is an alerting feature. By the time the number is known the provider has been paid, and the agent is free to make the next call while the alert is still being routed. Enforcement has to sit on the request path, before the upstream request is made.
The awkward part is that the cost of a call is not known until it returns. Token counts depend on the response. The working pattern is to reserve an estimate before forwarding and settle the true figure afterwards: the reservation is what blocks the next call in a burst, and the settlement is what keeps the ledger honest.
- Reserve against a conservative estimate, not an optimistic one.
- Make the reservation atomic, so concurrent calls cannot both pass the same last dollar.
- Release the reservation if the upstream call fails before it costs anything.
- Settle the actual figure once the response is accounted, and let the ledger correct itself.
Concurrency is where naive budgets fail
Read the balance, compare it to the limit, then write the new balance: three steps, and a burst of parallel agent calls interleaves them freely. Ten concurrent requests can each read the same remaining balance and each conclude there is room. The overspend is silent, and it is worst under exactly the conditions a budget exists for.
The fix is to make the check and the decrement one indivisible operation — a script or transaction the datastore runs atomically — so the answer to “is there room” is produced by the same operation that consumes it. This is a correctness property, not an optimisation, and it deserves a test that runs the calls concurrently rather than one after another.
Choose the scope of each limit deliberately
A single global cap is easy to reason about and nearly useless in practice: one runaway agent consumes the allowance of every well-behaved one, and the operator learns about it when everything stops. Limits should exist at the level where an incident should be contained.
Per-agent limits contain a single bad deployment. Per-tenant limits contain a single customer. Per-provider limits stop a pricing change on one model from draining the budget for all of them. Windowed limits — per minute as well as per month — are what catch a loop before the monthly figure is reached, because a monthly cap alone will happily be exhausted in an afternoon.
- Per agent: contains one bad deployment or one compromised credential.
- Per tenant: contains one customer, and is the boundary a bill is drawn on.
- Per provider or model: stops one expensive route absorbing everything.
- Per short window: the only limit that catches a runaway loop quickly.
Decide what happens at the limit, and fail closed on spend
A refusal has to be unambiguous. Return a distinct, documented status the caller can act on, say which limit was reached and when it resets, and never degrade the request silently to a cheaper model — a silent substitution turns a cost control into a correctness bug that surfaces days later as a quality complaint.
When the budget store itself is unreadable, spend should fail closed. This is the opposite of the usual availability instinct, and it is deliberate: an unknown balance is not evidence of a balance, and the cost of wrongly refusing a call is a retry, while the cost of wrongly allowing one is unbounded. Controls whose failure is merely inconvenient can fail open; controls whose failure is unbounded should not.
Make spend answerable, not just observable
A dashboard number answers the operator's question. It does not answer a customer's, an auditor's, or a counterparty's, because all of those require believing the operator's own system. A per-call record that names the agent, the provider, the decision and the cost — and that is signed — can be checked by someone with no access to the system that produced it.
That record is also what makes a limit reviewable after the fact: whether the cap was reached, whether it was raised, by whom, and when. A budget nobody can reconstruct is a budget that will be argued about.
Frequently asked questions
Short answers to the questions that matter.
What is the difference between a rate limit and a budget?
A rate limit bounds how many calls happen per unit of time; a budget bounds what they cost. They fail differently: a thousand cheap calls can pass a budget and should hit a rate limit, and a single very large context window can pass a rate limit and should hit a budget. Production agents need both, because neither one bounds the other.
How do you budget when a call's cost is unknown until it returns?
Reserve a conservative estimate before forwarding, then settle the true cost once the response is accounted. The reservation is what stops the next call in a burst; the settlement is what keeps the ledger accurate. Reserving nothing and reconciling later means the burst is already paid for.
Should a budget refusal fail open or fail closed?
Closed. An unreadable budget store is not evidence that there is money left, and the downside is asymmetric: a wrongly refused call costs a retry, while a wrongly allowed one is unbounded. Note that this is the opposite of the right default for some availability controls — the decision belongs per control, not per system.
Do streaming responses and retries break budget accounting?
They complicate it rather than break it. A stream's cost is only final at the end, so the reservation has to cover the worst case and settle on completion, including on an aborted stream. Retries must be counted individually — treating a retried call as free is how a retry loop becomes a denial-of-wallet event.
Primary sources