Three independent cost ceilings
A single shared budget is a fairness failure wearing the costume of a cost control.
Spend is a design problem
Every call to a hosted model is a purchase. That single fact should shape the architecture around it, and it usually does not — budget tends to arrive as an alert on a dashboard, discovered by whoever reads the invoice. By then the design decisions that determined the bill are already shipped.
We treat the ceiling as part of the feature. Our server function will not call the provider until three independent limits have been checked.
The three ceilings
- A per-user call limit. The coarsest control, and the one that stops a broken client or a scripted loop from spending a month of budget in an afternoon.
- A per-user monthly spend limit. Denominated in money rather than calls, because not all calls cost the same and a request with a long input is not equivalent to a short one.
- A global backstop. The last line, which exists to bound total exposure if the first two are wrong.
They are checked in that order, cheapest first, and all of them are checked on the server before the provider is contacted. A limit enforced in the interface that renders the button is not a limit; it is a suggestion to anyone willing to open developer tools.
Why the middle one is usually missing
Most systems that bother with cost control at all implement the third ceiling and stop. One global budget, one alert, done. It looks responsible and it fails in a specific, predictable way.
A single shared budget means the heaviest user determines everyone else's experience. One person, or one bug, exhausts the pool and the capability goes dark for the entire user base — including people who have paid for it and used it twice. That is not a cost control. It is an availability and fairness failure that happens to reduce your bill, which is why it survives review: the metric it was introduced to move does move.
The per-user ceiling is what makes the global one safe to have. It converts a shared resource that any individual can drain into a set of individual allowances that fail independently.
Record what you are billed, not what you sent. It is tempting to estimate spend from request size, because it is available before the call. Do not. Providers return actual input and output token counts on every response; write those. An estimate drifts from the invoice, and a ledger that disagrees with the bill is worse than no ledger, because it will be trusted right up until the moment it matters.
What this costs you
Three ceilings mean three places to get the accounting wrong, a table to migrate, and a monthly reset to reason about. It is genuinely more work than a global cap.
It is worth it because the alternative failure is invisible until it is severe. A global-only budget behaves perfectly under normal load and collapses precisely when the product is doing well — which is the moment you least want to explain to paying users why the feature they bought has stopped working for reasons that have nothing to do with them.
The next generation of software will be intelligent.
ScentID Solutions is building toward that future — one platform, one integration, and one intelligent solution at a time.
Work With Us