Multi-Tenant LLM Cost Isolation: Why One Bad User Should Not Ruin Your Margin
In a multi-tenant AI product, one user with a large document or a stuck retry loop can consume 10x your expected API budget in a single session. Here is how to isolate that risk at the architecture level.
by Orviupdated 3 min read#multi-tenant #llm-cost #saas
It's 3am. Your Slack webhook fires with a cost alert for your AI SaaS product. In the last 4 hours, a single customer's API session consumed $8,400 in Claude API calls. Your margin just evaporated. You dig into the logs and find: a 120-page PDF upload, a retry loop in your code that resubmitted the document 14 times due to a transient network error, and zero throttling in place to stop it.
By the time you kill the session, the damage is done. The customer will dispute the charge. Your payment processor will reverse it. You'll refund them out of goodwill. And now you're rebuilding your cost isolation logic while losing sleep.
This is not a rare edge case. Multi-tenant AI products face a unique cost explosion risk: one user's spike can consume your entire monthly budget, and your observability tools will only tell you what went wrong after the bill arrives.
The architecture mistake that creates this exposure
Most AI SaaS platforms start with a simple cost control: track API spend per user in a database, set a monthly budget, and check it before the request fires. It sounds solid. It almost never works at scale.
Here's the typical implementation:
async function checkBudget(userId, estimatedCost) { const user = await db.query("SELECT spend_this_month FROM users WHERE id = $1", [userId]); if (user.spend_this_month + estimatedCost > user.budget_cap) { throw new BudgetExceededError(); } await db.query( "UPDATE users SET spend_this_month = spend_this_month + $1 WHERE id = $2", [estimatedCost, userId] );}It passes code review. It works in staging. It fails in production the moment a single user fires concurrent requests — which is exactly the retry-loop and multi-tab scenario that causes cost spikes in the first place. Two requests read spend_this_month before either one's UPDATE commits, both see the user under budget, both proceed, and the user's actual spend ends up double what the cap allowed. The read-then-write sequence has a race condition baked into it, and it gets worse under load, not better — the exact opposite of what you want from a safety mechanism.
The correct architecture: atomic reservation, not read-then-write
The fix is to make the check-and-increment a single atomic operation instead of two separate database round trips. Redis's INCRBYFLOAT (or Postgres's UPDATE ... RETURNING in a single statement) does this correctly:
async function checkBudget(userId, estimatedCost, cap) { const key = `spend:${userId}:${currentMonth()}`; const newTotal = await redis.incrbyfloat(key, estimatedCost); if (parseFloat(newTotal) > cap) { await redis.incrbyfloat(key, -estimatedCost); // roll back the reservation throw new BudgetExceededError(); } return newTotal;}This closes the race: the increment and the check happen as one operation from Redis's perspective, so two concurrent requests can't both read a stale "under budget" state. Whichever request's increment lands second sees the already-updated total and gets correctly rejected.
Layering a hard circuit breaker on top
Per-request budget checks stop steady overspend, but they don't stop a retry cascade fast enough on their own — if a bug causes 50 retries in the same second, each one costing a few cents, you want the loop killed after a handful of failures, not after it's individually confirmed budget-exceeded 50 times. A circuit breaker per user (or per session) that trips after N consecutive rejections or M requests in a short window adds that second layer:
async function circuitBreakerCheck(userId) { const failures = await redis.get(`breaker:${userId}`); if (parseInt(failures) >= MAX_CONSECUTIVE_FAILURES) { throw new CircuitOpenError(); // stop trying entirely, don't just reject }}Isolation at the architecture level means every tenant's spend is tracked and enforced independently, with atomic counters that don't race under concurrency, and a circuit breaker that shuts down runaway loops faster than a per-request check alone would catch them. Building this correctly — atomic budget counters, per-tenant isolation, and circuit-breaking — is a small amount of code that's easy to get wrong exactly once, at exactly the wrong time.
Where noburn fits
The tools discussed here handle observability, routing or evaluation, and all of them act once the LLM call has been made. noburn acts before it: one check() in your own code, placed before the provider call, returns allow or block against a project, per-user or per-run cap, so an over-budget call is never sent. It is not a proxy: your OpenAI or Anthropic client and its connection stay as they are. How the caps resolve, and the reason each one returns, is on the homepage.
Per-user caps and usage quotas keep one customer from spending another's budget, and Pro adds passthrough billing through Stripe. The Free plan covers 10,000 recorded calls a month with no card (current plans: pricing). Setup takes one install and one check: the quickstart, then per-user budgets and agent runs in the SDK reference.