Rate limiting an LLM API by request count takes about ten minutes: pull in a sliding-window limiter, cap it at 100 requests per minute per user, done. Rate limiting by dollar spend is a different problem, and most teams discover the difference the hard way — a user well under the request cap racks up $600 in a single afternoon because each of their requests carries a 40,000-token document instead of a 200-token chat message.
Why request count is the wrong unit
A request-per-minute limiter treats every call as equivalent. In practice, LLM cost is dominated by tokens, not requests, and token count per request can vary by two orders of magnitude within the same endpoint. A support-chat feature might see 95% of requests average 300 tokens and 5% average 30,000 tokens (users pasting in a full email thread or log file). Cap requests at 100/minute and the 5% cohort still blows through budget, because the limiter was never measuring the thing that costs money.
The fix is to rate limit on estimated dollar cost per window, not request count. That requires three pieces: a way to estimate cost before the call fires, a counter that tracks spend per user and per project, and a gate that rejects the call when the counter is exceeded.
Estimating cost before the call
You don't need exact cost — you need a conservative estimate that's cheap to compute. For most SDKs this means counting tokens in the prompt with the same tokenizer the model uses (tiktoken for OpenAI, Anthropic's token-counting endpoint for Claude), multiplying by the model's per-token rate, and padding for an assumed output length since you don't know how long the response will be until it's finished.
def estimate_cost(model: str, prompt_tokens: int, expected_output_tokens: int = 500) -> float:
rate = PRICING[model] # {"input": 0.000003, "output": 0.000015} per token, e.g.
return prompt_tokens * rate["input"] + expected_output_tokens * rate["output"]Overestimate the output tokens rather than underestimate — a rejected call that would have been cheap is a minor inconvenience; an approved call that blows past budget is the failure mode you're trying to prevent.
The budget counter
The counter needs two properties that a naive SELECT spend FROM users WHERE id = ? followed by an UPDATE doesn't have: it needs to be atomic under concurrent requests from the same user, and it needs a reset window (daily, monthly, or a rolling window depending on your plan structure).
An atomic increment-and-check in Redis handles both:
async function checkAndReserve(userId, estimatedCost, budgetCap) {
const key = `budget:${userId}:${currentWindow()}`;
const newTotal = await redis.incrbyfloat(key, estimatedCost);
await redis.expire(key, WINDOW_SECONDS);
if (parseFloat(newTotal) > budgetCap) {
await redis.incrbyfloat(key, -estimatedCost); // release the reservation
return { allowed: false, spent: newTotal - estimatedCost };
}
return { allowed: true, spent: newTotal };
}The reserve-then-release pattern matters: you increment first and roll back on rejection, rather than checking then incrementing, because the check-then-act sequence has a race condition — two concurrent requests can both pass the check before either has incremented the counter.
Per-user and per-project limits together
Most production systems need both: a per-user cap (stop one customer from overspending on their own plan) and a per-project cap (stop the aggregate of all users on a project from exceeding what that project is provisioned for). Run both checks before the call fires, and reject if either is exceeded:
const [userCheck, projectCheck] = await Promise.all([
checkAndReserve(`user:${userId}`, cost, userBudgetCap),
checkAndReserve(`project:${projectId}`, cost, projectBudgetCap),
]);
if (!userCheck.allowed || !projectCheck.allowed) {
// roll back whichever one succeeded before rejecting the call
}This is the part that turns into real engineering work once you have more than a handful of users: two atomic counters, a rollback path when one check fails and the other passed, and a reset schedule that has to stay consistent across both. It's a small amount of logic that's easy to get subtly wrong under concurrency, which is the main reason teams reach for a pre-built enforcement layer instead of writing and maintaining it themselves.
Where noburn fits
The tools compared in this article handle observability, routing, or evaluation — all of which operate after the LLM call completes. noburn operates before it. It wraps your existing OpenAI, Anthropic, LangChain, and the Vercel AI SDK client, estimates the token cost of each call, and blocks it if the calling user or project has exceeded their budget. Nothing in this comparison does that at a self-serve price point.
Per-user metering lets you enforce separate limits per end-customer, and Stripe passthrough lets you bill them for their LLM usage without writing a billing layer yourself. The free tier covers 50,000 requests per month. Documentation and SDKs are at noburn.dev/docs.