How to Implement LLM API Rate Limiting by User and Project
Rate limiting LLM APIs by request count is easy. Rate limiting by dollar spend is harder. Here is a practical implementation that enforces per-user and per-project spend limits without a managed proxy.
by Orviupdated 3 min read#rate-limiting #llm-api #budget-enforcement
Rate limiting an LLM API by request count takes about ten minutes: pull in a sliding-window limiter, cap it at 100 requests per minute per user, done. Rate limiting by dollar spend is a different problem, and most teams discover the difference the hard way — a user well under the request cap racks up $600 in a single afternoon because each of their requests carries a 40,000-token document instead of a 200-token chat message.
Why request count is the wrong unit
A request-per-minute limiter treats every call as equivalent. In practice, LLM cost is dominated by tokens, not requests, and token count per request can vary by two orders of magnitude within the same endpoint. A support-chat feature might see 95% of requests average 300 tokens and 5% average 30,000 tokens (users pasting in a full email thread or log file). Cap requests at 100/minute and the 5% cohort still blows through budget, because the limiter was never measuring the thing that costs money.
The fix is to rate limit on estimated dollar cost per window, not request count. That requires three pieces: a way to estimate cost before the call fires, a counter that tracks spend per user and per project, and a gate that rejects the call when the counter is exceeded.
Estimating cost before the call
You don't need exact cost — you need a conservative estimate that's cheap to compute. For most SDKs this means counting tokens in the prompt with the same tokenizer the model uses (tiktoken for OpenAI, Anthropic's token-counting endpoint for Claude), multiplying by the model's per-token rate, and padding for an assumed output length since you don't know how long the response will be until it's finished.
def estimate_cost(model: str, prompt_tokens: int, expected_output_tokens: int = 500) -> float: rate = PRICING[model] # {"input": 0.000003, "output": 0.000015} per token, e.g. return prompt_tokens * rate["input"] + expected_output_tokens * rate["output"]Overestimate the output tokens rather than underestimate — a rejected call that would have been cheap is a minor inconvenience; an approved call that blows past budget is the failure mode you're trying to prevent.
The budget counter
The counter needs two properties that a naive SELECT spend FROM users WHERE id = ? followed by an UPDATE doesn't have: it needs to be atomic under concurrent requests from the same user, and it needs a reset window (daily, monthly, or a rolling window depending on your plan structure).
An atomic increment-and-check in Redis handles both:
async function checkAndReserve(userId, estimatedCost, budgetCap) { const key = `budget:${userId}:${currentWindow()}`; const newTotal = await redis.incrbyfloat(key, estimatedCost); await redis.expire(key, WINDOW_SECONDS); if (parseFloat(newTotal) > budgetCap) { await redis.incrbyfloat(key, -estimatedCost); // release the reservation return { allowed: false, spent: newTotal - estimatedCost }; } return { allowed: true, spent: newTotal };}The reserve-then-release pattern matters: you increment first and roll back on rejection, rather than checking then incrementing, because the check-then-act sequence has a race condition — two concurrent requests can both pass the check before either has incremented the counter.
Per-user and per-project limits together
Most production systems need both: a per-user cap (stop one customer from overspending on their own plan) and a per-project cap (stop the aggregate of all users on a project from exceeding what that project is provisioned for). Run both checks before the call fires, and reject if either is exceeded:
const [userCheck, projectCheck] = await Promise.all([ checkAndReserve(`user:${userId}`, cost, userBudgetCap), checkAndReserve(`project:${projectId}`, cost, projectBudgetCap),]);if (!userCheck.allowed || !projectCheck.allowed) { // roll back whichever one succeeded before rejecting the call}This is the part that turns into real engineering work once you have more than a handful of users: two atomic counters, a rollback path when one check fails and the other passed, and a reset schedule that has to stay consistent across both. It's a small amount of logic that's easy to get subtly wrong under concurrency, which is the main reason teams reach for a pre-built enforcement layer instead of writing and maintaining it themselves.
Where noburn fits
The tools discussed here handle observability, routing or evaluation, and all of them act once the LLM call has been made. noburn acts before it: one check() in your own code, placed before the provider call, returns allow or block against a project, per-user or per-run cap, so an over-budget call is never sent. It is not a proxy: your OpenAI or Anthropic client and its connection stay as they are. How the caps resolve, and the reason each one returns, is on the homepage.
Per-user caps and usage quotas keep one customer from spending another's budget, and Pro adds passthrough billing through Stripe. The Free plan covers 10,000 recorded calls a month with no card (current plans: pricing). Setup takes one install and one check: the quickstart, then per-user budgets and agent runs in the SDK reference.