Compare

Everyone in this category measures. Almost nobody refuses.

Three kinds of tool get sold as LLM cost control, and they solve different problems. This is where each one actually sits, with sources, and every side-by-side we have written underneath.

The three layers.

The line that matters is not which tool has more features. It is where the decision happens: after the provider has billed you, at a proxy your traffic has to pass through, or in your own code before the request exists.

Observability

Langfuse · LangSmith · Arize Phoenix · Traceloop

Records the call once it has already returned
Cost arrives as a trace attribute and a chart
Alerting is after the fact, on spend already committed
No documented cap that refuses a request
Priced per trace or per ingested unit, so measuring costs more as you grow

Gateways

LiteLLM · Portkey

LiteLLM checks a Redis spend counter and returns a budget error before forwarding
Portkey expires a virtual key once its budget limit is reached
Both require your traffic to pass through their proxy on every call
The proxy becomes a dependency in your request path and a hop in your latency
Portkey's budget limits are Enterprise-plan only and are not retroactive
Portkey was acquired by Palo Alto Networks in May 2026 and now ships inside Prisma AIRS

noburn

A pre-flight check, not a proxy

One check call returns allow or block before you contact the provider
You keep your provider SDK and your direct connection to it
Project, per-end-user and per-run caps resolve in the same call
A blocked call is never billed, because it is never made
If noburn is unreachable, onError decides whether you fail open or closed

Side by side.

noburn loses three rows here. It holds no traces, runs no evals and cannot be self-hosted today, and a table that hid that would not be worth reading.

What each tool does about spend. “Partial” means it does this with a condition worth reading.
Capability	LangfuseObservability	LangSmithObservability	HeliconeObservability	LiteLLMGateway	PortkeyGateway	noburnPre-flight cap
Refuses a call before it is sent	No	No	PartialCost-based rate limits	YesBudget error before forwarding	YesKey expires at its limit	Yes
No proxy in your request path	Yes	Yes	PartialAsync logging, or proxy mode	NoAll traffic routes through it	NoAll traffic routes through it	Yes
Per-end-user spend caps	No	No	PartialVia custom properties	YesPer key, user or team	PartialOne virtual key per customer	Yes
Per-run caps for a single agent run	No	No	No	NoBudgets are per key, not per run	No	Yes
Traces, spans and debugging	Yes	Yes	Yes	PartialLogs, plus integrations	Yes	NoRecords decisions, not traces
Evaluations and prompt experiments	Yes	Yes	Partial	No	Partial	NoNot what it is for
Enforcement without an enterprise plan	NoNo enforcement at any tier	NoNo enforcement at any tier	Partial	YesOpen source	NoEnterprise plan only	Yes
Self-hostable	Yes	PartialEnterprise self-host	Yes	Yes	PartialOpen-source gateway	NoHosted only today
Actively developed	Yes	Yes	NoMaintenance mode since March 2026	Yes	PartialAcquired, now inside Prisma AIRS	Yes

The row that only noburn fills.

Set a budget and send calls until one is refused. Nothing here is a recording — it is the same decision the SDK makes.

Try itPOST /v1/check · runs before the provider call

Run budget

$0.0000 spent$0.5000 left$0.0000 refused

No calls yet. Send one and watch the budget resolve.

Where noburn does not compete

It is a cap, not a tracing stack.

noburn does not replace Langfuse or LangSmith. It holds no traces, renders no spans and runs no evals, and if you need those you should keep the tool you have. It sits in front of them: the check decides whether the call happens, and whatever you already use records the calls that do.

Against a gateway the trade is explicit. LiteLLM and Portkey can enforce a budget, and in exchange every request routes through their proxy. noburn asks for one extra call and leaves your provider connection alone. If you already run a gateway and are happy with it, that is a reasonable place to cap spend.

Tooling comparisons

Cost, enforcement and architecture, tool against tool.

LLM Gateway Comparison 2026: LiteLLM, Portkey, Helicone, and What Each Actually DoesFour tools, four different bets on what the gateway layer should do. Here is a full side-by-side of architecture, pricing, enforcement capabilities, and who each one is actually built for.2026-05-27

Helicone vs Portkey in 2026: Which One Actually Enforces Your BudgetBoth tools promise LLM cost control. One is in maintenance mode after an acquisition. The other locks enforcement behind enterprise pricing. Here's what the comparison actually looks like.2026-05-22

LiteLLM Alternatives in 2026: What Production Teams Are Actually UsingLiteLLM works well at small scale. At production scale — concurrent users, multi-tenant billing, high RPS — teams run into three structural limits. Here's what they're switching to.2026-05-19

LangSmith vs Langfuse in 2026: Which One for Production Agent TeamsLangSmith and Langfuse both promise LLM observability. The pricing, enforcement story, and self-hosting options look very different. Here is the full comparison.2026-08-08

Traceloop vs Helicone: LLM Observability at Agent ScaleBoth tools track LLM calls and aggregate costs. The architecture, self-hosting story, and enforcement capabilities differ significantly. Here is how they compare for teams running agents.2026-06-17

Arize Phoenix vs LangSmith: Open Source vs Managed LLM ObservabilityPhoenix is open-source and OpenTelemetry-native. LangSmith is managed and LangChain-native. For teams choosing between a self-hosted and a managed observability stack, the trade-offs go deeper than hosting.2026-06-04

Datadog vs LangSmith for LLM Monitoring: What Each Tool Actually CoversDatadog added LLM observability features. LangSmith was built for it from the start. The overlap is smaller than the marketing suggests and the gaps in each direction are significant.2026-06-24

Braintrust vs LangSmith: Which One to Use for LLM Evaluation in 2026LangSmith is the right tool for LangChain teams and fast production tracing. Braintrust is the right tool for everything else. Here is exactly where the line is.2026-05-27

Weights & Biases vs Braintrust: ML Experiment Tracking vs LLM EvaluationW&B was built for ML training runs. Braintrust was built for LLM prompt experiments. Both have expanded into each other's territory. Here is what each actually does well in 2026.2026-05-31

Vercel AI SDK vs LangChain: Which Framework for Your Next AI FeatureVercel AI SDK is lean, streaming-first, and Next.js-native. LangChain is broad, chain-based, and Python-primary. The right choice depends on what you're building more than team preference.2026-06-22

Model and provider cost

What the calls themselves cost, before any tool is involved.

OpenAI vs Anthropic API Pricing in 2026: Real Cost Per Million TokensModel pricing changes constantly. Here is a current breakdown of what you actually pay per million tokens across OpenAI and Anthropic, with worked examples for common agent workloads.2026-05-30

Claude vs GPT-4o: Which Is Actually Cheaper for Your WorkloadThe headline price per million tokens does not tell the full story. Output-heavy workloads, context caching, and batch pricing change the effective cost by 30-60%. Here is how to model it properly.2026-06-01

GPT-4o mini vs Claude Haiku: Which Small Model Is Worth Using in ProductionBoth are the cheap tier of their respective families. Both have different strengths on different task types. This is a practical comparison using real tasks, not benchmark scores.2026-06-03

AWS Bedrock vs Azure OpenAI: Enterprise LLM Cost and ComplianceBoth platforms let enterprise teams run frontier models inside their cloud perimeter. The pricing models, compliance posture, and vendor lock-in implications look different under scrutiny.2026-05-27

Sources

Every claim made about a named competitor on this page was checked against that vendor’s own documentation in August 2026. This category moves quickly. If something here has gone out of date, it is a bug.

Cap it before you measure it.

The free plan needs no card, and the check is one call in front of the provider you already use.

Start free See how it works