compare · claims checked against vendor docs, August 2026
Three places a spend decision can happen.
This isn’t a scoreboard. Tracing tools, gateways and noburn sit at different points in the request path, and most teams end up running two of them.
TL;DR: observability tools (Langfuse, LangSmith, Arize Phoenix, Traceloop) record what a call cost after the provider has billed it, and cannot refuse it. Gateways (LiteLLM, Portkey) can refuse an over-budget call, but only when every call routes through their proxy. noburn refuses the call from inside your own code with one pre-flight check and no proxy; it has no traces, evaluations or self-hosting, so it sits in front of a tracing tool rather than replacing one.
The three layers
Observability
Langfuse · LangSmith · Arize Phoenix · Traceloop
Tells you what a call cost, after the provider has billed it.
- Records the call once it has already returned
- Cost arrives as a trace attribute and a chart
- Alerting is after the fact, on spend already committed
- No documented cap that refuses a request
- Priced per trace or per ingested unit, so measuring costs more as you grow
Gateways
LiteLLM · Portkey
Can genuinely cap spend, if every call is routed through their proxy.
- LiteLLM checks a Redis spend counter and returns a budget error before forwarding
- Portkey expires a virtual key once its budget limit is reached
- Both require your traffic to pass through their proxy on every call
- The proxy becomes a dependency in your request path and a hop in your latency
- Portkey's budget limits are Enterprise-plan only and are not retroactive
- Portkey was acquired by Palo Alto Networks in May 2026 and now ships inside Prisma AIRS
noburn
A pre-flight check, not a proxy
Decides before the call is sent, from inside your own code path.
- One check call returns allow or block before you contact the provider
- You keep your provider SDK and your direct connection to it
- Project, per-end-user and per-run caps resolve in the same call
- A blocked call is never billed, because it is never made
- If noburn is unreachable, onError decides whether you fail open or closed
Side by side
yes with a condition no
| capability | LangfuseObservability | LangSmithObservability | HeliconeObservability | LiteLLMGateway | PortkeyGateway | noburnPre-flight cap |
|---|---|---|---|---|---|---|
| Refuses a call before it is sent | ○ | ○ | ◐Cost-based rate limits | ●Budget error before forwarding | ●Key expires at its limit | ● |
| No proxy in your request path | ● | ● | ◐Async logging, or proxy mode | ○All traffic routes through it | ○All traffic routes through it | ● |
| Per-end-user spend caps | ○ | ○ | ◐Via custom properties | ●Per key, user or team | ◐One virtual key per customer | ● |
| Per-run caps for a single agent run | ○ | ○ | ○ | ○Budgets are per key, not per run | ○ | ● |
| Traces, spans and debugging | ● | ● | ● | ◐Logs, plus integrations | ● | ○Records decisions, not traces |
| Evaluations and prompt experiments | ● | ● | ◐ | ○ | ◐ | ○Not what it is for |
| Enforcement without an enterprise plan | ○No enforcement at any tier | ○No enforcement at any tier | ◐ | ●Open source | ○Enterprise plan only | ● |
| Self-hostable | ● | ◐Enterprise self-host | ● | ● | ◐Open-source gateway | ○Hosted only today |
| Actively developed | ● | ● | ○Maintenance mode since March 2026 | ● | ◐Acquired, now inside Prisma AIRS | ● |
noburn loses three rows on purpose: no traces, no evals, no self-hosting yet. It sits in front of the tracing tool you already run.
Where noburn does not compete
noburn does not replace Langfuse or LangSmith. It holds no traces, renders no spans and runs no evals, and if you need those you should keep the tool you have. It sits in front of them: the check decides whether the call happens, and whatever you already use records the calls that do.
Against a gateway the trade is explicit. LiteLLM and Portkey can enforce a budget, and in exchange every request routes through their proxy. noburn asks for one extra call and leaves your provider connection alone. If you already run a gateway and are happy with it, that is a reasonable place to cap spend.
Tooling comparisons
Cost, enforcement and architecture, tool against tool.
- LLM Gateway Comparison 2026: LiteLLM, Portkey, Helicone, and What Each Actually DoesFour tools, four different bets on what the gateway layer should do. Here is a full side-by-side of architecture, pricing, enforcement capabilities, and who each one is actually built for.
- Helicone vs Portkey in 2026: Which One Actually Enforces Your BudgetBoth tools promise LLM cost control. One is in maintenance mode after an acquisition. The other locks enforcement behind enterprise pricing. Here's what the comparison actually looks like.
- LiteLLM Alternatives in 2026: What Production Teams Are Actually UsingLiteLLM works well at small scale. At production scale — concurrent users, multi-tenant billing, high RPS — teams run into three structural limits. Here's what they're switching to.
- LangSmith vs Langfuse in 2026: Which One for Production Agent TeamsLangSmith and Langfuse both promise LLM observability. The pricing, enforcement story, and self-hosting options look very different. Here is the full comparison.
- Traceloop vs Helicone: LLM Observability at Agent ScaleBoth tools track LLM calls and aggregate costs. The architecture, self-hosting story, and enforcement capabilities differ significantly. Here is how they compare for teams running agents.
- Arize Phoenix vs LangSmith: Open Source vs Managed LLM ObservabilityPhoenix is open-source and OpenTelemetry-native. LangSmith is managed and LangChain-native. For teams choosing between a self-hosted and a managed observability stack, the trade-offs go deeper than hosting.
- Datadog vs LangSmith for LLM Monitoring: What Each Tool Actually CoversDatadog added LLM observability features. LangSmith was built for it from the start. The overlap is smaller than the marketing suggests and the gaps in each direction are significant.
- Braintrust vs LangSmith: Which One to Use for LLM Evaluation in 2026LangSmith is the right tool for LangChain teams and fast production tracing. Braintrust is the right tool for everything else. Here is exactly where the line is.
- Weights & Biases vs Braintrust: ML Experiment Tracking vs LLM EvaluationW&B was built for ML training runs. Braintrust was built for LLM prompt experiments. Both have expanded into each other's territory. Here is what each actually does well in 2026.
- Vercel AI SDK vs LangChain: Which Framework for Your Next AI FeatureVercel AI SDK is lean, streaming-first, and Next.js-native. LangChain is broad, chain-based, and Python-primary. The right choice depends on what you're building more than team preference.
Model and provider cost
What the calls themselves cost, before any tool is involved.
- OpenAI vs Anthropic API Pricing in 2026: Real Cost Per Million TokensModel pricing changes constantly. Here is a current breakdown of what you actually pay per million tokens across OpenAI and Anthropic, with worked examples for common agent workloads.
- Claude Sonnet 5.5 vs GPT-6.1 Sol: Which Is Actually Cheaper for Your WorkloadThe headline price per million tokens does not tell the full story. Claude Sonnet 5.5 and GPT-6.1 Sol now list at the same rates, so output ratio, caching, batch pricing and token counts decide the bill. Here is how to model it properly.
- GPT-6 Luna vs Claude Haiku 5.5: Which Small Model Is Worth Using in ProductionBoth are the cheap tier of their respective families, and they now list at the same price for most prompts. They have different strengths on different task types. This is a practical comparison using real tasks, not benchmark scores.
- AWS Bedrock vs Azure OpenAI: Enterprise LLM Cost and ComplianceBoth platforms let enterprise teams run frontier models inside their cloud perimeter. The pricing models, compliance posture, and vendor lock-in implications look different under scrutiny.
Sources
Every claim made about a named competitor on this page was checked against that vendor’s own documentation in August 2026. This category moves quickly. If something here has gone out of date, it is a bug.