AI Agent Routing / Gateway Cost Comparator
Compare AI gateway and LLM routing costs across leading platforms. Model gateway fees, LLM token spend, caching savings, routing savings, and margin pricing.
LLM workload
Gateway platform
Core gateway is free. Persistent logs limited (100k free, 10M paid per gateway). Logpush billed at $0.05/million on Workers Paid. Provider token costs pass through.
Counted only for self-hosted option.
Pricing model
Total monthly cost
—
Net LLM spend / mo
—
Gateway fee / mo
—
12-month total
—
Raw LLM spend / mo
—
Cache savings / mo
—
Routing savings / mo
—
Self-host compute / mo
—
Engineering / mo
—
Pricing to hit margin
Required monthly revenue
—
Price per 1k requests
—
Price per 1M tokens
—
Cost comparison by gateway provider
Same workload and LLM spend across all gateway options.
| Gateway / plan | Base | Overage | Logpush | Compute | Eng. | Total/mo |
|---|---|---|---|---|---|---|
| Cloudflare AI Gateway Core gateway is free. Persistent logs limited (100k free, 10M paid per gateway). Logpush billed at $0.05/million on Workers Paid. Provider token costs pass through. | — | — | — | — | — | — |
| Portkey AI Gateway $49/mo base with 100k recorded logs. +$9 per 100k logs up to 3M. OSS self-hosted available. Provider token costs pass through. | — | — | — | — | — | — |
| Helicone AI Gateway $79/mo Pro with ~1M requests. Free tier for 10k/mo. Zero markup on provider tokens. Apache-2.0 self-host option. | — | — | — | — | — | — |
| LiteLLM Enterprise (managed) Enterprise Basic starts around $250/mo. Premium around $2,500/mo. Pricing by annual request volume and support level. OSS core is free self-hosted. | — | — | — | — | — | — |
| Braintrust Gateway Gateway currently free in beta. Starter plan free with 1M spans + 1 GB data; Pro $249/mo. Tracing and eval platform separate. | — | — | — | — | — | — |
| Zuplo (API gateway for AI) $0 free tier with 100k requests/mo; Builder plan $25/mo with scalable limits. Token-based rate limits and policies. | — | — | — | — | — | — |
| TrueFoundry AI Gateway $499/mo Pro includes 1M requests. Enterprise custom. MCP gateway, governance, and routing included. | — | — | — | — | — | — |
| Kong AI Gateway (Konnect) ~$105/service/mo plus usage; AI Gateway add-on; model proxy $100/model/mo in Konnect. Enterprise custom. Best for existing Kong users. | — | — | — | — | — | — |
| Self-hosted LiteLLM / Bifrost OSS Open-source self-hosted: no per-request gateway fees. You pay compute, storage, and engineering time. Enterprise license optional. | — | — | — | — | — | — |
How the estimate works
- Raw LLM spend = requests × ((input tokens ÷ 1M × input price) + (output tokens ÷ 1M × output price)).
- Cache savings = raw LLM spend × cache hit% × cache cost reduction%.
- Routing savings = raw LLM spend × routing savings%.
- Net LLM spend = raw LLM spend − cache savings − routing savings.
- Gateway fee = base fee + request overage + logpush. Many tiers have zero gateway fee.
- Margin pricing = total monthly cost ÷ (1 − margin%).
When to use each gateway
- • Cloudflare for free core gateway if already on Cloudflare Workers/Pages.
- • Portkey when you need managed gateway + prompt management + guardrails.
- • Helicone for open-source-friendly gateway with zero provider markup.
- • LiteLLM / Bifrost OSS if you want self-hosting and multi-provider standardization.
- • Braintrust when gateway traffic should feed evals and release checks.
- • TrueFoundry / Kong for enterprise governance, MCP, or existing API-management stacks.
Frequently asked questions
What does an AI gateway actually save?▼
Three main levers: (1) caching reduces repeated identical calls, (2) routing picks cheaper or faster providers for the same model class, and (3) rate limits and retries stop runaway spend from loops or bad deploys.
Do gateway fees replace LLM provider fees?▼
No for most gateways. Cloudflare, Helicone, Portkey, and Braintrust pass through provider token costs with zero markup; you pay the gateway/platform fee on top. LiteLLM OSS and Bifrost OSS have no per-token gateway fee beyond your own infrastructure.
When is a self-hosted gateway cheaper than managed?▼
At high request volume, managed log/request fees and seat costs dominate. Self-hosting LiteLLM or Bifrost shifts cost to compute and SRE time. Break-even often appears above a few million requests per month, especially with heavy caching.
How do I model caching savings?▼
Set a cache hit percentage and a cost-reduction percentage. A 20% cache hit with 80% cost reduction means 16% of otherwise-billable LLM spend is avoided. The tool subtracts that from the raw LLM estimate and shows net LLM spend.
Which gateway should I start with?▼
Cloudflare AI Gateway is free and generous for apps already on Cloudflare. Helicone and Portkey are strong managed options with observability. LiteLLM/Bifrost are good if you need self-hosting or multi-provider standardization.
What about routing savings?▼
Smart routing can move traffic to cheaper provider keys (e.g. AWS Bedrock vs direct Anthropic), use smaller models for simple prompts, or fall back to cached responses. Savings are typically 5–20% of LLM spend.
Prices are directional estimates based on public list pricing for AI gateway / LLM routing platforms. Confirm current rates before budgeting. LLM provider token costs are passed through by most gateways; this tool focuses on gateway fees, infrastructure, and savings from caching/routing. Some vendors quote custom Enterprise pricing. Last updated: 2026-08-10.