LiteLLM Alternatives (2026): 9 Gateways for When You Do Not Want to Run the Proxy Yourself

The best LiteLLM alternative depends on why you are leaving. If you no longer want to run a proxy at all, move to a hosted gateway: Constellation Gate AI if you also want prompt-injection screening and an audit log nobody can edit, Vercel AI Gateway or Cloudflare AI Gateway for plain routing at list price. For a managed console with guardrails, Portkey. If you still want to self-host but want something lighter, Bifrost or TensorZero. If the traffic must never leave your network, stay on LiteLLM. Everything below was checked on vendor pages and GitHub on 23 September 2026.

LiteLLM earned its place: open source, one OpenAI-compatible API to more than 140 providers, virtual keys, budgets and teams for nothing. This page is about the moment you realise "free" meant "free, plus a server you now own".

Why people leave LiteLLM

• It is a stack, not a script. Virtual keys need a Postgres database; LiteLLM's own docs list "Need a postgres database" as step one. Its production guide says to run Redis 7 or newer "as soon as you run more than one proxy instance", to give each pod 1 vCPU and 4 GiB of memory, and to run a separate worker deployment for scheduled jobs like budget resets. Sensible, and four things to keep alive.
• Upgrades never stop. LiteLLM tagged 27 releases on GitHub in the 30 days to 23 September 2026. That speed is why new models arrive fast; it also means someone tests upgrades every week.
• The patching is yours. GitHub lists 14 security advisories for LiteLLM published between April and August 2026, three rated critical: an authentication bypass through an OIDC cache-key collision, SQL injection in API key verification, and an authentication bypass through host-header injection. All are patched. That is normal for popular software; the point is that on a self-hosted proxy, applying the patch is your job, and the proxy holds every provider key you own.
• The supply-chain scare. On 24 March 2026 two malicious LiteLLM versions (1.82.7 and 1.82.8) were uploaded to PyPI with a credential stealer inside. LiteLLM's own incident note says the attacker likely used credentials stolen via a compromised dependency in its CI, that customers on the official proxy Docker image were not affected because it pins dependencies, and that anyone who installed those versions with pip should rotate every secret. Handled openly, and a lesson in what "you host it" includes.
• The governance features are Enterprise. LiteLLM's docs list SSO (free for up to five users), SCIM, organisations and delegated admin roles, key rotation, secret-manager integration, and audit logs of admin actions as Enterprise features, priced on request.

If none of these bite, keep LiteLLM. If two of them do, read on.

Before you migrate: harden the LiteLLM you have

Moving is work too. If you are staying for now, these cost nothing:

1. Run the official Docker image, pinned to a version, rather than pip install litellm. It was the path the March incident did not reach.
2. Watch the advisories. Set GitHub to notify you of the repository's security alerts.
3. Keep the admin UI and management endpoints off the public internet. Several of this year's advisories sit in admin, MCP and test endpoints.
4. Keep provider keys in a secret manager, not in the config file, and rotate them after any incident you are unsure about.
5. Set budgets per virtual key. They are in the open-source version.

Self-hosted vs hosted gateway: what you are actually trading

Self-hosted keeps every request on infrastructure you control, with no licence fee; you pay in hours for the database, cache, uptime, upgrades, patches and your own guardrails. A hosted or managed LLM gateway takes all of that away and puts a third party in the path instead.

The useful question is not "which is cheaper" but "who notices when it breaks at eleven at night". For a one-person business or a small agency, that is usually nobody, so hosted with a free tier now, self-hosted when you have someone to run it. If you are new to the category, what an LLM gateway does, in plain English is the place to start.

The comparison

Checked on each vendor's own pricing page, documentation or GitHub repository on 23 September 2026. "Fee on top" is what is added to the model provider's list price.

Gateway	Who runs it	What you operate	Fee on top	Screens for prompt injection	SSO and audit log	Free to start
LiteLLM	You	Proxy, Postgres, Redis, worker	0%	Guardrails you configure	Enterprise licence (SSO free to 5 users)	Open source, MIT core
Constellation Gate AI	Gate	Nothing	0% ("the same price you would pay the provider directly"), or your own keys	Yes; flags on Free, blocks on Pro	Hash-chained, anchored audit trail on every plan	20,000 recorded requests/mo
Vercel AI Gateway	Vercel	Nothing	0%, "no markup and no platform fee on tokens"	No	Budgets built in; traces paid	Monthly free credit, subset of models
Cloudflare AI Gateway	Cloudflare	Nothing	0% with your own keys; 5% on Unified Billing credits	Guardrails billed as Workers AI; DLP free	Logs, 100k free	Core features free
Portkey	Portkey, or you (open source)	Nothing on hosted	None; plan fee	Deterministic free; LLM and partner guardrails on $49	SSO and org audit logs on Enterprise	10k logs/mo
OpenRouter	OpenRouter	Nothing	5.5% on pay-as-you-go credits (Standard)	No	Activity log	Free models
Helicone	Helicone, or you (open source)	Nothing on hosted	None; plan fee	No	Observability logs	10k requests/mo
Bifrost (Maxim)	You	Go binary or container	0%	Via plugins	Governance on enterprise	Open source, Apache 2.0
TensorZero	You	Rust gateway plus your database	0%	No	Your own database	Open source, Apache 2.0
Kong AI Gateway	You, or Kong Konnect	Kong Gateway	Not a token reseller	Regex allow/deny prompt guard	Via Kong	AI plugins free since 3.6

If you want nothing to run: the hosted alternatives

Constellation Gate AI

Gate is the alternative for people whose real problem was never routing. Affiliate link: we earn a commission on Pro seats at no cost to you, and it pays for the testing, not the verdict. It calls itself "the accountability layer for AI" and sits "between your agent and the model". It is hosted; there is no database, no Redis and no container on your side. You pay for tokens through Gate at "the same price you would pay the provider directly", or bring your own keys and keep paying OpenAI or Anthropic as now.

What it takes off your plate, measured against the list above:

• The server and the patching. Gone. Gate runs it.
• The guardrails you were going to wire up. Every request and response is screened for prompt injection. Gate's own report (arXiv:2606.02959) puts it at 95.4% of attacks caught at a one-in-a-hundred false-positive cap, with a 53 ms median. Vendor figures; our own test reports on 30 September.
• The audit log behind the Enterprise licence. Every request is written into a hash-chained log anchored to Constellation's Digital Evidence layer, on the free plan. It records what your AI sent and received, not just admin actions, in a form nobody can quietly edit (what a real AI audit trail needs).
• Some of the bill. Gate losslessly removes repeated file reads and duplicated context before forwarding: 20% or more fewer tokens on agent workloads by its own figures, 23% in its Rocket Resume case study. Its cache is exact-match, not semantic.

It also does something LiteLLM does not try to do: the Gate Connect app puts Claude Code, Codex, OpenCode, Cursor, Claude Desktop and ChatGPT behind the gateway while your existing subscription keeps paying for the model (Claude Code setup, Codex setup). Free is $0 with 20,000 recorded requests a month, basic compression and the audit trail. Pro is $20 per user per month for blocking, redaction, spend limits and full compression. Enterprise adds a private cloud or VPC deployment, which is the route if you want Gate's features without the shared service. Our full Gate review has the rest, and our Gate page has the setup.

Try it on one tool. Start free with Gate: no card, nothing to deploy, one change of address. Free records, Pro blocks: the free plan shows you a flagged request; stopping it is Pro.

Best for: anyone who left LiteLLM because running it took time they wanted back, and whose AI reads e-mail, web pages or documents nobody on the team wrote (why that is the risk).

Vercel AI Gateway

If routing at list price is all you used LiteLLM for, this is the shortest move. Vercel states "AI Gateway charges no markup and no platform fee on tokens", and there is no fee on bring-your-own-key either, though that needs the paid tier (you buy credits first) and you pay card-processing fees. Budgets per team, project, key or member are built in. Traces and team-wide zero data retention are paid add-ons. No screening.

Best for: developers who deploy on Vercel and want LiteLLM's one-endpoint convenience without the server.

Cloudflare AI Gateway

Free for the core features: analytics, caching and rate limiting, with 100,000 stored logs on the free Workers plan and 10 million per gateway on paid. Your own provider keys cost nothing; credits bought through Unified Billing carry a 5% fee. DLP scanning is free; guardrails run on Workers AI and are billed as inference.

Best for: anyone on Cloudflare who wants caching and rate limits in front of their own keys for free.

OpenRouter

The opposite of LiteLLM: one OpenRouter key for hundreds of models on their servers. The Standard plan adds a 5.5% platform fee on pay-as-you-go purchases. It routes and logs; it does not screen. If the fee is what bothers you there too, our OpenRouter alternatives comparison covers the zero-markup options.

Best for: trying many models under one key, at low spend.

Helicone (observability first; free to 10,000 requests, Pro $79 a month, open-source version available) sits in the table for completeness; pick it if the spend dashboard was all you used LiteLLM for.

If you still want to self-host: the lighter alternatives

Portkey

Portkey is the closest like-for-like, because it has both halves: an open-source gateway you can host yourself and a managed console you can pay for instead. Developer is free with 10,000 logs a month and deterministic guardrails; Production is $49 a month with 100,000 logs, LLM and partner guardrails and role-based access control; Enterprise adds SSO, org-wide audit logs and private-cloud or VPC hosting.

Best for: teams leaving LiteLLM for operational reasons who want the same open-source core with someone else running the hosted version.

Bifrost

An Apache 2.0 gateway from Maxim, written in Go, with one OpenAI-compatible API to more than 20 providers, failover, load balancing, semantic caching and MCP support. Its README claims about 11 microseconds of added latency at 5,000 requests per second and calls itself "50x faster than LiteLLM". Vendor benchmark, but a Go binary is genuinely less to run than a Python proxy with a Prisma engine. Clustering and advanced governance sit in the enterprise edition.

Best for: engineers who want to keep self-hosting but spend less on compute and latency.

TensorZero

An Apache 2.0 platform with a Rust gateway ("<1ms p99 latency overhead", its claim), plus observability, evaluations, A/B testing and fine-tuning workflows on top, storing data in your own ClickHouse or Postgres. Self-hosted only. More than a LiteLLM replacement: the point if you were about to bolt five tools on, overkill if you only wanted routing.

Best for: teams building an AI product who want the gateway and the experiment loop in one place.

Kong AI Gateway

Kong made its AI plugins, including AI Proxy and AI Prompt Guard, free and open source from Kong Gateway 3.6. The prompt guard uses regular-expression allow and deny lists, which catches patterns you already know.

Best for: organisations that already have Kong and people who run it.

LiteLLM vs Portkey

Both have an MIT-licensed gateway you can host. The practical difference is what you buy when you stop wanting to host: Portkey sells the managed service, LiteLLM sells an Enterprise licence for software you still operate. SSO and org-wide audit logs are paid on both. Leaving over operations: Portkey's hosted plan is a direct swap. Leaving over the Enterprise line: Portkey draws a similar one.

LiteLLM vs OpenRouter

Opposite trades. LiteLLM: your server, your keys, no fee, all the work. OpenRouter: their server, their key, 5.5% on credits, no work. Neither screens traffic. If you want OpenRouter's convenience without its fee, Vercel and Gate both pass provider prices through at 0%.

When LiteLLM is still the right answer

If your contracts say the traffic may not pass through another processor, if you need the proxy inside your own network with zero third parties, and you have an engineer who is happy to own it, LiteLLM remains a strong choice, and so do Kong or Portkey's open-source gateway. Gate's Enterprise private-cloud option narrows the gap, but it is a sales conversation, not a toggle.

Which one

• You are tired of running the server and your AI reads things you did not write: Gate. Screening and an uneditable log, nothing to host.
• You are tired of running the server and only need routing: Vercel AI Gateway, or Cloudflare with your own keys.
• You want a managed console with guardrails: Portkey.
• You need a record of what your AI sent and received, without buying an Enterprise licence first: Gate; the hash-chained audit trail is on the free plan.
• You run Claude Code, Codex or Cursor on a subscription and want them fronted: Gate, via Gate Connect.
• You want to keep self-hosting, but lighter and faster: Bifrost.
• You want the gateway plus evaluations: TensorZero. Already run Kong: Kong AI Gateway.
• No third party, ever, and you have someone to run it: stay on LiteLLM, and do the five hardening steps above.

Not sure it is worth the move? Put your monthly spend into the AI API cost calculator and see what the token savings and a 0% fee are worth against the hours you spend on the proxy.

Your first fifteen minutes

Following the rule we apply to every tool: real input, one output, hard stop.

1. Minutes 0 to 3. Write down how many hours you or your developer spent on LiteLLM last month: upgrades, restarts, the advisory you had to read. Put a rate on it. That is the real price of "free".
2. Minutes 3 to 8. Create a free account on the hosted gateway your reason points to. For most readers of this site that is Gate: no card, one key, and Gate Connect covers Claude Code in one toggle.
3. Minutes 8 to 13. Point one app or tool that currently calls LiteLLM at the new base URL and key. Leave LiteLLM running. Do one real task.
4. Minutes 13 to 15. Open the dashboard. Note requests recorded, cost, and (on Gate) tokens saved and anything flagged. Stop.

Output: one tool on a hosted gateway, LiteLLM untouched as your fallback, and two numbers to compare in a week: the hours LiteLLM costs you and what the same traffic costs elsewhere. Free records, Pro blocks: if the first flagged request shows up before the week is out, the $20 Pro seat is a simple decision. Start free with Gate (affiliate link; we earn a commission on Pro seats).

Fees, features and security advisories checked on each vendor's own pricing page, documentation or GitHub repository on 23 September 2026. Gate's security and compression figures are vendor-published; our own field-test numbers follow on 30 September. If a number above is out of date, tell us and we will fix it with a dated note.