Five open source AI gateway options you can self-host are Tyk AI Studio Community, LiteLLM, Bifrost, Portkey Gateway and Agent Router, formerly Envoy AI Gateway. Checked on 30 October 2026, the stable releases include Tyk AI Studio 2.2.2, LiteLLM 1.106.6 and Bifrost HTTP 2.4.0. The decisive difference is what the free edition enforces: Tyk AI Studio Community provides LLM access and reporting, while budget enforcement, guardrail execution and semantic caching require Tyk AI Studio Enterprise.
Last updated: 30 October 2026. All linked sources were checked on this date.
Disclosure: Tyk publishes this comparison and makes Tyk AI Studio; other products are assessed against their own documentation and repositories.
How do open source AI gateway licences and limits compare?
The table separates self-hosting from documented air-gapped operation. “Budgets enforced free?” means a monetary spending limit that can block requests; token quotas are identified separately. Release dates follow the publisher's release records. Prereleases are excluded from the stable-release column. “Not documented” means we found no vendor documentation for it on 30 October 2026, not that the capability is absent.
How did we compare these open source LLM gateways?
This comparison examines five projects through a licence and deployment lens. The review uses licence files, product documentation and release records checked on 30 October 2026. The review did not execute gateway deployments, benchmark latency or test isolation. Configuration examples below come from documentation; the acceptance procedure is a proposed evaluation method.
An AI gateway mediates access to AI services; an LLM gateway or LLM proxy is the model-request part of that role. An MCP gateway governs access to tools, resources and prompts. Those capabilities can belong to different components and editions. For the broader terminology, see what an AI gateway does.
Four questions determine the comparison. Which source licence covers the deployed component? Which free features actively block traffic? Which management services remain outside the customer's network? Which stable version implements the claimed behavior? A repository badge or product-wide feature list cannot answer all four.
For air-gapped evaluation, identify every dependency: image and package registries, databases, model servers, embedding services, identity providers, licence checks, telemetry and model-price updates. A locally hosted proxy can still send requests to an external model or management service. LiteLLM's outbound-request inventory illustrates why offline configuration must cover more than the proxy process.
Budget reporting, monetary enforcement and token quotas receive separate treatment. A token allowance does not establish a currency budget. A spending dashboard does not establish blocking. A configured limit also needs an explicit answer for concurrent requests, delayed usage accounting and unavailable state storage.
What does Tyk AI Studio Community include?
Tyk AI Studio Community is available under AGPL-3.0. The edition matrix includes the LLM proxy, embedders and cost tracking in Community. Tyk documents Docker, Kubernetes and Linux installation for Community and Enterprise.
Tyk AI Studio distinguishes two cache capabilities. The Community LLM Response Cache provides exact matching. Semantic matching requires the Enterprise Advanced LLM Cache feature and an Enterprise licence, as the cache documentation states explicitly. A general claim that “Tyk has caching” therefore needs an edition and a matching method.
Tyk AI Studio Community records budget information but does not stop requests when a limit is exceeded. Budget enforcement requires Enterprise. The model router and semantic router also require Enterprise.
Tyk AI Studio's filter documentation makes another operational distinction: Community does not execute configured filters, so content passes through unchanged. Guardrail and filter execution require Enterprise. The audit trail, outbound webhooks and fine-grained roles and permissions are also Enterprise capabilities.
How can you start Tyk AI Studio Community?
The official Community quickstart uses these commands:
git clone https://github.com/TykTechnologies/ai-studio.git
cd ai-studio/quickstart
docker compose -f ./ce/compose.yaml up -dThese are setup instructions, not results from a deployment test. For a repeatable evaluation, record the checked-out commit and resolved container versions before testing edition boundaries. Pin the chosen artifacts when moving from evaluation to an operational deployment.
Which MCP capabilities belong to the separate Tyk Gateway?
Tyk Gateway's open-source core uses MPL-2.0, with commercial code excluded from that licence. Remote MCP proxies became generally available in Gateway 5.13 and are available across Gateway licences, including open source. These are Gateway capabilities; installing AI Studio Community does not establish their deployment.
Tyk Gateway policies control JSON-RPC methods and named tools, resources and prompts. The policy documentation defines four rate-limit levels: global policy, MCP proxy, JSON-RPC method and named primitive. Tyk Gateway supports Streamable HTTP only, including SSE responses within that transport; stdio and legacy HTTP+SSE are unsupported.
Per-primitive OAuth2 scope enforcement and discovery filtering are available in Tyk Gateway 5.14. Gateway 5.16 adds MCP 2026-07-28 support alongside 2025-11-25 and Origin validation. Check the release notes for the edition that includes each feature before relying on it.
REST API to MCP requires Enterprise and Gateway 5.15 or later, with a Tyk OAS REST API as the source. Kubernetes management through TykMcpProxyDefinition requires Operator 1.4.0 or later. Tyk's air-gapped installation guide covers Gateway deployment; Tyk's AI Studio documentation does not describe a full offline installation.
What does LiteLLM provide in the open-source edition?
LiteLLM's licence file applies MIT terms outside the separately licensed enterprise directory. The edition documentation includes virtual keys, basic budgets, spend tracking, an administration interface and the MCP gateway in the open-source gateway.
LiteLLM's free budget enforcement has a required dependency: PostgreSQL. The budget guide states that a database-less deployment does not enforce the documented budgets. Even a configured global max_budget fails open without the database-backed spend total. A configuration file containing a budget is therefore insufficient evidence that spending is capped.
LiteLLM documents Docker and Kubernetes deployment. Production guidance separates persistent data from shared runtime state: PostgreSQL holds keys, configuration and spend records, while Redis is required when running multiple instances. The router documentation covers load balancing, retries, timeouts, cooldowns and provider fallbacks.
LiteLLM's caching documentation distinguishes exact request matching from semantic matching and lists Redis, Valkey and Qdrant semantic backends. Exact caches use the complete request to identify a match. The documentation cautions against treating semantic caching as interchangeable with exact caching for multi-turn or agent workloads, where a changed conversation can require a different response.
LiteLLM provides a documented offline mode through LITELLM_OFFLINE_MODE=true. That setting suppresses specified automatic outbound lookups; configured model providers, guardrails, logging integrations and identity services remain separate network dependencies. Offline deployment still requires staging the relevant packages and data and selecting reachable upstream services.
LiteLLM's MCP gateway supports tools, prompts and resources, with access controls associated with keys and teams. The documented upstream transports include Streamable HTTP, SSE and stdio. Stdio servers are disabled by default and require LITELLM_ENABLE_MCP_STDIO=true.
LiteLLM Enterprise adds controls including audit logs, delegated administration, secret-manager integrations and some advanced budget scopes. The guardrail boundary is also specific: custom guardrails and Presidio are in the OSS framework, while several named built-in callback integrations require Enterprise. Treating every guardrail integration as either free or paid would misstate those documented differences.
What does Bifrost provide in the open-source edition?
Bifrost's public project uses Apache-2.0, with binary and Docker installation. Bifrost documents retries and fallbacks and exact and semantic caching among open-source capabilities.
Bifrost's budget documentation describes enforcement across applicable provider, virtual-key, team and customer budgets. A failed applicable budget blocks a request. The examples also show usage being recorded after a request, so budget enforcement should not be interpreted as a guarantee that completed spend can never cross the configured amount.
Bifrost's deployment boundary matters for shared budgets. The multinode guide says that multiple OSS nodes sharing a PostgreSQL configuration backend are unsupported because important configuration, usage and budget state remains in memory. The OSS multinode approach uses shared file configuration; Enterprise adds runtime state synchronization. Multiple running replicas and one coordinated budget are different properties.
Bifrost publishes an explicit air-gapped deployment procedure. Pricing and model-parameter datasheets must be supplied locally when their normal endpoints are unreachable, or startup fails. The MCP library catalogue can be loaded locally or have synchronization disabled. Local file locations must be configured through config.json; the management UI and configuration API accept HTTP or HTTPS locations instead.
Bifrost's MCP gateway FAQ explicitly confirms open-source availability. The MCP documentation describes Bifrost as a client of upstream servers and a server for downstream clients. HTTP 2.4.0 adds prompts, resources and support for MCP 2026-07-28, according to the release notes.
Bifrost's Enterprise documentation separately lists identity provisioning, fine-grained access controls, audit logs and clustering. The built-in Enterprise guardrail system is another paid boundary. An open-source gateway and an Enterprise governance feature should not be merged into one unqualified feature claim.
Can Portkey Gateway be self-hosted without the commercial platform?
Portkey Gateway is available under the MIT licence. The repository documents Node.js, Bun, Docker and Docker Compose deployment. Portkey's open-source documentation includes fallbacks, load balancing and retries.
Portkey requires careful release qualification. The latest tagged stable release checked is 1.15.2, published on 12 January 2026, while the current README labels Gateway 2.0 prerelease. Documentation describing Portkey's broader platform cannot automatically establish the contents of that stable package.
Portkey's 24 March 2026 Production Gateway announcement says budget limits, semantic caching and other capabilities moved into open source. The announcement also describes an MCP registry and request, token and cost policies. Those statements describe the announced 2.0 scope; as of 30 October 2026, 2.0 is still a prerelease and the latest stable tag is 1.15.2.
The current repository also includes a guardrail configuration example. Describing Portkey OSS as having no guardrails would therefore be inaccurate. However, the README and edition matrix retain boundaries that do not fully align with the 2.0 announcement. The unresolved question is which features execute in the exact build being deployed.
Portkey's commercial hybrid deployment has a different ownership boundary. The architecture documentation places the data plane in the customer's environment and the administration, configuration and analytics control plane with Portkey. A customer-hosted gateway does not make that control plane self-hosted.
Portkey's current deployment documentation states that new fully air-gapped deployments are no longer offered, while existing air-gapped deployments remain supported. That describes the commercial offering. We found no documented offline procedure for the standalone OSS gateway.
What does Agent Router, formerly Envoy AI Gateway, provide?
Agent Router is the project's current name following the September 2026 rename announcement. The project retains Apache-2.0 licensing. The renamed repository does not require renaming the existing CRDs, API group, container images or aigw command.
Agent Router documents a standalone CLI for Linux and macOS and a Kubernetes deployment on Envoy Gateway. The Kubernetes architecture uses Agent Router as a control plane. Provider fallback is configured through route backends and retry policies.
Agent Router's QuotaPolicy manages upstream token consumption against provider backends and models. The documentation describes Redis-backed counters and rejection when the relevant quotas are exhausted. Those cumulative token quotas are distinct from consumer rate limits. We found no documentation of native monetary (currency) budgets.
Agent Router's MCPRoute API supports server aggregation, tool filtering and OAuth-based access controls over Streamable HTTP. Protocol support still needs a version qualifier: the 1.2 release notes describe stateless MCP 2026-07-28 support as work in progress, not enabled functionality.
The 1.2.0 GitHub release was published on 7 October 2026. The self-hosting guide establishes local and Kubernetes operation; we found no documented air-gapped installation procedure.
What does open source not include?
Open source identifies the licensing of particular code. Feature availability still needs an edition, release and deployment topology. LiteLLM's enterprise directory exclusion and Tyk Gateway's commercial-code exception illustrate why a repository's headline licence should not be applied to every bundled capability.
Enforcement also needs a more precise description than “supports budgets.” Tyk AI Studio Community supplies reporting without blocking. LiteLLM requires database-backed spend state. Bifrost documents limits on coordinated OSS runtime state. These are materially different operating conditions.
A management action and a request decision also produce different evidence. Tyk AI Studio's Enterprise audit trail records management actions; the documentation explicitly separates that trail from LLM proxy traffic. An audit requirement should name which events must be recorded, who can read the records and how long the records must survive.
Finally, open source does not establish ownership of a separately hosted management service. Portkey's hybrid architecture is one documented example. Evaluate the gateway process, management plane and model endpoint as separate parts of the deployment. Apply the same evidence standard to every vendor, including Tyk.
Operational ownership needs its own checklist. Who approves new models, rotates credentials, restores configuration and updates the gateway? Can those actions be completed during a management-service outage? Which team maintains database backups and the model-cost catalogue? These questions turn a self-hosting requirement into an operating plan and make the effort behind a free software licence visible before adoption.
When is Tyk not the right choice?
Tyk AI Studio Community is not the right edition when mandatory controls must all execute without a commercial licence. Tyk's edition matrix places budget enforcement, guardrails, semantic caching and model routing in Enterprise. Those requirements should be settled before an evaluation treats Community as an operationally equivalent substitute.
Tyk Gateway is also not a direct match for an MCP deployment that requires native stdio or legacy HTTP+SSE. Tyk documents Streamable HTTP only. A bridge introduces another component to deploy and evaluate; the gateway's SSE response support does not change the transport boundary.
Tyk's Gateway air-gap guide should not be treated as approval for an entire AI Studio stack. If fully offline AI Studio operation is a procurement prerequisite, confirm the complete topology and its outbound dependencies with Tyk.
For a small application that only needs one model endpoint and no shared policy, an additional gateway may introduce more configuration and operational work than the application requires. Establish the policy or ownership problem first. A gateway evaluation becomes more useful when the requirements name the controls that must run centrally.
How do you choose the best LLM gateway for your deployment?
Choose the best LLM gateway by turning requirements into observable decisions. Start with the mandatory licence and network boundaries, then evaluate the released edition against the same request set. The following procedure is an evaluation plan, not a report of tests performed for this article.
- Freeze the candidate. Record the repository commit, release tag, container digest, edition and enabled plugins. Keep an announced feature separate from a capability present in the selected artifact.
- Define the network boundary. List every permitted destination. Prestage dependencies, block other outbound connections, then test startup, model calls, administration, restarts and configuration changes. Record any dependency that prevents operation.
- Exercise the budget decision. Send an allowed request, exhaust a deliberately small allowance, then send another request. Check both the client response and upstream logs. Repeat with concurrent requests and multiple replicas.
- Exercise identity boundaries. Use two identities with different model and tool permissions. Verify discovery and direct invocation separately. Check that a caller cannot bypass a hidden tool by sending the tool name directly.
- Exercise degradation. Interrupt shared state, the primary model endpoint and selected external integrations. Record whether each operation blocks, continues or falls back, and compare that behavior with the written requirement.
- Retain decision evidence. Save configuration, expected outcomes, actual responses and relevant logs together. Distinguish a documented feature, an observed result and an unresolved question.
For a concrete budget exercise, configure two applications with separate allowances. Exhaust application A's allowance while leaving application B below its limit. The proposed requirement is that new model requests from A are denied while authorized requests from B continue. Record whether A's denied request reached the upstream provider, rather than relying only on the gateway's displayed spending total. Repeat after a restart and during concurrent traffic. A pass should describe the observed scope and any allowed accounting delay; an unexplained result remains an open question.
For an air-gap exercise, include maintenance as well as model requests. Restart every required component, change a policy, restore a saved configuration and perform the planned upgrade with external access blocked. Record which files and services had to be staged inside the boundary. This produces a deployment-specific answer that a general “self-hosted” label cannot supply.
Use a compact record for each required control:
Product and edition:
Release, commit and image digest:
Required control:
Configuration and dependencies:
Allowed-request result:
Denied-request result:
Upstream request observed:
Restart and concurrency results:
Remaining verification:For migration-specific requirements, see LiteLLM alternatives and the self-hosted Portkey alternative comparison. For agent-to-tool governance, continue with the enterprise MCP gateway comparison. Use the resulting requirements to decide whether Tyk AI Studio Community, AI Studio Enterprise and the separate Tyk Gateway match the deployment you intend to operate.
What are common questions about open source AI gateways?
What is an open source AI gateway?
An open source AI gateway is software with an open-source licence that mediates requests to AI services. An LLM gateway focuses on model requests; MCP gateway capabilities govern tool access. Check the licence and edition for each component, because some repositories contain separately licensed Enterprise code.
What is the best AI gateway for self-hosting?
The best AI gateway for a deployment is the released edition that satisfies the required policies and network boundaries. Compare spending enforcement, management-plane ownership, identity controls, protocol support and recovery behavior against explicit acceptance criteria. This comparison does not assign a universal winner.
Which open source LLM gateways enforce budgets for free?
LiteLLM enforces basic budgets with PostgreSQL, and Bifrost documents OSS budget enforcement. Bifrost's multinode state limitations still apply. Tyk AI Studio Community reports spending; Enterprise enforces budgets.
Can a self-hosted LLM gateway run air-gapped?
Air-gapped operation requires the gateway and every necessary dependency to work within the isolated network. LiteLLM documents offline configuration, and Bifrost documents an air-gap procedure. Tyk documents Gateway installation; AI Studio's documentation does not describe a full offline installation.
Is Tyk AI Studio open source?
Tyk AI Studio Community is AGPL-3.0. Model routing, semantic routing, semantic caching and guardrail execution require Enterprise. The separate Tyk Gateway open-source core uses MPL-2.0.
Does an open source AI gateway include an MCP gateway?
MCP support must be checked by component, edition and protocol version. Tyk provides remote MCP proxies through the separate Tyk Gateway. LiteLLM and Bifrost document MCP gateways. Agent Router documents MCPRoute, while Portkey's announced 2.0 MCP scope, which is in a prerelease.
