Antirion

Every model, half a millisecond away.

Antirion is the fastest open-source AI gateway. One Rust binary, console included, between your apps and every model provider.

Get startedStar on GitHub

Speaks OpenAI, Anthropic, Gemini and Bedrock, and calls every one of these, or any OpenAI-compatible server such as vLLM.

  • OpenAI
  • Anthropic
  • Amazon Bedrock
  • Vertex AI
  • Gemini
  • Azure OpenAI
  • Mistral
  • Cohere
  • Groq
  • Together
  • Fireworks
  • DeepSeek
  • xAI
  • Cerebras
  • DeepInfra
  • Perplexity

01/Performance

Overhead you can’t see.

Antirion, Bifrost and LiteLLM on the same dedicated machines, interleaved run by run, against an upstream that answers at once.

37×

lower overhead

p50 at 5,000 RPS: 0.53 ms against 19.7 ms

4×

the sustained rate

on 4 vCPUs: 14,000 RPS against 3,500 RPS

6.5×

less memory

peak at 5,000 RPS: 215 MiB against 1,389 MiB

Gateway overhead, median, simple chat at 5,000 requests per second. Lower is better.

Antirion

0.53 ms

Bifrost 2.2.4

19.7 ms

LiteLLM 1.103.1

every request failed

Under heavier load, one c7i.xlarge
ScenarioAntirionBifrost 2.2.4LiteLLM 1.103.1
Simple chat at 10,000 RPS, overhead p50 / p990.61 ms / 1.50 ms12.1 s / 19.1 s100% errors
Invalid key rejected at 20,000 RPS, p50 / p990.30 ms / 0.75 ms1.86 s / 2.49 s100% errors
Unknown model rejected at 20,000 RPS, p50 / p990.57 ms / 1.27 ms17.7 s / 27.4 s100% errors

Read the methodRun it on your own AWS account

02/Console

The console is in the binary.

The gateway, the Admin API and the web console ship together. No sidecars, no separate UI to deploy, no runtime to install.

The Antirion console’s Overview: spend, requests, tokens, latency, error and cache hit rates, request volume and latency charts, team spend, top models and provider headroomThe Antirion console’s Overview: spend, requests, tokens, latency, error and cache hit rates, request volume and latency charts, team spend, top models and provider headroom

03/Drop-in

Change one line.

Clients speak OpenAI, Anthropic, Gemini or Bedrock, so the official SDKs, LangChain and the Vercel AI SDK work as they are. Antirion translates between the dialects, so any client can call any provider.

  1. 1Run the container. The console opens on port 8081.
  2. 2Add a provider and a Model. Name the Model chat, the name your apps send.
  3. 3Create a key. Point your SDK at port 8080.
1import os2from openai import OpenAI3 4client = OpenAI(-    base_url="https://api.openai.com/v1",+    base_url="http://localhost:8080/v1",6    api_key=os.environ["ANTIRION_API_KEY"],7)8 9reply = client.chat.completions.create(10    model="chat",11    messages=[{"role": "user", "content": "Hello"}],12)
app.py1 line changed

04/Platform

Production-ready out of the box.

Everything a platform team puts in front of its models, in the open-source build. There is no enterprise edition.

Routing that keeps you up

Fallback chains, weighted splits, least cost, least latency, tiers and classifiers, with retries, hedging and circuit breakers. Any client can call any provider: dialects and streams are translated.

Routing policy: rules by team and project with fallback and least-latency targets, and each rule’s share of trafficRouting policy: rules by team and project with fallback and least-latency targets, and each rule’s share of traffic

Spend under control

Rate limits and budgets in dollars or tokens on the organization, team, project, key and Model, with projections, run-out dates and a ledger behind every number.

Budgets: projected spend for the month and team budgets, one of them at 91%Budgets: projected spend for the month and team budgets, one of them at 91%

Every request, explained

The route taken, every attempt and why it failed, tokens, cost, cache status and guardrail verdicts. Prometheus, OTLP and log sinks for the rest of your stack.

Request log with a request that retried Anthropic and fell back to OpenAI, with its route, tokens, cost ledger and guardrail resultsRequest log with a request that retried Anthropic and fell back to OpenAI, with its route, tokens, cost ledger and guardrail results

Safety at the edge

PII, secrets, regexes, JSON schemas, tool permissions, moderation and LLM judges, on requests, responses and streams. Block, redact with format-preserving masks, flag or divert.

Safety and PII: the rule that redacts personal data, each kind it checks with its placeholder and its matchesSafety and PII: the rule that redacts personal data, each kind it checks with its placeholder and its matches

Standalone or cluster

An embedded store, or many nodes on PostgreSQL and Valkey.

node-1readynode-2readynode-3ready
PostgreSQLstoreValkeycounters, cache

MCP and agent gateways

Tool servers and A2A agents behind the same keys, budgets and log.

githubpostgres
Antirion
plannersupport
MCP serversA2A agents

Exact and semantic cache

Per Model, team or key, with hit rates and the spend each cache saved.

33.4%

hit rate, last 24 hours

~$49.05

saved

SSO, SCIM and audit

OIDC, SAML and GitHub sign-in, custom roles, change requests.

OIDCSAMLGitHubSCIMTOTPWebAuthn

Verifiable releases

SBOM, Sigstore signatures and SLSA provenance. Distroless, non-root.

  • SBOM
  • Sigstore signature
  • SLSA provenance
  • Distroless, non-root

Config as one document

YAML, API or console, schema-checked, with diffs and rollback.

version: 1server: { listen: "0.0.0.0:8080" }admin: { listen: "0.0.0.0:8081" }store: { backend: embedded }
  • #42Routing: fallback to openaicurrent
  • #41Budget: growth to $90Roll back
  • #40Guardrail: pii_redact onRoll back

05/Deploy

Your infrastructure, your data.

A single node with an embedded store, or a cluster on PostgreSQL and Valkey. Requests go from your network straight to your providers.

  • Docker

    A standalone node with an embedded store

    $docker run -p 8080:8080 -p 8081:8081 -v antirion-data:/var/lib/antirion eliasbiondo/antirion

  • Compose

    Two nodes on PostgreSQL and Valkey

    $docker compose -f compose.cluster.yaml up -d

  • Kubernetes

    The Helm chart, with autoscaling

    $helm install antirion deploy/helm/antirion --set mode=cluster

  • Binary

    Linux, amd64 and arm64, signed

    $antirion serve --config antirion.yaml

06/Questions

Asked often.

Something else? Open an issue on GitHub.

Cross over in thirty seconds.

From the first container to the first request through the gateway, the guide walks every step.

Read the getting-started guide