Bifrost

v1.0.0Cloud Servicesstable

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

ai-gatewaygatewaygateway-servicesgenerative-aiguardrails
Share:
5,134
Stars
0
Downloads
0
Weekly
0/5

What is Bifrost?

Bifrost is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to fastest enterprise ai gateway (50x faster than litellm) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k rps.

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

This server falls under the Cloud Services category on MCPgee, the world's largest MCP server directory with 33,000+ servers.

Features

  • Fastest enterprise AI gateway (50x faster than LiteLLM) with

Use Cases

Enterprise AI gateway with adaptive load balancing
Support 1000+ models with guardrails
maximhq

Maintainer

LicenseApache-2.0
Languagego
Versionv1.0.0
UpdatedMay 22, 2026
Statushealthy
Maintenanceactive

Works with

ClaudeOpenAIwindowsmacoslinux

Installation

Manual Installation

npx bifrost

Configuration

Configuration Details

Config File

claude_desktop_config.json

Performance

Response Metrics

Response Time< 200ms
ThroughputMedium

Resource Usage

Memory UsageLow
CPU UsageLow

How to Set Up and Use Bifrost

Bifrost is a high-performance enterprise AI gateway written in Go that exposes a single OpenAI-compatible endpoint in front of 23+ LLM providers, delivering sub-100 µs overhead at 5,000 requests per second. It handles automatic failover, adaptive load balancing, intelligent semantic response caching, budget caps, rate limiting, and MCP-based external tool integration — all configurable through a built-in web UI, a REST API, or JSON config files. Organizations use Bifrost to centralize model access, enforce guardrails, and gain Prometheus-native observability across every AI call in their stack.

Prerequisites

  • Docker (recommended) or a Go 1.21+ toolchain for source builds
  • API keys for the LLM providers you want to route through (OpenAI, Anthropic, Google, etc.)
  • Port 8080 available on the host (configurable)
  • An MCP-compatible client (Claude Desktop, Claude Code, etc.) to use the MCP integration feature
  • Optional: Prometheus + Grafana for metrics dashboards
1

Run Bifrost via Docker

The quickest way to start Bifrost is with Docker. The container exposes the gateway and web UI on port 8080.

docker run -p 8080:8080 -v $(pwd)/data:/app/data maximhq/bifrost
2

Or install and run with npx

If you prefer not to use Docker, you can launch Bifrost directly with npx without a global install step.

npx -y @maximhq/bifrost
3

Open the web UI and add your first provider

Navigate to the web UI at http://localhost:8080. Go to Providers, click Add Provider, select your LLM vendor (e.g., OpenAI), and paste your API key. Bifrost stores keys securely using environment variable references.

# Open: http://localhost:8080
4

Configure load balancing and failover

In the web UI under Routing, define a route that maps an incoming model alias (e.g., gpt-4o) to one or more provider-model pairs with priority weights. Enable automatic failover so Bifrost retries a secondary provider if the primary fails or rate-limits.

5

Enable MCP tool integration

Bifrost's MCP integration lets AI models routed through the gateway call external tools (filesystem, web search, databases). Enable it in the MCP section of the web UI or via the JSON config, then point your MCP client at Bifrost's endpoint.

6

Configure your MCP client to use Bifrost

Add Bifrost as the AI gateway in your claude_desktop_config.json so all model calls from your client pass through the gateway's load balancer and guardrails.

{
  "mcpServers": {
    "bifrost": {
      "command": "npx",
      "args": ["-y", "@maximhq/bifrost"],
      "env": {
        "BIFROST_PORT": "8080"
      }
    }
  }
}

Bifrost Examples

Client configuration

claude_desktop_config.json snippet for launching Bifrost via npx and connecting Claude Desktop to it.

{
  "mcpServers": {
    "bifrost": {
      "command": "npx",
      "args": ["-y", "@maximhq/bifrost"],
      "env": {
        "BIFROST_PORT": "8080"
      }
    }
  }
}

Prompts to try

Sample prompts once Bifrost is running and your MCP client is connected.

- "Route this request through the cheapest available model that supports 128k context."
- "Show me the current rate-limit status for my OpenAI provider in Bifrost."
- "Set a monthly budget cap of $50 on the Anthropic provider."
- "Explain what semantic response caching means in Bifrost and how it saves tokens."

Troubleshooting Bifrost

Gateway returns 502 Bad Gateway for all requests

Verify the upstream provider API key is valid and that the provider endpoint is reachable from the Bifrost container. Check container logs with `docker logs <container-id>` for upstream error details.

Web UI shows no metrics after several minutes of traffic

Ensure the Bifrost data directory is writable (the -v flag in the docker run command must point to a directory the container user can write to). Check that Prometheus scrape config points to http://localhost:8080/metrics.

MCP tools not appearing in connected client

Confirm MCP integration is enabled in the Bifrost web UI under Settings > MCP. Restart the MCP client after saving changes, and verify the client config URL matches the running Bifrost port.

Frequently Asked Questions about Bifrost

What is Bifrost?

Bifrost is a Model Context Protocol (MCP) server that fastest enterprise ai gateway (50x faster than litellm) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k rps. It connects AI assistants to external tools and data sources through a standardized interface.

How do I install Bifrost?

Follow the installation instructions on the Bifrost GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.

Which AI clients work with Bifrost?

Bifrost works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.

Is Bifrost free to use?

Yes, Bifrost is open source and available under the Apache-2.0 license. You can use it freely in both personal and commercial projects.

Browse More Cloud Services MCP Servers

Explore all cloud services servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.

Quick Config Preview

{ "mcpServers": { "bifrost": { "command": "npx", "args": ["-y", "bifrost"] } } }

Add this to your claude_desktop_config.json or .cursor/mcp.json

Read the full setup guide →

Ready to use Bifrost?

Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.

33,000+ ServersFree & Open SourceStep-by-Step Guides