|
| 1 | +#AI Bridge |
| 2 | + |
| 3 | +>[!NOTE] |
| 4 | +>AI Bridge is currently an_experimental_ feature. |
| 5 | +
|
| 6 | +AI Bridge is a smart proxy for AI. It acts as a man-in-the-middle between your users' coding agents / IDEs |
| 7 | +and AI providers like OpenAI and Anthropic. By intercepting all the AI traffic between these clients and |
| 8 | +the upstream APIs, AI Bridge can record user prompts, token usage, and tool invocations. |
| 9 | + |
| 10 | +AI Bridge solves 3 key problems: |
| 11 | + |
| 12 | +1.**Centralized authn/z management**: no more issuing & managing API tokens for OpenAI/Anthropic usage. |
| 13 | + Users use their Coder session or API tokens to authenticate with`coderd` (Coder control plane), and |
| 14 | +`coderd` securely communicates with the upstream APIs on their behalf. Use a single key for all users. |
| 15 | +2.**Auditing and attribution**: all interactions with AI services, whether autonomous or human-initiated, |
| 16 | + will be audited and attributed back to a user. |
| 17 | +3.**Centralized MCP administration**: define a set of approved MCP servers and tools which your users may |
| 18 | + use, and prevent users from using their own. |
| 19 | + |
| 20 | +##Setup |
| 21 | + |
| 22 | +**Required**: |
| 23 | + |
| 24 | +1. A**Premium** license |
| 25 | +1. Feature must be[enabled](#activation) |
| 26 | +1. One or more[provider](#providers) API keys must be configured |
| 27 | + |
| 28 | +###Activation |
| 29 | + |
| 30 | +To enable this feature, activate the`aibridge` experiment using an environment variable or a CLI flag. |
| 31 | +Additionally, you will need to enable AI Bridge explicitly: |
| 32 | + |
| 33 | +```sh |
| 34 | +CODER_EXPERIMENTS="aibridge" CODER_AIBRIDGE_ENABLED=true coder server |
| 35 | +# or |
| 36 | +coder server --experiments=aibridge --aibridge-enabled=true |
| 37 | +``` |
| 38 | + |
| 39 | +_If you have other experiments enabled, separate them by commas._ |
| 40 | + |
| 41 | +###Providers |
| 42 | + |
| 43 | +AI Bridge currently supports OpenAI and Anthropic APIs. |
| 44 | + |
| 45 | +**API Key**: |
| 46 | + |
| 47 | +The single key used to authenticate all requests from AI Bridge to OpenAI/Anthropic APIs. |
| 48 | + |
| 49 | +-`CODER_AIBRIDGE_OPENAI_KEY` or`--aibridge-openai-key` |
| 50 | +-`CODER_AIBRIDGE_ANTHROPIC_KEY` or`--aibridge-anthropic-key` |
| 51 | + |
| 52 | +**Base URL**: |
| 53 | + |
| 54 | +The API to which AI Bridge will relay requests. |
| 55 | + |
| 56 | +-`CODER_AIBRIDGE_OPENAI_BASE_URL` or`--aibridge-openai-base-url`, defaults to`https://api.openai.com/v1/` |
| 57 | +-`CODER_AIBRIDGE_ANTHROPIC_BASE_URL` or`--aibridge-anthropic-base-url`, defaults to`https://api.anthropic.com/` |
| 58 | + |
| 59 | +If you're using_[Google Vertex AI](https://cloud.google.com/vertex-ai?hl=en)_,_[AWS Bedrock](https://aws.amazon.com/bedrock/)_, or others, |
| 60 | +you may specify the base URL(s) above to the appropriate API endpoint. |
| 61 | + |
| 62 | +--- |
| 63 | + |
| 64 | + |
| 65 | +>[!NOTE] |
| 66 | +>See[Supported APIs](#supported-apis) section below for a comprehensive list. |
| 67 | +
|
| 68 | +##Collected Data |
| 69 | + |
| 70 | +AI Bridge collects: |
| 71 | + |
| 72 | +- The last`user` prompt of each request |
| 73 | +- All token usage |
| 74 | +- All tool invocations |
| 75 | + |
| 76 | +##Implementation Details |
| 77 | + |
| 78 | +`coderd` runs an in-memory instance ofhttps://github.com/coder/aibridge. In future releases we will support running external instances for higher throughput and complete memory isolation from`coderd`. |
| 79 | + |
| 80 | +<details> |
| 81 | +<summary>See a diagram of how AI Bridge interception works</summary> |
| 82 | + |
| 83 | +```mermaid |
| 84 | +
|
| 85 | +sequenceDiagram |
| 86 | + actor User |
| 87 | + participant Client |
| 88 | + participant Bridge |
| 89 | +
|
| 90 | + User->>Client: Issues prompt |
| 91 | + activate Client |
| 92 | +
|
| 93 | + Note over User, Client: Coder session key used<br>as AI token |
| 94 | + Client-->>Bridge: Sends request |
| 95 | +
|
| 96 | + activate Bridge |
| 97 | + Note over Client, Bridge: Coder session key <br>passed along |
| 98 | +
|
| 99 | + Note over Bridge: Authenticate |
| 100 | + Note over Bridge: Parse request |
| 101 | +
|
| 102 | + alt Rejected |
| 103 | + Bridge-->>Client: Send response |
| 104 | + Client->>User: Display response |
| 105 | + end |
| 106 | +
|
| 107 | + Note over Bridge: If first request, establish <br>connection(s) with MCP server(s)<br>and list tools |
| 108 | +
|
| 109 | + Note over Bridge: Inject MCP tools |
| 110 | +
|
| 111 | + Bridge-->>AIProvider: Send modified request |
| 112 | +
|
| 113 | + activate AIProvider |
| 114 | +
|
| 115 | + AIProvider-->>Bridge: Send response |
| 116 | +
|
| 117 | + Note over Client: Client is unaware of injected<br>tools and invocations,<br>just receives one long response |
| 118 | +
|
| 119 | + alt Has injected tool calls |
| 120 | + loop |
| 121 | + Note over Bridge: Invoke injected tool |
| 122 | + Bridge-->>AIProvider: Send tool result |
| 123 | + AIProvider-->>Bridge: Send response |
| 124 | + end |
| 125 | + end |
| 126 | +
|
| 127 | + deactivate AIProvider |
| 128 | +
|
| 129 | + Bridge-->>Client: Relay response |
| 130 | + deactivate Bridge |
| 131 | +
|
| 132 | + Client->>User: Display response |
| 133 | + deactivate Client |
| 134 | +``` |
| 135 | + |
| 136 | +</details> |
| 137 | + |
| 138 | +##MCP |
| 139 | + |
| 140 | +[Model Context Protocol (MCP)](https://modelcontextprotocol.io/docs/getting-started/intro) is a mechanism for connecting AI applications to external systems. |
| 141 | + |
| 142 | +AI Bridge can connect to MCP servers and inject tools automatically, enabling you to centrally manage the list of tools you wish to grant your users. |
| 143 | + |
| 144 | +>[!NOTE] |
| 145 | +>Only MCP servers which support OAuth2 Authorization are supported currently. In future releases we will support[optional authorization](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization#protocol-requirements). |
| 146 | +>[_Streamable HTTP_](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports#streamable-http) is the only supported transport currently. In future releases we will support the (now deprecated)[_Server-Sent Events_](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports#backwards-compatibility) transport. |
| 147 | +
|
| 148 | +AI Bridge makes use of[External Auth](../admin/external-auth/index.md) applications, as they define OAuth2 connections to upstream services. If your External Auth application hosts a remote MCP server, you can configure AI Bridge to connect to it, retrieve its tools and inject them into requests automatically - all while using each individual user's access token. |
| 149 | + |
| 150 | +For example, GitHub has a[remote MCP server](https://github.com/github/github-mcp-server?tab=readme-ov-file#remote-github-mcp-server) and we can use it as follows. |
| 151 | + |
| 152 | +```bash |
| 153 | +CODER_EXTERNAL_AUTH_0_TYPE=github |
| 154 | +CODER_EXTERNAL_AUTH_0_CLIENT_ID=... |
| 155 | +CODER_EXTERNAL_AUTH_0_CLIENT_SECRET=... |
| 156 | +# Tell AI Bridge where it can find this service's remote MCP server. |
| 157 | +CODER_EXTERNAL_AUTH_0_MCP_URL=https://api.githubcopilot.com/mcp/ |
| 158 | +``` |
| 159 | + |
| 160 | +See the diagram in[Implementation Details](#implementation-details) for more information. |
| 161 | + |
| 162 | +You can also control which tools are injected by using an allow and/or a deny regular expression on the tool names: |
| 163 | + |
| 164 | +```bash |
| 165 | +CODER_EXTERNAL_AUTH_0_MCP_TOOL_ALLOW_REGEX=(.+_gist.*) |
| 166 | +CODER_EXTERNAL_AUTH_0_MCP_TOOL_DENY_REGEX=(create_gist) |
| 167 | +``` |
| 168 | + |
| 169 | +In the above example, all tools containing`_gist` in their name will be allowed, but`create_gist` is denied. |
| 170 | + |
| 171 | +The logic works as follows: |
| 172 | + |
| 173 | +- If neither the allow/deny patterns are defined, all tools will be injected. |
| 174 | +- The deny pattern takes precedence. |
| 175 | +- If only a deny pattern is defined, all tools are injected except those explicitly denied. |
| 176 | + |
| 177 | +In the above example, if you prompted your AI model with "list your available github tools by name", it would reply something like: |
| 178 | + |
| 179 | +>Certainly! Here are the GitHub-related tools that I have available: |
| 180 | +> |
| 181 | +>1.`bmcp_github_update_gist` |
| 182 | +>2.`bmcp_github_list_gists` |
| 183 | +
|
| 184 | +AI Bridge marks automatically injected tools with a prefix`bmcp_` ("bridged MCP"). It also namespaces all tool names by the ID of their associated External Auth application (in this case`github`). |
| 185 | + |
| 186 | +##Tool Injection |
| 187 | + |
| 188 | +If a model decides to invoke a tool and it has a`bmcp_` suffix and AI Bridge has a connection with the related MCP server, it will invoke the tool. The tool result will be passed back to the upstream AI provider, and this will loop until the model has all of its required data. These inner loops are not relayed back to the client; all it seems is the result of this loop. See[Implementation Details](#implementation-details). |
| 189 | + |
| 190 | +In contrast, tools which are defined by the client (i.e. the[`Bash` tool](https://docs.claude.com/en/docs/claude-code/settings#tools-available-to-claude) defined by_Claude Code_) cannot be invoked by AI Bridge, and the tool call from the model will be relayed to the client, after which it will invoke the tool. |
| 191 | + |
| 192 | +If you have the`oauth2` and`mcp-server-http` experiments enabled, Coder's own[internal MCP tools](mcp-server.md) will be injected automatically. |
| 193 | + |
| 194 | +##Troubleshooting |
| 195 | + |
| 196 | +- Too many tools |
| 197 | +- Coder MCP tools not being listed |
| 198 | +- External Auth tools not being injected |
| 199 | + |
| 200 | +##Supported APIs |
| 201 | + |
| 202 | +API support is broken down into two categories: |
| 203 | + |
| 204 | +-**Intercepted**: requests are intercepted, audited, and augmented - full AI Bridge functionality |
| 205 | +-**Passthrough**: requests are proxied directly to the upstream, no auditing or augmentation takes place |
| 206 | + |
| 207 | +Where relevant, both streaming and o |
| 208 | + |
| 209 | +###OpenAI |
| 210 | + |
| 211 | +**Intercepted**: |
| 212 | + |
| 213 | +-[`/v1/chat/completions`](https://platform.openai.com/docs/api-reference/chat/create) |
| 214 | + |
| 215 | +**Passthrough**: |
| 216 | + |
| 217 | +-[`/v1/models(/*)`](https://platform.openai.com/docs/api-reference/models/list) |
| 218 | +-[`/v1/responses`](https://platform.openai.com/docs/api-reference/responses/create)_(Interception support coming in**Beta**)_ |
| 219 | + |
| 220 | +###Anthropic |
| 221 | + |
| 222 | +**Intercepted**: |
| 223 | + |
| 224 | +-[`/v1/messages`](https://docs.claude.com/en/api/messages) |
| 225 | + |
| 226 | +**Passthrough**: |
| 227 | + |
| 228 | +-[`/v1/models(/*)`](https://docs.claude.com/en/api/models-list) |