Claude Code Behind an LLM Gateway: What Breaks and Why

A team I know moved to a gateway on a Friday and spent the following Tuesday trying to work out why their token spend had roughly tripled with no change in how anyone was working. Nothing had broken. Their gateway was silently removing a marker on one block of the request, and every turn was being billed as though the conversation had never been cached.

Gateways are a reasonable and often necessary piece of enterprise infrastructure, and the material written about them concentrates almost entirely on what you gain. This post is the other half: the two-variable distinction that decides who gets billed, the features that stop working when the base URL points anywhere but the Anthropic API, and the caching failure that is expensive precisely because it is invisible.

What a gateway actually buys

The case for one is genuinely strong, and it is worth stating plainly before the caveats. A gateway gives an organisation one place to manage five things:

• Credentials — the provider key stays server-side and developers hold gateway credentials instead, so offboarding revokes one credential rather than hunting for keys.
• Usage tracking — attribute usage by developer or team regardless of which provider served the request.
• Cost controls — enforce budgets and rate limits in one place rather than per machine.
• Audit logging — log every model request for compliance.
• Provider switching — change the provider in gateway configuration without touching a single developer machine.

Four of those five hold whether the upstream is the Anthropic API or a cloud provider. The fifth has a condition: switching providers without reconfiguring machines depends on the gateway exposing a single Anthropic-format endpoint whatever the upstream. A gateway that exposes a provider's own format ties every client configuration to that provider, which quietly removes the benefit people most often cite when choosing one.

Two variables, and only one of them changes who pays

This is the distinction I have seen misread most often, and it decides both billing and usage limits. Setting the base URL alone does not replace a subscription; adding a credential does.

# These two configurations look similar and bill completely
# differently. This is the single most important distinction
# on this page.

# 1. Base URL ONLY.
#    Requests route through the gateway, but a saved claude.ai
#    login stays the ACTIVE CREDENTIAL — so the subscription's
#    usage limits and billing still apply.
ANTHROPIC_BASE_URL=https://gateway.internal.example.com

# 2. Base URL PLUS a gateway credential (or an apiKeyHelper).
#    The credential REPLACES the subscription login for that
#    session. Subscription limits no longer apply, and the
#    traffic is billed per token to whoever owns the credential
#    the gateway forwards.
ANTHROPIC_BASE_URL=https://gateway.internal.example.com
ANTHROPIC_AUTH_TOKEN=<the developer's gateway credential>

# A gateway passing case 1 through to Anthropic must forward the
# OAuth capability in the anthropic-beta header, or the saved
# login stops working.

The rollout is four steps and the last one is the one people skip

The sequence is the same whichever gateway product you choose, and it is short:

1. Deploy the gateway and give it your provider credential so it can authenticate the requests it forwards.
2. Issue each developer their own gateway credential, so usage attributes correctly and offboarding revokes exactly one thing.
3. Distribute the base URL and credential through a managed settings file and your secrets tooling, so developers configure nothing themselves.
4. Have every developer check for the configuration inside Claude Code, so distribution problems surface before anyone depends on the gateway.

What stops working when the base URL moves

Pointing the base URL at a non-first-party host changes several behaviours at once. None of them produce an error that names the gateway as the cause, which is what makes this list worth keeping:

# Features that stop working, or quietly change behaviour,
# when ANTHROPIC_BASE_URL points somewhere that is not
# api.anthropic.com:

Remote Control      unavailable outright. Unset the variable to
                    use it. (Before v2.1.196 it was allowed.)

Tool search         disabled by default, because most proxies do
                    not forward tool_reference blocks. Every MCP
                    tool definition then loads UPFRONT into the
                    system prompt. Set ENABLE_TOOL_SEARCH=true to
                    override — requests FAIL on proxies that
                    cannot handle it.

1M context          Claude Code cannot verify support, so a
                    Sonnet 5 session budgets 200K. Pick the 1M
                    entry in the model picker to get it back.

Plan checks         Claude Code stops checking usage-credit
                    requirements for extended context; the
                    gateway decides whether the request succeeds.

Auto-compaction     An unrecognised model ID compacts at the
                    window Claude Code ASSUMES for that ID.

Cache markers       See below — the expensive one.

The caching failure, which is the expensive one

Claude Code keeps the conversation's own cache breakpoints in place, so a gateway that forwards them still caches your conversation normally. The problem sits on a separate block: Claude Code appends system context mid-conversation, such as file-change notices, and before v2.1.237 it marked that block for caching through gateways too. A gateway that silently removed the marker left the entire conversation billed as uncached input on every single turn.

As of v2.1.237, Claude Code no longer marks that block for caching through a gateway, so it shows up as uncached input on every request — a small, predictable cost rather than a catastrophic one. That is the current, correct behaviour and it is the fix for the old failure. If you are behind a gateway and your numbers look wrong, check your Claude Code version before you start investigating the gateway.

Choosing between a gateway and the alternatives

A gateway is not the only way to get centralised control, and the alternatives have genuinely different trade-offs. The question is which properties you actually need, because each option gives up something different.

Three ways to centralise, and what each costs:

The maintenance nobody budgets for

The trade that matters most is not technical, it is organisational: the gateway becomes infrastructure your organisation operates. Claude Code adds capabilities with every release, and a gateway that does not forward them breaks the corresponding feature. That means the gateway product has to be kept current as Claude Code evolves — not once at rollout, but continuously.

The protocol reference exists precisely for this: it documents the endpoints, the headers to forward and the feature pass-through a gateway needs to get right. Whoever owns the gateway should be reading it against each Claude Code release, because the failure mode is not an outage. It is a feature that quietly stops working for everyone at once, and a support conversation that starts with a developer saying this used to work.

Run a gateway if you need credentials centralised, budgets enforced in one place, or an audit log you own. Go in knowing it is a component you now maintain, that a handful of features will be unavailable behind it, and that its most expensive failure mode is silent. Check your Claude Code version first when the numbers look wrong, watch the cache ratio rather than the invoice, and put someone's name against keeping the gateway current — because the alternative is discovering the gap three releases later.