LLM Gateway

Access models across providers with one LangSmith API key while tracing calls and enforcing spend and data-protection policies.

Use one LangSmith API key to call models across configured providers. Switch providers by changing the model ID, while the LLM Gateway traces every model call and applies centralized governance policies.

Beta: The LLM Gateway is in beta.

What the gateway provides

• One key, multiple providers: Developers authenticate with a LangSmith API key instead of storing provider keys locally.
• One request format, multiple models: Use Chat Completions, Messages, or Responses with models across configured providers.
• Built-in observability: Every gateway call appears as a LangSmith trace.
• Central governance: Apply spend limits, rate limits, and data policies.

See what a request looks like

Once an administrator has completed the setup, a gateway call is a standard Chat Completions request authenticated with a workspace-scoped LangSmith API key. This example calls the US gateway on LangSmith Cloud and assumes the workspace has an Anthropic provider secret:

export LANGSMITH_API_KEY="lsv2_..._....cbed3e"

curl https://gateway.smith.langchain.com/v1/chat/completions \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"anthropic/claude-sonnet-4-6","messages":[{"role":"user","content":"Hello!"}]}'

A 200 response confirms that the gateway, your LangSmith API key, permissions, and the selected provider secret are configured correctly. To make this request yourself in cURL, Python, TypeScript, or Deep Agents, follow the quickstart.

Use the standard API

Choose the request format already used by your application. The format does not limit which configured provider you can call.

API format	Endpoint
OpenAI Chat Completions	POST /v1/chat/completions
Anthropic Messages	POST /v1/messages
OpenAI Responses	POST /v1/responses

Set model to a provider-prefixed bring-your-own-key ID such as openai/gpt-5.4-mini, anthropic/claude-opus-5, or azure/<deployment-name>, or use a Gateway Credits model slug such as moonshotai/kimi-k3.

To call a saved model configuration, such as one that points to a custom OpenAI-compatible endpoint, use custom/<name>. The model ID determines the upstream route. When the selected provider uses a different native format, the gateway translates the request and response.

The gateway runs on LangSmith Cloud in every region, on BYOC, and on self-hosted LangSmith. On BYOC and self-hosted, the same paths sit behind the /gateway prefix, such as POST /gateway/v1/chat/completions. For the hostname to use in each case, see Check availability. Self-hosted installations also need an administrator to enable the gateway in the Helm chart.

For base URLs, examples, and translation behavior, see API formats.

Choose how credentials are managed

Option	Upstream credential	Setup and billing
Bring your own provider account	An administrator stores the provider key in workspace Provider Secrets.	The provider bills usage to your provider account.
Gateway Credits	LangChain owns the upstream credential.	No provider secret is required. Invocations are billed to your LangSmith account.

Go further

Quickstart

Make a request with cURL, Python, or TypeScript, then view its trace.

Administrator setup

Enable the gateway, add provider credentials, and grant developer access.

Need to use the gateway with prompts stored in the Prompt Hub? See Prompt Hub with the gateway for how to route Prompt Hub model calls through the gateway using two environment variables.

Need provider-native request and response behavior? Use Direct model access to bypass the standardization layer. This is an advanced alternative to the standard API.

For further questions, contact LangChain support.

---

Connect these docs to your agent of choice via MCP for real-time answers.

Edit this page on GitHub or file an issue.