Switch to OptScale AI#

Use OptScale AI to route existing LLM application requests through it without rewriting application logic. In most cases, switching to OptScale AI Gateway requires updating only the connection values used by the existing SDK or HTTP client.

This guide focuses on code and configuration changes after your organization is set up on the platform. For administrator setup (organization, providers, access), complete First Steps first.

Migration flow: your application sends OpenAI-compatible requests through OptScale AI Gateway to a registered AI provider

Capabilities unlocked after switch#

Moving traffic to OptScale AI Gateway adds platform-level controls that a standalone proxy or direct vendor integration typically does not provide:

Unified provider catalog — One OpenAI-compatible entry point for OpenAI, Anthropic, Google, Ollama, and other registered vendors.
Virtual keys and access control — Organization-scoped credentials with allowed providers per user.
Policies and guardrails — Request filtering, PII redaction, safety checks, and timeouts applied before and after provider calls. See Policies + Guardrails.
Routing rules — Weighted targets and fallbacks managed in Admin UI instead of in application code.
MCP servers — Approved tools for Chat and agent workflows. See MCP Servers.
FinOps and observability — Per-request Traces, token usage, cost, and Dashboards for the same traffic your applications send through the gateway.

External tools and custom applications share this path—see AI request flow. For IDE and agent clients, see Connect OpenCode, Connect Cursor, and Connect Claude.

When to switch#

Consider routing application traffic through OptScale AI Gateway when:

Teams use multiple vendor APIs and want one governed entry point.
Raw provider keys in repos or environment files are a security or rotation risk.
You need organization-wide budgets, usage visibility, or audit trails per request.
Compliance requires infrastructure-level guardrails on prompts and responses.
Agent or IDE clients should use the same providers, policies, and observability as production APIs—see Connect OpenCode, Connect Cursor, and Connect Claude.

Before you begin#

Confirm the following platform prerequisites are in place:

An organization exists and is selected in Chat or Admin UI.
At least one Active provider is registered under Admin UI → Providers.
Users who send API traffic have allowed providers assigned in AI Access.

Copy connection values from Chat → Connect this model to external tools → OPENAI-COMPATIBLE tab so you do not construct URLs manually. See Get connection values from Chat.

Update Connection Settings#

For most OpenAI-compatible clients, migration requires only updating the connection values.

Replace:

# Direct OpenAI
client = openai.OpenAI(
    api_key="sk-...",
    base_url="https://api.openai.com/v1"
)

with:

# OptScale AI Gateway
client = openai.OpenAI(
    api_key="<optscale_ai_api_key>",
    base_url="<optscale_ai_base_url>"
)

Where:

<optscale_ai_api_key> — the API key value
<optscale_ai_base_url> — the Base URL value

Both values and model name are available in Chat → Connect this model to external tools → OPENAI-COMPATIBLE tab. See Get connection values from Chat. Use the Model name as the value of the model parameter:

response = client.chat.completions.create(
    model="<optscale_ai_model_name>",
    messages=[
        {"role": "user", "content": "Hello!"}
    ]
)

OpenAI-Compatible Example#

Install the OpenAI Python SDK:

pip install openai

Connect to OptScale AI Gateway:

from openai import OpenAI

client = OpenAI(
    api_key="<optscale_ai_api_key>",
    base_url="<optscale_ai_base_url>"
)

response = client.chat.completions.create(
    model="<optscale_ai_model_name>",
    messages=[
        {"role": "user", "content": "Hello!"}
    ]
)

print(response.choices[0].message.content)

In Chat, open Connect this model to external tools and go to the CODE EXAMPLES tab. There you can find an example with <optscale_ai_api_key>, <optscale_ai_base_url>, and <optscale_ai_model_name> already replaced with actual values.

Anthropic-Compatible Example#

Install the Anthropic Python SDK:

pip install anthropic

Connect to OptScale AI Gateway:

from anthropic import Anthropic

client = Anthropic(
    api_key="<optscale_ai_api_key>",
    base_url="<optscale_ai_base_url>"
)

response = client.messages.create(
    model="<optscale_ai_model_name>",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello!"}
    ]
)

print(response.content[0].text)

In Chat, open Connect this model to external tools and go to the CODE EXAMPLES tab. There you can find an example with <optscale_ai_api_key>, <optscale_ai_base_url>, and <optscale_ai_model_name> already replaced with actual values.

Other clients and frameworks#

Many libraries accept a custom base URL and API key:

LangChain — Pass base_url and api_key to ChatOpenAI, or set OPENAI_API_BASE and OPENAI_API_KEY to the OptScale AI values from Chat.
LiteLLM — Set api_base to the OptScale AI Base URL and api_key to your virtual key; use the Model name from Chat as the model identifier.
HTTP clients — Send POST requests to {base_url}/chat/completions with Authorization: Bearer <optscale_ai_api_key> and a JSON body that includes model and messages.

Verify the Connection#

After updating the connection values:

Send a test request from the application.
Check that the response is returned from the selected model.
Open OptScale AI monitoring pages and verify that the request appears in Traces and Usage and Cost.

Troubleshooting#

Table 1: Common OptScale AI Gateway migration issues — authentication, base URL, model name, streaming, and TLS
Issue	Possible cause	Resolution
Authentication error	The API key is incorrect or expired	Copy a new API key from Connect model to external tools
Connection error	The Base URL is incorrect	Check that the Base URL was copied exactly as shown in Connect model to external tools
Model not found	The model name is incorrect	Use the model name from Connect model to external tools
Tool calls fail	The selected model does not support tool calls	Select a model that supports tool execution
Streaming does not work	Streaming is not supported by the selected model or integration	Check model capabilities and integration settings
TLS error	The gateway URL does not use a trusted HTTPS certificate	Configure HTTPS with a trusted certificate

Notes and Limitations#

OptScale AI Gateway must be available over HTTPS with a trusted certificate.
The selected model must support the features used by the application, such as streaming or tool calls.

First Steps — Organization, provider, and access setup.
External Tools — Connection values from Chat and IDE integration guides.
Connect OpenCode — Route OpenCode agent chats through the gateway.
Connect Cursor — Point Cursor IDE at the OpenAI-compatible proxy.
Connect Claude — Configure Claude Code and Claude Desktop.
Architecture Overview — AI request flow — How requests move through policies and providers.
Core Services — AI Access — Users, virtual keys, and allowed providers.
Core Services — Providers — Provider connections and routing rules.
Core Services — Traces — Per-request debugging after cutover.
Core Services — Usage and Cost — Token usage and spend after cutover.