Providers#

Open Providers to connect external AI services used for Chat and API requests. From this page, you can register provider connections, enable available models, configure per-provider limits, and manage how requests are routed between providers. For advanced routing scenarios, see Routing Rules and Routing Best Practices.

Use this page to:

• Add or update a provider connection.
• Enable models and assign provider access on the add form or in AI Access — Allowed providers so they are available to users.
• Configure budgets, token limits, and request limits for individual providers.
• Troubleshoot missing providers or models before configuring routing.

The page contains two tabs:

• Providers — Manage provider connections, available models, and per-provider limits.
• Routing Rules — Configure request matching, routing targets, priorities, weighted traffic distribution, and failover behavior.

After configuring a provider, verify that requests are routed as expected on Traces.

Providers overview#

What are Providers?#

Providers are connections between the platform and external AI services. A provider stores the information required to communicate with an AI service, such as authentication credentials, connection settings, and the models that the service exposes.

Each provider represents a single connection to a specific AI service or deployment. For example, you can configure providers for OpenAI, Azure OpenAI, Anthropic, Google Vertex AI, Amazon Bedrock, Ollama, or other supported APIs.

After a provider is configured, you can:

• enable or disable the models available through the provider;
• control which users, teams, or agents can use it through AI Access;
• apply budgets and rate limits;
• include the provider in routing rules that automatically direct requests.

Multiple providers can be configured within the same organization. This allows you to combine different AI services, separate production and development environments, use providers in different regions, or route requests based on cost, performance, availability, or other criteria. See When you need one provider vs several.

Provider connection vs models#

A provider connection registers how OptScale AI reaches a vendor (API type, key, base URL). After you save, the platform discovers models for that connection on the provider detail page.

Saving the provider form alone does not expose models in Chat. Complete the provider workflow: enable models, then grant access on the add form or in AI Access.

When you need one provider vs several#

One provider is enough for a single vendor endpoint (for example one OpenAI account or one Ollama host).

Add additional provider rows when you need:

• A second vendor (for example OpenAI and Anthropic).
• Separate credentials or base URLs for production vs development.
• A backup connection for routing rule fallbacks.

Use a unique Name per row even when the vendor is the same.

Provider workflow#

Use this workflow to take a provider from registration to live traffic. Procedural details are covered in Provider lifecycle. Routing behavior is explained in How routing rules work.

1. Add provider — Create the provider connection with credentials and endpoint settings on the Providers tab. Wait until the provider status becomes Active. See Add provider.

2. Enable models — On the provider detail page, enable the models that the organization can use. Disabled models are unavailable in Chat, routing rules, and API requests. Enabling a model does not grant user access. See Enable or disable a model.

3. Grant access — During provider creation, you can assign the provider to Teams, Employees, or Agents, or enable Assign to new users automatically. Otherwise, assign the provider in AI Access — Allowed providers for each user who needs it. A request can use a provider only when the provider is Active, the model is enabled, and the provider is assigned to the requesting user.

4. Optional: provider limits and routing rules — Configure provider limits if you need per-connection quotas. Add routing rules for failover, weighted targets, or scoped traffic steering when a single provider selection is not sufficient. Routing rules do not bypass AI Access or other access controls. See When you need a routing rule.

5. Verify — Verify the configuration end to end by following Verify provider. Test Chat or API requests, then optionally inspect Traces to confirm that requests are routed as expected.

Providers page#

Use the Providers tab to register and manage vendor endpoints. Use Routing Rules when traffic must prefer a target, fall back on failure, or split across models—see When you need a routing rule.

Providers tab#

Use the Providers tab to create, review, and manage provider connections. The table helps you identify connections, verify Active status, manage usage limits, and edit or delete providers.

• To register a connection, see Add provider.
• Click a provider Name to open the provider detail page (summary, limits, and available models).

Routing Rules tab#

The Routing Rules tab lives in the same Admin area as Providers. Use it to review and manage rules that select provider/model destinations for matching Chat and API traffic.

For concepts, procedures, and the detail page, see Routing Rules. For scenario examples, see Routing best practices.

Provider lifecycle#

Before you begin#

Complete these checks before you open the add form:

1. An organization is selected in the header.
2. You have permission to manage providers for that organization.

Add provider#

1. In the Admin UI, open Providers.
2. On the Providers tab, click + ADD.

3. Complete the provider form:

   • Name — A unique name for this provider. Use different names for multiple configurations of the same vendor (for example, OpenAI – Production and OpenAI – Development).
   • API type — Select the AI provider integration that matches the service you want to connect. When you select an API type, OptScale AI automatically populates the Base URL field with the provider's default endpoint. The sidebar also displays Where to get your API key, linking to provider-specific documentation that explains how to obtain the required API key. See Supported AI providers for the complete list of supported integrations.
   • API key — The credential used to authenticate with the provider. Leave this field empty only if the endpoint does not require authentication.
   • Base URL — The provider endpoint URL used for outbound requests (for example, https://api.your-provider.com).
   • Description — Optional notes for administrators. Markdown is supported.
   • Insecure — Disable SSL certificate verification for this provider. Enable this option only for trusted internal or test endpoints.
   • Assign to new users automatically — Automatically adds this provider to Allowed providers for employees who join the organization after the provider is saved.
   • Teams, Employees, Agents — Optional. Grant access immediately by assigning the provider to one or more teams, employees, or agents. Alternatively, configure access later in AI Access — Allowed providers.

4. Click SAVE and wait until the provider status becomes Active. If validation fails, see Troubleshooting.

5. Optional: Repeat these steps to add additional providers for different vendors or environments. See When you need one provider vs several.

After you save#

Registration is not complete until models are enabled and access is granted:

1. Click the provider Name to open the detail page.
2. In Available Models, enable the models users should access.
3. If you did not assign Teams, Employees, or Agents on the add form, open AI Access and configure allowed providers for each user who needs this connection.
4. Optional: add routing rules for failover or load balancing across multiple connections.
5. Verify with a Chat message or API request.

Verify provider#

Verify that the provider is configured correctly and can process requests end to end:

1. Navigate to Providers menu → Providers tab. In the providers table, verify that the provider status is Active.

2. Navigate to Providers menu → Providers tab, then click the provider name. Verify that the required models appear under Available Models and that Enabled is turned on for each model users should be able to access.

3. Navigate to AI Access menu. Find the test user and review the Allowed providers column. Verify that the provider is listed for the user, or that the user has access through Teams, Employees, or Agents configured when the provider was created.

4. Ask the test user to sign in to Chat using their credentials. Open the Chat URL for your deployment. Verify that the user can select the provider and one of its enabled models, then successfully complete a test request.

5. Navigate to Analytics menu → Traces. Open the test request and verify that the expected Provider and Model appear in Request details.

For an introduction to the initial setup, see First Steps — Add your first provider.

Troubleshooting#

Provider does not become Active

• Verify the API key is valid and has vendor-side permissions for the models you plan to use.
• Verify the Base URL matches vendor documentation (scheme, host, port, and path).
• Confirm outbound network access from OptScale AI to the vendor (corporate proxy, firewall, DNS).
• For self-hosted endpoints, try Insecure only when using HTTP or a private CA on a trusted network.
• Check clock skew and proxy TLS interception on restricted networks.

No models on the detail page

• Click REFRESH on the provider detail page after save.
• Confirm the vendor account exposes models at the configured Base URL.
• Re-save the provider after correcting API type or Base URL.

Provider Active but missing in Chat

• Enable models on the provider detail page.
• Assign the provider on the add form or in Allowed providers for that user—organization registration alone is not enough.
• Confirm the user has the correct organization selected.

Models enabled but request fails

• Inspect the failure in Traces.
• Check provider limits for exhausted budget, token, or request quotas.
• Confirm policies and guardrails are not blocking the traffic.

Provider details#

Provider detail page#

Click a provider in the Providers table to open the provider detail page.

The Summary section contains provider metadata, including the provider name, API type, status, base URL, and description. EDIT (to modify or delete the provider) and REFRESH (to reload page data) actions are available.

The Limits panel displays per-provider budget, token, and request quotas with reset intervals, current usage, and last/next reset dates. Click the pencil icon next to the Limits caption to edit limits. See Configure provider limits.

The Available Models section lists models discovered for the provider. The table displays each model Name, Created at, and an Enabled toggle. Use Search to filter the model list by name. To turn models on or off, see Enable or disable a model.

Enable or disable models#

Use the Enabled column in Available Models to control which models from a provider are offered in Chat, routing rules, and API requests.

1. Open Providers, select the Providers tab, and click a provider name to open its detail page.

2. In the Available Models section, locate the model (use Search if needed).

3. Turn the model on or off using one of the following options:

• Individual model — Use the Enabled toggle in the model row.
• All models — Use the Enabled toggle in the column header to enable or disable every model in the table at once.

Disabled models remain listed on the provider detail page but are not selectable elsewhere in the platform. Users can access enabled models only when the provider is also assigned through Allowed providers in AI Access.

Configure provider limits#

Per-provider Limits control budget spend, token volume, and API request count for traffic routed through a provider connection. Limits can be edited from the Providers tab or the provider detail page.

1. Open Providers and select the Providers tab.

2. Start editing using either option:

• Provider detail page — Click a provider Name to open its detail page, then click the pencil icon next to the Limits caption.

• Providers table — In the Limits column for the provider row, click the pencil icon.

3. In the Edit limits form, configure any combination of the following limits:

• Budget — Set the Budget limit and Budget reset duration.
• Tokens — Set the Token limit and Token reset duration.
• Requests — Set the Request limit and Request reset duration.

4. Click SAVE to apply the limits, or CANCEL to discard your changes.

5. Optional: click CLEAR LIMITS to remove all configured quotas for the provider.

Edit provider#

1. Open the provider for editing using one of the following options:

• On the Providers tab, click Edit in the provider row.

• On the provider detail page, click EDIT.

2. Review the pre-populated configuration and update the editable fields as needed.

3. Click SAVE to apply the changes.

Optional actions:

• Click CANCEL to return without saving changes.

• Use DELETE to remove the provider. See Delete provider for step-by-step instructions.

Changes can affect which models are available in Chat after the provider is refreshed. To make the provider available to users, verify that it is assigned through Allowed providers in AI Access.

Delete provider#

1. Open Providers and select the Providers tab.

2. Remove the provider using one of the following options:

• Use the Delete row action on the provider in the table on the Providers tab, or

• Click the provider name to open the detail page, click EDIT, then click DELETE on the edit form.

3. Confirm the deletion when prompted.

The provider is removed from the organization and is no longer available for Chat, routing rules, or AI Access assignments.