Providers

A provider is where the agent’s model lives: an endpoint, a credential, and a default model. Configure one or more on the Setup page’s Providers tab; each conversation and each | osai search can pick any of them, and one is the default. This page covers the four kinds, the options that matter for local models, failover, and how to make the app record what a request costs.

Add a provider

Press Add provider and fill in the form. The fields every kind shares:

Save, then press Test on the provider’s row: the app runs a real completion through the whole chain and reports the latency and the model that answered. On the free tier you can configure one provider.

The four kinds

OpenAI-compatible

Anything that exposes /v1/chat/completions: OpenAI itself, DeepSeek, Groq, xAI, Mistral, OpenRouter, Together, Fireworks, and the local servers - Ollama, LM Studio, vLLM, llama.cpp, sglang. The base URL includes the /v1 segment; the app appends /chat/completions and /models:

The key is sent as a bearer token. Extra headers - OpenRouter’s HTTP-Referer, a corporate proxy’s tenant header - go in the Extra HTTP headers field, one Name: value per line; never a secret.

Anthropic

Claude through the native Messages API. Leave the base URL blank for https://api.anthropic.com; if you set one, do not include /v1 - the app appends /v1/messages itself. The key is sent as x-api-key. Prompt caching is its own control on the form and is on by default for this kind; it needs no beta flag. Anthropic beta flags takes a comma-separated anthropic-beta header value, only for features Anthropic documents as requiring one.

Azure OpenAI

Your resource’s OpenAI v1 surface. The base URL is the resource endpoint without a path - https://my-resource.openai.azure.com - and the model field carries the deployment name. Three ways to authenticate:

• API key - the resource’s key in the API key field.
• Entra ID app registration - set the tenant id and client id; the API key field then carries the app registration’s client secret, which the app exchanges for a bearer token and refreshes ahead of expiry.
• Managed identity - switch Keyless on. No secret is stored anywhere; the search head must be an Azure VM or App Service with a managed identity that can reach the resource. Not available on Splunk Cloud.

AWS Bedrock

The Converse API, so one provider serves every Converse-capable model family - Claude, Amazon Nova, Llama, Mistral. Set the AWS region; the endpoint is derived from it unless you override the base URL with a VPC endpoint. Authenticate with a Bedrock API key in the API key field (sent as a bearer token), or set an AWS access key ID to sign requests with SigV4, in which case the API key field carries the secret access key. No AWS SDK or CLI is needed on the search head.

Local models

Self-hosted models are a first-class case, and four fields on the form exist for them:

• Context window - the model’s context length in tokens. When set, long conversations are trimmed to fit before each call instead of failing at the provider. Load models fills it in automatically where the provider reports it (LM Studio, vLLM, OpenRouter). Strongly recommended for Ollama and LM Studio.
• Reserve output tokens - how much of that window to hold back for the reply when trimming.
• Default max tokens - caps each reply so one runaway answer cannot consume a small model’s whole window.
• Request timeout and Stream idle timeout - the total cap on one call (including the quiet period before the first token, which on a local model is prompt processing), and the maximum silence between stream events once the model has started answering. Raise the first for slow local providers; the second is what cuts a dead connection quickly.

Local and private addresses are supported, including over a Tailscale network. A cloud instance-metadata address is refused, even when a hostname resolves to one; the check runs when the agent connects, not when the provider is saved.

Reliability and failover

Transient failures - rate limits, 5xx responses, dropped connections, timeouts - retry with exponential backoff, honoring the provider’s Retry-After. Tune it per provider with Max attempts and Retry backoff. A provider that keeps failing trips a circuit breaker and is skipped for a short cooldown; switch it off per provider if you would rather always try.

Fallback providers names an ordered chain: when this provider stays unavailable after its retry budget, the request is served by the next provider in the chain, using that provider’s own default model, and the response reports who actually answered. Retries and failovers never fire mid-answer: once part of a response has streamed, a failure surfaces instead of restarting the answer behind your back.

A redirect is never followed: if a provider answers with one, the request fails and names where the endpoint was pointing, and nothing - the credential included - is sent to that address. A chat request never legitimately redirects, so this only shows when something between you and the provider is misconfigured. The app’s TLS client applies current Go defaults: a certificate with a SHA-1 signature or an RSA key under 2048 bits is refused.

Default system prompt

A provider’s Default system prompt is appended beneath the app’s built-in playbook, under an “Operator instructions” heading, on every conversation that uses this provider and has no prompt of its own. Use it for tone, house conventions, and environment context - not to re-teach tool usage. A deployment-wide default lives on the Settings tab; see Configuration.

Prices

Enter Input price and Output price in USD per million tokens - the denominator every vendor publishes - and the app records a cost on every request this provider serves. Add the cache-read and cache-write rates if your provider bills cached prompt tokens differently. The app ships no price list on purpose: published list prices go stale and almost nobody pays them, and the rate you enter is the one that is right for you. For a self-hosted model, enter 0 for both: zero is recorded as $0.00, while blank records “cost unknown”. See Costs and limits.

Where configuration is stored

Providers are [provider:<name>] stanzas in the app’s os_ai_agent.conf, written to local/ by the Setup page; the API key is a reference to Splunk’s encrypted credential store. In a search head cluster, both replicate to every member. On a self-managed search head you can also author a provider by hand - the bundled README/os_ai_agent.conf.spec is the full schema - for example the smallest possible Ollama setup:

1[settings]
2default_provider = local-ollama
3
4[provider:local-ollama]
5kind = openai_compat
6base_url = http://localhost:11434/v1
7default_model = llama3.2:3b