Providers

A provider is where the agent’s model lives: an endpoint, a credential, and a default model. Configure one or more on the Setup page’s Providers tab; each conversation and each | osai search can pick any of them, and one is the default. This page covers the four kinds, the options that matter for local models, proxies, failover, and how to make the app record what a request costs.

Add a provider

Press Add provider and fill in the form. The fields every kind shares:

A stored key is bound to where it is sent and to what it is used as, so an edit that changes either asks for the key again: the base URL (clearing it included), the kind, the AWS region, turning Insecure TLS on (it skips certificate verification; see below), and for Azure OpenAI the way the key authenticates - switching between an API key and an Entra ID client secret, changing a client secret’s tenant or client ID, or turning managed identity off. Turning managed identity on, or Insecure TLS back off, does not ask for it: neither can send the key anywhere new.

Save, then press Test on the provider’s row: the app runs a real completion through the whole chain and reports the latency and the model that answered. On the free tier you can configure one provider.

Every configured provider is open to everyone who holds use_os_ai_agent: there is no per-provider permission, and no permission on a provider’s stanza narrows it. If some providers are expensive, restrict who holds the capability (see Security).

The four kinds

OpenAI-compatible

Anything that exposes /v1/chat/completions: OpenAI itself, DeepSeek, Groq, xAI, Mistral, OpenRouter, Together, Fireworks, and the local servers - Ollama, LM Studio, vLLM, llama.cpp, sglang. The base URL includes the /v1 segment; the app appends /chat/completions and /models:

The key is sent as a bearer token. Extra headers - OpenRouter’s HTTP-Referer, a corporate proxy’s tenant header - go in the Extra HTTP headers field, one Name: value per line; never a secret.

Anthropic

Claude through the native Messages API. Leave the base URL blank for https://api.anthropic.com; if you set one, do not include /v1 - the app appends /v1/messages itself. The key is sent as x-api-key. Prompt caching is its own control on the form and is on by default for this kind, so every turn of an agentic run re-reads the tool definitions, the system prompt and the earlier conversation at the cached-input rate; it needs no beta flag. Turn it off only for a gateway in front of the Anthropic API that rejects the cache marker. Anthropic beta flags takes a comma-separated anthropic-beta header value, only for features Anthropic documents as requiring one.

Azure OpenAI

Your resource’s OpenAI v1 surface. The base URL is the resource endpoint without a path - https://my-resource.openai.azure.com - and the model field carries the deployment name. Three ways to authenticate:

• API key - the resource’s key in the API key field.
• Entra ID app registration - set the tenant id and client id; the API key field then carries the app registration’s client secret, which the app exchanges for a bearer token and refreshes ahead of expiry.
• Managed identity - switch Keyless (managed identity) on. No secret is stored anywhere; the search head must be an Azure VM or App Service with a managed identity that can reach the resource. Leave the client ID empty to use the host’s system-assigned identity, or set it to a user-assigned identity’s client ID. That is a different Azure object from an app registration, so when you switch over from an app registration, replace its client ID or clear it; the form points this out while the client ID may still be the app registration’s. Not available on Splunk Cloud.

AWS Bedrock

The Converse API, so one provider serves every Converse-capable model family - Claude, Amazon Nova, Llama, Mistral. Set the AWS region; the endpoint is derived from it unless you override the base URL with a VPC endpoint. Authenticate with a Bedrock API key in the API key field (sent as a bearer token), or set an AWS access key ID to sign requests with SigV4, in which case the API key field carries the secret access key; for temporary IAM credentials, add the AWS session token too - they expire, so prefer long-term keys or a Bedrock API key for an always-on search head. No AWS SDK or CLI is needed on the search head.

Prompt caching is off by default for Bedrock, because only some model families accept cache checkpoints and Bedrock rejects the whole request for the others. Turn it on and the app marks a request only when its model is one Bedrock caches - Claude 3.5 Sonnet v2, 3.7 Sonnet, and 4.5 or later (not 4.0 or 4.1), and Amazon Nova - so a conversation or | osai model= that picks another model on the same provider runs uncached instead of failing. An application inference profile whose ARN does not name the model family is taken at your word: turn caching on only if the profile fronts a model that caches.

Local models

Self-hosted models are a first-class case, and four fields on the form exist for them:

• Context window - the model’s context length in tokens. When set, long conversations are trimmed to fit before each call instead of failing at the provider. Load models fills it in automatically where the provider reports it (LM Studio, vLLM, OpenRouter). Strongly recommended for Ollama and LM Studio.
• Reserve output tokens - how much of that window to hold back for the reply when trimming.
• Default max tokens - caps each reply so one runaway answer cannot consume a small model’s whole window.
• Request timeout and Stream idle timeout - the total cap on one call (including the quiet period before the first token, which on a local model is prompt processing), and the maximum silence between stream events once the model has started answering. Raise the first for slow local providers; the second is what cuts a dead connection quickly.

Local and private addresses are supported, including over a Tailscale network. A cloud instance-metadata address is refused, even when a hostname resolves to one; the check runs when the agent connects, not when the provider is saved.

Behind a proxy

Provider requests follow the proxy Splunk itself is configured with - [proxyConfig] in server.conf, which the server/httpsettings/proxysettings REST endpoint also writes - for chats, the background worker, and the Setup page’s Test and Load models. The agent reads it from the search head’s configuration files, so it needs no extra capability.

• https_proxy carries https:// providers, and falls back to http_proxy when unset; http_proxy carries http:// ones.
• Only hosts matching proxy_rules (every host when it is unset) use the proxy, and never hosts matching no_proxy. Entries are host or domain names (a name covers its whole domain), addresses, or CIDR ranges.
• localhost and loopback addresses always go direct, so a model on the search head itself is unaffected. A model elsewhere on your network goes through the proxy unless no_proxy lists it: if your search head already has [proxyConfig] and a provider’s base URL names an Ollama or vLLM server on the LAN or an internal gateway, add that host, its domain or its address range to no_proxy.
• Azure managed identity fetches its token directly from the host’s identity endpoint, never through the proxy.
• Proxy credentials go on the proxy URL (http://user:password@proxy:3128) and are never sent to the provider.
• The metadata-address safeguard (see Local models) still applies: a base URL naming a cloud metadata address, or a name that resolves to one on the search head, is refused before the request reaches the proxy.

A proxy setting the agent cannot use - a SOCKS proxy, or a URL that does not parse - fails the request with a message naming the setting; the request never goes around the proxy. To keep every provider off the proxy, set use_splunkd_proxy = false in the [settings] stanza of os_ai_agent.conf (file only; see Configuration).

Reliability and failover

Transient failures - rate limits, 5xx responses, dropped connections, timeouts - retry with exponential backoff, honoring the provider’s Retry-After. Tune it per provider with Max attempts and Retry backoff. A provider that keeps failing trips a circuit breaker and is skipped for a short cooldown; switch it off per provider if you would rather always try.

Fallback providers names an ordered chain: when this provider stays unavailable after its retry budget, the request is served by the next provider in the chain, using that provider’s own default model, and the response reports who actually answered. Retries and failovers never fire mid-answer: once part of a response has streamed, a failure surfaces instead of restarting the answer behind your back.

A redirect is never followed: if a provider answers with one, the request fails and names where the endpoint was pointing, and nothing - the credential included - is sent to that address. A chat request never legitimately redirects, so this only shows when something between you and the provider is misconfigured. The app’s TLS client applies current Go defaults: a certificate with a SHA-1 signature or an RSA key under 2048 bits is refused. Insecure TLS, on the form for every kind, turns certificate verification off for that one provider - meant for a self-signed corporate proxy during an evaluation. Leave it off in production: with verification off, a stored key goes to whichever certificate answers for the provider’s name, which is why turning it on asks for the key again.

When a provider answers 401 or 403 to its stored API key, the app posts a Splunk bulletin message naming the provider, linked to the Providers tab; it clears after the next request the provider accepts. See Troubleshooting.

Default system prompt

A provider’s Default system prompt is appended beneath the app’s built-in playbook, under an “Operator instructions” heading, on every conversation that uses this provider and has no prompt of its own. Use it for tone, house conventions, and environment context - not to re-teach tool usage. A deployment-wide default lives on the Settings tab; see Configuration.

Prices

Enter Input price and Output price in USD per million tokens - the denominator every vendor publishes - and the app records a cost on every request this provider serves. Add the cache-read and cache-write rates if your provider bills cached prompt tokens differently. The app ships no price list on purpose: published list prices go stale and almost nobody pays them, and the rate you enter is the one that is right for you. For a self-hosted model, enter 0 for both: zero is recorded as $0.00, while blank records “cost unknown”. See Costs and limits.

Where configuration is stored

Providers are [provider:<name>] stanzas in the app’s os_ai_agent.conf, written to local/ by the Setup page; the API key is a reference to Splunk’s encrypted credential store (api_key_ref, which only ever names the provider’s own entry). In a search head cluster, both replicate to every member. A hand-written stanza may carry an inline api_key = ... instead, for local evaluation only: it sits in the file in plain text, readable by anyone who can read the file. On a self-managed search head you can also author a provider by hand - the bundled README/os_ai_agent.conf.spec is the full schema - for example the smallest possible Ollama setup:

1[settings]
2default_provider = local-ollama
3
4[provider:local-ollama]
5kind = openai_compat
6base_url = http://localhost:11434/v1
7default_model = llama3.2:3b