The app charges nothing per request; what a conversation costs is whatever your provider bills for the tokens it served. OS AI Agent records those tokens on every request, turns them into dollars when you tell it your rates, shows the totals to administrators and to each user, and can refuse work past a budget you set.
What is recorded
Every finished chat request writes a usage record: the user, provider, model, conversation, and job; input, output, cache-read, cache-write, and reasoning tokens; a total computed under the serving provider’s own accounting convention (OpenAI-compatible and Azure report cached tokens inside the input count, Anthropic and Bedrock alongside it); the number of tool calls; and, if the provider is priced, the cost. The rates in force are frozen onto each record, so repricing a provider later never rewrites past spend.
The Token Usage dashboard aggregates the records for administrators - spend by user and day, the most expensive conversations, prompt-cache hit rate and the dollars it saved, requests served by more than one provider - and the chat window shows each user their own last 24 hours. See Dashboards.
| osai runs write the same records: every form of the command submits its run through the handler the chat page uses, so it is counted like a chat turn, and its turn event in the activity trail (index=_internal sourcetype=os_ai_agent:audit event=turn surface=spl) carries tokens and cost as well.
Pricing a provider
On the Providers tab, enter a provider’s Input price and Output price in USD per million tokens - the denominator every vendor publishes - and optionally the cache-read and cache-write rates. From then on every request that provider serves records a cost.
The app ships no price list, deliberately. Published list prices go stale between releases and almost nobody pays them: enterprise agreements, committed-use discounts, and per-region Azure and Bedrock rates all differ from the public page. The rate you enter is the one that is right for you. For a self-hosted model, enter 0 for both: zero is recorded as $0.00, while blank records “cost unknown”, which is a different claim. Requests served by an unpriced provider record tokens and no cost, and the Token Usage dashboard reports how many records are unpriced so the gap is visible.
Spend limits
On the Settings tab, four guardrails. The concurrency cap is on out of the box; the three daily budgets are off until you set them:
| Limit | What it bounds |
|---|---|
| Concurrent chat runs per user | How many runs one user may have unfinished at once - queued, executing, or paused on an approval card (default 10). Stops one runaway client from starving everyone else; a run waiting for approval holds its slot until it is approved, denied, or cancelled. |
| Daily tokens per user | Total tokens one user may consume in the last 24 hours. |
| Daily cost per user | Recorded spend one user may reach in the last 24 hours. |
| Daily cost, whole deployment | Recorded spend across every user in the last 24 hours. |
A user at or above a budget has new runs refused with a message naming the limit, and the refusal is recorded in the activity trail, so “who was turned away, and why” is a search. Three properties of the enforcement are worth knowing:
- The window rolls. Budgets are enforced over the last 24 hours, not a calendar day - a calendar day needs a timezone to be meaningful and resets at a moment users learn to wait for.
- They are guardrails, not exact quotas. A request’s cost is not known until it finishes, so spend already under way is invisible to the next check; the concurrent-run cap bounds how far that can go.
- Cost budgets count priced providers only. Spend on an unpriced provider is invisible to a cost cap by construction - price your providers first.
Budgets apply to every surface that starts a run - Chat, the REST API, alert investigations, and | osai, whose runs are chat jobs: a budget refuses one at submission, the search returns no rows, and the job inspector names the limit. Splunk’s own scheduling and concurrency limits still bound how often a scheduled search fires. If the usage ledger cannot be read, budgets fail open: the run proceeds and the reason is logged, because a cost guardrail that stops the product when KV Store is briefly unavailable would be worse than the overspend it prevents.
Keeping cost down
- Set the context window on local providers, so long conversations are trimmed rather than failing at the provider.
- Leave tool-result compaction on (the default): older bulky tool results go to the model as short placeholders.
- Use prompt caching where your provider offers it, and enter the cache rates so the savings show up on the dashboard.
- Pin cheap models for routine conversations by giving them their own provider entry and choosing it in the chat header.