Configuration

Providers, the license, and the query templates each have their own tab on the Setup page. The situational guidance notes are on the Guidance tab, and everything else an operator tunes is on the Settings tab: the chat defaults, the spend limits, how long the app keeps what it stores, the activity trail, and the log level. All of it lives in os_ai_agent.conf.

A field left blank keeps the built-in default, and the help under each field says what that default is. The search command and the REST handlers read a change on their next request; the background chat worker reads it at the start of its next pass - within seconds when it is idle, and once the pass in progress ends when it is busy. On Splunk Cloud the tab is the only way to set these values; on a self-managed search head you can also edit the file:

$SPLUNK_HOME/etc/apps/os_ai_agent/local/os_ai_agent.conf

An app upgrade overwrites default/, so a value set there is lost on the next release; local/ always wins and survives upgrades, and the Setup page writes local/ for you. The bundled README/os_ai_agent.conf.spec is the exhaustive schema.

Chat defaults

The [chat] stanza, plus the global provider timeout in [settings].

The approval mode, the system prompt, and the tools are a new conversation’s starting point. They are copied into a conversation when it is created - from the chat page, the REST API, or the alert action - wherever the request names no value of its own. Changing one affects conversations started afterwards; an existing conversation keeps what it was created with, and its own approval mode stays the user’s to change. A complete import is the one create that does not take them: it reproduces a conversation, so the archive’s settings are kept exactly, empty ones included. A REST caller asks for the same with apply_defaults: false on the create.

1[chat]
2default_approval_mode = approve_all
3default_system_prompt = Prefer index=app_prod for application logs. Answer in British English.
4default_tools = splunk_search, profile_data, list_indexes, verify_spl, get_spl_command, current_time

Spend limits

The [limits] stanza. Every budget is off by default and enforced over a rolling 24-hour window, not a calendar day. Cost budgets count only providers you have priced. See Costs and limits for what is counted and what is not.

Retention

The [retention] stanza, which only a full Splunk administrator (admin_all_objects) may change - the app’s own administrator role sees it read-only. Chat state lives in KV Store - conversations, messages, finished job records, usage rows, feedback - and KV Store never ages anything out on its own. These settings, in days, bound it; the chat worker applies the cutoffs about once an hour, and on a search head cluster exactly one member sweeps each hour. 0 keeps forever.

The Health dashboard shows every collection’s size next to these settings.

Activity trail

The [audit] stanza, which only a full Splunk administrator (admin_all_objects) may change - the app’s own administrator role sees it read-only. The trail is one JSON event per agent turn, per tool call, and per refusal, written to $SPLUNK_HOME/var/log/splunk/os_ai_agent_audit.log, which Splunk’s own default monitor indexes into _internal as sourcetype=os_ai_agent:audit. There is no index to provision. It records what nothing else can: which conversation changed a dashboard, which tool call made it, whether a person approved it or a policy did, and what the agent was refused permission to do. The Activity Trail dashboard reads it.

Logging

The [logging] stanza.

Situational guidance

The Guidance tab, which writes [guidance:<name>] stanzas. A note is a short piece of guidance the agent appends to its system prompt when a request touches what the note covers, selected by whole-word match against the user’s message - so a note that applies to a tenth of requests costs a tenth of what it would as a permanent prompt. Nine ship with the app: alert-hygiene, dashboard-tokens, field-extraction, metric-indexes, null-handling, search-performance, subsearch-limits, time-bounds, and tstats-and-acceleration. All nine are about SPL and Splunk itself; none describes a particular product’s data, which is what your own notes are for.

The tab lists every note, shipped and your own, with the words that bring it into a request. From there you can:

• Add note - your own house rule: a name, a title, the “applies when” words, and the note text in Markdown. Turn on Every request for a rule that genuinely applies to all work; it is then paid for on every provider call.
• Customize a shipped note. The form opens on the shipped text, and what you save replaces it. Restore shipped brings the original back.
• Turn off a note, shipped or your own, without losing its text, and Turn on to bring it back.
• Delete a note of your own.

The list is the agent’s own view, not a reading of the file. A note the agent will not use says so, and why: it has no title or no text, it has no “applies when” words and is not marked for every request, or it is larger on its own than the guidance budget for one turn, which the agent would otherwise skip silently on every request. If a customized shipped note is refused, the row says the shipped text is still the one in effect; a customized note that is only too large has still replaced the shipped one, so the row says that neither is sent. Disable situational guidance and the guidance budget are on the Settings tab.

In the file, a note is a stanza with a title, the words that put it in scope, and a body in Markdown; a stanza named after a shipped note replaces it, and disabled = true on such a stanza removes it:

1[guidance:payments-index]
2title = Payments application logs live in index=payments, not index=app
3match = payments, checkout, cart
4body = The payments team's services log to index=payments with sourcetype=payments:api. \
5  Latency is in the field duration_ms. Never search index=app for them.
6
7[guidance:alert-hygiene]
8disabled = true

Multi-line bodies continue a line with a trailing backslash. Set always = true on a note that genuinely applies to every request - it is then paid for on every provider call. A note’s name is lowercase letters, digits, -, and _; the agent compares names without regard to letter case, so a hand-written [guidance:Payments-Index] is the note payments-index on the tab.

Connection to splunkd

The [splunk] stanza governs how the app reaches the local splunkd REST API for credential lookups and the agent’s own tools - not how it reaches your provider. The defaults match a standard install and pin the connection to splunkd’s own certificate on loopback; you will not normally change them. README/os_ai_agent.conf.spec documents request_timeout, tls_ca_file, tls_cert_file, tls_server_name, tls_server_name_from_cert, and tls_insecure in full.

The chat worker

The background input that runs chat jobs is os_ai_agent_worker://default in the app’s inputs.conf, enabled with a 60-second interval. It does no external egress until a job exists. Manage it like any modular input under Settings - Data inputs; disabling it stops asynchronous chat and the retention sweep, and the Health dashboard and health endpoint say so.