Kubernetes Search

Configuration

Most of Kubernetes Search is configured from the Setup tab - clusters, credentials, and whether the cache is on. The settings on this page are the rest: runtime values you rarely need to touch, but that are there when you want to tune request timeouts, fan-out concurrency, cache size, or log verbosity.

They live in k8s_search.conf. Edit the copy under local/, never the shipped default/:

$SPLUNK_HOME/etc/apps/os_k8s_search/local/k8s_search.conf

An app upgrade overwrites default/, so a value set there is lost on the next release; local/ always wins and survives upgrades. Set only the keys you want to change - anything you leave out keeps its default. Each search reads the file when it runs, so a change takes effect on your next search.

The bundled README/k8s_search.conf.spec is the exhaustive schema; this page covers the settings worth tuning in practice.

Kubernetes API requests

[kubernetes] controls how the commands talk to your cluster API servers.

SettingDefaultWhat it does
api_request_timeout30sCaps a single non-streaming API call (discover, list, get) from | k8s, | k8sevents, and | k8sdescribe. A wall-clock bound from the search side, so a hung connection can’t leave a search stuck.
logs_api_request_timeout30sThe same bound for | k8slogs. A value of 0 means “use the default”, not “unbounded”. Raise it only for a large tail= or since= read.
fan_out_concurrency8How many clusters are queried at once during a context=* fan-out. The per-search concurrency= argument overrides it. Raise it for very wide fleets; lower it to keep CPU off a small search head.
max_log_line_bytes1048576The largest single container log line | k8slogs returns, in bytes (1 MiB). Raise it if you see a “token too long” error in the search log - usually a one-line JVM stack trace or an init-container bootstrap.
local/k8s_search.conf ini
1[kubernetes]
2fan_out_concurrency = 16
3api_request_timeout = 1m

Cache

[cache] tunes the on-disk result cache. The defaults suit most installs; see Concepts - caching for how the cache behaves and the per-search cache= override.

SettingDefaultWhat it does
enabledtrueMaster on/off switch. false runs every search uncached. The Setup tab toggles this too.
dir$SPLUNK_HOME/var/run/os_k8s_search/cacheWhere cached responses live on the search head.
discovery_ttl10mHow long the cluster’s API discovery (its groups and kinds) is cached. Discovery is schema, not state, so a long TTL is safe.
list_default_ttl30sDefault freshness for list queries (| k8s kind=pods, | k8sevents) when a search doesn’t set cache=.
get_default_ttl10sDefault freshness for single-object gets (| k8s kind=pods/name, | k8sdescribe).
max_size_bytes268435456Soft cap on total cache size, in bytes (256 MiB). The oldest entries are evicted when it is exceeded.
max_entries4096Soft cap on the number of cached entries, enforced the same way.

To flush the cache immediately - for example right after rotating a credential, rather than waiting out the TTL - use the Clear cache button on the Setup page. On a Splunk Cloud search head where var/run/ isn’t writable, set enabled = false; the app also falls back to running uncached on its own if it can’t write.

Logging

[logging] controls the search-command log, the per-search log you read when diagnosing a problem (see Troubleshooting).

SettingDefaultWhat it does
levelinfoMaster log level: trace, debug, info, warn, error, or fatal. Raise it to debug while you reproduce an issue; the change takes effect on your next search.
filek8s_search.logLog file. A relative path lands in the search’s dispatch directory, which Splunk cleans up; an absolute path goes where you point it and is yours to rotate.
levels.<component>-Raise the level for one component without flooding the rest. For example, levels.core/kubernetes/api = debug traces just the API calls, and levels.core/httpx = debug the HTTP transport and TLS. The full component list is in README/k8s_search.conf.spec.
local/k8s_search.conf ini
1[logging]
2level = info
3levels.core/kubernetes/api = debug

Connection to splunkd

[splunk] governs how the app reaches the local splunkd REST API to resolve the credentials a search needs - not how it reaches your Kubernetes clusters (that is on the Clusters page). The defaults match a standard Splunk install, and you won’t normally change them.

SettingDefaultWhat it does
request_timeout30sCaps a single REST call to the local splunkd.
tls_server_name(empty)The hostname the splunkd certificate must match. Set it only when you have replaced splunkd’s certificate with one that carries a proper DNS SAN.
tls_insecurefalseSkip certificate verification for that local call. For an install with a custom certificate the app cannot otherwise verify; the call is still authenticated by the Splunk session.

The remaining keys (tls_ca_file, tls_cert_file, tls_server_name_from_cert) point at the splunkd CA and certificate and pin the connection on loopback. Leave them at their defaults on a stock Splunk install; README/k8s_search.conf.spec documents them in full.

License

[license] holds your license key.

SettingDefaultWhat it does
key(empty)The signed key from your order confirmation. Empty is the free tier. Prefer entering it on the Setup page, which writes it here for you.

See Licensing for what a key unlocks.