Kubernetes Search

Release history

Version history for the Kubernetes Search app

Kubernetes Search versions independently of the Collectord agent and the Monitoring apps, using standard semantic versioning.

1.2.0

  • New: A license covers a number of users, the Splunk users holding run_k8s_search (Splunk administrators are not counted), and registers any number of clusters. The License tab shows the licensed users beside the users holding the capability; more users than the license covers raise a warning there and in a bulletin message, and searches keep running.
  • Security: Built with Go 1.27.2, which fixes vulnerabilities in the Go standard library that scanners report against Go 1.27.1, in the HTTP/1 and HTTP/2 client and server, TLS and MIME header parsing
  • Security: A link that sets a dashboard input in its URL (?form.namespace=…) can no longer add arguments or commands to the dashboard’s searches
  • Security: A link that sets the cluster in its URL can no longer show a link or an image of its own in the Incident Snapshot heading
  • The free tier serves one user, the first to run a search, which an admin can change on the License tab: 1.1 allowed any number of users
  • Bug fix: On Splunk Free every Kubernetes Search command that reads a cluster, and adding a cluster, failed with a capability error, and the health check logged an error every 5 minutes. Splunk Free’s admin, which Splunk reports with no capabilities, now holds the app’s capabilities, and the health check skips with one INFO line
  • Bug fix: Resource Hygiene’s Admission Control tab showed a webhook configuration with no webhooks as a webhook with failurePolicy Fail
  • Bug fix: The Events dashboard counts a warning about an object with oom-kill in its name, such as a back-off of a pod named cb-oom-kill-web, as an OOM kill
  • Bug fix: The Node and Pod Definition tabs and the CoreDNS Corefile panel on System Components show no heading, and the Corefile panel’s description is missing
  • Bug fix: Splunk reports auth_server_fingerprint in kubeclusters.conf and kubeauth.conf as an invalid key every time it starts

Requires Splunk Enterprise 9.4, 10.0, 10.2, 10.4, or 10.6 on a search head, or Splunk Cloud, and Kubernetes 1.20 or later. Restart Splunk after upgrading.

1.1.0

  • New: Minted tokens: a cluster can run searches with ten-minute tokens for a read-only service account instead of its stored credential
  • New: Settings tab on the Setup page for request timeouts, fan-out concurrency, the longest log line | k8slogs reads, and the log level, so they can be set on Splunk Cloud
  • New: Unused PVCs tab on the Networking & Storage dashboard (Kubernetes 1.37 and later)
  • New: The Events dashboard shows OOM kills, evictions, and node conditions first, in a Hot Spots table
  • New: Splunk messages for license problems and for clusters that refuse their stored credentials
  • New: Audit events in _internal for Setup changes, credential use, and minted tokens
  • New: Support for Splunk Enterprise 10.6
  • Security: A cluster administrator can no longer obtain a stored credential they cannot read. Re-save credentials stored by 1.0.0 to bind them to their API server
  • Security: A shared credential is no longer returned to a one-line | rest call. Users who can search the cluster can still obtain it; minted tokens, per-user credentials and impersonation prevent that
  • Security: Trace-level logging no longer writes credentials to the search log
  • Values with commas must be quoted in search commands, for example labels="app=web,tier=frontend", and a parameter given twice is refused
  • API server certificates with SHA-1 signatures or RSA keys shorter than 2048 bits are refused
  • Bug fix: | k8slogs fails the whole search when one container’s logs cannot be read
  • Bug fix: | k8sevents and | k8sdescribe report the wrong count and times for events created through the events.k8s.io API
  • Bug fix: On Kubernetes 1.27 to 1.29, | k8s cannot find some kinds, such as hpa, pdb, and custom resources
  • Bug fix: The Networking & Storage claim panels are empty for users who cannot list PersistentVolumes
  • Bug fix: Some pods and endpoints are counted with the wrong readiness
  • Bug fix: Dashboards show no value instead of 0 on a healthy cluster
  • Bug fix: The Setup page does not work for a role with only edit_k8s_search_clusters, or next to OS AI Agent or Search for AWS on the same search head

Requires Splunk Enterprise 9.4, 10.0, 10.2, 10.4, or 10.6 on a search head, or Splunk Cloud, and Kubernetes 1.20 or later. Restart Splunk after upgrading.

1.0.0

Initial release.

  • Live search commands. | k8s (list and get resources), | k8slogs (pod logs), | k8sevents (cluster events), | k8sdescribe (an object with its events), and | k8syaml (format results as YAML). See the Command reference.
  • Multi-cluster. Register multiple clusters and target them with context=, including parallel fan-out across a glob or all clusters, with a cluster field on every row.
  • Authentication. Bearer token and client-certificate credentials, kept in Splunk’s encrypted credential store. Import a kubeconfig to register a cluster quickly.
  • Access control. Dedicated roles and capabilities, plus three credential models - shared, per-user, and Kubernetes impersonation - for controlling whose RBAC a search runs under. See Access control.
  • Dashboards. Cluster health, system components, workloads, incident snapshot, reliability troubleshooting, events, logs explorer, resource hygiene, networking and storage, and per-pod and per-node detail views. See Dashboards.
  • Caching. Short-lived on-disk result cache with per-search cache= override and _cache_hit / _cache_age_seconds reporting.
  • Search head clustering support (paid license), with cluster registrations and credentials replicated across members.
  • Free tier: one cluster on a single standalone search head, no license key required.

Requires Splunk Enterprise 9.4, 10.0, 10.2, or 10.4 on a search head, and Kubernetes 1.20 or later. See Requirements for the full compatibility matrix.

Known issues and limitations

  • AWS IAM / EKS authentication is not yet available. Authenticate to EKS with a bearer token; native IAM support is planned for a future release.
  • Pod logs are a point-in-time snapshot. | k8slogs has no live follow - re-run to see newer lines.
  • Events are bounded by Kubernetes retention, about an hour by default.
  • Impersonation is unavailable to scheduled and saved searches, which run as nobody with no user identity to impersonate.