Kubernetes Search

Access control

Two independent questions govern who sees what:

  1. Can this Splunk user run the commands at all? Answered by Splunk roles and capabilities.
  2. Whose Kubernetes permissions does the search run under? Answered by the credential model you choose for each cluster.

Get both right and Kubernetes Search gives analysts controlled, audited read access to clusters without handing out kubectl or cluster credentials.

Splunk roles and capabilities

The app defines two capabilities:

CapabilityGates
run_k8s_searchRunning | k8s, | k8slogs, | k8sevents, | k8sdescribe, and managing your own per-user credentials.
edit_k8s_search_clustersRegistering, editing, and deleting cluster definitions and importing kubeconfigs.

And two roles that bundle them:

RoleInheritsAddsFor
role_k8s_search_useruserrun_k8s_searchAnyone who queries clusters.
role_k8s_search_adminrole_k8s_search_useredit_k8s_search_clusters, list_storage_passwordsWhoever owns the connections.

A full Splunk administrator (admin_all_objects) can do everything without an explicit role. Note that a plain role_k8s_search_user does not get list_storage_passwords: search commands resolve their credential through a system-context handler, so users run searches without being able to read the credential store directly.

Credential models

Each cluster uses one of three models. They differ in whose Kubernetes RBAC a search runs under and in how well one user is isolated from another. This matters most on clusters where different people should see different things.

Shared default credential

One credential is configured for the cluster and every Splunk user’s searches use it. Simplest to set up; no per-user configuration.

The shared credential is extractable. Any Splunk user who can run searches can arrange to read the token or key and use it outside Splunk with its full Kubernetes RBAC. Treat a shared credential as readable by everyone who has run_k8s_search, and scope it to least privilege accordingly - read-only, no Secrets. It is the right choice for a low-sensitivity, read-only cluster where everyone is entitled to the same view.

Per-user credentials

Each Splunk user registers their own token for the cluster, under their own account. A user’s credential is isolated from other users - the app refuses cross-user reads - so people query with their own Kubernetes RBAC. A user can still extract their own token (it is theirs), but not anyone else’s. Use this when users genuinely have different Kubernetes permissions and you want each search to reflect the searcher’s own access. Per-user credentials require a paid license.

Impersonation

The cluster holds one service-account credential that is allowed to impersonate users. At search time the app stamps the Splunk username into the request’s Impersonate-User header, and the Kubernetes API server applies that user’s RBAC. The service-account bearer never reaches the search process, and each user is limited to exactly what their Kubernetes identity allows. This is the strongest isolation. Impersonation requires a paid license.

Set it up carefully:

  • The service account needs the impersonate verb on users only - never on groups. Kubernetes Search stamps the user, not groups.
  • Bind your Kubernetes RBAC to Splunk usernames, since that is what gets impersonated.
  • A search with no real Splunk-user identity - a scheduled or saved search runs as nobody - is refused impersonation. Impersonation is for interactive analyst queries.

Choosing a model

Runs underOne user can see another’s data?Credential reaches the user?License
SharedOne shared identitySame view for allYes - extractable, full RBACFree
Per-userEach user’s own tokenNoOwn token onlyPaid
ImpersonationEach user’s K8s identityNo (API server enforces)NoPaid

As a rule: shared for a read-only cluster everyone may see equally, impersonation for sensitive or multi-tenant clusters where the API server should be the one enforcing per-user access, and per-user when impersonation isn’t available but users still need distinct RBAC.

Auditing usage

Because Kubernetes Search gives live read access to clusters, you’ll want to see who ran what. There are two audit trails.

In Splunk. Every search is recorded in the audit log, with the user and the SPL they ran, so you can review Kubernetes Search activity directly:

index=_audit action=search
| search search="*| k8s*" OR search="*k8slogs*" OR search="*k8sevents*" OR search="*k8sdescribe*"
| table _time user search

In Kubernetes. With impersonation, each API request reaches the cluster under the impersonated Splunk username, so the cluster’s own API-server audit log attributes activity to the individual user - per-user accountability in the cluster’s trail, not just in Splunk. With a shared credential, requests appear under the single shared identity instead, which is one more reason to prefer impersonation where attribution matters.