Monitoring OpenShift

Splunk field extraction

When you want Splunk to parse fields out of your container logs at search time, target the extraction at the right containers by matching on Collectord’s source value. Every container log Collectord forwards carries a structured source that encodes the container ID, container name, image name, pod name, namespace, and stream:

text
1/openshift/{openshift_container_id}/{openshift_container_name}/{openshift_image_name}/{openshift_pod_name}/{openshift_namespace}.{docker_stream}

Use that structure in props.conf with wildcards (*) for individual segments and ... to skip multiple — that way you can scope extractions to a single image, a specific container name, or any combination.

For example, to apply an nginx access-log extraction to every container running the nginx image, regardless of container ID, container name, or stream:

props.conf ini
1[source::/openshift/.../nginx:*/*/*]
2EXTRACT-nginx-ingress-controller-http = ^(?P&lt;remote_addr&gt;[^ ]+)\s+\-\s+\[(?P&lt;proxy_add_x_forwarded_for&gt;[^\]]+)\]\s+\-\s+(?P&lt;remote_user&gt;[^ ]+)\s+\[(?P&lt;time_local&gt;[^\]]+)[^"\n]*"(?P&lt;request&gt;[^"]+)"\s+(?P&lt;status&gt;\d+)\s+(?P&lt;body_bytes_sent&gt;\d+)\s+"(?P&lt;http_referer&gt;[^"]+)"\s+"(?P&lt;http_user_agent&gt;[^"]+)"\s+(?P&lt;request_length&gt;\d+)\s+(?P&lt;request_time&gt;[^ ]+)\s+\[(?P&lt;proxy_upstream_name&gt;[^\]]+)]\s+(?P&lt;upstream_addr&gt;[^\s]+)\s+(?P&lt;upstream_response_length&gt;\d+)\s+(?P&lt;upstream_response_time&gt;[^\s]+)\s+(?P&lt;upstream_status&gt;\d+)$</code></pre>

If you’d rather override the source or sourcetype on a per-pod or per-project basis instead — and key your extractions off those — see Splunk Indexes.