Outcold Solutions - Monitoring Kubernetes, OpenShift and Docker in Splunk

Forwarding Kubernetes logs to ElasticSearch and OpenSearch

Configuration

Configurations

collectorforkubernetes-elasticsearch.yaml

Deploying on OpenShift clusters

For OpenShift clusters you need to add privileged SCC to the service account.

Save content of the following file to collectorforkubernetes-scc.yaml file.

And apply it with oc apply -f collectorforkubernetes-scc.yaml.

kind: SecurityContextConstraints
apiVersion: security.openshift.io/v1
metadata:
  name: collectorforkubernetes
allowHostDirVolumePlugin: true
allowHostIPC: true
allowHostNetwork: true
allowHostPID: true
allowHostPorts: true
allowPrivilegeEscalation: true
allowPrivilegedContainer: true
readOnlyRootFilesystem: false
allowedCapabilities:
  - '*'
allowedUnsafeSysctls:
  - '*'
fsGroup:
  type: RunAsAny
runAsUser:
  type: RunAsAny
seLinuxContext:
  type: RunAsAny
supplementalGroups:
  type: RunAsAny
seccompProfiles:
  - '*'
users:
  - system:serviceaccount:collectorforkubernetes:collectorforkubernetes
volumes:

List of Created Kubernetes Objects

Configuration file collectorforkubernetes-elasticsearch.yaml creates several Kubernetes Objects.

  • Namespace collectorforkubernetes.
  • ClusterRole collectorforkubernetes with limited capabilities to get, list and watch deployed objects. Collectord uses this information to enrich logs and stats with Kubernetes specific metadata.
  • ServiceAccount collectorforkubernetes is used to connect to Kubernetes API.
  • ClusterRoleBinding collectorforkubernetes to bind a service account to a cluster role.
  • ConfigMap collectorforkubernetes-elasticsearch delivers configuration files for collectord.
  • DaemonSet collectorforkubernetes-elasticsearch allows to deploy collectord on each node.
  • Deployment collectorforkubernetes-elasticsearch is a single collectord instance, that needs to forward data from the whole cluster once.

Read commentaries in collectorforkubernetes-elasticsearch.yaml file to get more deep details on all configurations and source of the logs.

  • Installation
    • Forwarding container logs, application logs, host logs and audit logs.
    • Test our solution with the embedded 30-day evaluation license.
  • Collectord Configuration
    • Collectord configuration reference for Kubernetes and OpenShift clusters.
  • Annotations
    • Changing the type and format of messages forwarded from namespaces, workloads and pods.
    • Forwarding application logs.
    • Multi-line container logs.
    • Field extraction for application and container logs (including timestamp extractions).
    • Hiding sensitive data, stripping terminal escape codes and colors.
  • Troubleshooting
  • FAQ and the common questions
  • License agreement
  • Pricing
  • Contact

About Outcold Solutions

Outcold Solutions provides solutions for monitoring Kubernetes, OpenShift and Docker clusters in Splunk Enterprise and Splunk Cloud. We offer certified Splunk applications, which give you insights across all container environments. We are helping businesses reduce complexity related to logging and monitoring by providing easy-to-use and easy-to-deploy solutions for Linux and Windows containers. We deliver applications, which help developers monitor their applications and help operators keep their clusters healthy. With the power of Splunk Enterprise and Splunk Cloud, we offer one solution to help you keep all the metrics and logs in one place, allowing you to quickly address complex questions on container performance.