Outcold Solutions - Monitoring Kubernetes, OpenShift and Docker in Splunk
Monitoring Docker in Splunk Monitoring Kubernetes in Splunk Monitoring OpenShift in Splunk Monitoring Linux in Splunk Monitoring Windows Containers in Splunk Forwarding Logs to Elasticsearch and OpenSearch Forwarding Logs to QRadar
Documentation Pricing Contact Us Blog Evaluation licenses Partners License Agreement Privacy Policy FAQ

Forwarding Kubernetes logs to ElasticSearch and OpenSearch

Configuration

Configurations

collectorforkubernetes-elasticsearch.yaml

Deploying on OpenShift clusters

For OpenShift clusters you need to add privileged SCC to the service account.

Save content of the following file to collectorforkubernetes-scc.yaml file.

And apply it with oc apply -f collectorforkubernetes-scc.yaml.

 1kind: SecurityContextConstraints
 2apiVersion: security.openshift.io/v1
 3metadata:
 4  name: collectorforkubernetes
 5allowHostDirVolumePlugin: true
 6allowHostIPC: true
 7allowHostNetwork: true
 8allowHostPID: true
 9allowHostPorts: true
10allowPrivilegeEscalation: true
11allowPrivilegedContainer: true
12readOnlyRootFilesystem: false
13allowedCapabilities:
14  - '*'
15allowedUnsafeSysctls:
16  - '*'
17fsGroup:
18  type: RunAsAny
19runAsUser:
20  type: RunAsAny
21seLinuxContext:
22  type: RunAsAny
23supplementalGroups:
24  type: RunAsAny
25seccompProfiles:
26  - '*'
27users:
28  - system:serviceaccount:collectorforkubernetes:collectorforkubernetes
29volumes:

List of Created Kubernetes Objects

Configuration file collectorforkubernetes-elasticsearch.yaml creates several Kubernetes Objects.

  • Namespace collectorforkubernetes.
  • ClusterRole collectorforkubernetes with limited capabilities to get, list and watch deployed objects. Collectord uses this information to enrich logs and stats with Kubernetes specific metadata.
  • ServiceAccount collectorforkubernetes is used to connect to Kubernetes API.
  • ClusterRoleBinding collectorforkubernetes to bind a service account to a cluster role.
  • ConfigMap collectorforkubernetes-elasticsearch delivers configuration files for collectord.
  • DaemonSet collectorforkubernetes-elasticsearch allows to deploy collectord on each node.
  • Deployment collectorforkubernetes-elasticsearch is a single collectord instance, that needs to forward data from the whole cluster once.

Read commentaries in collectorforkubernetes-elasticsearch.yaml file to get more deep details on all configurations and source of the logs.

  • Installation
    • Forwarding container logs, application logs, host logs and audit logs.
    • Test our solution with the embedded 30-day evaluation license.
  • Collectord Configuration
    • Collectord configuration reference for Kubernetes and OpenShift clusters.
  • Annotations
    • Changing the type and format of messages forwarded from namespaces, workloads and pods.
    • Forwarding application logs.
    • Multi-line container logs.
    • Field extraction for application and container logs (including timestamp extractions).
    • Hiding sensitive data, stripping terminal escape codes and colors.
  • Troubleshooting
  • FAQ and the common questions
  • License agreement
  • Pricing
  • Contact

About Outcold Solutions

Outcold Solutions provides solutions for monitoring Kubernetes, OpenShift and Docker clusters in Splunk Enterprise and Splunk Cloud. We offer certified Splunk applications, which give you insights across all container environments. We are helping businesses reduce complexity related to logging and monitoring by providing easy-to-use and easy-to-deploy solutions for Linux and Windows containers. We deliver applications, which help developers monitor their applications and help operators keep their clusters healthy. With the power of Splunk Enterprise and Splunk Cloud, we offer one solution to help you keep all the metrics and logs in one place, allowing you to quickly address complex questions on container performance.

Red Hat
Splunk
AWS