Monitoring Docker, OpenShift and Kubernetes - Version 5.7 - Journald input

Version 5.7 of our applications and Collectord includes bug fixes and a new input that allows you to forward logs directly from Journald.

Journald input

For OpenShift clusters, we recommended using rsyslog to forward messages from journald to /var/log/message. Now you can uninstall rsyslog if you don’t need it anymore and forward messages directly from journald.

You can find the reference for the journald input in configurations for Docker, Kubernetes, and OpenShift:

As follows:

[input.journald]

# disable host level logs
disabled = false

# root location of log files
path = /rootfs/var/log/journal/

# when reach end of journald, how often to pull
pollingInterval = 250ms

# if you don't want to forward journald from the beginning,
# set the oldest event in relative value, like -14h or -30m or -30s (h/m/s supported)
startFromRel =

# override type
type = kubernetes_host_logs

# specify Splunk index
index =

# sample output (-1 does not sample, 20 - only 20% of the logs should be forwarded)
samplingPercent = -1

# sampling key (should be regexp with the named match pattern `key`)
samplingKey =

# set output (splunk or devnull, default is [general]defaultOutput)
output =

In the case of Kubernetes and OpenShift clusters, include it in your ConfigMap in file 002-daemonset.conf. If you are upgrading from the previous version of the application, we recommend specifying

startFromRel = -1h

This will tell Collectord to start reading journald from only one hour behind. Considering that you have already forwarded all the host logs from /var/log/messages, this will minimize the amount of forwarded journald logs from the first start and cause fewer duplications in Splunk.

Links

You can find more information about other minor updates by following links below.

Release notes

Upgrade instructions

Installation instructions

Outcold Solutions provides solutions for monitoring Kubernetes, OpenShift and Docker clusters in Splunk Enterprise and Splunk Cloud. We offer certified Splunk applications, which give you insights across all container environments. We are helping businesses reduce complexity related to logging and monitoring by providing easy-to-use and easy-to-deploy solutions for Linux and Windows containers. We deliver applications, which help developers monitor their applications and help operators keep their clusters healthy. With the power of Splunk Enterprise and Splunk Cloud, we offer one solution to help you keep all the metrics and logs in one place, allowing you to quickly address complex questions on container performance.