Monitoring Windows Containers

Configuration

Collectord configuration

Join Rules

By default, Collectord joins a line with the previous one if it starts with whitespace. When you need something different — say, a Java stack trace that should attach to its preceding log line — define a custom join rule.

Take this sample application output:

text
1[2017-09-04T06:28:05,664][WARN ][MyComponent]
2java.security.AccessControlException: access denied
3  at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_131]
4  at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_131]
5[2017-09-04T06:28:05,664][WARN ][MyComponent] another message

Match every container whose name contains my_app, and treat any line starting with [YYYY- as the start of a new event:

ini
1[pipe.join::my_app]
2matchRegex.docker_container_name = .+my_app.+
3patternRegex = ^\[\d{4}-

collector.conf Reference

  1# collectord configuration file
  2#
  3# Run collectord with flag -conf and specify location of the configuration file.
  4#
  5# You can override all the values using environment variables with the format like
  6#   COLLECTOR__<section>__<key>=<value>
  7# As an example you can set dataPath in [general] section as
  8#   COLLECTOR__general__dataPath=C:\\some\\path\\data.db
  9# This parameter can be configured using -env-override, set it to empty string to disable this feature
 10
 11[general]
 12
 13# location for the database
 14# is used to store position of the files and internal state
 15; dataPath = .\data\
 16
 17# log level (trace, debug, info, warn, error, fatal)
 18; logLevel = info
 19
 20# http server gives access to two endpoints
 21# /healthz
 22# /metrics
 23; httpServerBinding = :8080
 24
 25# telemetry report endpoint, set it to empty string to disable telemetry
 26; telemetryEndpoint = https://license.outcold.solutions/telemetry/
 27
 28# license check endpoint
 29; licenseEndpoint = https://license.outcold.solutions/license/
 30
 31# license server through proxy
 32; licenseServerProxyUrl =
 33
 34# license
 35; license =
 36
 37# docker daemon hostname is used by default as hostname
 38# use this configuration to override
 39; hostname =
 40
 41
 42[general.docker]
 43
 44# path to docker root folder
 45; dockerRootFolder = C:\ProgramData\Docker\
 46
 47
 48# Log files
 49[input.files]
 50
 51# disable container logs monitoring
 52; disabled = false
 53
 54# root location of docker files
 55; path = C:\ProgramData\Docker\containers\
 56
 57# glob matching pattern for log files
 58; glob = *\*-json.log*
 59
 60# files are read using polling schema, when reach the EOF how often to check if files got updated
 61; pollingInterval = 250ms
 62
 63# how often to look for the new files under logs path
 64; walkingInterval = 5s
 65
 66# include verbose fields in events (file offset)
 67; verboseFields = false
 68
 69# override type
 70; type = wincontainers_logs
 71
 72# read hardlinks instead of original files
 73# on Windows that allows not to block the original directory, when
 74# files are getting deleted by daemon
 75; hardlinks = true
 76; hardlinksPath = C:\ProgramData\Docker\containers\
 77
 78
 79[input.files::docker_service]
 80
 81# disable host level logs
 82; disabled = false
 83
 84# root location of docker files
 85path = C:\ProgramData\Docker\
 86
 87# regex matching pattern
 88match = ^service.txt$
 89
 90# limit search only on one level
 91recursive = false
 92
 93# files are read using polling schema, when reach the EOF how often to check if files got updated
 94pollingInterval = 250ms
 95
 96# how often o look for the new files under logs path
 97walkingInterval = 5s
 98
 99# include verbose fields in events (file offset)
100verboseFields = false
101
102# override type
103type = wincontainers_host_logs
104
105# field extraction
106extraction = ^\[(?P<timestamp>[^\]]+)\]\[(?P<docker_service_component>[^\]]+)\]\[(?P<docker_service_level>[^\]]+)\]\s(.*)$
107# extractionMessageField =
108
109# timestamp field
110timestampField = timestamp
111
112# format for timestamp
113# the layout defines the format by showing how the reference time, defined to be `Mon Jan 2 15:04:05 -0700 MST 2006`
114timestampFormat = 15:04:05.000
115
116# Adjust date, if month/day aren't set in format
117timestampSetMonth = true
118timestampSetDay = true
119
120# timestamp location (if not defined by format)
121timestampLocation = Local
122
123
124# Splunk output
125[output.splunk]
126
127# Splunk HTTP Event Collector url
128; url =
129
130# Splunk HTTP Event Collector Token
131; token =
132
133# Allow invalid SSL server certificate
134; insecure = false
135
136# Path to CA cerificate
137; caPath =
138
139# CA Name to verify
140; caName =
141
142# Events are batched with the maximum size set by batchSize and staying in pipeline for not longer
143# than set by frequency
144; frequency = 5s
145; batchSize = 768K
146
147# Splunk through proxy
148; proxyUrl =
149
150
151# Pipe to join events (container logs only)
152[pipe.join]
153
154# disable joining event
155; disabled = false
156
157# Maximum interval of messages in pipeline
158; maxInterval = 100ms
159
160# Maximum time to wait for the messages in pipeline
161; maxWait = 1s
162
163# Maximum message size
164; maxSize = 100K
165
166# Default pattern to indicate new message (should start not from space)
167; patternRegex = ^[^\s]
168
169
170# Define special event join patterns for matched events
171# Section consist of [pipe.join::<name>]
172# [pipe.join::my_app]
173## Set match pattern for the fields
174#; matchRegex.docker_container_image = my_app
175#; matchRegex.docker_stream = stdout
176## All events start from '[<digits>'
177#; patternRegex = ^\[\d+
178
179
180# Stats for Windows Containers
181[input.wincontainer_stats]
182
183; disabled = false
184
185# how often to collect stats
186; statsInterval = 30s
187
188# override type
189; type = wincontainers_stats
190
191
192# Not supported on Windows
193[input.docker_events]
194; disabled = true
195
196
197# Not supported on Windows
198[input.system_stats]
199; disabled = true
200
201
202# Not supported on Windows
203[input.proc_stats]
204; disabled = true

What is next?