Configuration

By default, collectorforlinux forwards logs from /var/log (including syslog files) and from journald. When your applications write logs somewhere else — /opt/<app>/logs/, /srv/<app>/, a mounted data volume — add an [input.files::<name>] section to 002-user.conf to pick them up.

The example below forwards *.log files from /opt/myapp/logs. Replace myapp with a name that identifies the source (for example, webportal or payments):

002-user.conf ini

 1# Input syslog(.\d+)? files
 2[input.files::mylogs]
 3
 4# disable host level logs
 5disabled = false
 6
 7# root location of log files
 8path = /opt/myapp/logs
 9
10# glob pattern
11glob = *.log
12
13# regex matching pattern (use it instead of glob pattern if you need more complicated filtering)
14# match = 
15
16# limit search only on one level
17recursive = false
18
19# files are read using polling schema, when reach the EOF how often to check if files got updated
20pollingInterval = 250ms
21
22# how often o look for the new files under logs path
23walkingInterval = 5s
24
25# include verbose fields in events (file offset)
26verboseFields = false
27
28# override type (source type)
29type = linux_host_logs
30
31# specify Splunk index
32index =
33
34# regexp to specify the beginning of the event line
35eventPattern = 
36
37# regexp field extraction
38extraction = 
39
40# timestamp field (if field extraction is used)
41timestampField =
42
43# format for timestamp
44# the layout defines the format by showing how the reference time, defined to be `Mon Jan 2 15:04:05 -0700 MST 2006`
45timestampFormat = Jan 2 15:04:05
46
47# Adjust date, if month/day aren't set in format
48timestampSetMonth = false
49timestampSetDay = false
50
51# timestamp location (if not defined by format)
52timestampLocation = Local
53
54# sample output (-1 does not sample, 20 - only 20% of the logs should be forwarded)
55samplingPercent = -1
56
57# sampling key for hash based sampling (should be regexp with the named match pattern `key`)
58samplingKey =
59
60# set output (splunk or devnull, default is [general]defaultOutput)
61output =
62
63# configure default thruput per second for for each container log
64# for example if you set `thruputPerSecond = 128Kb`, that will limit amount of logs forwarded
65# from the single container to 128Kb per second.
66thruputPerSecond =
67
68# Configure events that are too old to be forwarded, for example 168h (7 days) - that will drop all events
69# older than 7 days
70tooOldEvents =
71
72# Configure events that are too new to be forwarded, for example 1h - that will drop all events that are 1h in future
73tooNewEvents =

Examples

Will be added in the future

Monitoring Linux

Log forwarding

Configuration

Examples