Splunk indexes

Out of the box, collectorfordocker writes everything to the default index of your HTTP Event Collector token. This page shows how to send logs and metrics to indexes of your own, and how to make the Monitoring Docker app search them.

Each token has an allow-list of indexes it can write to, and one of them is the default when the sender doesn’t pick one. The Monitoring Docker app, in turn, assumes those indexes are searchable by default for your Splunk role: main is the obvious example.

If you forward to an index your role doesn’t search by default, the dashboards come up empty, even though Splunk indexes the data.

You have two ways to fix that. The first is to add the index to Indexes searched by default for your role, under Settings › Access Control › Roles:

The second is to update the search macros the app uses, so the dashboards search your indexes explicitly. Edit them in Splunk Web under Settings › Advanced search › Search macros. The Macros panel on the app’s Setup › Configuration page links the ones you are most likely to change. Or override $SPLUNK_HOME/etc/apps/monitoringdocker/default/macros.conf with a local/macros.conf:

Every events-index macro builds on a single base macro, macro_docker_base, so you typically set the index list only once:

1macro_docker_base = (index=docker_stats OR index=docker_logs)

If Collectord sends stats to a Splunk metrics index, macro_docker_metrics_base names it for the metrics-index dashboards and alerts. It is (index=*) by default: every metrics index your role can search. See Metrics index.

For finer-grained control, such as pinning a specific datatype to a specific index and sourcetype, override the individual macro instead:

1macro_docker_stats_host = (index=docker_stats sourcetype=docker_stats_v2_host)

The macros worth knowing about:

Before 26.10, the app also defined macro_docker_stats, which no dashboard read. In a search of your own, use macro_docker_stats_host for host records and macro_docker_stats_cgroup for container records instead.

Using dedicated indexes for different types of data

For most hosts, it’s worth splitting logs from metrics: they have different access patterns, different volumes, and different retention needs. A common layout is:

• docker_logs for events, container logs, and host logs;
• docker_stats for proc and system metrics.

You can go further and pin every datatype Collectord forwards to its own index.

The big practical win: dedicated indexes let you set different retention policies for logs and metrics independently.

Override the destination index per input in the Collectord configuration. Collectord Configuration shows how to apply the changes:

 1[input.system_stats]
 2index = docker_stats
 3
 4[input.proc_stats]
 5index = docker_stats
 6
 7[input.net_stats]
 8index = docker_stats
 9
10[input.net_socket_table]
11index = docker_stats
12
13[input.mount_stats]
14index = docker_stats
15
16[input.disk_stats]
17index = docker_stats
18
19[input.files]
20index = docker_logs
21
22[input.app_logs]
23index = docker_logs
24
25[input.journald]
26index = docker_logs
27
28[input.files::logs]
29index = docker_logs
30
31[input.docker_events]
32index = docker_logs

[input.files::syslog] ships off. If you turn it on (see Host logs), give it the same index as [input.journald].