Splunk HTTP Event Collector

Collectord sends its data to Splunk through the HTTP Event Collector (HEC). Use the [output.splunk] settings on this page to verify HEC’s certificate, go through a proxy, spread the load across endpoints, or send to more than one Splunk cluster.

Configure HTTP Event Collector secure connection

Splunk ships HEC with a self-signed certificate that no client can verify, so Collectord either accepts it unverified or verifies a certificate of your own. The settings for both live under [output.splunk] in the configuration.

Verify the HEC certificate

Splunk’s default HEC certificate (SplunkServerDefaultCert) names no host in a Subject Alternative Name, so no client can verify it against the host it connects to. Collectord refuses it with x509: certificate relies on legacy Common Name field, use SANs instead, whatever caPath and caName say. With Splunk’s default certificate, set insecure = true.

To verify the connection instead, give HEC a certificate whose Subject Alternative Name names the host in Collectord’s url, and point caPath at the CA that signed it.

A minimal config that accepts the license, points at HEC and verifies its certificate against your CA:

1[general]
2acceptLicense = true
3
4[output.splunk]
5url = https://hec.example.com:8088/services/collector/event/1.0
6token = B5A79AAD-D822-46CC-80D1-819F80D7BFB0
7caPath = /config/hec-ca.pem

Layer the config and the CA onto the base image:

1FROM registry.outcold.solutions/outcoldsolutions/collectorfordocker:26.10.1
2
3COPY 002.conf /config/002.conf
4COPY hec-ca.pem /config/hec-ca.pem

Set caName only when that certificate names the host differently from the url. The name must still be one of the certificate’s Subject Alternative Names.

Run the resulting image as the installation guide shows, and Collectord talks to HEC over a fully verified TLS connection.

HTTP Event Collector incorrect index behavior

HEC rejects events whose target index isn’t on the token’s allow-list. Once you start overriding indexes with annotations, it’s easy to mistype a name or forget to enable a new index on the token. incorrectIndexBehavior sets how Collectord reacts:

• RedirectToDefault, the default, re-routes the rejected event to the token’s default index, so nothing is lost.
• Drop drops the event outright. Use it when you’d rather see gaps in Splunk than have unrouted events polluting the default index.
• Retry keeps retrying. Use it only when you can fix the index on the Splunk side quickly. Otherwise the affected pipeline (for example, process stats) stalls for the entire host.

1[general]
2acceptLicense = true
3
4[output.splunk]
5url = https://hec.example.com:8088/services/collector/event/1.0
6token = B5A79AAD-D822-46CC-80D1-819F80D7BFB0
7incorrectIndexBehavior = Drop

Build the image with the embedded configuration:

1FROM registry.outcold.solutions/outcoldsolutions/collectorfordocker:26.10.1
2
3COPY 002.conf /config/002.conf

Run it following the installation guide.

Using proxy for HTTP Event Collector

If your network forces outbound traffic through a proxy, point Collectord at it with proxyUrl. When the proxy itself terminates TLS, bake its CA into the image too:

1[general]
2acceptLicense = true
3
4[output.splunk]
5url = https://hec.example.com:8088/services/collector/event/1.0
6token = B5A79AAD-D822-46CC-80D1-819F80D7BFB0
7proxyUrl = http://proxy.example:4321
8caPath = /config/proxie-ca.pem

Build the image with the embedded configuration:

1FROM registry.outcold.solutions/outcoldsolutions/collectorfordocker:26.10.1
2
3COPY 002.conf /config/002.conf
4COPY proxie-ca.pem /config/proxie-ca.pem

Run it following the installation guide.

Using multiple HTTP Event Collector endpoints for Load Balancing and Fail-over

When you have several HEC endpoints, typically a heavy forwarder pool or a dedicated indexer cluster, list them all: Collectord spreads the load across them and survives failures. A “failure” here means a connection error or any HTTP status >= 500.

urlSelection picks one of three algorithms:

• random picks a random URL on first send, and a different random one after every failure. This is the default.
• round-robin starts at the first URL and advances one position on every failure.
• random-with-round-robin picks a random URL on first send, then goes round-robin from there on every failure.

With round-robin and random-with-round-robin, the Collectord instances on a failed URL all move to the same next one, so restarting the endpoints one by one can gather most instances on one of them. No algorithm moves an instance back to an endpoint that returns: the endpoint restarted last takes instances only as others fail or Collectord restarts.

 1[general]
 2acceptLicense = true
 3
 4[output.splunk]
 5urls.0 = https://hec1.example.com:8088/services/collector/event/1.0
 6urls.1 = https://hec2.example.com:8088/services/collector/event/1.0
 7urls.2 = https://hec3.example.com:8088/services/collector/event/1.0
 8
 9urlSelection = random
10
11token = B5A79AAD-D822-46CC-80D1-819F80D7BFB0

Build the image with the embedded configuration:

1FROM registry.outcold.solutions/outcoldsolutions/collectorfordocker:26.10.1
2
3COPY 002.conf /config/002.conf

Run it following the installation guide.

Before 26.10, the default was random-with-round-robin. A configuration file copied from an earlier release names it and keeps it: change it to random.

Enable indexer acknowledgement

By default, HEC tells Collectord a payload was accepted as soon as it lands on the receiver, not when an indexer has persisted it. For stronger delivery guarantees, turn on Indexer acknowledgment on the token and on Collectord.

Each thread keeps up to 64 batches per index in flight while their acknowledgements arrive, so acknowledgement doesn’t limit how fast Collectord sends.

1[general]
2acceptLicense = true
3
4[output.splunk]
5url = https://hec.example.com:8088/services/collector/event/1.0
6ackUrl = https://hec.example.com:8088/services/collector/ack
7token = B5A79AAD-D822-46CC-80D1-819F80D7BFB0
8ackEnabled = true
9ackTimeout = 3m

Build the image with the embedded configuration:

1FROM registry.outcold.solutions/outcoldsolutions/collectorfordocker:26.10.1
2
3COPY 002.conf /config/002.conf

Run it following the installation guide.

Behind a load balancer, acknowledgement needs each Collectord instance’s connections kept on one indexer, by source-IP or connection affinity. Collectord doesn’t keep cookies, so cookie-based sticky sessions don’t work. Without affinity, Collectord posts batches again before their acknowledgements arrive.

Before 26.10, each thread waited for Splunk to acknowledge a batch before it posted the next one to that index, which cost throughput.

Client certificates for collector

If your HEC endpoint requires mTLS, embed the client certificate and key in the image and point Collectord at them:

1[general]
2acceptLicense = true
3
4[output.splunk]
5url = https://hec.example.com:8088/services/collector/event/1.0
6token = B5A79AAD-D822-46CC-80D1-819F80D7BFB0
7clientCertPath = /config/client-cert.pem
8clientKeyPath = /config/client-cert.key

Support for multiple Splunk clusters

A Docker host can forward to more than one Splunk cluster: say, a primary indexing tier and a separate tier for a security team. Define a named output alongside the default:

1[output.splunk::prod1]
2url = https://prod1.hec.example.com:8088/services/collector/event/1.0
3token = AF420832-F61B-480F-86B3-CCB5D37F7D0D

Any setting the named output doesn’t specify falls back to the one on output.splunk.

Then send specific containers to the secondary cluster with an annotation such as collectord.io/output=splunk::prod1.