Troubleshooting

On OpenShift, swap kubectl for oc and use the collectorforopenshift-syslog namespace.

Verify configuration

When something looks off, the first thing to do is run collectord verify from inside a Collectord pod. It checks the configuration - license, container runtime, file inputs, the journal - and reports each item as OK or FAILED. It doesn’t connect to the syslog server: when Collectord can’t reach it, the Collectord log shows cannot open connection or failed to write warnings.

Start by listing the Collectord pods:

1$ kubectl get pods -n collectorforkubernetes-syslog
2NAME                                           READY     STATUS    RESTARTS   AGE
3collectorforkubernetes-syslog-addon-857fccb8b9-t9qgq   1/1       Running   1          1h
4collectorforkubernetes-syslog-bwmwr                    1/1       Running   0          1h
5collectorforkubernetes-syslog-xbnaa                    1/1       Running   0          1h

Collectord runs as two workloads - a DaemonSet (collectorforkubernetes-syslog) and a single Deployment add-on (collectorforkubernetes-syslog-addon). Run verify against the add-on, a DaemonSet pod on a control-plane node (it reads the API server’s audit logs) and one on a worker, so every code path is exercised (kubectl get pods -n collectorforkubernetes-syslog -o wide shows which node each pod runs on):

1$ kubectl exec -n collectorforkubernetes-syslog collectorforkubernetes-syslog-addon-857fccb8b9-t9qgq -- /collectord verify
2$ kubectl exec -n collectorforkubernetes-syslog collectorforkubernetes-syslog-bwmwr -- /collectord verify
3$ kubectl exec -n collectorforkubernetes-syslog collectorforkubernetes-syslog-xbnaa -- /collectord verify

Before 26.10, a second DaemonSet, collectorforkubernetes-syslog-master, ran on the control-plane nodes. See Upgrade from 26.04 to 26.10.

Each command prints a report that starts with Version = 26.10.1 and the environment, then one section per area with one line per check: + for OK, x for FAILED, and - for a check that doesn’t apply. The file inputs of a DaemonSet pod on a worker node read:

1  File Inputs:
2  - input(audit-logs): not read on this node
3    its labels do not match nodeSelector "node-role.kubernetes.io/control-plane"
4  + input(logs): OK
5    path /rootfs/var/log/

The API server’s audit logs are on the control-plane nodes (on OpenShift, the nodeSelector is node-role.kubernetes.io/master), so a worker doesn’t read them, and that line counts no error. On a control-plane node the line reads + input(audit-logs): OK, or x input(audit-logs): FAILED until the API server writes its audit logs to /var/log/kube-apiserver/.

When a check fails, the report ends with the number of failures, such as Errors: 1.

If you fix a real configuration error, kubectl apply -f ./collectorforkubernetes-syslog.yaml won’t restart the running pods. Delete them so the workloads recreate them with the new config: kubectl delete pods --all -n collectorforkubernetes-syslog.

Describe command

When the same setting can be defined on a pod, its workload, the namespace, a CRD Configuration, and the ConfigMap, it can be hard to track which value Collectord is actually using. The collectord describe command resolves a pod’s effective configuration and prints every field. Run it from inside any Collectord pod:

1kubectl exec -n collectorforkubernetes-syslog collectorforkubernetes-syslog-4gjmc -- /collectord describe --namespace default --pod postgres-pod --container postgres

Starting with version 26.04, describe also tags each resolved field with its origin in square brackets:

• [pod] - the value comes from a pod annotation
• [namespace] - the value comes from a namespace annotation
• [configuration:<name>] - the value comes from a Collectord CRD Configuration resource (the <name> matches the resource name)

This makes it easy to trace which level of the configuration hierarchy is winning when the same syslog.collectord.io/ annotation is defined at multiple levels - for example, when a CRD-level default is being overridden by a pod-level annotation, or when a namespace annotation is unexpectedly routing logs to a different output:

1$ kubectl exec -n collectorforkubernetes-syslog collectorforkubernetes-syslog-fqhmv -- /collectord describe --namespace webportal --pod audit-logger-774675c89c-rpfwx | grep '\['
2logs-type [pod] = audit_logs
3volume.1-logs-name [pod] = data
4volume.1-logs-glob [pod] = *.log

Especially useful when debugging why a pod is routing to an unexpected output, using the wrong sourcetype, or picking up a field extraction you didn’t expect.

Collect diagnostic information

When you open a support case, attach a diagnostic bundle so we can reproduce the issue without a back-and-forth. The bundle includes performance, memory, and telemetry metrics, host Linux information, and the Collectord configuration - sensitive output settings are stripped out.

Run all four steps below.

1. Collect diagnostics information run following command

Pick any Collectord pod and run collectord diag. The command takes a few minutes:

1kubectl exec -n collectorforkubernetes-syslog collectorforkubernetes-syslog-bwmwr -- /collectord diag --stream 1>diag.tar.gz

You can extract the archive yourself to see exactly what’s in it - performance and memory profiles, basic telemetry metrics, host Linux info, and license metadata.

2. Collect logs

1kubectl logs -n collectorforkubernetes-syslog --timestamps collectorforkubernetes-syslog-bwmwr  1>collectorforkubernetes.log 2>&1

3. Run verify

1kubectl exec -n collectorforkubernetes-syslog collectorforkubernetes-syslog-bwmwr -- /collectord verify > verify.log

4. Prepare tar archive

1tar -czvf collectorforkubernetes-$(date +%s).tar.gz verify.log collectorforkubernetes.log diag.tar.gz