ElasticSearch and OpenSearch

Annotations reference

Every annotation Collectord recognizes for the ElasticSearch / OpenSearch output, grouped by the datatype it controls. All annotations use the elasticsearch.collectord.io/ prefix because this product sets annotationsSubdomain = elasticsearch. For usage, examples, and patterns, see the companion Annotations page.

  • General annotations
    • elasticsearch.collectord.io/index - change the datastream for all the data forwarded for this Pod (metrics, container logs, application logs)
    • elasticsearch.collectord.io/host - change the host for all the data forwarded for this Pod (metrics, container logs, application logs)
    • elasticsearch.collectord.io/output - change the output to devnull or elasticsearch
    • elasticsearch.collectord.io/userfields.{fieldname} - attach custom fields to events
  • Annotations for container logs
    • elasticsearch.collectord.io/logs-index - change the datastream for the container logs forwarded from this Pod
    • elasticsearch.collectord.io/logs-host - change the host for the container logs forwarded from this Pod
    • elasticsearch.collectord.io/logs-eventpattern - set the regex identifying the event start pattern for Pod logs
    • elasticsearch.collectord.io/logs-replace.{N}-search - define the search pattern for the replace pipe
    • elasticsearch.collectord.io/logs-replace.{N}-val - define the replace pattern for the replace pipe
    • elasticsearch.collectord.io/logs-hashing.{N}-match - the regexp for a matched value
    • elasticsearch.collectord.io/logs-hashing.{N}-function - hash function (default is sha256, available adler-32,crc-32-ieee,crc-32-castagnoli,crc-32-koopman,crc-64-iso,crc-64-ecma,fnv-1-64,fnv-1a-64,fnv-1-32,fnv-1a-32,fnv-1-128,fnv-1a-128,md5,sha1,sha256,sha384,sha512)
    • elasticsearch.collectord.io/logs-extraction - define the regexp for fields extraction
    • elasticsearch.collectord.io/logs-extractionMessageField - specify the field name for the message (by default first unnamed ground in regexp)
    • elasticsearch.collectord.io/logs-timestampfield - define the field for timestamp (after fields extraction)
    • elasticsearch.collectord.io/logs-timestampformat - define the timestamp format
    • elasticsearch.collectord.io/logs-timestampsetmonth - define if month should be set to current for timestamp
    • elasticsearch.collectord.io/logs-timestampsetday - define if day should be set to current for timestamp
    • elasticsearch.collectord.io/logs-timestamplocation - define timestamp location if not set by format
    • elasticsearch.collectord.io/logs-joinpartial - join partial events
    • elasticsearch.collectord.io/logs-joinmultiline - join multiline logs (default value depends on [pipe.join] disabled)
    • elasticsearch.collectord.io/logs-escapeterminalsequences - escape terminal sequences (including colors)
    • elasticsearch.collectord.io/logs-override.{N}-match - match for override pattern
    • elasticsearch.collectord.io/logs-override.{N}-index - override datastream for matched events
    • elasticsearch.collectord.io/logs-output - change the output to devnull or elasticsearch (this annotation can’t be specified for stderr and stdout)
    • elasticsearch.collectord.io/logs-disabled - disable any log processing for this container (this annotation can’t be specified for stderr and stdout)
    • elasticsearch.collectord.io/logs-sampling-percent - specify the % value of logs that should be forwarded to ElasticSearch
    • elasticsearch.collectord.io/logs-sampling-key - regexp pattern to specify the key for the sampling based on hash values
    • elasticsearch.collectord.io/logs-ThruputPerSecond - set the thruput for this container, maximum number of logs per second, for example 128Kb, 1024b
    • elasticsearch.collectord.io/logs-TooOldEvents - duration of events from now to past that are considered too old and should be ignored, for example 168h, 24h
    • elasticsearch.collectord.io/logs-TooNewEvents - duration of events from now to the future that are considered too new and should be ignored, for example 1h, 30m
    • elasticsearch.collectord.io/logs-whitelist - allow configuring a pattern for log messages, only log messages matching this pattern will be forwarded to ElasticSearch
    • elasticsearch.collectord.io/logs-userfields.{fieldname} - attach custom fields to events
    • Specific for stdout, with the annotations below you can define configuration specific for stdout
      • elasticsearch.collectord.io/stdout-logs-index
      • elasticsearch.collectord.io/stdout-logs-host
      • elasticsearch.collectord.io/stdout-logs-eventpattern
      • elasticsearch.collectord.io/stdout-logs-replace.{N}-search
      • elasticsearch.collectord.io/stdout-logs-replace.{N}-val
      • elasticsearch.collectord.io/stdout-logs-hashing.{N}-match
      • elasticsearch.collectord.io/stdout-logs-hashing.{N}-function
      • elasticsearch.collectord.io/stdout-logs-extraction
      • elasticsearch.collectord.io/stdout-logs-extractionMessageField
      • elasticsearch.collectord.io/stdout-logs-timestampfield
      • elasticsearch.collectord.io/stdout-logs-timestampformat
      • elasticsearch.collectord.io/stdout-logs-timestampsetmonth
      • elasticsearch.collectord.io/stdout-logs-timestampsetday
      • elasticsearch.collectord.io/stdout-logs-timestamplocation
      • elasticsearch.collectord.io/stdout-logs-joinpartial
      • elasticsearch.collectord.io/stdout-logs-joinmultiline
      • elasticsearch.collectord.io/stdout-logs-escapeterminalsequences
      • elasticsearch.collectord.io/stdout-logs-override.{N}-match
      • elasticsearch.collectord.io/stdout-logs-override.{N}-index
      • elasticsearch.collectord.io/stdout-logs-sampling-percent
      • elasticsearch.collectord.io/stdout-logs-sampling-key
      • elasticsearch.collectord.io/stdout-logs-ThruputPerSecond
      • elasticsearch.collectord.io/stdout-logs-TooOldEvents
      • elasticsearch.collectord.io/stdout-logs-TooNewEvents
      • elasticsearch.collectord.io/stdout-logs-whitelist
    • Specific for stderr, with the annotations below you can define configuration specific for stderr
      • elasticsearch.collectord.io/stderr-logs-index
      • elasticsearch.collectord.io/stderr-logs-host
      • elasticsearch.collectord.io/stderr-logs-eventpattern
      • elasticsearch.collectord.io/stderr-logs-replace.{N}-search
      • elasticsearch.collectord.io/stderr-logs-replace.{N}-val
      • elasticsearch.collectord.io/stderr-logs-hashing.{N}-match
      • elasticsearch.collectord.io/stderr-logs-hashing.{N}-function
      • elasticsearch.collectord.io/stderr-logs-extraction
      • elasticsearch.collectord.io/stdout-logs-extractionMessageField
      • elasticsearch.collectord.io/stderr-logs-timestampfield
      • elasticsearch.collectord.io/stderr-logs-timestampformat
      • elasticsearch.collectord.io/stderr-logs-timestampsetmonth
      • elasticsearch.collectord.io/stderr-logs-timestampsetday
      • elasticsearch.collectord.io/stderr-logs-timestamplocation
      • elasticsearch.collectord.io/stderr-logs-joinpartial
      • elasticsearch.collectord.io/stderr-logs-joinmultiline
      • elasticsearch.collectord.io/stderr-logs-escapeterminalsequences
      • elasticsearch.collectord.io/stderr-logs-override.{N}-match
      • elasticsearch.collectord.io/stderr-logs-override.{N}-index
      • elasticsearch.collectord.io/stderr-logs-sampling-percent
      • elasticsearch.collectord.io/stderr-logs-sampling-key
      • elasticsearch.collectord.io/stderr-logs-ThruputPerSecond
      • elasticsearch.collectord.io/stderr-logs-TooOldEvents
      • elasticsearch.collectord.io/stderr-logs-TooNewEvents
      • elasticsearch.collectord.io/stderr-logs-whitelist
  • Annotations for events (can be applied only to namespaces)
    • elasticsearch.collectord.io/events-index - change the datastream for the events of specific namespace
    • elasticsearch.collectord.io/events-host - change the host for the events of specific namespace
    • elasticsearch.collectord.io/events-userfields.{fieldname} - attach custom fields to events
    • elasticsearch.collectord.io/events-output - (26.04.1+) change the output for the events published on the namespace
  • Annotations for application logs
    • elasticsearch.collectord.io/volume.{N}-logs-name - name of the volume attached to Pod
    • elasticsearch.collectord.io/volume.{N}-logs-index - target datastream for logs forwarded from the volume
    • elasticsearch.collectord.io/volume.{N}-logs-output - configure the output for the logs forwarded from the volume
    • elasticsearch.collectord.io/volume.{N}-logs-host - change the host for logs forwarded from the volume
    • elasticsearch.collectord.io/volume.{N}-logs-eventpattern - change the event pattern defining new event for logs forwarded from the volume
    • elasticsearch.collectord.io/volume.{N}-logs-replace.{N}-search - specify the regex search for replace pipe for the logs
    • elasticsearch.collectord.io/volume.{N}-logs-replace.{N}-val - specify the regex replace pattern for replace pipe for the logs
    • elasticsearch.collectord.io/volume.{N}-logs-hashing.{N}-match - the regexp for a matched value
    • elasticsearch.collectord.io/volume.{N}-logs-hashing.{N}-function - hash function (default is sha256, available adler-32,crc-32-ieee,crc-32-castagnoli,crc-32-koopman,crc-64-iso,crc-64-ecma,fnv-1-64,fnv-1a-64,fnv-1-32,fnv-1a-32,fnv-1-128,fnv-1a-128,md5,sha1,sha256,sha384,sha512)
    • elasticsearch.collectord.io/volume.{N}-logs-extraction - specify the fields extraction with the regex the logs
    • elasticsearch.collectord.io/volume.{N}-logs-extractionMessageField - specify the field name for the message (by default first unnamed ground in regexp)
    • elasticsearch.collectord.io/volume.{N}-logs-timestampfield - specify the timestamp field
    • elasticsearch.collectord.io/volume.{N}-logs-timestampformat - specify the format for timestamp field
    • elasticsearch.collectord.io/volume.{N}-logs-timestampsetmonth - define if month should be set to current for timestamp
    • elasticsearch.collectord.io/volume.{N}-logs-timestampsetday - define if day should be set to current for timestamp
    • elasticsearch.collectord.io/volume.{N}-logs-timestamplocation - define timestamp location if not set by format
    • elasticsearch.collectord.io/volume.{N}-logs-glob - set the glob pattern for matching logs
    • elasticsearch.collectord.io/volume.{N}-logs-match - set the regexp pattern for matching logs
    • elasticsearch.collectord.io/volume.{N}-logs-recursive - set if walker should walk the directory recursive
    • elasticsearch.collectord.io/volume.{N}-logs-override.{N}-match - match for override pattern
    • elasticsearch.collectord.io/volume.{N}-logs-override.{N}-index - override datastream for matched events
    • elasticsearch.collectord.io/volume.{N}-logs-sampling-percent - specify the % value of logs that should be forwarded to ElasticSearch
    • elasticsearch.collectord.io/volume.{N}-logs-sampling-key - regexp pattern to specify the key for the sampling based on hash values
    • elasticsearch.collectord.io/volume.{N}-logs-ThruputPerSecond - set the thruput for this container, maximum number of logs per second, for example 128Kb, 1024b
    • elasticsearch.collectord.io/volume.{N}-logs-TooOldEvents - duration of events from now to past that are considered too old and should be ignored, for example 168h, 24h
    • elasticsearch.collectord.io/volume.{N}-logs-TooNewEvents - duration of events from now to the future that are considered too new and should be ignored, for example 1h, 30m
    • elasticsearch.collectord.io/volume.{N}-logs-whitelist - allow configuring pattern for log messages, only log messages matching this pattern will be forwarded to ElasticSearch
    • elasticsearch.collectord.io/volume.{N}-logs-userfields.{fieldname} - attach custom fields to events
    • elasticsearch.collectord.io/volume.{N}-logs-maxholdafterclose - how long Collectord can hold file descriptors open for files in PVC after pod is terminated (duration 5s, 1800s)
    • elasticsearch.collectord.io/volume.{N}-logs-onvolumedatabase - boolean flag to enable on a volume database for this volume, in case if this volume might be used on more than one host