Forwarding Audit Logs

OpenShift 4.x has audit log enabled by default. No additional configurations are required.

Our solution provides detailed Audit dashboards. By default, OpenShift does not provide audit logs. You can enable them by following the instructions from the OpenShift documentation Master and Node Configuration/Advanced Audit.

You need to enable audit logs only on Masters. For that, you need to edit the master-config.yaml file.

sudo vi /etc/origin/master/master-config.yaml

And add configuration to keep, for example, audit logs for the last 10 days or a maximum of 3 files with the size of 100 megabytes.

auditConfig:
  auditFilePath: "/var/lib/origin/openpaas-oscp-audit/openpaas-oscp-audit.log"
  enabled: true
  maximumFileRetentionDays: 10
  maximumFileSizeMegabytes: 100
  maximumRetainedFiles: 3
  policyFile: "/etc/origin/master/audit-policy.yaml"
  logFormat: json

The collectord automatically forwards logs from /var/lib/origin/openpaas-oscp-audit/; no additional configuration is required in the collectord.

We recently updated our configuration to use /var/lib/origin/openpaas-oscp-audit/ instead of /var/log. OpenShift, starting from version 3.10, does not map /var/log inside the container with the master API.

You need to create an audit-policy.yaml in the specified path.

sudo vi /etc/origin/master/audit-policy.yaml

Use our example as a reference. We hide most of the system activity.

Another good example of the audit-policy.yaml file is the audit profile used by GCE.

apiVersion: audit.k8s.io/v1beta1
kind: Policy
rules:
  # Do not log from kube-system accounts
  - level: None
    userGroups:
    - system:serviceaccounts:kube-system
    - system:serviceaccounts:openshift-infra
    - system:serviceaccounts:kube-service-catalog
    - system:serviceaccounts:openshift-template-service-broker
    - system:serviceaccounts:openshift-sdn
    - system:serviceaccounts:openshift-node
    - system:serviceaccounts:openshift-ansible-service-broker
    - system:serviceaccounts:openshift-monitoring
  - level: None
    users:
    - system:apiserver
    - system:kube-scheduler
    - system:volume-scheduler
    - system:kube-controller-manager
    - system:node
    - system:openshift-master

  # Do not log from collector
  - level: None
    users:
    - system:serviceaccount:collectorforopenshift:collectorforopenshift

  # Don't log nodes communications
  - level: None
    userGroups:
    - system:nodes

  # Don't log these read-only URLs.
  - level: None
    nonResourceURLs:
    - /healthz*
    - /version
    - /swagger*

  # Log configmap and secret changes in all namespaces at the metadata level.
  - level: Metadata
    resources:
    - resources: ["secrets", "configmaps"]

  # A catch-all rule to log all other requests at the request level.
  - level: Request

Restart the services after making changes.

sudo /usr/local/bin/master-restart api
sudo /usr/local/bin/master-restart controllers

If you are using OpenShift 3.9 or below, the command to restart the services is sudo systemctl restart atomic-openshift-master-api atomic-openshift-master-controllers

The application has a macro that defines how to find the audit logs: macro_openshift_audit_logs.

(`macro_openshift_host_logs` "audit.k8s.io")

Links

• Installation
  • Start monitoring your OpenShift environments in under 10 minutes.
  • Automatically forward host, container and application logs.
  • Test our solution with the embedded 30 days evaluation license.
• Collectord Configuration
  • Collectord configuration reference.
• Annotations
  • Changing index, source, sourcetype for namespaces, workloads and pods.
  • Forwarding application logs.
  • Multi-line container logs.
  • Fields extraction for application and container logs (including timestamp extractions).
  • Hiding sensitive data, stripping terminal escape codes and colors.
  • Forwarding Prometheus metrics from Pods.
• Audit Logs
  • Configure audit logs.
  • Forwarding audit logs.
• Prometheus metrics
  • Collect metrics from control plane (etcd cluster, API server, kubelet, scheduler, controller).
  • Configure the collectord to forward metrics from the services in Prometheus format.
• Configuring Splunk Indexes
  • Using not default HTTP Event Collector index.
  • Configure the Splunk application to use not searchable by default indexes.
• Splunk fields extraction for container logs
  • Configure search-time field extractions for container logs.
  • Container logs source pattern.
• Configurations for Splunk HTTP Event Collector
  • Configure multiple HTTP Event Collector endpoints for Load Balancing and Fail-overs.
  • Secure HTTP Event Collector endpoint.
  • Configure the Proxy for HTTP Event Collector endpoint.
• Monitoring multiple clusters
  • Learn how to monitor multiple clusters.
  • Learn how to set up ACL in Splunk.
• Streaming OpenShift Objects from the API Server
  • Learn how to stream all changes from the OpenShift API Server.
  • Stream changes and objects from OpenShift API Server, including Pods, Deployments or ConfigMaps.
• License Server
  • Learn how to configure a remote License URL for Collectord.
• Monitoring GPU
• Alerts
• Troubleshooting
• Release History
• Upgrade instructions
• Security
• FAQ and the common questions
• License agreement
• Pricing
• Contact