[helm/elastic-agent] upgrade helm chart to collect rotated logs

Upgates the elastic-agent helm chart to allow the collection of
rotated logs, including the GZIP-compressed logs.

Author: AndersonQ

changelog/fragments/1762936912-upgrade-helm-chart-to-collect-rotated-logs.yaml

@@ -0,0 +1,32 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: feature
+
+# Change summary; a 80ish characters long description of the change.
+summary: upgrade helm chart to collect rotated logs
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
+#description:
+
+# Affected component; a word indicating the component this changeset affects.
+component: helm-chart
+
+# PR URL; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: https://github.com/elastic/elastic-agent/pull/11129
+
+# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+issue: https://github.com/elastic/elastic-agent/issues/10889

deploy/helm/elastic-agent/README.md

@@ -87,6 +87,7 @@ The chart built-in [kubernetes integration](https://docs.elastic.co/integrations
 | kubernetes.containers.state.enabled | bool | `true` | enable containers state stream (kube-state-metrics) [ref](https://www.elastic.co/guide/en/beats/metricbeat/8.11/metricbeat-metricset-kubernetes-state_container.html) |
 | kubernetes.containers.state.vars | object | `{}` | containers state stream vars |
 | kubernetes.containers.logs.enabled | bool | `true` | enable containers logs stream [ref](https://www.elastic.co/docs/current/integrations/kubernetes/container-logs) |
+| kubernetes.containers.logs.rotated_logs | bool | `false` | enable rotated logs ingestion [ref](https://www.elastic.co/docs/reference/integrations/kubernetes/container-logs#ingesting-rotated-container-logs) |
 | kubernetes.containers.logs.vars | object | `{}` | containers logs stream vars |
 | kubernetes.containers.audit_logs.enabled | bool | `false` | enable containers audit logs stream [ref](https://www.elastic.co/docs/current/integrations/kubernetes/audit-logs) |
 | kubernetes.pods.metrics.enabled | bool | `true` | enable pods metric stream (kubelet) [ref](https://www.elastic.co/docs/current/integrations/kubernetes/kubelet#pod) |

deploy/helm/elastic-agent/templates/integrations/_kubernetes/_kubernetes_logs_containers.tpl

@@ -17,12 +17,19 @@ Config input for container logs
     namespace: {{ .Values.kubernetes.namespace }}
   use_output: {{ .Values.kubernetes.output }}
   streams:
+  {{- if $.Values.kubernetes.containers.logs.rotated_logs }}
+  - id: kubernetes-container-logs-${kubernetes.pod.uid}-${kubernetes.container.name}
+    gzip_experimental: true
+    paths:
+      - '/var/log/pods/${kubernetes.namespace}_${kubernetes.pod.name}_${kubernetes.pod.uid}/${kubernetes.container.name}/*.log*'
+  {{ else }}
   - id: kubernetes-container-logs-${kubernetes.pod.name}-${kubernetes.container.id}
+    paths:
+      - '/var/log/containers/*${kubernetes.container.id}.log'
+  {{ end }}
     data_stream:
       dataset: kubernetes.container_logs
       type: logs
-    paths:
-      - '/var/log/containers/*${kubernetes.container.id}.log'
     prospector.scanner.symlinks: {{ dig "vars" "symlinks" true .Values.kubernetes.containers.logs }}
     parsers:
       - container:

deploy/helm/elastic-agent/values.yaml

@@ -151,6 +151,8 @@ kubernetes:
       # -- enable containers logs stream [ref](https://www.elastic.co/docs/current/integrations/kubernetes/container-logs)
       # @section -- 2 - Kubernetes integration
       enabled: true
+      # -- enable rotated logs ingestion [ref](https://www.elastic.co/docs/reference/integrations/kubernetes/container-logs#ingesting-rotated-container-logs)
+      rotated_logs: false
       # -- containers logs stream vars
       # @section -- 2 - Kubernetes integration
       vars: {}