Application topology (#114)

Author: paliwalparitosh

CHANGELOG.md

@@ -1,5 +1,17 @@
 # Change Log
 
+# 2025-06-17
+### Added
+- Introduced a new DaemonSet that uses eBPF (Extended Berkeley Packet Filter) to capture TCP connection logs and builds application/network topology representing workload to workload relationships within the Kubernetes cluster.
+  - To be able to run the required eBPF program, the pods needs to run in privileged mode but restricting to CAP_BPF capability only.
+- New helm variable to control the resource limits at individual logan workloads.
+- Enables OKE infra discovery and service logs collection (default)
+- OCI Console integration supporting new features:
+  - Topology : New Views (Infra and Network) along with Platform.
+  - View Insights for Workloads including capabilities to view the detailed spec of a workload, monitor the changes to the spec of a workload, create in-line labels for issues etc.
+
+### Changed
+- `kubernetesClusterID` (in the Helm chart) is now a mandatory field. *(This is not backward compatible.)*
 
 ## 2025-03-19
 ### Added

README.md

@@ -36,7 +36,7 @@ It does extensive enrichment of logs, metrics and object information to enable c
 
 ## Get Started :rocket:
 
-:stop_sign: Upgrading to a major version (like 2.x to 3.x)? See [upgrade](#upgrading-to-a-major-version) section below for details. :warning:
+:stop_sign: Upgrading to a major version (like 3.x to 4.x)? See [upgrade](#upgrading-to-a-major-version) section below for details. :warning:
 
 ### Pre-requisites
 
@@ -367,6 +367,34 @@ Refer [here](#3c-import-dashboards).
 
 ### Upgrading to a major version
 
+#### 3.6.0 to 4.0.0
+
+For changes in this release, refer to [CHANGELOG.md](CHANGELOG.md)
+
+##### Upgrade instructions
+
+1. Update IAM Policies:
+    * This version requires additional policy statements for infrastructure discovery.
+    * See the pre-requisites section in the [README](../README.md#0-pre-requisites) for details.
+
+1. As mentioned in the change log, this version introduces a new DaemonSet that uses eBPF (Extended Berkeley Packet Filter) to capture TCP connection logs and builds application/network topology representing workload to workload relationships within the Kubernetes cluster.
+    * To be able to run the required eBPF program, the pods needs to run in privileged mode but restricting to CAP_BPF capability only.
+    * In your environment, if you have any restrictions with respect to running pods in privileged mode, you may need to adjust your cluster configuration accordingly.
+
+2. Upgrade the Helm chart:
+
+      ```sh
+      # fetch latest (4.x) helm repo for oci
+      helm repo update oci-onm
+
+      # fetch the current release configuration
+      helm get values <release-name> -n <namespace> > override_values.yaml
+
+      # Upgrade the helm chart
+      helm upgrade <release-name> oci/oci-onm -n <namespace> -f override_values.yaml
+      ```
+
+
 #### 2.x to 3.x
 
 One of the major changes introduced in 3.0.0 is refactoring of helm chart where major features of the solution got split into separate sub-charts. 2.x has only support for logs and objects collection using Fluentd and OCI Logging Analytics and this is now moved into a separate chart oci-onm-logan and included as a sub-chart to the main chart oci-onm. This is a breaking change w.r.t the values.yaml and any customisations that you might have done on top of it. There is no breaking change w.r.t functionality offered in 2.x. For full list of changes in 3.x, refer to [changelog](CHANGELOG.md). 

charts/logan/Chart.yaml

@@ -5,7 +5,7 @@ apiVersion: v2
 name: oci-onm-logan
 description: Charts for sending Kubernetes platform logs, compute logs, and Kubernetes Objects information to OCI Logging Analytics.
 type: application
-version: 3.6.0
+version: 4.0.0
 appVersion: "3.0.0"
 
 dependencies:

charts/logan/templates/_helpers.tpl

@@ -1,5 +1,5 @@
 
-# Copyright (c) 2023, 2024, Oracle and/or its affiliates.
+# Copyright (c) 2023, 2025, Oracle and/or its affiliates.
 # Licensed under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl.
 
 # tpl render function
@@ -43,6 +43,11 @@
   {{- end -}}
 {{- end -}}
 
+#ociLAClusterEntityID
+{{- define "logan.ociLAClusterEntityID" -}}
+  {{ include "common.tplvalues.render" ( dict "value" .Values.ociLAClusterEntityID "context" .) }}
+{{- end -}}
+
 #kubernetesClusterName
 {{- define "logan.kubernetesClusterName" -}}
   {{- if .Values.kubernetesClusterName -}}

charts/logan/templates/discovery-cronjob.yaml

@@ -1,3 +1,5 @@
+# Copyright (c) 2025, Oracle and/or its affiliates.
+# Licensed under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl.
 {{- $authtype := .Values.authtype | lower }}
 {{- $resourceNamePrefix := .Values.global.resourceNamePrefix }}
 {{- $kubernetesClusterName := (include "logan.kubernetesClusterName" .) }}

charts/logan/templates/fluentd-daemonset.yaml

@@ -1,4 +1,4 @@
-# Copyright (c) 2023, 2024, Oracle and/or its affiliates.
+# Copyright (c) 2023, 2025, Oracle and/or its affiliates.
 # Licensed under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl.
 
 ---
@@ -69,7 +69,9 @@ spec:
           {{- if .Values.extraEnv }}
           {{- toYaml .Values.extraEnv | nindent 10 }}
           {{- end }}
-        {{- if .Values.resources }}
+        {{- if .Values.resourceOverrides.fluentdDaemonset }}
+        resources: {{- toYaml .Values.resourceOverrides.fluentdDaemonset | nindent 10 }}
+        {{- else if .Values.resources }}
         resources: {{- toYaml .Values.resources | nindent 10 }}
         {{- end }}
         volumeMounts:

charts/logan/templates/fluentd-deployment.yaml

@@ -1,4 +1,4 @@
-# Copyright (c) 2023, 2024, Oracle and/or its affiliates.
+# Copyright (c) 2023, 2025, Oracle and/or its affiliates.
 # Licensed under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl.
 
 ---
@@ -61,7 +61,9 @@ spec:
           {{- if .Values.extraEnv }}
           {{- toYaml .Values.extraEnv | nindent 10 }}
           {{- end }}
-        {{- if .Values.resources }}
+        {{- if .Values.resourceOverrides.fluentdDeployment }}
+        resources: {{- toYaml .Values.resourceOverrides.fluentdDeployment | nindent 10 }}
+        {{- else if .Values.resources }}
         resources: {{- toYaml .Values.resources | nindent 10 }}
         {{- end }}
         volumeMounts:

charts/logan/templates/logs-configmap.yaml

@@ -1,8 +1,9 @@
-# Copyright (c) 2023, 2024, Oracle and/or its affiliates.
+# Copyright (c) 2023, 2025, Oracle and/or its affiliates.
 # Licensed under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl.
-
+---
 {{- $kubernetesClusterName := (include "logan.kubernetesClusterName" .) }}
 {{- $kubernetesClusterId := (include "logan.kubernetesClusterId" .) }}
+{{- $ociLAClusterEntityID := (include "logan.ociLAClusterEntityID" .) }}
 apiVersion: v1
 kind: ConfigMap
 metadata:
@@ -86,12 +87,16 @@ data:
        @type record_transformer
        enable_ruby true
        <record>
+       {{- if eq $name "tcpconnect" }}
+       oci_la_metadata ${{"{{"}}"Kubernetes Cluster Name":"{{ $kubernetesClusterName }}", "Kubernetes Cluster ID": "{{ $kubernetesClusterId }}", "Kubernetes Cluster Entity ID": "{{ $ociLAClusterEntityID }}" {{- range $k, $v := $logDefinition.metadata }},{{ $k | quote }}: {{ $v | quote -}} {{- end }}{{"}}"}}
+       {{- else }}
        {{- if $logDefinition.metadata }}
        oci_la_metadata ${{"{{"}}"Kubernetes Cluster Name":"{{ $kubernetesClusterName }}", "Kubernetes Cluster ID": "{{ $kubernetesClusterId }}" {{- range $k, $v := $logDefinition.metadata }},{{ $k | quote }}: {{ $v | quote -}} {{- end }}{{"}}"}}
        {{- else if $.Values.fluentd.kubernetesSystem.metadata }}
        oci_la_metadata ${{"{{"}}"Kubernetes Cluster Name":"{{ $kubernetesClusterName }}", "Kubernetes Cluster ID": "{{ $kubernetesClusterId }}" {{- range $k, $v := $.Values.fluentd.kubernetesSystem.metadata }},{{ $k | quote }}: {{ $v | quote -}} {{- end }}{{"}}"}}
        {{- else }}
        oci_la_metadata ${{"{{"}}"Kubernetes Cluster Name":"{{ $kubernetesClusterName }}", "Kubernetes Cluster ID": "{{ $kubernetesClusterId }}" {{- range $k, $v := $.Values.metadata }},{{ $k | quote }}: {{ $v | quote -}} {{- end }}{{"}}"}}
+       {{- end -}}
        {{- end }}
        {{- if $logDefinition.ociLALogGroupID }}
        oci_la_log_group_id "{{ $logDefinition.ociLALogGroupID }}"

charts/logan/templates/tcpconnect-daemonset.yaml

@@ -0,0 +1,75 @@
+# Copyright (c) 2025, Oracle and/or its affiliates.
+# Licensed under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl.
+
+---
+{{- if .Values.enableTCPConnectLogs }}
+{{- $authtype := .Values.authtype | lower }}
+{{- $imagePullSecrets := .Values.image.imagePullSecrets }}
+{{- $resourceNamePrefix := (include "logan.resourceNamePrefix" .) }}
+apiVersion: apps/v1
+kind: DaemonSet
+metadata:
+  name: {{ $resourceNamePrefix }}-logan-tcpconnect
+  namespace: {{ include "logan.namespace" . }}
+  labels:
+    app: {{ $resourceNamePrefix }}-logan-tcpconnect
+    version: v1
+spec:
+  selector:
+    matchLabels:
+      app: {{ $resourceNamePrefix }}-logan-tcpconnect
+      version: v1
+  template:
+    metadata:
+      labels:
+        app: {{ $resourceNamePrefix }}-logan-tcpconnect
+        version: v1
+    spec:
+      serviceAccountName: {{ include "logan.serviceAccount" . }}
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        effect: NoSchedule
+      - key: node-role.kubernetes.io/control-plane
+        effect: NoSchedule
+      {{- if $imagePullSecrets }}
+      imagePullSecrets:
+      - name: {{ .Values.image.imagePullSecrets }}
+      {{- end}}
+      containers:
+      - name: {{ $resourceNamePrefix }}-logan-tcpconnect
+        image: {{ .Values.image.url }}
+        command:
+        - /bin/bash
+        - -c
+        - --
+        args:
+        - /usr/bin/tcpconnect -e
+        - -i 30
+        env:
+        - name: K8S_NODE_NAME
+          valueFrom:
+            fieldRef:
+              apiVersion: v1
+              fieldPath: spec.nodeName
+        {{- if .Values.resourceOverrides.tcpconnectDaemonset }}
+        resources: {{- toYaml .Values.resourceOverrides.tcpconnectDaemonset | nindent 10 }}
+        {{- else if .Values.resources }}
+        resources: {{- toYaml .Values.resources | nindent 10 }}
+        {{- end }}
+        imagePullPolicy: {{ default "IfNotPresent" .Values.image.imagePullPolicy }}
+        # The container runs in privileged mode, but with only the CAP_BPF capability enabled. 
+        # This allows it to execute the required BPF programs while maintaining a minimal security footprint.
+        securityContext:
+          capabilities:
+            add:
+            - CAP_BPF
+          privileged: true
+        terminationMessagePath: /dev/termination-log
+        terminationMessagePolicy: File
+        tty: true
+      dnsPolicy: ClusterFirst
+      restartPolicy: Always
+      schedulerName: default-scheduler
+      securityContext: {}
+      terminationGracePeriodSeconds: 30
+{{- end }}

charts/logan/values.schema.json

@@ -7,7 +7,8 @@
     "image",
     "ociLANamespace",
     "ociLALogGroupID",
-    "fluentd"
+    "fluentd",
+    "ociLAClusterEntityID"
   ],
   "properties": {
     "image": {
@@ -64,6 +65,9 @@
       "type": "string",
       "minLength": 3,
       "maxLength": 63
+    },
+    "ociLAClusterEntityID": {
+      "type": "string"
     }
   }
 }