Install (Enterprise Edition)
SaaS

Installing Agent in K8s

Created：2024-07-03 Last Modified：2024-08-23

This document was translated by ChatGPT

#1. Introduction

This chapter provides a detailed guide on how to deploy the DeepFlow Agent in your K8s cluster.

After deploying the DeepFlow Agent in the K8s cluster, it will automatically and non-intrusively collect observability data (AutoMetrics, AutoTracing, AutoProfiling) from the Pods in the K8s cluster and automatically inject K8s resources and K8s custom labels tags into all observability data (AutoTagging).

#2. Deployment Topology

Deployment Topology

#3. Define Your Scenario

DeepFlow observability covers two types of K8s clusters:

Attached container clusters: These are container clusters where the Nodes use cloud servers learned by DeepFlow from a certain cloud platform.
Independent container clusters: These are container clusters where the Nodes are not associated with cloud servers learned by DeepFlow from a cloud platform.

For attached container clusters, you need to automatically or manually input the container cluster information on the Resources - Cloud Platform page of DeepFlow so that the DeepFlow Agent in the container cluster can be correctly recognized by the DeepFlow Server.

For independent container clusters, the DeepFlow Server can automatically create the cluster after receiving the first DeepFlow Agent registration information from the cluster, without the need for operations personnel to intervene.

注意

If the cloud servers of the K8s cluster have already been automatically learned and recorded by the cloud platform, then the K8s cluster must be considered an attached container cluster of that cloud platform. Otherwise, DeepFlow's observability data tagging will not be able to determine the cloud platform to which the data belongs, leading to data tag confusion.

#4. Deploying Agent in Attached Container Clusters

#4.1 Input Attached Container Cluster

For managed K8s clusters on the following public clouds, DeepFlow can automatically learn and input K8s cluster information through public cloud APIs without manual operation:

Alibaba Cloud
Baidu Cloud
AWS
QingCloud

The automatic input results of attached container clusters can be viewed on the DeepFlow web page under Resources - Resource Pool - Cloud Platform - Attached Container Clusters.

For self-built K8s clusters by public cloud tenants or managed K8s clusters outside the above public clouds, you need to manually input the attached container cluster information on the DeepFlow Resources - Resource Pool - Cloud Platform - Attached Container Clusters page:

Input Attached Container Cluster

Before deploying the DeepFlow Agent, you also need to obtain 3 ID values from the DeepFlow web page and configure them in the DeepFlow Agent's configuration file.

#4.2 Obtain teamId

teamId is the Team ID, used to identify the organization to which the DeepFlow Agent belongs. Refer to the following steps:

Obtain teamId (Team ID)

#4.3 Obtain agentGroupID

agentGroupId is the Collector Group ID, used to identify the collector group to which the DeepFlow Agent belongs. Refer to the following steps:

Obtain agentGroupId (Collector Group ID)

提示

The purpose of creating a collector group is to configure different operating policies for grouped management of DeepFlow Agents.

#4.4 Obtain deepflowK8sClusterID

deepflowK8sClusterID is the Container Cluster ID, used to identify the attached container cluster to which the DeepFlow Agent belongs. Refer to the following steps:

Obtain deepflowK8sClusterID (Container Cluster ID)

#4.5 Install Agent Using Helm

Initialize the Helm Repo, create a values-custom.yaml file, and write the key field information required for DeepFlow Agent deployment into the file:

helm repo add deepflow https://deepflowio.github.io/deepflow
helm repo update deepflow # use `helm repo update` when helm < 3.7.0
cat << EOF > values-custom.yaml
deepflowServerNodeIPS:
- agent.cloud.deepflow.yunshan.net
teamId: "t-xxxxxxxxxx"  # Mandatory. Filling with the 'teamId' which could be found in DeepFlow Cloud.
agentGroupID: "g-xxxxxxxxxx"  # Optional. Filling with the 'agentGroupID' which could be found in DeepFlow Cloud.
deepflowK8sClusterID: "d-xxxxxxxxxx"  # Mandatory to standalone Kubernetes cluster. Filling with the Kubernetes ClusterID which could be found in DeepFlow Cloud.
image:
  repository: deepflowee/deepflow-agent
  pullPolicy: Always
  tag: v6.5
EOF

1
2
3
4
5
6
7
8
9
10
11
12
13

helm repo add deepflow https://deepflow-ce.oss-cn-beijing.aliyuncs.com/chart/stable
helm repo update deepflow # use `helm repo update` when helm < 3.7.0
cat << EOF > values-custom.yaml
deepflowServerNodeIPS:
- agent.cloud.deepflow.yunshan.net
teamId: "t-xxxxxxxxxx"  # Mandatory. Filling with the 'teamId' which could be found in DeepFlow Cloud.
agentGroupID: "g-xxxxxxxxxx"  # Optional. Filling with the 'agentGroupID' which could be found in DeepFlow Cloud.
deepflowK8sClusterID: "d-xxxxxxxxxx"  # Mandatory to standalone Kubernetes cluster. Filling with the Kubernetes ClusterID which could be found in DeepFlow Cloud.
image:
  repository: hub.deepflow.yunshan.net/public/deepflow-agent
  pullPolicy: Always
  tag: v6.5
EOF

1
2
3
4
5
6
7
8
9
10
11
12
13

The key field values are explained as follows:

Field	Purpose	Value	Exception Description
`deepflowServerNodeIPS`	The address of the DeepFlow Server that the DeepFlow Agent will connect to	`agent.cloud.deepflow.yunshan.net`	If this address is incorrect, the DeepFlow Agent will fail to register
`teamId`	The `Team ID`, used by the DeepFlow Server to determine the organization to which the DeepFlow Agent belongs	Obtained from the DeepFlow page	If this ID value is incorrect, the DeepFlow Agent will fail to register
`agentGroupID`	The Collector Group ID, used by the DeepFlow Server to issue operating policies	Obtained from the DeepFlow page	If this ID value is invalid, the DeepFlow Server will issue the default group operating policy
`deepflowK8sClusterID`	The Container Cluster ID, used by the DeepFlow Server to determine the attached container cluster to which the DeepFlow Agent belongs	Obtained from the DeepFlow page	If this ID is incorrect, the DeepFlow Agent will fail to register

Install the deepflow-agent using Helm:

helm install deepflow-agent -n deepflow deepflow/deepflow-agent --create-namespace \
    -f values-custom.yaml

1
2

#5. Deploying Agent in Independent Container Clusters

Before deploying the DeepFlow Agent, you need to obtain 2 ID values from the DeepFlow web page and configure them in the DeepFlow Agent's configuration file.

#5.1 Obtain teamId

teamId is the Team ID, used to identify the organization to which the DeepFlow Agent belongs. Refer to the following steps:

Obtain teamId (Team ID)

#5.2 Obtain agentGroupID

agentGroupId is the Collector Group ID, used to identify the collector group to which the DeepFlow Agent belongs. Refer to the following steps:

Obtain agentGroupId (Collector Group ID)

提示

The purpose of creating a collector group is to configure different operating policies for grouped management of DeepFlow Agents.

#5.3 Install Agent Using Helm

Initialize the Helm Repo, create a values-custom.yaml file, and write the key field information required for DeepFlow Agent deployment into the file:

helm repo add deepflow https://deepflowio.github.io/deepflow
helm repo update deepflow # use `helm repo update` when helm < 3.7.0
cat << EOF > values-custom.yaml
deepflowServerNodeIPS:
- agent.cloud.deepflow.yunshan.net
teamId: "t-xxxxxxxxxx"  # Mandatory. Filling with the 'teamId' which could be found in DeepFlow Cloud.
agentGroupID: "g-xxxxxxxxxx"  # Optional. Filling with the 'agentGroupID' which could be found in DeepFlow Cloud.
clusterNAME: "k8s-cluster-1" # Optional. Filling with a Kubernetes cluster name which you want.
image:
  repository: deepflowee/deepflow-agent
  pullPolicy: Always
  tag: v6.5
EOF

1
2
3
4
5
6
7
8
9
10
11
12
13

helm repo add deepflow https://deepflow-ce.oss-cn-beijing.aliyuncs.com/chart/stable
helm repo update deepflow # use `helm repo update` when helm < 3.7.0
cat << EOF > values-custom.yaml
deepflowServerNodeIPS:
- agent.cloud.deepflow.yunshan.net
teamId: "t-xxxxxxxxxx"  # Mandatory. Filling with the 'teamId' which could be found in DeepFlow Cloud.
agentGroupID: "g-xxxxxxxxxx"  # Optional. Filling with the 'agentGroupID' which could be found in DeepFlow Cloud.
clusterNAME: "k8s-cluster-1" # Optional. Filling with a Kubernetes cluster name which you want.
image:
  repository: hub.deepflow.yunshan.net/public/deepflow-agent
  pullPolicy: Always
  tag: v6.5
EOF

1
2
3
4
5
6
7
8
9
10
11
12
13

The key field values are explained as follows:

Field	Purpose	Value	Exception Description
`deepflowServerNodeIPS`	The address of the DeepFlow Server that the DeepFlow Agent will connect to	`agent.cloud.deepflow.yunshan.net`	If this address is incorrect, the DeepFlow Agent will fail to register
`teamId`	The `Team ID`, used by the DeepFlow Server to determine the organization to which the DeepFlow Agent belongs	Obtained from the DeepFlow page	If this ID value is incorrect, the DeepFlow Agent will fail to register
`agentGroupID`	The Collector Group ID, used by the DeepFlow Server to issue operating policies	Obtained from the DeepFlow page	If this ID value is invalid, the DeepFlow Server will issue the default group operating policy
`clusterNAME`	The name of the container cluster for user identification, used by the DeepFlow Server to automatically create the container cluster and ID	Named flexibly as needed	If different K8s clusters' DeepFlow Agent configurations have the same `clusterNAME`, it will cause registration anomalies

Install the deepflow-agent using Helm:

helm install deepflow-agent -n deepflow deepflow/deepflow-agent --create-namespace \
    -f values-custom.yaml

1
2