This document was translated by ChatGPT

1. Data Flow

2. Configure the Application

Currently, DeepFlow supports integration of Profile data (continuous profiling data) based on Pyroscope (opens new window), Golang pprof, and Java Jfr.

2.1 Based on Pyroscope SDK

DeepFlow currently supports receiving Profile data sent via the Pyroscope SDK. You can find SDKs for supported languages in the Pyroscope SDKs (opens new window) documentation and instrument your code accordingly.

After modifying the code, set the target sending address to the DeepFlow Agent via environment variables in the application runtime environment. For example, in a K8S deployment, add the following environment variables to the deployment file:

env:
  - name: PYROSCOPE_SERVER_ADDRESS
    value: http://deepflow-agent.deepflow/api/v1/profile

In addition, to identify different data sources in DeepFlow, you need to explicitly specify the application name by adding the following environment variable:

env:
  - name: PYROSCOPE_APPLICATION_NAME
    value: application-demo # FIXME: your application name

2.2 Based on Golang pprof

Profile data collected via Golang pprof can also be sent to DeepFlow, but you need to manually add the logic for active reporting. You can use "net/http/pprof" (opens new window) to collect Profile data and expose a data download service. After the client requests the /debug/pprof/profile endpoint to obtain the target pprof data, send it to DeepFlow.

Below is a sample code snippet for building the reporting logic:

func main() {
  // Note: the actual URL used here is `/api/v1/profile/ingest`
  deepflowAgentAddress = "http://deepflow-agent.deepflow/api/v1/profile/ingest"
  var pprof io.Reader // FIXME: This is an example, please first obtain pprof data from `/debug/pprof/profile`
	err = sendProfileData(pprof, deepflowAgentAddress)
	if err != nil {
		fmt.Println(err)
	}
}

// Build and send the request
func sendProfileData(pprof io.Reader, remoteURL string) error {
	bodyBuf := &bytes.Buffer{}
	bodyWriter := multipart.NewWriter(bodyBuf)
	pprofWriter, err := bodyWriter.CreateFormFile("profile", "profile.pprof")
	if err != nil {
		return err
	}

	_, err = io.Copy(pprofWriter, pprof)
	if err != nil {
		return err
	}

	err = bodyWriter.Close()
	if err != nil {
		return err
	}

	u, err := url.Parse(remoteURL)
	if err != nil {
		return err
	}

	q := u.Query()
	q.Set("spyName", "gospy")                            // hardcode, no need to modify
	q.Set("name", "application-demo")                    // FIXME: your application name
	q.Set("unit", "samples");                            // FIXME: unit, see explanation below
	q.Set("from", strconv.Itoa(int(time.Now().Unix())))  // FIXME: profile start time
	q.Set("until", strconv.Itoa(int(time.Now().Unix()))) // FIXME: profile end time
	q.Set("sampleRate", "100")                           // FIXME: actual profile sampling rate, 100=1s/10ms, i.e., sample every 10ms
	u.RawQuery = q.Encode()

	req, err := http.NewRequest(http.MethodPost, u.String(), bodyBuf)
	if err != nil {
		return err
	}
	req.Header.Set("Content-Type", bodyWriter.FormDataContentType())

	// Send the request
	client := http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		return err
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return fmt.Errorf("server returned non-OK status: %s", resp.Status)
	}

	return nil
}

2.3 Based on Java Async Profiler

For Java applications, we support receiving Profile data in JFR (opens new window) format. Use Java's built-in jcmd (opens new window) or async-profiler (opens new window) to collect Profile data, generate Jfr format, and send it to DeepFlow.

Below is a sample code snippet for building the reporting logic:

import okhttp3.*;
import okio.BufferedSink;
import okio.GzipSink;
import okio.Okio;

import java.io.IOException;

public class Sender {
	// Note: the actual URL used here is `/api/v1/profile/ingest`
    private static final String DEEPFLOW_AGENT_ADDRESS = "http://deepflow-agent.deepflow/api/v1/profile/ingest";

    public static void main(String[] args) throws IOException {
        sendProfileData(DEEPFLOW_AGENT_ADDRESS);
    }

    private static void sendProfileData(String remoteURL) throws IOException {
        OkHttpClient client = new OkHttpClient();

        byte[] data = new byte[] {}; // FIXME: This is an example, please use a profiling tool to obtain Jfr content before reporting
        MediaType mediaType = MediaType.parse("application/octet-stream");

        RequestBody requestBody = new RequestBody() {
            @Override
            public MediaType contentType() {
                return mediaType;
            }

			// Use Gzip compression for transmission
            @Override
            public void writeTo(BufferedSink sink) throws IOException {
                BufferedSink gzipSink = Okio.buffer(new GzipSink(sink));
                gzipSink.write(data);
                gzipSink.close();
            }
        };

        HttpUrl.Builder urlBuilder = HttpUrl.parse(remoteURL).newBuilder();
        urlBuilder.addQueryParameter("name", "application-demo"); // FIXME: your application name
        urlBuilder.addQueryParameter("spyName", "javaspy"); // hardcode, no need to modify
        urlBuilder.addQueryParameter("format", "jfr"); // hardcode, no need to modify
        urlBuilder.addQueryParameter("unit", "samples"); // FIXME: unit, see explanation below
        urlBuilder.addQueryParameter("from", String.valueOf(System.currentTimeMillis() / 1000)); // FIXME: profile start time
        urlBuilder.addQueryParameter("until", String.valueOf(System.currentTimeMillis() / 1000));// FIXME: profile end time
        urlBuilder.addQueryParameter("sampleRate", "100"); // FIXME: actual profile sampling rate, 100=1s/10ms, i.e., sample every 10ms
        String urlWithQueryParams = urlBuilder.build().toString();

        Request request = new Request.Builder()
                .url(urlWithQueryParams)
                .post(requestBody)
                .build();

        try (Response response = client.newCall(request).execute()) {
            if (response.isSuccessful()) {
                System.out.println("Profile data sent successfully.");
            } else {
                System.err.println("server returned non-OK status: " + response.code());
            }
        }
    }
}

2.4 Reporting Parameter Description

Name	Type	Description
name	string	Application name, used to identify reported data. You can also add custom labels to distinguish different deployment specs of the same application, e.g., application-demo{region="cn",deploy="prod"}
spyName	string	Used to indicate the type of reported data. For Golang applications, fixed as gospy; for Java applications, fixed as javaspy
format	string	Profile data format. Golang pprof format is pprof (default), Java Jfr format is jfr
unit	string	Unit of measurement. Different sampling types have different units. See here (opens new window) for details. For cpu, use samples; for memory, use bytes; others are similar
from	int	Profile start time, Unix timestamp (seconds)
until	int	Profile end time, Unix timestamp (seconds)
sampleRate	int	Actual profile sampling rate

3. Configure DeepFlow

Refer to the Configure DeepFlow section to complete the DeepFlow Agent configuration and enable the data integration port.

4. Experience with Demo

Use the following commands to quickly deploy a demo and experience the continuous profiling capability in DeepFlow:

kubectl apply -f https://raw.githubusercontent.com/deepflowio/deepflow-demo/main/DeepFlow-CP-Demo/deepflow-cp-golang-demo.yaml

kubectl apply -f https://raw.githubusercontent.com/deepflowio/deepflow-demo/main/DeepFlow-CP-Demo/deepflow-cp-java-demo.yaml

Then, for the community edition, refer to the Continuous Profiling - View Data section to access the data generated by continuous profiling.