• Integration
  • Output
  • Exporter

Prometheus Remote Write

Created：2023-12-07 Last Modified：2024-06-28

This document was translated by ChatGPT

#1. Functionality

Using Prometheus Remote Write, you can export metrics generated by DeepFlow to external platforms. This allows you to continue leveraging the Prometheus ecosystem, such as viewing metrics and configuring alerts through Prometheus.

#2. Metrics Overview

Within DeepFlow, metrics can be categorized into two types:

  • Application Performance Metrics: Refer to details
    • Corresponds to flow_metrics.application* table data in ClickHouse
  • Network Performance Metrics: Refer to details
    • Corresponds to flow_metrics.network* table data in ClickHouse

#3. Prometheus Remote Write

For protocol format, refer to Prometheus's pb file definition: https://github.com/prometheus/prometheus/blob/main/prompb/remote.proto

#4. DeepFlow Server Configuration Guide

Add the following configuration under the Server settings to enable metric export:

ingester:
  exporters:
    - protocol: prometheus
      enabled: true
      endpoints: [http://127.0.0.1:9091/receive, http://1.1.1.1:9091/receive]
      data-sources:
        - flow_metrics.application_map.1s
      # - flow_metrics.application_map.1m
      # - flow_metrics.application.1s
      # - flow_metrics.application.1m
      # - flow_metrics.network_map.1s
      # - flow_metrics.network_map.1m
      # - flow_metrics.network.1s
      # - flow_metrics.network.1m
      queue-count: 4
      queue-size: 100000
      batch-size: 1024
      flush-timeout: 10
      tag-filters:
      export-fields:
        - $tag
        - $metrics
      extra-headers:
        key1: value1
        key2: value2
      export-empty-tag: false
      export-empty-metrics-disabled: false
      enum-translate-to-name-disabled: false
      universal-tag-translate-to-name-disabled: false
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

#5. Detailed Parameter Description

Field Type Required Description
protocol string Yes Fixed value prometheus
data-sources string Yes Values from flow_metrics.* data, does not support flow_log.* data
endpoints string Yes Remote receiving address, remote write receiving address, randomly selects one that can send successfully
batch-size int No Batch size, sends in batches when this value is reached. Default: 1024
extra-headers map No Header fields for remote HTTP requests, such as tokens for authentication
export-fields string Yes Currently does not support $k8s.label, recommended configuration: [$tag, $metrics]

Refer to detailed configuration

#6. Quick Practice Demo

exporters:
  - protocol: prometheus
    data-sources:
      - flow_metrics.application_map.1s
    endpoints: [http://localhost:1234/receive]
    export-fields:
      - $tag
      - $metrics
1
2
3
4
5
6
7
8
  • Restart DeepFlow Server, and after a short wait, you should see the output results at the RemoteWrite receiver as shown in the image