v6.4 CE Release Notes

Created:2024-02-27 Last Modified:2024-07-05

This document was translated by ChatGPT

#1. v6.4.9 [2024/01/18]

#1.1 New Features

  • API
    • SQL API adds the Derivative pre-operator to calculate the difference for Prometheus Counter type metrics, thereby calculating the rate.
    • SQL API adds TopK and Any operators to obtain the high-frequency or any value of a specified Tag.
  • CLI
    • Support debugging eBPF Socket Data using deepflow-ctl.
  • Agent

#1.2 Optimizations

#2. v6.4.8 [2024/01/11]

N/A

#3. v6.4.7 [2024/01/04]

#3.1 New Features

  • AutoTagging
    • Enhanced the ability to parse HTTP2 compressed headers, supporting decompression and restoration of cBPF and eBPF kprobe data.

#4. v6.4.6 [2023/12/28]

#4.1 Optimizations

  • AutoProfiling
    • When enabling Profiling, there is no need to generate symbol table files in the Pod where the Java process is located.
  • Server
    • Added trace_id_index integer column in ClickHouse as an index column for the trace_id field, supporting the extraction of Timestamps to accelerate TraceID searches.
  • Wasm
    • Wasm Plugin supports dynamic loading in the Agent.
  • CLI
    • Added list command: deepflow-ctl domain additional-resource list --type <resource_type> --name <resource_name>.

#5. v6.4.5 [2023/12/21]

#5.1 New Features

  • AutoMetrics
    • MySQL, PostgreSQL, and Redis protocol data support obfuscation, which can be enabled through the Agent's obfuscate-enabled-protocols (thanks to eddietanshuo: FR (opens new window)).
  • AutoTagging
    • Support injecting custom auto-grouping tags into all observability data, achieving the ability to automatically take the first non-empty tag by combining multiple tag fields and ignoring subsequent tag columns.
  • Integration
    • Server supports exporting metrics through the Prometheus RemoteWrite protocol (thanks to chenjiandongx: PR (opens new window)).

#5.2 Optimizations

  • Server
    • Optimized K8s Label synchronization mechanism, supporting setting regular expression filters for interested tags and limiting the maximum length of tag values.
    • Upgraded ClickHouse to v23.8 (LTS).

#6. v6.4.4 [2023/12/07]

#6.1 New Features

  • AutoMetrics
    • Support parsing Geneve tunnel encapsulation (adapted to Kube-OVN).
  • AutoTagging
    • Support precise setting of Pod tags for all eBPF observability data of HostNetwork Pods.
    • Added gprocess process information tags for continuous profiling data.
    • Support synchronization of Huawei public cloud RDS and Redis resources.
    • Support synchronization of Baidu public cloud Redis resources.
  • AutoProfiling
    • Support displaying a universal CPU flame graph of all processes on a server, down to the thread granularity.
    • Generate symbol tables for Java processes in a staggered manner to avoid high load caused by clustering, supporting the modification of the Agent's java-symbol-file-refresh-defer-interval configuration item to adjust the base interval of the delay.
  • Server
    • Support balancing the data sending Server based on the amount of data sent by the Agent, improving the data volume balance of ClickHouse (thanks to jiekun: FR (opens new window)).

#6.2 Optimizations

#7. v6.4.3 [2023/11/23]

#7.1 New Features

  • AutoMetrics
    • Support parsing all encrypted application protocols over TLS, not limited to HTTP (thanks to dirtyren: FR (opens new window)).
  • AutoTagging
    • Support synchronization of tag information for cloud servers in Tencent public cloud and Huawei public cloud, and support synchronization of resource set information for cloud servers in Alibaba public cloud.

#7.2 Optimizations

  • AutoMetrics
    • Simplified connection metrics in flow logs, removing redundant rtt_client_avg and rtt_server_avg.
    • Optimized the default parsing port for DNS traffic, adding 5353, see the Agent's l7-protocol-ports configuration item.
  • AutoTagging
    • Deprecated the Others type in l7_protocol (application protocol), merging HTTP_TLS and HTTP2_TLS into HTTP and HTTP2, adding is_tls in call logs to indicate whether it is encrypted traffic.

#8. v6.4.2 [2023/11/09]

#8.1 New Features

  • AutoTracing
    • Support extracting trace_id from SofaRPC Payload (Hessian encoding, TreeMap structure).
    • Support extracting TraceID fields injected by SkyWalking and OpenTelemetry in Kafka messages.
    • Support parsing SkyWalking sw3 Header (thanks to Wadud-Ma: FR (opens new window), PR (opens new window)).
  • AutoTagging
    • Support extracting the topic_name field from Kafka messages and assigning it to the request_resource in call logs (thanks to mickeyzzc: FR (opens new window)).
    • Support extracting endpoints from HTTP URLs and assigning them to call logs and application performance metrics data, supporting the configuration of http-endpoint-extraction extraction rules for the Agent (thanks to DomineCore: FR (opens new window)).

#8.2 Optimizations

  • AutoMetrics
    • Moved the implementation of the ProtobufRPC protocol from Rust code to the Wasm Plugin.
    • Used eBPF to obtain the Accept/Connect types of Sockets, correcting the direction of call logs and application performance metrics.
  • Agent
    • Used the TCP protocol to transmit the Agent's own logs.
  • Server
    • Merged self-monitoring metrics of Agent and Server into a single table deepflow_system.deepflow_system in ClickHouse.
  • API
    • Support deleting cloud platforms by name.

#8.3 Community PRs

#9. v6.4.1 [2023/10/26]

#9.1 New Features

  • AutoMetrics
    • Support parsing the MongoDB protocol (thanks to mickeyzzc: FR (opens new window), PR (opens new window)).
    • In flow logs and network performance metrics, system latency (srt, srt_max) supports recording the latency of ICMP traffic and correcting the direction of ICMP traffic records using ICMP Echo messages.
  • AutoTagging
    • Added pod_group_type (K8s workload type) tag field to all observability data, extending the values of auto_service_type to represent the type of K8s workload.
    • Added coroutine ID fields syscall_coroutine_0, syscall_coroutine_1 to call logs.

#9.2 Community PRs

#10. v6.4.0 [2023/10/12]

#10.1 Optimizations

  • Server
    • ClickHouse uses Array(LowCardinality(String)) instead of Array(String) to optimize the read and write performance of low cardinality fields, such as tag_names, metrics_names, etc.
    • Profiling data supports plaintext (non-compressed) storage in ClickHouse (thanks to tailerong: FR (opens new window)).
  • CLI

#10.2 Community PRs