Ctrlb团队发布开源日志分析工具，将海量日志压缩99.9%

Ctrlb团队发布了开源日志分析工具ctrlb-decompose，可将原始日志行转化为结构化模式、统计数据和异常标志1。该工具通过时间戳提取、CLP编码、Drain3聚类、变量提取与分类、统计累积、异常检测及评分与关联等处理流程实现数据规模缩减1。功能演示表明，1,247,831行日志可被压缩为43个模式，实现99.9%的压缩率1。

ctrlb-decompose支持多种运行方式和输出格式1。用户可将其作为命令行工具、WASM浏览器版本或Rust库使用，并可通过brew install、dpkg、git clone或Claude Code插件等方式安装1。该工具支持人类可读的ANSI终端输出、针对大语言模型优化的markdown格式以及JSON格式1。底层技术采用DDSketch量分位数、HyperLogLog++基数估计、储层采样和Drain3前缀树聚类等算法1。

Ctrlb-decompose, an open-source log analysis tool, has been released to compress massive volumes of raw log data into structured patterns, statistical summaries, and anomaly flags 1. The tool achieves a 99.9% reduction in data scale, enabling more efficient processing of logs before sending them to large language models 1.

The tool operates through a multi-stage processing pipeline that extracts timestamps, applies CLP encoding, performs Drain3 clustering, extracts and categorizes variables, accumulates statistics, detects anomalies, and generates relevance scores 1. A demonstration showed the tool compressing 1,247,831 log lines into just 43 patterns, achieving the stated compression rate 1. Users can access ctrlb-decompose as a command-line interface, a WebAssembly browser version, or a Rust library 1. Output formats include human-readable ANSI terminal display, LLM-optimized markdown, and JSON 1. Installation options include Homebrew, dpkg, git clone, or integration as a Claude Code plugin 1.

The tool leverages advanced statistical techniques including DDSketch quantile sketching, HyperLogLog++ cardinality estimation, and reservoir sampling to efficiently process large log datasets 1. These technologies, combined with Drain3 prefix tree clustering, enable the rapid identification and consolidation of similar log entries 1.