Didi's logs lived in Elasticsearch, written twice — once to ES for search, once to HDFS for analysis — at petabyte scale. Moving both onto ClickHouse cut machine cost 30% and made queries 4 times faster.

Didi’s logs were primarily stored in Elasticsearch. To support full-text search, ES leans on tokenization and inverted indexing — and Didi found this created “significant bottlenecks in write throughput.” It also meant storing both the raw text and the inverted index for everything, which pushed up storage costs and memory requirements at the same time.

Under the old architecture, that wasn’t even the whole write path: log data had to be written to both Elasticsearch, for real-time queries, and HDFS, for Spark to analyze later. Two independent pipelines for the same data.

Post-migration, Didi’s ClickHouse logging cluster exceeds 400 physical nodes, with peak write traffic over 40 GB/s. It serves approximately 15 million queries a day, with peak QPS around 200 — and does it for 30% less machine cost than the equivalent Elasticsearch setup. Query speed improved by about 4 times, with most queries completing within a second at the P99 mark.

The architecture behind those numbers is deliberately split: a Log cluster (over 300 nodes) for detailed log storage, and a separate Trace cluster (over 40 nodes) for trace data extracted from logs via materialized views. Physically isolating the two stops expensive log queries — the kind that scan for a pattern with a LIKE clause — from slowing down the high-QPS trace queries running alongside them.

Didi’s own comparison of before and after is precise about what changed structurally: the old design “required users to maintain two independent writing pipelines, doubling resource consumption.” The new one collapses that into a single pipeline — Flink writes log data directly into the ClickHouse Log cluster, and materialized views extract trace information from those same logs, writing it into the Trace cluster asynchronously through distributed tables. Cold data still lands in HDFS eventually, but that’s handled by background threads inside the Log cluster rather than by a second application-level write.

Didi didn’t make this bet alone — the team cited other companies, including JD.com, Ctrip, and Bilibili, as having already built log storage successfully on ClickHouse, and that precedent gave them confidence to attempt their own migration. That’s a real hedge worth naming: this wasn’t a novel architecture invented from scratch, it was a pattern proven elsewhere and adapted to Didi’s scale.

The honest limitation, in Didi’s own words: “significant cluster resource consumption still occurs when handling fuzzy queries.” Their fix for that is still ahead of them — the team names secondary indexing, ZSTD compression, and storage-computation separation as directions they intend to explore next, not problems already solved.

Collapsing two write pipelines into one costs more up front in redesign than swapping the database underneath — but it’s usually where the real savings were hiding.