Pinterest moved a 4 TB HBase table serving 14,000 reads and 400 writes per second onto TiDB with no downtime, verifying 99.999% consistency between the two systems before finally deprecating the HBase table.

At Pinterest, HBase powered critical online services including Zen (a graph database), UMS (a wide-column datastore), and Ixia (a near-real-time secondary indexing service) — but the HBase ecosystem couldn’t serve the company’s needs for the next three to five years, thanks to high operational cost, complexity, and missing functionality like secondary indexes and transactions. After evaluating more than 10 storage backends and benchmarking three finalists with shadow traffic, Pinterest picked TiDB. The first real test was a table backing Ixia: 4 TB of data serving 14,000 read queries and 400 write queries per second.

Migrating that table without downtime meant ruling out the obvious shortcuts first.

Once the table was fully cut over, Pinterest measured a 3x-5x reduction in p99 read latency — from 500ms down to 60ms for this use case — along with read-after-write consistency, one of the specific goals for use cases migrating off Ixia. Comparing a second dump of both HBase and TiDB after reconciliation showed 99.999% consistency between the tables. The architecture also got simpler, in terms of the number of components involved, which the team expects will directly aid debugging future production issues.

The chosen approach combined a bulk snapshot ingestion with a staged dual-write handoff. First, an HBase snapshot was converted to a base64-encoded CSV dump in S3 and ingested into TiDB using Lightning’s local mode, while async dual writes to TiDB began in the background — kept in “shadow mode” so TiDB’s latency couldn’t affect the live service’s SLA. Next, parallel MapReduce jobs snapshotted both HBase and TiDB, found mismatched rows, read each mismatch’s latest value from HBase, and wrote it into TiDB — a reconciliation loop run daily to track and close the mismatch count. Only once double writes were fully synchronous and reconciliation was clean did Pinterest flip TiDB to the primary write path, keeping HBase as an async, best-effort fallback before finally stopping writes to HBase and deprecating the table entirely.

Getting there wasn’t friction-free. TiDB Lightning 5.3.0 didn’t support automatic TLS certificate refresh, and Pinterest’s internal certificate service rotated certificates every 12 hours — a mismatch that caused failed ingestion jobs and was hard to debug from the logs alone, until PingCap helped trace and later fixed it upstream. Lightning’s local mode also consumed enough resources during ingestion to affect a separate table sharing the same cluster, requiring short- and long-term fixes to placement rules. The MapReduce connector needed scalability and TLS fixes of its own just to snapshot a 4 TB table in reasonable time. After those fixes, the full 4 TB ingested in about 8 hours, with one round of reconciliation and verification taking roughly 7 more.

“Zero downtime” is a claim about the read path, not the effort behind it — the actual work is building enough reconciliation and rollback capacity that you can afford to be wrong at every intermediate stage.