THE MIGRATION LINE

All systems · status board

Every metric
breaks eventually.

Real companies, real systems, real thresholds crossed. Each row is an engineering migration retold as the incident it actually was — what climbed, what broke, what they shipped instead. Facts traced to source, never altered.

37Migrations logged

100Tech categories

24yYears covered

Ruby sitar-agent on a Sparkey datastore ⟶ Java sitar-agent on a WAL-mode SQLite datastore Airbnb Airbnb's config-delivery sidecar, sitar-agent, was fully rewritten from Ruby to Java in 2024 — and its local datastore moved off a Sparkey store that had to lock the whole file on every write.

In-house ORM sharding + index database ⟶ Vitess vindexes Etsy Etsy's ~1,000-shard MySQL cluster routed every query through a single index database — a full-site outage waiting to happen. Five years later it runs on Vitess.

Compute Managed Batch (CMB), a homegrown batch queueing system ⟶ Kueue, on Kubernetes-native Titus Netflix Netflix built Compute Managed Batch in 2018 because nothing in the Kubernetes ecosystem did what it needed. Seven years later, the open-source Kueue project did all of that and the features CMB never shipped — so Netflix moved millions of batch jobs onto it in a four-week production migration.

Gradle ⟶ Bazel Airbnb Airbnb's JVM monorepo held tens of millions of lines of Java, Kotlin, and Scala. Moving it off Gradle and onto Bazel took 4.5 years.

Enzyme ⟶ React Testing Library (RTL) Airbnb Airbnb used an LLM-driven pipeline to migrate 3.5K React test files from Enzyme to RTL in 6 weeks instead of an estimated 1.5 years.

Scheduled worker services on MySQL (OLTP) ⟶ ELT pipeline on an OLAP warehouse (Snowflake) Canva Canva counts billions of content usages a month to pay creators. Counting them one database round trip at a time on MySQL was O(N) and took over a day — so they moved the counting into an OLAP warehouse.

Static-analysis boundaries (Packwerk) ⟶ Running-code isolation Shopify Four years after shipping Packwerk, Shopify hit the wall: a package with zero Packwerk violations still crashed when run in isolation. The retrospective is an honest accounting of what static analysis missed.

Amazon Web Services ⟶ Own hardware 37signals After 15 years on the cloud, 37signals concluded the promised savings never came and the team never shrank — so they bought their own servers and left.

Clustered search-system storage ⟶ Husky (decoupled compute + object storage) Datadog Datadog's event platform outgrew its clustered search-system storage. Husky is the third-generation rebuild — a columnar store that decouples compute from storage on commodity object storage — and this is the design that guarantees exactly-once ingestion into it.

Airflow ⟶ Dagster + dbt Discord Discord outgrew Derived, its five-year-old in-house orchestrator, and picked Dagster plus dbt against four hard criteria.

Go ⟶ Rust Vercel Having decided to move Turborepo from Go to Rust, Vercel's team rejected a ground-up rewrite. Instead they ran both languages at once, porting piece by piece behind a Rust entry point they call the Rust-Go-Rust sandwich -- and hit a segfaulting Alpine Linux binary along the way.

Legacy payments in the Rails monolith ⟶ Product-agnostic payment orchestration platform Airbnb Airbnb's payment logic was welded to stay reservations, so every new product — Experiences, Resolution Center — meant reimplementing payments. This is the rebuild into an idempotent, product-agnostic orchestration platform.

Snapshot replication ⟶ CDC-based service-aligned data platform Canva Canva was taking a full snapshot of every service database every 24 hours to feed a 5-petabyte Snowflake warehouse. When a snapshot started taking longer than 24 hours to extract, the whole schedule fell apart.

Nginx (NGINX/OpenResty + Lua) ⟶ Pingora (Rust) Cloudflare Cloudflare evaluated forking NGINX, adopting another proxy, or starting clean — and built Pingora in Rust, cutting CPU by 70% and memory by 67% at over a trillion requests a day.

AWS SQS + Kinesis (data pipelines) ⟶ Apache Kafka + Apache Flink DoorDash Every one of DoorDash's data pipelines worked, yet data arrived late and cost too much. The culprit was the shape of the collection, not any single pipe.

Ruby pre-receive hooks (in the Rails monolith) ⟶ A dedicated Go service GitHub Every git push to GitHub ran Ruby hooks that spent about 880 milliseconds loading dependencies before doing any work — so pushing nothing at all took over two seconds.

Zookeeper cluster (Kafka) ⟶ New Zookeeper cluster Airbnb Airbnb's biggest Kafka cluster stored its metadata in a shared Zookeeper ensemble that nobody owned, so the team moved it to a dedicated one with no downtime and no data loss.

Apache HBase (on-premises) ⟶ Google Cloud Bigtable Box Box moved 600 TB of file metadata off three on-premises HBase clusters onto two Bigtable clusters with zero downtime, by writing to both databases synchronously until it trusted the new one.

Python monolith (Metaserver) ⟶ Managed microservices platform (Atlas) Dropbox Dropbox tried breaking its 3-million-line Python monolith into services, decided most teams shouldn't run services at all, and built a managed platform instead.

Python 2 ⟶ Go Khan Academy How an end-of-life deadline turned into a full backend rewrite

Single-tenant Confluence ⟶ Stateless multi-tenant on AWS (Project Vertigo) Atlassian Confluence Cloud's real ceiling wasn't CPU or memory — it was database connections, and one busy tenant could exhaust them for everyone. Vertigo rebuilt the architecture around that fact.

Native iOS + Android apps ⟶ React Native Shopify After three app experiments showed 95-99% code sharing and a team that felt twice as productive, Shopify decided all new mobile apps would be built in React Native.

Delta sync (full document on every change) ⟶ Incremental sync Figma Figma studied operational transforms and CRDTs, then built a simpler centralized model that syncs one property at a time instead of shipping document state around.

Manual code review and regex-based lint checks ⟶ LibCST-powered lint framework and Pyre-driven codemods Instagram Instagram's Python server is several million lines, edited by hundreds of engineers shipping hundreds of commits a day. A single shared-state linter couldn't keep up, so they rebuilt static analysis as infrastructure.

PostgreSQL + Citus ⟶ ClickHouse Cloudflare Cloudflare's HTTP analytics pipeline hit a wall at 15 queries per second on a Postgres/Citus cluster. They rebuilt it on ClickHouse, shut down 12 database nodes, and deleted tens of thousands of lines of Go, SQL, Bash and PHP.

Self-hosted GitHub Enterprise Server ⟶ GitHub Enterprise Cloud GitHub Etsy stopped running its own GitHub Enterprise Server and moved repositories to GitHub Enterprise Cloud, trading infrastructure self-reliance for third-party service.

Mocha (test runner) ⟶ Jest Airbnb Airbnb's Mocha suite took 12+ minutes in CI, 45 minutes locally, and flaked on 12% of builds. Jest cut it to 4m30s and ~1% flake — once they found the 480ms hiding in their spec helper.

Rails monolith ⟶ Service-oriented architecture (Thrift + Dropwizard) Airbnb Airbnb's Rails monolith had no strongly typed service contracts and shipped every payload as bulky JSON. This is how they standardized on Thrift IDLs and generated Java services so a new one could take production traffic in weeks.

Ruby on Rails monolith ⟶ Client-rendered React SPA on a v2 API Airbnb Swapping one small template for a React component broke unrelated parts of Airbnb's search page — and even its API response. The hunt for why ended in a rearchitecture away from Rails-delivered pages.

OpenSSL (patched) ⟶ BoringSSL Cloudflare Cloudflare's edge ran a patched OpenSSL fork alongside a separate Go stack that existed only to handle TLS 1.3. It folded both into BoringSSL.

MongoDB ⟶ Cassandra Discord How Discord rebuilt message storage before MongoDB's single replica set hit its limit

Go (performance-critical services) ⟶ Rust Dropbox Dropbox rewrote Magic Pocket's storage engine in Rust and it worked so well that almost nobody learned it — and that turned out to be the problem.

Ad-hoc manual MongoDB migrations ⟶ An automated migration panel Coinbase A Coinbase MongoDB migration meant finding one of a handful of engineers with production access to hand-run a script over tens of millions of records.

The tracelet-based HHVM JIT compiler ⟶ A profile-guided, region-based HHVM JIT compiler Meta (Facebook) HHVM's original JIT compiled PHP and Hack one small, type-guessed fragment at a time, re-checking types at every boundary. A three-year profile-guided redesign cut CPU usage for running Facebook by about 15 percent.

Oracle billing database, in Netflix data center ⟶ AWS — Cassandra for subscriber data, MySQL for payments Netflix On January 4, 2016, right before Netflix expanded into 130 new countries, its billing infrastructure went 100% AWS cloud-native — splitting one Oracle database into Cassandra for subscribers and MySQL for payments, because payment processing needed ACID transactions.

PostgreSQL ⟶ MySQL Uber Uber traced write amplification, verbose replication, a corruption bug and painful upgrades back to one Postgres design choice — indexes pointing at physical disk locations — and moved to MySQL/InnoDB.

Obidos monolith ⟶ SOA (Service-Oriented Architecture) Amazon By 2001 the Obidos monolith behind amazon.com had become a deployment bottleneck, and a famous internal mandate broke it into services owned by two-pizza teams.