Netflix moved its decade-old Eureka-and-Ribbon IPC architecture onto an Envoy-based service mesh, co-developing a new Envoy feature with Kinvolk — On-Demand Cluster Discovery — specifically so services could migrate with zero configuration changes required.

Netflix began migrating to AWS in 2008, years before most cloud-native tooling existed — the CNCF itself wasn’t founded until 2015. With no off-the-shelf options, Netflix built its own service discovery (Eureka) and client-side load-balancing IPC layer (Ribbon), driven by a design philosophy of avoiding single points of failure that a 2012 Christmas Eve outage reinforced further. The interface was simple and durable: a service advertised a VIP (or Secure VIP) name and port to Eureka, and IPC clients resolved that name to a set of hosts directly, without routing through a central load balancer.

Netflix set two hard constraints on the migration: keep the existing VIP-plus-secure abstraction that had served the company well, and make the migration itself automated and seamless rather than requiring service owners to do work. That collided immediately with an Envoy limitation — clusters must be predefined in a proxy’s own config. Netflix evaluated three ways to populate that config: asking service owners to define every cluster their service talks to, which failed because owners often don’t know what auxiliary services or libraries call under the hood; auto-generating Envoy config from each service’s existing call graph, workable for existing services but not for a brand-new one; and pushing every cluster to every proxy, which back-of-envelope math showed would mean pushing millions of endpoints to each individual proxy — simple, but not remotely feasible.

Because Eureka remained the single source of truth for VIP and instance status throughout, Netflix could run some services on mesh and others still directly on Eureka/Ribbon simultaneously, migrating individual IPC clients to connect through the local proxy at their own pace rather than all at once. The tradeoff Netflix accepted openly: on-demand lookup adds latency to a cluster’s first request, unacceptable for a small number of genuinely low-latency-sensitive use cases, which Netflix simply exempted from the pattern — those services either predefine their clusters up front or prime connections before the real first request, rather than being forced into on-demand discovery that doesn’t fit their needs.

A migration constraint like “zero configuration changes” is worth treating as non-negotiable enough to co-build new infrastructure for — the cost of getting hundreds of service owners to hand-configure anything correctly, at scale, is almost always higher than the cost of building the automation yourself.