Etsy built every feature twice — once for the website, once for the API — in a PHP world with no concurrency. So they rebuilt the API from scratch, and taught PHP to run requests in parallel.

Etsy’s second-generation API was a JSON REST API, tightly coupled to the database schema, so powerful that when the first iPad app launched the team didn’t have to write a single new endpoint. Clients could request multiple resources at once and trim responses to just the fields they needed. But that power hid a structural problem: much of Etsy’s logic was implemented twice — once in the website code, then rebuilt in the API for the iOS and Android apps.

The web codebase had also drifted into bad habits over years of an evolving MVC architecture: data fetched during template rendering, and business logic living in templates. The API served AJAX while the backend was PHP, and there was no single place holding logic reusable by both.

When the team re-examined the requirements, the constraint was latency — what users feel as the wait from request to response. Citing Ilya Grigorik’s “breaking the 1000 milliseconds time to glass,” they noted that mobile network speed left only about 100 milliseconds of server-side budget to stay in play. The API needed reusable components shared between web and apps, so features could be expressed once at the API level.

The hard part was the runtime itself. Etsy came from a sequential, shared-nothing PHP world with no built-in concurrency. The question that shaped everything else: how do you parallelize and reuse work while keeping the network footprint low, all without leaving the PHP they already knew how to run?

Extending the existing API framework was ruled out early — it simply couldn’t support the shared-component approach, so the team decided to abandon it and rebuild from scratch as “API v3.” For concurrency, the tempting industry answer was a runtime built for it, like the NodeJS event loop. Instead they chose to start from what they were good at and stay in PHP, borrowing the server-side composition idea from Netflix and eBay’s ql.io: a second layer of endpoints that consume Etsy’s own API and aggregate the results, so a client makes just one request.

The concurrency mechanism came out of a hack-week experiment. In 2013, after a teammate tweeted that “curl_multi_info_read() is my new event loop,” two engineers from Etsy’s core team showed they could get real concurrency in the HTTP layer through parallel cURL calls driven by curl_multi_info_read. The HTTP layer was an appealing place to do it, because routing, load balancing, and caching solutions already existed there.

On top of cURL they added dependency tracking between calls — “proxies” — running each request when its proxy became unblocked, event-loop style, all encapsulated in one component they call the curl callback orchestrator. Meta-endpoints let the server act as a client of its own API, collapsing many resource fetches into a single request from the actual client, which is the biggest bottleneck for a responsive mobile interface.

Building it out meant more than a scheduler. v3 minimized each endpoint’s job to declaring its route, its input expectations and output guarantees, and its implementation — pushing StatsD monitoring, type checks, route compilation, and authentication into the framework. Two new concepts governed access: “perspectives” (on whose behalf a call is made — Public, Member, Shop, Admin, Infrastructure, Application) and “services” (from where it’s made — Ajax, Admin, internal, Apps, third-party). An API compiler generated the Apache routes and the PHP and JavaScript client code, verified up to date by Jenkins before every deploy.

The trade-off is inherent in the design: concurrency now rides on real HTTP subrequests through the composition layer, and the server is deliberately made more complex so the client can stay simple. Etsy took that bargain on purpose — more control and an extra layer for code execution, in exchange for the client making one request instead of many.

Concurrency doesn’t require a new language — sometimes it’s cheaper to hide an event loop inside the stack you already run and let developers keep writing code that looks sequential.