HHVM's original JIT compiled PHP and Hack one small, type-guessed fragment at a time, re-checking types at every boundary. A three-year profile-guided redesign cut CPU usage for running Facebook by about 15 percent.

In early 2013, Facebook replaced its original HipHop Compiler — which statically compiled PHP to C++ — with the HipHop Virtual Machine, HHVM, a just-in-time compiler for PHP and Hack. HHVM’s initial JIT didn’t use profile-guided optimizations. Instead, it compiled the program into tracelets: maximal type-specialized bytecode sequences built by inspecting the live VM state at one execution point. A tracelet ended either when the JIT needed type information it couldn’t infer or obtain by inspecting local variables and stack slots, or at a branch instruction whose direction it couldn’t predict.

The project started in summer 2013 and was incrementally released through 2014 and 2015. As of this 2016 post, Facebook says the redesign “continues to be the basis for further improving performance,” and that overall it “has improved the CPU usage for running Facebook by about 15 percent.” The gains arrived in stages: enabling longer straight-line traces by default in the first half of 2014 delivered “about a 5 percent reduction in CPU usage” in Facebook’s webservers by the end of that year, and by spring 2015 the JIT could handle regions with arbitrary control flow.

The redesign turned the JIT into a multi-gear system. The first, profiling gear reused the existing tracelet JIT: by simply inserting a counter after the type guards in each tracelet, HHVM could cheaply collect block-execution frequencies and type information for input values while still serving live production traffic. Once enough profile data accumulated, a second, optimizing gear retranslated the machine code into larger compilation regions that could span what used to be several independent tracelets.

That profiling framework went on to enable a list of concrete optimizations Facebook names directly: code layout, dynamic method dispatch, partial function inlining, guard relaxation, specialization of array operations for different array and element types, and predicting the position of elements inside PHP arrays.

The hard part wasn’t the profiling — it was that the entire JIT backend had been built around tracelet assumptions. Because tracelets were intrinsically acyclic with no merge points at the bytecode level, many analysis and optimization passes had been written simpler than a general compiler would need — no iterative data-flow analysis required. Generalizing the JIT to arbitrary regions meant most of those optimization passes had to be fixed or rewritten from scratch, which Facebook singles out as the task it knew would be “very challenging” going in.

Even after three years of incremental releases, the company frames the redesign as ongoing rather than finished: “we believe there are still plenty of opportunities to further leverage profiling information,” including better static branch-likelihood decisions, improved spill-code placement, and smarter layout for the whole translation cache.

A JIT compiler is only as good as the size of the region it’s allowed to see — feed it real execution data, and a fragment-by-fragment translator can finally optimize across the fragments.