HipHop, PHP, and the Evolution of Language

A lively little discussion developed over the past few days on the DC-PHP developers mailing list. We have a very active developers group here in the DC area – much larger than most cities, in fact. Part of what makes our group great is the diversity of background and experience that is in the group.

This was front and center over the past few days when one of our members, Hans, offered his opinions on Facebook’s new HipHop for PHP product. We have already expressed our intent to help make WordPress compliant with HipHop, something that will be beneficial to major WordPress sites like TechCrunch, Mashable, VentureBeat,, the NFL Blogs, the NY Times blogs, the Cheezeburger network (LOLcats, FAILBlog, etc) that carry large amounts of traffic. I hope to be able to consult with some of these organizations on moving into a HipHop system once my head is wrapped around it and WordPress is compliant.

Photo by Josh Hunter
Photo by Josh Hunter

Hans is an extraordinary developer. I have never met him personally, but his depth of knowledge on issues of security and scalability is downright frightening. He offered his own opinion of HipHop on the mailing list and so I’m going to pick on him a bit:

This HipHop thing is interesting, perhaps in much the same way as HipHop music: it feels like a hack. — And I mean that respectfully in both cases; I like hip-hop music, and appreciate how it pays homage to R&B roots, remixing/reinterpreting them, etc; and I think that the idea of taking one language and building it out to something else is also something I should support. After all, I’ve embroiled myself in code generation tools (e.g. Propel) that are operating on the same philosophical groundwork. But I also believe that there’s a general rule like “if you need code generation, there’s something wrong [in your design or in the tools you’ve chosen or …]” … so those tools also feel like hacks.

In all of life, there is an evolution that happens. One iteration of something becomes better with improvements over time. This has happened on a micro level inside PHP. Without PHP 3 there would be no PHP 4. Without PHP 4, there would be no PHP 5. Ben Ramsey talked about this evolution before Christmas.

Why is it a hack to improve upon the tools used with a language? Is it a hack to use Memcached with PHP? Is it a hack to run on nginx instead of Apache or to implement FastCGI? All of these are third party software or extensions outside of PHP. So how is HipHop any different?

That’s all fair, but I feel like the problem here is that somewhere a long, long time ago, Facebook *must* have realized that they were going to have scaling problems. Long before they started having a problem, someone *must* have thought “maybe a compile-at-runtime language isn’t the right solution here”. I guess to me this cross-compiler is just a public way to admit that PHP is not the right tool for the job, but they’re stuck with all these developers that only know PHP so it was somehow cheaper to engineer a way to change PHP to C++ than it was to retrain developers on C++ (or, probably more realistic, Java).

I responded in that conversation with an only slightly edited response. While I appreciate, and always have appreciated, his frank, honest, high level view of PHP, web security, web applications, etc., he strikes me as somewhat naive and puritanical.

What I can say is *I*, along with dozens of other technology people in and out of DC, in and out of PHP, never look at our initial ideas as scaling ideas. We look at them as ideas and experiments to see if they have legs. In fact, I’d go so far as to say it is counter-productive to think about scale before thinking of concievability (is that a word?).

There’s a reason why Rails (God help us) is popular. It’s a great prototyping tool. You stand up an app quickly and let it into the wild to see if it has legs. Does it go? What are the market influences? What are the
pros and cons? Do we have to adjust?

After a concept is proven, then a solid dev team with solid tech leadership brings in their expertise to see if the idea can be built into something sustainable. As a sidebar, please take a read of Brad Feld’s very awesome
post from a few years ago “The first 25,000 Users are Irrelevant“.

My point is, it’s silly and a waste of resources for startup people to start thinking about how big they might get maybe 5 years down the road. I think you’d find out that, in most cases, successful technology, web-based companies happened by some dumb luck. Twitter. Facebook. Name-the-popular-app. Dumb luck.

Hey, I’d even argue that when too much comp-sci brain energy goes into an app, you get things like Wolfram Alpha. Cool. But useless. And not nimble enough to actually do the scaling necessary to need all that comp-sci engineering prowess.

Balance, my friend. Balance.

Facebook (and others) start with PHP because PHP is fairly ubiquitous and easy as pie to drop into production. However, there is a point of no return where you are committed to PHP and that’s where HipHop comes in.

Personally, I wish we had HipHop when I was at b5media. We had a ton of scaling problems with PHP and we were running fully clustered Apache servers (25 deep, if I recall), sharded MySQL across 6ish database servers, and we had massive I/O bottlenecks. We ran eAccelerator and Memcached and had squid-based load balancing and damn if Grey’s Anatomy or the Oscar’s didn’t pin our entire network on more than one occasion. What could have happened with an alternate to opcode caching. What could have happened if I had resources to put on developing C++ binaries of our frequently used PHP libraries.

I’ll tell you. It would have rocked. We were already committed to PHP. We were already committed to WordPress. And when the company started, we were all volunteer resources. There was no assumption that our idea had legs or I think everyone on the team would have quit our jobs immediately and put everything into building that company. It took a year to get there.

This is, for better or for worse, the way companies get started in the real world.

11 Replies to “HipHop, PHP, and the Evolution of Language”

  1. Hey Aaron – I think you’re absolutely right about the virtues of a quick and agile prototype. And for any startup, even on as successful as Facebook, the pressures to get to the very next stage are so substantial, that long term planning really never happens. And it shouldn’t either – my guess is Facebook would have never been successful if it were developed in Java, as Hans suggests, because it would have taken several times as much money and time.

    I know you’re a PHP guy, but I want to point out that Rails really does offer a tremendous advantage here. While it is much more resource intensive than PHP, it can be compiled into Java relatively easily for instant, long term scalability.

  2. As I am quoted above (and somewhat selectively), I did feel I should probably mention here that Aaron and I are really talking about different things. My perspective is much more from traditional software development; Aaron comes from more of a startup background (and admittedly that is more relevant here).

    To be fair, Michael, I never suggested Facebook should have *started out* writing code in Java. I am a huge fan of prototypes and I concur that PHP (like Rails) is a good prototyping framework. I do, however, want to suggest the possibility that the PHP version also have been treated as a prototype. I believe it’s important to be able to discard prototypes. It feels like HipHop, more than anything else, is about holding on to a prototype. (Granted that’s really unfair in FB’s case, given the amount of third-party code out their that’s written for that “prototype”.)

    Anyway, while I’m ok with Aaron’s opinion of me being puritanical & naive, I do want to make it clear that I’m not against HipHop or Facebook — or any of Facebook’s open-source tools. I do think that it provokes some good discussion about prototyping and choosing the right tools for a job, though, and I feel those are worth discussing.

    Deep down, I think I just feel strongly that the reason to choose PHP is the deployment paradigm and *not* the language. So, I just don’t see a reason to use a tool like HipHop other than rescuing an ailing PHP application.

    *Had* they started with Ruby, it does seem likely that they could have moved to leverage JRuby on the JVM and they could have used existing tools rather than having to rebuild the compiler/engine themselves. At the end of the day that does /sound/ like it would have been cheaper. Of course, hindsight is 20/20. I’m more interested in how other people choose to use HipHop than how Facebook uses it; I’m sure it makes perfect sense for them. I just think it’s important to say “prototypes should be thrown away”.

    1. “Prototypes should be thrown away” ignores the fact that they’re *never* thrown away. Even Twitter, despite their move to Scala, still has a lot of Rails on nonperformant parts.

      Prototyping, in general, is always the worst reason to adopt a language because, in practice, it’s a lie. It was a lie when people said it to get in the door at Yahoo! and various enterprise places (and then became entrenched), It‘s a lie when people argue for Rails development (and then had to hire consultants), etc. It’s even a lie in most RAD tools—when you use prototyping to build a UI, you either never leave/break it (Interface Builder), or you drop it once the part you needed has been built (Windows Forms Designer).

      1. The quote was a (loose) reference to Fred Brooks’ Mythical Man Month, but it’s a fair point that prototypes typically evolve and never fully disappear. I’m sure the same could be said about Facebook’s use of PHP. What I was trying to suggest is that technologies should be allowed to change. From all I’ve read of Facebook’s technology platform they understand this really well. Yes, it sounds to me like when you have a monolithic web app, things are already “too far gone” to make those sort of design changes, but whatever — Facebook with all their various technologies in play definitely seems to understand that some tools are better for some jobs than others. I’m really not talking about Facebook, I’m talking about new PHP projects, and trying to suggest (not very successfully) that there’s something to be said for *not* ending up in a situation where you do have a monolithic codebase that can only be migrated wholesale. (And I think that discussing how one designs a project to make that possible would be an interesting discussion.)

        So maybe “discard prototypes” is too strong a phrase. But I think there’s a value to remaining objective here, to reviewing, and to entertaining the idea that maybe the technology that started it all is actually limiting the solution. For example, the way the Twitter folks said, (something like) “you know, we actually needed long-running processes [and Ruby had a hard time with that]”.

        Obviously, it’s purely inflammatory to suggest that PHP is a good prototyping tool ;) — I’ve certainly deployed plenty of production code on PHP. But I think there’s a valid point missing in a lot of this HipHopped-up retoric, which is: revisit your prototype decisions. And maybe even, if you can afford it, “throw away your prototype”.

      2. That’s sort of the whole Agile philosophy in a sense… Quickly build something that works properly, test it, make it reusable if appropriate, deliver and keep moving.

        A lot of the decision on language (and prototypes) comes down to a decision on technical debt. If you’re using PHP, you’re incurring some debt by developing with a slower language at runtime. But you’re avoiding other debt by being able to develop, test and deploy quicker.

        HipHop, to me, is a code debt refinancing tool. Used well, it could save you. Used poorly, you’re not paying down the debt at all, or possibly even making it worse.

        (Aside: If there was a selection of good C++ web frameworks/libraries, HipHop may not have been as necessary.)

        I’m curious to look at HipHop for one-time translations from PHP to C++ and then maintaining the C++ code from there. By the sounds of it, though, that’s not how it’s being used at FB.

        Also, if you can do per-class translation or run a fully compiled app but then plug in individual classes in PHP, it could be pretty nice from a rapid development standpoint. I assume that FB is doing this.

        1. I’m curious to see some translated code, but I don’t think that HipHop produces code that is manageable in C++ is any way. More likely, it produces a collection of string maps and weird inferred types, which is probably why only a 50% performance increase is achieved.

          Side note: Wt ( is a very descent native C++ web toolkit.

    2. Hans,

      I understand what you’re saying about picking the wrong tools, but now that HipHop exists, it seems there are fewer reasons not to use PHP (or Rails => JRuby). While you can claim Facebook made a mistake, their resolution means other startups can’t make the same mistake. In general, I think we should be working toward brining scalability to the most agile language, as there is a clear value to doing so (the same can be said of projects like Grails, the other way around).

      Ultimately I don’t see much sense in objecting to something feeling like a “hack”. What is a hack, but a rift between dogma and implementation? Programming languages are tools, not philosophies. We shouldn’t allow ourselves to be limited by the aesthetics of a technical solution.

  3. Hans said
    “Deep down, I think I just feel strongly that the reason to choose PHP is the deployment paradigm and *not* the language. So, I just don’t see a reason to use a tool like HipHop other than rescuing an ailing PHP application.”

    A tool like HipHop will become the defacto runtime for php, when it combines its compilation with its runtime in an on the fly JIT compile. Then you get your deploy model back (just edit and reload), and the performance gains. Keep your deployment scheme, keep your language… get performance gains. Win win win win win. Am I missing something here? The community needs to jump behind this.

    I don’t understand why people keep saying things like “this is nice but its only for those who are cpu bound” and “other parts of web applications could be better optimized”.It seems like an argument built for misdirection. If you could improve php by 50% you could (theoretically reslice the hardware) and shut off a large chunk of the internet with no lost capacity. Sure, the apps will still be IO bound, but for the ones that required more than one httpd to fire up all those hungry mod_php’s they will now need less boxes, less cpu, and will get to dispatching those IO requests faster, and processing the results faster.

    Yes. I’m saying we’d need less power plants. This is a big deal.

Comments are closed.