Solving the WordPress Traffic Overload Problem

Anyone who’s been around WordPress for a “Digg effect” or other massive influx of traffic knows that it can be a real problem. From a technical standpoint, the problem is that PHP is entirely loaded into memory for every pageload. That includes the 99% of PHP that is not being used to actually render the page.

On low traffic sites, this problem is not necessarily noticed. It doesn’t have a huge impact. However when there are hundreds of requests hitting a server in a single second, that kind of overhead builds up very fast.

There are solutions to this sort of thing and depending on what the scale of the environment is, some might be more excessive than necessary. The WP Super Cache plugin is a quick solution that will cause pages loaded on WordPress to be cached meaning that if subsequent page loads can pul the HTML from the cache without having to load the overhead of PHP as well, everyone wins. On the more extreme end, server configurations can be made to send requests for different types of content (for instance, images) to specialized servers optimized for that content type.

Very geeky stuff. It’s important to note that WordPress gets a black eye all the time for it’s caching mechanisms and ability to handle the load of a “Digg effect”, etc. In fact, Instapundit Glenn Reynolds is the latest to take a stinging swipe at WordPress and trust me when I say, we heard it loud and clear.

At b5media (where I’ll be leaving as the Director of Technology soon), we’ve had to deal with this as well and have managed to develop really sound solutions to some of these problems. However, for WordPress as a whole, it is a well recognized problem that not everyone can solve by following in our footsteps (or WordPress.com footsteps).

We’re going to do what we can to help solve this problem once and for all as two of our developers, Mark Jaquith and Brian Layman will be mentoring a Google Summer of Code intern to develop a robust caching engine for WordPress. We hope that this exercise will result in a more reliable (and sane!) caching mechanism.

Integrated Caching Solutions will improve WordPress’s speed and reliablity out of the box and allow people to “Digg Proof” their sites without the struggle of installing plugins on a site that is virtually unreachable. (Source: WordPress Google Summer of Code 2008)

Glenn, I hope that the work that Mark, Brian and our intern will be doing will improve the WordPress problem. In the meantime, let me know if I can help you with anything (though I believe you are using Movable Type). It is a known issue and it’s one that needs to be solved and hopefully some steps can be made toward that this summer.

Downloading Blog World Expo


Photo by Kris Krug

What a fantastic week in Vegas last week. People have asked me what my feelings were on it and I keep summing things up in one way: It was the only single time when all of the blogoshpere came together at one time, in one place and enjoyed it.

There are well over 2000 photos on Flickr right now tagged blogworld or blogworldexpo. Most of them display geeks ability to party and of course, it was Vegas – partying was expected.

But more so were the people I met. When I first started in blogging, the Instapundit Glenn Reynolds was sort of my idol. Not so much these days as I avoid politics like the plague, but it was awesome to meet him and share a brief few minutes talking about blogging.


Photo by Brian Solis

Also very cool was the chance to connect with Wendy Piersall. I am apparently now her BFF. :-)

A drunken evening was spent at the bar with Marshall Kirkpatrick – what a guy! And I can’t forget about old friends Jim Turner and Tris Hussey and new friends Jeremy Pepper, Brian Solis and MyBlogLog‘s Robyn Tippins – love that girl!

Of course, there’s hundreds of people I could list… like Rick Klau who I’ve finally had the pleasure of meeting… but this entry would be too long.

Again, the value of this, like many conferences, was in the hallways. Better conversations. More productivity. B2B opportunities. It was also really productive to have the political bloggers in the same room with tech and business bloggers in the same room with social media networks in the same room as military bloggers, advertisers, service providers, etc. It’s hard to get everyone in the same room, but it happened in Vegas.