Battle of the Titans: Premium Theme Framework Smackdown


Criteria

The criteria used to judge each theme are below. Each theme has been judged objectively based on findings and given a grade. How and why I judge these criteria are described below.

Cost

The cost of a theme may or may  be a breaking point for some people. Most of the themes are reasonably priced, but are an important metric in the decision making process.

File Size

This is a big deal because, file size is in addition to everything else WordPress loads. The code that is required to load into the WordPress runtime should also be fairly small. A tragic benchmark in this cases is the new WordPress 3.0 Twentyten theme which weighs in at a hefty 1.6MB. (WordPress 3.0 is currently in beta1). For my judging, I want a theme below 800k in total size.

Database Calls

The number of database calls made in a theme is a critical data point. While a lot of database calls may not show up in small sites, a large number can quickly bog down a site of even intermediate size. Additional database calls can be created with multiple loops, widgets (depending on what is being done in the widget) or options created via update_option().

While users won’t see it, update_option() will create an option in the options table but will not set autoload. When WordPress loads, all autoload options are pulled in one database call and those that are not, but are needed in the theme, will generate an additional database call. Ideally, a theme makes no more than 10 database calls per page load. More than that may require database tuning for busy blogs.

Note: This does not factor in database calls from the WordPress Admin. Only from the front end of the blog where users will be.

Page Load

Page load is a defining, critical datapoint for large blogs. Keep in mind that in this stress test, I have preloaded 10,000 blog posts and 10,000 pages into WordPress. The results, in some cases, are alarming and I’d highly recommend theme developers put significant efforts into improving in this area. The results of the test are guaged on a local install, with no network latency, all applications closed except Firefox 3.6 and the web server and database server restarted after each test to ensure no throttled performance. Numbers were ascertained using Firebug.

Hooks

Hooks are used extensively among all themes. This is a useful way to allow plugin developers to extend the theme. The use of filters allow modification of the theme without modification of core code so, in many ways, filters are more important in my judging than actions.

Localization/Internationalization (i18n)

All of our theme developers live here in the United States. How their perspective outside of the United States relates to their theme development is gleaned by whether or not their theme contains the necessary components to allow for translation of the theme into different languages. This is a huge initiative in the WordPress community and it is unacceptable for any theme to not follow suit in their development.

Security

While it is impossible to do an exhaustive security audit for each theme, there was one key requirement for this test. For exhaustive security research, I recommend contacting Mark Jaquith and hiring him. He is the best WordPress security guy I know. For my tests, I looked for the use of the WordPress security API functions esc_url(), esc_attr(), esc_html() and esc_js(). Use of these functions throughout the theme (frontend and backend) provide most of the security that is needed and is the cornerstone of this audit.

BuddyPress Compatibility

Though there is disagreement in the theme community about providing built-in support for BuddyPress, it is the single fastest growing aspect of the WordPress consulting world. From experience, I can tell you that over 60% of my recent consulting inquiries have all required a BuddyPress component. In my opinion, failure to support BuddyPress out of the box is a huge mistake. Sadly, all of the four themes fail in this regard, though one does offer a third party option to provide support.

Built In SEO

To most of these providers, SEO is critically important and all four provide some level of built in SEO support. I, personally, feel it is less important than it used to be as Google’s new Caffeine algorithm update focuses more on page load (see above) than keyword and metadata optimization. However, this data point examines how much emphasis and built in support the theme provides for SEO.

WordPress 2.9 Post Thumbnail Support

In WordPress 2.9, a default post thumbnail feature was baked into core. This recognized the evolution of blogging from a traditional top down reading approach to magazine layouts. Many of these themes were instrumental in helping that evolution take place. As a result, these themes have often provided their own image thumbnailing features. Compatibility with WordPress 2.9’s functionality ensures forward compatibility and avoids duplication of efforts, however. For this test, I want to see the theme provide support for the Post Thumbnail feature, but does not mark off points for providing backwards compatibility to their legacy models.

WordPress 3.0 Navigation Support

Since WordPress 3.0 is not out yet, this criteria is less important than others. In WordPress 3.0, custom navigation support (read: build your own navigation easily) is baked into core. Forward thinking developers have considered this and provide support now for when WordPress 3.0 becomes available in a month or so.

Raw SQL Usage

Never, ever, ever, ever use raw SQL in themes. It’s better if used with the Database class provided by WordPress but why should a theme have it’s own tables? If it does have its own tables then raw SQL is needed, but for this audit, I am biased toward the idea that there should never be a need. Store data in the options table if you need to. That way you can use WordPress API like get_option() to retrieve values… and you can minimize database calls as listed above. You also avoid potential security problems like SQL injection. If the theme includes SQL, I am automatically failing that theme on this criteria.

Grading Scale

  • A+ = 100
  • A = 95
  • A- = 90
  • B+ = 89
  • B = 85
  • B – = 80
  • C+ = 79
  • C = 75
  • C- = 70
  • F = 0

On to Thesis 1.7!