Technosailor.com

Being the Technology Architect for a startup is a dangerous job. Deadly dangerous. It involves sleepless nights and extra-careful sensory skills to navigate the landmines laid in marriages. Often times, instead of watching Lost or re-runs of The Wire with my wife, I find myself solving technical problems and ensuring the technical side of enterprise survives the night. We don’t talk about specific statistical numbers in public, but I can assue you that they are quite robust and that brings significant landmines to the process.

You may have noted some moments where we have experienced sluggishness or even downtime. While we are working through these issues, the reality is is that an enterprise of our size has demanding requirements.

As a part of our strategy, and particularly because we are an investment-backed corporation, we are required to provide reports to our investors concerning the health and status of the network. Shortly after recieving funding, we contracted a well known development company to build a tool that would allow us to see our network on a birds-eye level. Stats, traffic, bandwidth, comments, posts, who’s talking about which of our blogs. All our blogs are represented. It’s quite a tool.

During the development of this project, we encountered a MySQL bug which affected the entire enterprise and forced us to adjust our approach to this tool and our development process. The bug occurs when a LEFT JOIN is performed on a field that has NULL values.

While this bug is remedied in MySQL 5.2+, there are a couple things to remember when developing in production:

1. Never run in production unless you have to. In our case, we had to to get real data
2. Barring #1, always run against a replication slave database. We replicate our primary database server to a secondary. In the case of our project, we ran against the primary database and in fact the bug caused our server to grind to a halt.
3. Test, test, test. Monitor, monitor, monitor. When introducing a new specimen to an ecosystem, you need to ensure that it won’t devour the native life

Pick up your copy of the WordPress Bible, a wildly popular resource for beginners and experts alike.

Written by Aaron Brazell on December 20, 2006

Popularity: 1% [?]