Data Center Power Outage

Technosailor and all of the b5media sites have been experiencing several critical issues lately. You’ve probably noticed it. It’s been a perfect storm of sorts with a bunch of unrelated, but highly important problems.

1) First, we discovered we had a degraded RAID array on one of our servers. Because this server was RAIDED we were not concerned about data loss, but had to replace the drive, wait for it to synchronize with the RAID and carry on.

2) That same RAID controller has wonky firmware. So we ran into problems with synchronizing that new drive. This is actually still an issue but hopefully near resolution.

3) We have an Apache crash on a nightly basis and I believe this will be resolved with an Apache upgrade. Working on at the moment.

4) the latest outage came as a result of a power outage at the data center. While this is highly abnormal and rare, the folks at WebNX have been great and highly responsive. We still have one server out of commission so all web traffic is being served by two servers, instead of three.

Interestingly, we’ve been having pinging and trackback issues and have not been able to identify the problem. Though I can’t guarantee anything, it would appear that having to restart servers this morning kicked the pings out of the queue suddenly. Because this is XML-RPC, I am not sure how it would relate or play into anything, but it was an interesting observation nonetheless.

• 
• Stumble it!

About the Author: Aaron Brazell is the lead editor of Technosailor.com and a social media expert. His passion is to see companies and individuals use the internet and web technologies wisely and effectively to promote their brands and companies. He is Business Development Manager for Lijit and he worked as Director of Technology at b5media from 2005-2008 and is currently an independent consultant.