Cloud Computing Does Not Spell the End for Common Sense I.T. Management

Sometimes I think I might be the only one who retains commons sense. Really. At least in the area of I.T. Management. Though we had our share of growing pains at b5media, the knowledge gained from working in an enterprise environment at Northrop Grumman was only accentuated by my tenure as the Director of Technology at b5media.

Unfortunately, some common best-use practices in developing infrastructure are often put aside by those with shiny object syndrome surrounding “cloud computing“.

Let me explain.

You may have noticed a severe hampering of many internet services over the weekend. The culprit was a rare, but yet heavy-duty outage of Amazon S3 (Simple Storage Service) cloud storage. S3 is used by many companies including Twitter, WordPress.com, FriendFeed, and SmugMug to name a few. Even more individuals are using S3 for online data backup or for small projects requiring always-on virtual disk space. Startups often use S3 due to the “always on” storage, defacto CDN and the inexpensive nature of the service… it really is cheap!

And that’s good. I’m a fan of using the cheapest, most reliable service for anything. Whatever gets you to the next level quickest and with as little output of dollars is good in my book, for the same reason I’m a fan of prototyping ideas in Ruby on Rails (but to be clear, after the prototype, build on something more reliable and capable of handling multi-threaded processes, kthxbai.)

However, sound I.T. management practice says that there should never be a single point of failure. Ever. Take a step back and map out the infrastructure. If you see anyplace where there’s only one of those connecting lines between major resource A and major resource B – start looking there for bottlenecks and potential company-sinking aggravation.

Thus was the case for many companies using S3. Depending on the use of S3, and if the companies had failover to other caches, some companies were affected more than others. Twitter for instance, uses S3 for avatar storage but had no other “cold cache” for that data rendering a service without user images – bad, but not deathly.

SmugMug shrugged the whole thing off (which is a far cry from the disastrous admission that “hot cache” was used very little when Amazon went down back in February), which I thought was a bit odd. Their entire company revolves around hosted photos on Amazon S3 and they simply shrugged off an 8 hour outage as “ok because everyone goes down once in awhile”. Yeah, and occasionally people get mugged in dark city streets, but as long as it’s not me it’s okay! Maybe it was the fact that the outage occurred on a Sunday. Who knows? To me, this sort of outage rages as a 9.5/10 on the critical scale. Their entire business is wrapped up in S3 storage with no failover. For perspective, one 8 hour outage in July constitutes 98.9% uptime – a far cry from five 9′s (99.999%) which is minimal mitigation of risk in enterprise, mission-critical services.

WordPress.com, as always, comes through as a shining example of a company who economically benefits from the use of S3 as a cold cache and not primary access or “warm cache”.

Let me stop and provide some definition. Warm (or hot) cache is always preferable to cold cache. It is data that has been loaded into memory or a more reasonably accessible location – but typically memory. Cold cache is a file based storage of cached data. It is less frequently accessed because access only occurs if warm cache data has expired or doesn’t exist.

WordPress.com has multiple levels of caching because they are smart and understand the basic premise of eliminating single point of failure. Image data is primarily accessed over their server cluster via a CDN, however S3 is used as a cold cache. With the collapse of S3 over the weekend, WordPress.com, from my checking, remained unaffected.

This is the basic principle of I.T. enterprise computing that is lost on so much of the “web world”. If companies have built and scaled (particularly if they have scaled!) and rely on S3 with no failover, shame on them. Does it give Amazon a black eye? Absolutely. however, at the end of the day SmugMug, WordPress.com, Friendfeed, Twitter and all the other companies utilizing S3 answer to their customers and do not have the luxury of pointing the finger at Amazon. If their business is negatively affected, they have no one to blame but themselves. The companies who understood this planned accordingly and were not negatively affected by the S3 outage. Those who weren’t were left, well, holding the bag.

Added: GNIP gets it, and they are new to the game. Even startups have no excuse.

Google File System: Much To Do About Nothing

Google had a much-hyped announcement tonight that, frankly, I’m missing the point of. Techcrunch covered it. Scoble Qik’d it live. I was one of numerous who took the bait out of curiosity and watched the announcement live until Scoble turned off his camera, or something.

honestly, folks, I don’t see what the point is. The product manager for this new service began the party by talking about how Google App Engine (Link dead until launch time) would be “easy to use and easy to scale”. The presentation then showed a very nervous developer trying to write up a simple Hello World script in Python.

Ok, here’s my problem. For the growing number of non-technical entrepreneurs, python is neither easy to use and the demonstration does not demonstrate easy to scale. At some point, the presenter stated that anyone could build applications using Google’s infrastructure that could be as big as Google’s own apps.

Forgive my cynicism.

This, my friends, is an Amazon S3 “me too”. There is not innovation here. There is nothing ground breaking here. It is yet another case of Google deciding that it can do things better than everyone else but with the exception of Search, Gmail and Google Adsense (the latter being questionable these days), I wonder how many of Google’s initiatives are really all that groundbreaking.

Then there’s the question of privacy. Google’s ever present incursion into deeper parts of lives should make every privacy nut cringe, and turn those who are not privacy nuts into privacy nuts. With the adoption of OpenSocial and now providing a platform for application development, Google’s hand continue to delve deeper into our deeply guarded private lives.

I’m skeptical here folks. From what I’ve seen, nothing is easy to get into here. Companies are not necessarily better off for using this infrastructure. The concept of threaded processes and optimized platforms for optimized content goes out the window with an S3 or a Google App Engine. And… The privacy concerns are very real.

Hold the phone. Let’s see what happens here.