Scale...

Scale...  Here's a fascinating (and not particularly technical) article about the scale of Amazon Web Services (AWS).  The whole thing is interesting, but a couple of points in particular jumped out at me:

• An educated guess places the total number of AWS servers at between 2.8 and 5.6 million servers.  If each of those was a standard 1U rack server, these would fill a standard equipment rack between 929 miles and 1,858 miles tall!  Each of those servers has many millions of transistors on its chips, so the total number of transistors in AWS is in the range of tens to hundreds of trillions.  That scale is just mind-boggling!
• Amazon designed their own networking gear, mainly to save costs.  To their own surprise, the network availability went up when they did this.  This is a surprise because by default you'd expect the Ciscos of the world to have better quality product than you could make yourself.  The Amazon engineers figured out why, though: it's because their equipment was much simpler, with only the features they actually needed.  Typical commercial network gear is larded up with an enormous variety of features, many of which interact with each other in complex and often poorly documented or understood ways.  I have quite a few personal experiences with that complexity that backs this up.  Still, I was surprised myself to read that Amazon boosted their availability with home-brew switches and routers.  You couldn't do this, though, unless you had a scale similar to AWS.  Most companies couldn't even imagine dedicating engineering teams for years to build that sort of gear.