Before I start shitting on VMware, I’d like to say kudos to them. They really kick started the virtualisation revolution, which has made the world of computing better. Since 2003, when vCenter (or Virtual Center as it was then known) was first introduced until now, we’ve seen a change from where almost all workloads had dedicated servers, to now where I hear System Administrators say often “Everything we run is virtualised” and “there is no reason not to virtualise”. In my short career as an application support engineer turned system administrator I’ve never commissioned a physical server for anything other than to run some virtualised platforms; ok I’ve only ever commissioned one physical server, but that alone proves my point.
But then, in 2006, Amazon launch Amazon Web Services (AWS) and there was at first a big change, but from the other end of the spectrum. Not as many workloads moved from virtualisation to Public Cloud infrastructure as those that moved from physical to virtual, but AWS has now become a dominating force in the world of computing.
There are two different models at play here, people call it Public vs. Private cloud, but that doesn’t really capture the difference in approaches you take when designing systems in these two virtualisation platforms. Heavily customised, slowly moving infrastructure vs. highly standardised fast paced instances; AWS encourages a standised, ephemeral nature of computing. Occasionally I see people complain about how AWS murdered their instance and how can they run a business where there is no SLA around individual server uptime. The fact is AWS has built their architecture with a very clear view of how they think computing should be done, and it’s changing the way we think about infrastructure.
You can build systems in a reliable, fault tolerant manner using VMware and AWS. You can also build them in a repeatable way in either system, allowing you to blow away and rebuild servers at will. You can also build them like pets, with massive uptimes that have long manual provisioning processes, but I think we all agree that’s the a bad idea. Only one product actively encourages you to do it the right way.
Fault Tolerance vs. High availability
AWS’s Elastic Compute Cloud (EC2) has a SLA of 99.95% for multiple Availability Zones (AZs) in one region. That means if more than one Availability Zone (AZ) is unavailable for more than 22 minute a month then you can claim a service credit. You can have instances in only one availability zone, so your instances may be unavailable for long periods at any time and this is considered normal. Some people find this surprising. For fault tolerance you should ensure you have instances in multiple AZs and, if possible, multiple regions.
One of VMware’s key features is vMotion, which allows you to move instances from one physical host to another without service interruption. I remember the first time I heard about this feature it blew my mind, it is an amazing feat of engineering. However it encourages people to ignore designing their system in a fault tolerant way and instead fall back on this high availability feature. Some applications don’t adhere nicely to this architecture, they do not natively support using say a load balancer to distribute requests across nodes. Such applications should be avoided at all costs.
Note what I’m talking about when I say fault tolerance should not be confused with VMware’s Fault Tolerance feature.
API all the things
What Amazon succeeds at most is in the API. More has been written on this which I won’t regurgitate, but I encourage you to read it. Someone in the office said AWS v1 SDK sucked the other day and I couldn’t help but laugh.
Here’s some Ruby I wrote to add a disk to a Virtual Machine, using the rbvmomi SDK which has been abandon.
The Perl code is just as heavy and difficult to write.
Here’s some pseudo code on how you do something similar in AWS, attaching an EBS volume to an EC2 instance.
The v1 SDK requires a bit more code, you have to loop through a list of every volume looking for one that matches.
Ultimately I’m not sure if the problem is entirely with the API itself, I think the amount of effort Amazon has put into its SDKs also goes a long way.
Platforms and Infrastructure
AWSs Platform as a Service (PaaS) offerings are another great differentiator to the VMware/Private Cloud offerings. I recently wrote some code to setup a staging database from a snapshot, it’s about 96 lines of code. I didn’t have to install or configure MySQL, I booted an RDS instance, seeded the database then deleted it and allowed it to create a final snapshot. The code (written in SparkleFormation which is a wrapper for CloudFormation) simply creates a new RDS instance from that snapshot.
Doing this with VMware would’ve taken me days, instead it took me hours.
Moving to a Public Cloud offering like AWS takes a lot of fore thought, you need to plan and re-architect your environment to provide resiliancy and flexibility. But the ultimate result, if your architecture and application are mature enough, is an environment that supports much faster iterations and more resiliant, fault tolerant designs. Still AWS is not always the right choice; Netflix recently spoke about their AWS migration project and noted they left their DVD business in their own datacenters, they don’t need the scalability AWS provides so the cost of migrating and running it in AWS doesn’t make sense.
But the AWS model is something not to be ignored:
- Build in Fault Tolerance not just HA. Not just because it’s more resilient but because it forces you to make cattle not pets.
- Have an API that’s easy to use and abstracts the complexity from the developer.
- Use PaaS to abstract complexity of building boilerplate services.