Cloud—plain and simple
Cloud, cloud, cloud—it’s what you’ve been hearing about the last several years in software. Most of what you’ve heard sounds great, except the parts that don’t — including the learning curve and all the terminology and mumbo jumbo — VMs, EC2, Docker, containers, serverless, document databases, and ‘architecting for the cloud’.
In this article I’ll explain some of the cloud basics. AWS is the leader in the cloud space and I’ll be using AWS exclusively in this article. Visit my website for tutorials on AWS.
Several years ago I began consciously transitioning away from monolithic application architectures to micro-services application architectures with the emergence of several JavaScript libraries and frameworks.
With a micro-services architecture, an application is separated to enable a more agile software iteration methodology. Successful software is about quickly iterating and responding to a feedback loop. Software requirements regularly change — sometimes they change while building a piece of functionality. Software built six months ago may not be required today. A monolithic application can be much more complex and limiting to update due to sequential dependencies.
Likewise, the nature of traditional on-premise hosting has many limitations. These limitations include the capital investment for the physical server space, servers, racks, switches, routers, cables, multiple high-speed bandwidth backbones, redundant power supplies, generators, tanks for fuel storage, 24/7 staff, etc, etc. Ordering and receiving physical hardware takes time — and can include delays. You also have to acquire software licenses, which is generally a complicated and confusing process (do you need seats/CALs/per-core licensing, yuck!).
Some organizations migrated away from the astronomical costs of traditional on-premise hosting for co-location hosting. This enabled organizations to securely place physical hardware in someone else’s data center, under lock and key in a cage (or multiple cages), and leverage the data center’s resources including networking, bandwidth, physical security, and power (many of the resources mentioned earlier). You still needed to buy physical servers, license software, and have an IT staff to maintain your hardware and software infrastructure, though. The benefit was that organizations eliminated the requirements of having physical space for your data center, physical security, dedicated and redundant bandwidth, redundant power supplies, generators, and tanks for fuel. Obviously this saved a tremendous amount of time and money.
Managed hosting emerged as a viable option too. This enabled customers to rent dedicated hardware, managed by skilled technicians and hosted in their data center. Managed hosting was the option I elected to use in a previous position in 2007. Even when I accurately forecasted traffic patterns I ‘over spent’ to accommodate for potential traffic spikes. I spent about a half-a-decade of my career stressing out over whether I had procured enough physical hardware to handle seasonal traffic spikes when I led the engineering team at an online costume retailer. Each August I guessed (I called it ‘forecasting’ to sound sophisticated) how many temporary seasonal servers would be required to support our busy traffic spike each year. Compared to cloud standards, the company I worked for paid exorbitant amounts of money to have additional temporary servers for three months each year in support of seasonal traffic spike. I vertically scaled databases from a set of small servers to a set of larger servers and horizontally scaled a bunch of web servers. If I guessed capacity incorrectly, the e-commerce application would experience lag — or could have possibly crashed under the traffic load. This could have been catastrophic as employees could have lost their jobs, including me! Fortunately, I guessed correctly and avoided those consequences. I hosted everything in a single data center , and if that single data center went down during the busy season it could have jeopardized the company’s ability to stay in business too. At times, that data center did have issues— fortunately, this never occurred during our busy season, but it was certainly a concern for me and something everyone with mission critical applications should consider.
With on-premise, co-location, and managed hosting services you have to plan well in advance to accommodate estimated traffic. When you need additional servers it can takes days, if not weeks, to increase your infrastructure requests.
Cloud alleviates many of the limitations of physical hardware in a single data center in favor of the flexibility of virtualization. Instead of taking weeks to stand up new physical servers you can spin up Virtual Machines (VMs) in minutes. The customer is responsible for managing their VMs, which are called Elastic Compute Cloud (EC2) instances in the AWS ecosystem. EC2 instances come in a variety of flavors and support various operating systems and configurations — from 1 vCPU and 0.5GiB of RAM (t2.nano) to dozens of vCPUs and hundreds of GiBs of RAM. EC2 is how I’ll be referring to VMs for the remainder of this article — specifically EC2 instances running Linux.
A vCPU is not necessarily equal to one physical CPU thread.
You can rent EC2 instances ‘On Demand’ and pay by the minute without having a long term commitment. If you’re willing to commit to having EC2 instances for a year or longer you can purchase reserved EC2 instances and spend less money.
Need a bunch of web servers and want to manually build everything out? Have at it! Spin up a bunch of EC2 instances and place them behind an Elastic Load Balancer. Establish event thresholds which trigger actions using AWS Cloud Watch when specified conditions are met. As an example, when one or more of your EC2 instances exceeds 70% CPU utilization for two consecutive minutes you can automatically have another EC2 instance created. This criteria is flexible. You could elect to reduce the number of EC2 instances if the EC2’s CPU utilization drops below a threshold of 30% for three consecutive minutes, for example — while always retaining a minimum of at least two EC2 instances. Furthermore, Cloud Watch can evaluate the health of your EC2 instances and remove an unhealthy instance from rotation when failing a health check — oh, and Cloud Watch can automatically spin up a replacement EC2 instance in its place.
AWS offers multiple physically separated data centers, connected by high-throughput fiber-optic connections and independent physical redundancies, named an Availability Zones (AZ). Multiple AZs make up a Region, a Region typically includes two or more AZs (remember, an AZ has multiple physically separated data centers). If you’re concerned with having EC2 instances in a single AZ you may spread them across multiple AZs within the same Region to ensure High Availability. You can build an entire virtual infrastructure yourself, or you may opt to use Elastic Beanstalk when using homogenous EC2 instances. This is considered a best practice for solutions including web server farms. Elastic Beanstalk creates and manages all the resources for you automatically —you only pay for the underlying resources (EC2 instances and the load balancers). Note — an Elastic Load Balancer costs about $16/mth ($0.0225/hr * 24 hrs * 30 = $16.20)
You can configure startup scripts when spinning up new EC2 instances, too. You can even use your own custom VM image (known as an AMI) to boot your EC2 instances from, which may include pre-installed software.
Mess something up? Delete your Elastic Beanstalk environment and create a new one, or just re-configure your EC2 instance types (vCPU and RAM) as desired.
Setup a CodePipeline and Code Deployment to automatically update your Elastic Beanstalk environment(s) when code is checked in. With Elastic Beanstalk you might be able to ‘lift and shift’ your application to the cloud with minimal refactoring. This expedites your initial cloud transition, however, you won’t immediately benefit from all the amazing services AWS offers. Regardless, once you’re on AWS you can begin incrementally refactoring applications to leverage other AWS services (CloudFront, S3, RDS, SNS, SQS, ECS, etc).
Sounds terrific! What are the downsides?
Like any disciple, there is a steep learning curve going to ‘the cloud’. Don’t underestimate the significance of this! You’ll need to begin learning the terminology (some of which you’ve been exposed to in this article). Begin understanding how things fit together in the cloud by learning about some of the basic AWS services available. I recommend trying to get ‘something’ working in AWS, a simple Node.js app in Elastic Beanstalk, for example.
Be patient. Trying too much too quickly may become overwhelming— frustrating you and possibly discouraging. AWS is sophisticated and complex, which you’ll appreciate as your infrastructure requirements grow!
You’ll also need to understand how pricing works so there aren’t surprises with your bill. Invest time to estimate your costs using this tool or contact an experienced professional like myself to help. This isn’t easy — it’s difficult and time consuming. Lastly, stay in a single Region while experimenting and REGULARLY check (perhaps DAILY) to make sure you haven’t left ‘temporary’ resources running — this happens to all of us and can be quite frustrating when you discover you’re paying for resources you’re not using (or when resources cost more than you estimated).
AWS offers many services which are free tier eligible for the first year, so for budget conscious experimenters ensure you’re using services within the free tier as much as possible. I encourage folks to periodically tinker with AWS while staying in the free tier for several months. Spin up an EC2 instance — terminate that EC2 instance — get hands on experience and get some small wins. Need help with this? Again, refer to some of my free tutorials — this is really cool, you’ll think! You got ‘something’ working. Continue tinkering and taking incremental steps.
Scaling
If you’re running EC2 instances in an auto-scaling environment, such as Elastic Beanstalk, when traffic increases you’ll notice it takes a few minutes for new EC2 instances to begin servicing traffic. This is due to the requirements of spinning up new EC2 instances, installing your code, passing health checks, and registering EC2 instances with the Elastic Load Balancer before being placed into rotation. Juxtaposing virtualization, which requires minutes to dedicated hardware, which requires days, isn’t a fair comparison. However, as you continue to use AWS your expectations change. While there are certainly techniques to improve usage of EC2 in auto-scaling environments (using golden AMIs pre-installed with your required software), EC2 instances simply cannot spin up at the speed containers can. Furthermore, you continually pay for EC2 instances and Elastic Load Balancers regardless of utilization. These two factors can make a significant impact on your architectural decisions — enter serverless.
Serverless
What is serverless? Containers…well, sometimes.
I’ll start by saying there are servers — AWS abstracts the servers from your purview. In the serverless architectural paradigm your responsibilities are limited to your application’s security and functionality. Serverless essentially means that you’re not responsible for the underlying servers (including hardware and operating system) and you’ll only be billed for usage. Unlike EC2 instances, where you’re paying regardless of whether you’re serving traffic or not, serverless uses a consumption billing model where you only pay for what you use.
Dropping a static client-side application (React/Vue/Angular and/or static assets — HTML, CSS, images) into an S3 (Simple Storage Service) bucket and putting a CloudFront (CDN) distribution in front of the S3 bucket is one of the easiest ways to get started with serverless.
Building and deploying a RESTful API service can be another matter. AWS offers a variety of serverless services — which continually appear to be growing. Some of the most popular serverless services include API Gateway, Lambda (functions), Aurora Serverless, and DynamoDB (key/value & document database). You can build a serverless application using these technologies, which appear to seamlessly scale — with very affordable pricing models.
However, I warn the learning curve is steep and there are many trade-offs you’ll find difficult to navigate through. I advise not initially using API Gateway, Lambda, and DynamoDB for an existing API service as ECS (Elastic Container Service) Fargate is a terrific serverless alternative (note ECS is not a serverless service, only the Fargate flavor of ECS is). You’ll probably need an Application Load Balancer with ECS Fargate which starts around $16/mth. There is a much lower learning curve for this serverless service compared with using API Gateway, Lamdba, and DynamoDB though.
If you want to go full-tilt ‘re-build our entire application serverlessly’ check out AWS Serverless Application Model (AWS SAM). AWS SAM is AWS’s open-source framework for building serverless applications. If you’re interested in exploring AWS SAM further you may get started using my AWS Serverless Repository Application with correlating code repositories.
API Gateway, Lambda, and DynamoDB Downsides:
The learning curve for these services is high. I can’t understate this —most everything about these specific services are completely different than traditional engineering activities. It’s essentially like building an application using a new type of language and a new development methodology — all at once. You can’t just lift-and-shift an existing application to a serverless architecture — you’ll have to re-write most of it or write something from scratch — preferably a prototype — which I highly recommend.
These services aren’t going to solve all your problems, either. They’ll also present several new challenges because the paradigm is different — you’ll have to figure out how to solve these new challenges.
To over simplify this topic, several underlying serverless architectures shard data structures and their data over multiple servers — which is a necessity to seamlessly scale infrastructure horizontally. As an example, AWS used to recommend randomizing prefixes for S3 objects for best performance. The brilliance of the engineering teams at AWS addressed this challenge and solved it for customers, so prefixing objects is no longer required. However, you can face similar challenges with DynamoDB hot partitions. Lambda functions experience cold-starts the first time they’re called and go cold after a threshold of idle time. A new container has to spin up, requiring a warm up period, which varies in time based on several factors including the language used, the libraries you reference, and how your code is structured. There aren’t always simple and concise answers to these challenges.
API Gateway requires you to enable CORS for your client to access API endpoints. Appropriate request and response headers must also exist in your Lambda functions.
Accessing DynamoDB tables can be done more easily by integrating with DynamoDB’s Document Client API. However, you can’t conduct batch operations using DynamoDB’s Document Client, so you have to use a different integration syntax. Scaling is complicated, folks. There are many other challenges you’ll face — none of which are insurmountable — though they are time consuming to understand and you’ll have to put in the work.
Many applications benefit from serverless architectures and I’ve used a serverless architecture for several projects. It’s not for everything, though. Be aware of the trade-offs and understand the investment of time required to achieve success.
I hope you learned something about cloud and enjoyed this article. Checkout my website to learn more about cloud.
