Checklist part 3: Scaling on Azure - The basics

When you start developing your new solution then scaling probably isn’t the first thing you think about. You first need to know how the market responds, what your chances of growing are or maybe you just forgot about it. And that’s alright. But eventually performance and resource utilization will hit its limit and scaling can become your nemesis. So, what are your options? With Microsoft Azure you have a wide range of platforms available to you, each with their own scaling capabilities.

Note that there is a lot more to scaling from a technical perspective, but it all starts with understanding what we’re talking about. Because even though scaling your solution is mostly a technical exercise, you still need a strategy and a plan just as if you would plan on growing/scaling your business.  

Where do you start? Do you just need more power? Do you want to scale dynamically and how fast are you growing? Is that growth local, regional or global? That’s a lot of questions, let’s get started with the basics and your options.

Scaling basics
Let’s first look at Vertical Scaling and Horizontal Scaling. Picture the following scenario: you started a transporting business where you transport people to the airport. Over time you notice your car is struggling to keep up as the weight of the luggage increases and you must decline customers as your car is simply full. You tell them to wait for the next round trip and before you know it you have a big queue standing in the parking lot.

This is basically a request and demand scenario where your car simply can’t keep up with the demand. We need to solve this.

Vertical Scaling
When we talk about vertical scaling we are talking about the scenario’s where you “scale up” and “scale down”. If you translate this to our transportation scenario then what we’re doing is “scaling up” the car. We’re getting a bigger car and maybe add a trailer to it. The downside here is that you will have to wait for your new car to be ready. You either have to plan this ahead and swap it outside of business hours or tell your customers to wait for a bit. But, once you have your car, you’re good to go!

Over time your business grows more and more and hits the maximum capacity of what your car can bring to the table. Now what? Your next step would be to get a bigger car, van or even a bus if the demand is there.

All this swapping of resources takes time and there is a limit to how far you can go. What if you do get that bus and you’re still not keeping up with demand? And what if your demand peaks between 10:00am and 14:00pm? Do you keep driving that expensive bus outside of business hours? If you’re growing steadily and you can predict your growth, then all this won’t be much a problem. But at some point, it may become one.

Basically, what you’re doing here is adding more power and this is exactly what you do when you vertically scale your resources. You’re adding more CPU and more memory to your system to cope with the demand. The action of upgrading generally comes with downtime (on Azure this is only seconds – minutes depending on the type of resource). This means you have to plan ahead, and you definitely don’t want to scale during peak hours.

On Azure vertical scaling can be done with virtually any resource. But as with the depicted scenario above, there’s a limit to it. And is a rule thumb: Scaling up doesn’t increase availability, you’re still focusing on one instance.

On the plus side, there is usually never an issue when scaling up and it’s very effective. You’re simply adding more power and giving your solution more room to breathe. If you haven’t thought of scaling when you started building your solution, then vertical scaling is the way to start. What you need to do is to determine your scaling strategy before you hit that limit. The great thing about Azure is that you can scale up, add more power and when you no longer need it you scale back down. The costs will scale as well because on Azure you only pay for what you use (or provision 😊)

You just need to plan this outside of business hours to minimize the impact which scaling actions have on user experience. Long story short: Great way to start!

App Service - Vertical Scaling

Horizontal Scaling
As opposed to vertical scaling, we have horizontal scaling.
With horizontal scaling we’re not looking at upgrading our current car. We’re simply adding more cars and drivers. This comes with other logistic challenges, but we don’t have to worry about the availability of the cars during upgrades and we can choose to use 5 cars during peak hours and only 2 during the rest of the day. Add some smart resource planning tooling into the mix and you can automatically send drivers a message when demand drops, and you don’t require there services for the rest of the day. Pretty flexible right? This is what we call horizontal scaling. We’re not adding more power (or upgrade) to one specific car, we’re adding more cars to the resource pool.

Now your challenge is to distribute the load over those cars. Do you evenly distributed passengers over all your cars?  Are all cars suited for all types of passengers?

If you translate this to Azure, then this is where things can get a little confusing. Configuring horizontal scaling works slightly different for different technology stacks. If you’re using Infrastructure as a Service you need to deploy multiple virtual machines (or instances if you’re using VM Scale Sets), configure a load balancer and manage your back-end pool.

If you’re using Platform as a Service (PaaS) such as Web Apps, then horizontal scaling is done as simple as sliding the bar to add more instances. You can automatically scale based on metrics such as CPU, Memory, IOPS or custom ones if you desire. And if you’re going serverless, don’t worry about it at all! Azure will do all that for you😊

App Service – Horizontal scaling

What’s the catch here? If you truly want to be dynamic in your horizontal scaling then you want to make sure your application is stateless. User sessions are cached outside of your solution and if you need to cache data never use memory or local storage but look into solutions such as Azure Redis Cache or Azure Service Bus.

Horizontal scaling also works well for micro services architectures and distributed systems. But if either of those is what you have been doing then it’s likely that you already looked into horizontal scaling or you have been doing it all along.

But, chances are you haven’t thought of this when you just started building your solution and depending on your application, going from stateful to stateless can be quite the project.

“Great, thanks for the information. Do I scale vertically or horizontally?”

When migrating your solution to Microsoft Azure then you have the option of both. If you have a pre-existing solution and the financial impact of going from stateful  to stateless is too big for now, then vertical scaling is your quick win. But as stated, you need to plan your maintenance windows. Your next step would be to gradually evolve your solution into a stateless piece of art before you hit vertical scaling limits on Azure.

If you’re going with Infrastructure as a Service, then it might take you a while to hit these limits on Azure. But don’t forget, high performance virtual machines generally don’t come cheap. What you want to do is do a comparison where you focus on the costs of the Virtual Machine (over a period) and compare those to the estimated costs of refactoring your application to support horizontal scaling.

The same goes with Platform as a Service but you’re likely to hit the limit there a bit quicker. Long story short: Vertical scaling is your quick win when moving onto Azure but if you plan on growing rapidly then horizontal scaling (or combination of both) will be your goal as this provides a higher level of availability and more flexibility (financially).

Global scaling
One of the things Microsoft Azure excels at is global availability. You can literally deploy your application all over the world, with the touch of a button. But what does this mean for your software architecture and your technical infrastructure? What you want to achieve when scaling your solution globally is local presence in a specific region. The technology stack you are using will impact the way you scale globally (Are you using IaaS, PaaS or Serverless?). What’s your business case? Are you going to re-deploy a copy of your architecture to a specific region and doubling your costs (note that pricing can be different per region), are you doing this for just one customer and what about compliance and regulations?

Global scaling considerations
When scaling globally there are three subjects that are trivial for your global scaling strategy;

• Finance;
• Latency;
• Compliance, local laws and regulation.

Finance
Simply grabbing a copy of your environment and redeploying it to another region will at least double your costs. If you’re currently providing a service to a hundred customers and you are scaling to another region for just one customer, then the business case doesn’t look ideal. And let’s be honest: this is usually what it looks like at the beginning (how sure are you that you will be onboarding a lot more customers in that specific region the next few weeks?).

This is where your strategy for scaling vertical and horizontal comes in. If you implemented either one of those, chances are you can scale back to the minimum required for just a few customers while maintaining availability. This is why you need to understand why you scale and when you scale. You want to be in control of your capacity requirements vs. the usage of your platform.

Latency
By far, your single most important asset when scaling globally is data. You want to store your data close to your users. Let’s first look at databases. When it comes to Microsoft Azure you have a few options. If you’re using IaaS and are hosting a MSSQL database on a virtual machine, then you’re likely limited to redeploying that virtual machine in another region with customer specific databases for that region, hosted on that machine. If you want to scale globally you want to look at Platform as a Service or the Serverless technologies that Microsoft Azure provides.

When it comes to databases services, we generally see two solutions being used widely:

Platform as a Service with Azure SQL
Azure SQL supports database sharding. Long story short: this will help you split your data into smaller pieces, allowing you to store those pieces in different locations and increase performance. 

Distributed database services
Distributed databases are built for horizontal scaling and global distribution. You can read and write your data to a specific location whilst still using the same technology and frontend (let’s say an Azure Web App). This sounds a lot like sharding but distributed databases services are much more than that in terms of replication, scaling and partitioning. Azure provides one of these services in the form of CosmosDB  and is considered serverless. Note that this probably requires a different database model as that you’re used to. But if you’re scaling globally, CosmosDB is one of those services that you want to consider using.

In additional to databases there is also “content” in general and the access / endpoint of that content. First of you want to store content close to your customers, without having to copy and paste it all over the globe. Long story short: you need a Content Delivery Network. Microsoft provides this in the form of Azure CDN Azure CDN provides seamless integration with most Azure PaaS services.

And last but not least you want to provide your users with a single endpoint, regardless of where they are. Depending on your use case, Azure Traffic Manager will provide just that for you. Based on DNS, traffic manager will route your users to the backend closest to them.

Compliance, local laws and regulation
Depending on where your customers are, there may be different requirements for the storage data. This is the subject that you want to have all the answers to when talking to your customer. As a rule of thumb, you want to store the data there where your customer is or at least in the same country / region where the same laws and regulations are applicable. You probably already want to do this to manage latency, this is just another reason to store the data close to your customers.

When it comes to compliance: Microsoft provides you with Trust Center where you can browse through all the compliance offerings Microsoft Provide. Additionally, Microsoft has provided thorough documentation on how Azure is built up from a security perspective. For instance: take a look at isolation which is one of those questions that comes back during many onboarding sessions. It is not so much that you can always influence the Azure Architecture, but it is trivial that you understand it and can explain it to your customers.

What’s important is that you look at how you distribute your data and that you can implement the technological requirements for that specific country. And in most cases if you go with the options as we described in the previous paragraph then you have those options. In rare cases there might be different requirements (government customers). In those cases, Microsoft has specific technologies such as isolation regions such as Azure Government and Azure Germany.

Wrap-Up
To wrap it up. There are many ways you can scale, and each strategy has different implications for your solution. What’s important to know is that there is always a quick win and whether you have through of scaling before hand or not, moving your solution to Azure will benefit you. All technologies are there for you to use and succeed in growing your business but it’s up to you to decide where your quick wins are and what your business case is for the short and long term.