Introduction

Windows Azure provides a variety of possibilities to automate management and deployment of Azure services, for example through scripting from the command line, programmatically calling the Management API or using tools such as Systems Center Operations Manager. In this blog post I introduce a pattern for building a scalable custom provisioning service for a Windows Azure application.
Fully automating the deployment procedures for an app is always a good idea in order to have a consistent and predictable process. This is specifically important if your app has a complex structure of services and/or you want to automate deployment where you anticipate often dynamic changes in the environment (e.g. when serving a large number of tenants and regularly need to adjust deployed resources to serve additional tenants).

Example Pattern

The following graphic provides a high level overview of the architecture of a provisioning service.

Figure 1 – Provisioning Service Overview

The provisioning service can receive requests from different clients. It can be used for manual provisioning of tenants by a management portal, automated from a monitoring solution for dynamic scaling or from the application itself. For example, the application might need to pro-actively create additional resources for planned workloads, based on predefined rules, business logic or schedule. In addition, the service can be deployed in another datacenter (DC) and can be used in a case of disaster to spin up the application on a secondary site. In one of my previous blogs I described a cross-datacenter disaster recovery solution that uses a provisioning service to redeploy the app in a secondary DC.

Once a request is received, the dedicated provisioning service takes care of the deployment and configuration of app services. A good practice is to manage deployment in scale units. Ideally, the scale unit for an application should be defined in relation to expected workload (e.g. number of users). The scale unit typically will define a number of different resources for specific workload – e.g. X number of a web roles, Y number of a worker role, additional storage, etc. necessary to serve a specific number of users. The provisioning service can perform deployments in scale units, i.e. provision the required resources for a specific number of scale units.

The service uses queue(s) to decouple the processing. Optionally, different queues can be used to manage different priorities or potentially to decouple provisioning of independent components of the app.

Worker roles are responsible for processing the queue messages/jobs and use the Azure Management API to programmatically create the necessary Azure resources, as well as to configure them as appropriate. Service packages and reference app data stored in Windows Azure Storage (WAS) is used during provisioning. The worker roles can be fanned out dynamically based on the current workload (i.e. queue size) to speed up deployment. For example, in the case of a disaster recovery failover to a secondary site for a large deployment, all resources need to be provisioned at once. In that case the worker roles can be scaled out in order to accelerate processing and reduce RTO.

The outcome of the provisioning jobs is persisted in a repository (e.g. SQL Database). This repository contains the entire "footprint" of the deployment (i.e. all currently provisioned resources). It can be used to list or visualize all resources being utilized, organized by app specific boundaries (e.g. currently running services in scale units per tenant).

Summary

In conclusion, I have provided a short overview of a pattern for automation of app deployment. In general, it is beneficial to fully automate deployment and abstracting it as a service allows you to use it in a consistent and unified way for initial deployment as well as for scaling in different scenarios: such as manual provisioning through a UI or automated as part of the app logic or from a monitoring solution. For sure, this is not the only option for implementation, but hopefully reading through this article has generated some ideas for your application.