Once upon a time in Azureland!

Premise

Let me tell you my story about Azure. While Azure is very resourceful now, it used to be very experimental during its first incarnation back in 2009/ 2010. There were only a handful of services available and very few bells n’ vessels to play with.

This story is from 2009/2010. During the summer of 2009, Accenture acquired a company called Origin Digital. As the name suggests, Origin Digital was into digital media. They had an online product that used to transcode the videos and images from any format to any other required format. They used to get tons of traffic during Thanksgiving and Holiday Season since more folks logged-on to transcode videos. During the 2008 Holiday season, Origin Digital saw a large spike that took down the online transcoder service since the on-premise data center in Virginia got overwhelmed.

While this was happening back in the US, a couple of very important events were happening in Redmond and India respectively.

• Event 1: While Amazon’s AWS was extremely popular, Microsoft decided to take a completely fresh approach of looking at the cloud computing with flagship Platform As Service Offering called Azure (it was then called as Windows Azure)
• Event 2:  Avanade and Accenture partnered with Microsoft and decided to form a Center Of Excellence (CoE) in Pune, India.

One of the first challenges given to Azure Factory was to take Origin Digital’s online transcoder service to Azure. Gauntlet was thrown!

Breaking Shackles

The idea was – since Azure provides scalability, elasticity and processing muscles on-demand – it should be able to handle the traffic spike that was observed by Origin Digital’s on-premise datacenter during the Holiday season.

Back then, Azure PaaS had very limited services like Cloud Services (Web Roles, Worker Roles), Azure SQL Database to handle relation data and Azure Storage to handle the unstructured data like files.

We decided to do this in 2 phases. In the first phase only “lift and shift” was targeted. The idea was to move the code as-is to Azure with as minimal changes as possible. The second phase was all about making the architecture conducive for Microsoft Azure.

As part of Phase 1, the web APIs and web applications were migrated and encapsulated into Web Roles and Windows Services were packaged as worker roles. The existing SQL database was migrated to Azure SQL. Code was required to be changed since the old code was storing the files into on-premise storage while for scalability and cheap cost reasons Azure Blob Storage was the best. While Azure Migration Assistant was not available back then, we were not only successfully able to lift and shift existing app to the cloud successfully but were able to share a lot of our lessons to the community. Remember these were early days and lessons learned were like goldmines since the industry was still struggling to understand what Azure is!

Phase 2 took a while as we designed and refactored for the cloud without a lot of documentation and patterns/practices. We were able to implement auto-scaling. Per problem statement, the business wanted a design that would instantiate more Encoder service instances looking at the number of jobs on queue and at CPU/Memory of other encoder instances. We wrote custom orchestrator service that would monitor the metadata emitted by each instance and add /remove worker role instances using Windows Azure Service Management APIs. (Remember this was early 2010 😊).

We had a lot of fun in making the service production-ready! This was pre-VSTS – when DevOps itself was a relatively new concept for the industry. Implementing CI/CD pipelines was interesting. Microsoft had published few cmdlets and PowerShell commands wrapping Service Management APIs that were configured in Team Foundation Server (TFS) to deploy the services, web apps into Azure.

The monitoring and logging story was interesting as well. Azure had Windows Azure Diagnostics (WAD) and these WAD agents used to run on all cloud service VMs. This agent used to collect all the diagnostics events, event logs, errors, crash dumps and upload them to the Azure Storage Account.

Conclusion

It was fascinating what we were able to achieve with so little! 

 

When I see new and shiny toys that engineers get to play with Azure these days- it puts a smile on my face. It is very apparent the vision Microsoft had for Azure was different and it transformed into this reliable platform after years of innovation and evolution. Glad that I was part of the journey and more to come! This Origin Digital story was published as a success story in MSDN magazine and was showcased by Microsoft in 2010.