Azure Endpoint Monitoring

Came across this cool feature on Windows Azure Management Portal called “Endpoint Monitoring”. The feature is still in preview, but worth giving a shout-out. Azure Org lately has hit this nice momentum of releasing features one after the other. They initially release the feature as preview and once it stabilizes, once enough hands are dirty and issues are ironed out, they GA it. This is a right way of releasing features to production, in my opinion.

Endpoint Monitoring, as the name suggests monitors if your web service/web site endpoint is up or not. The idea is, you provide an endpoint to Azure configuration and they call back that endpoint periodically and maintain the log for the same. The good thing is, you can make Azure call back your endpoint from different datacenters across the geography. This helps in scenarios where the endpoint is up in let’s say Chicago DC, but is down in Dublin.

Endpoint monitoring lets you monitor the availability of HTTP or HTTPS endpoints from geo-distributed locations. You can test an endpoint from up to 3 geo-distributed locations at a time(for now). A monitoring test fails if the HTTP response code is greater than or equal to 400 or if the response takes more than 30 seconds. An endpoint is considered available if its monitoring tests succeed from all the specified locations.

In Configuration tab of your service/website on Azure management portal, you will see the Endpoint Monitoring section.

 

Image

As you see above, two endpoints have been configured called Ping-Chicago, Ping-Dublin. This means whichever endpoint you provide there would be called back periodically from Chicago and Dublin.

The results of the endpoint monitoring are shown on the Dashboard as below:

Image

The detailed log can be found by clicking on the endpoint hyperlink

Image

A typical ping endpoint code should ping all the dependencies the service relies on. E.g. if service uses a SQL Azure database, Azure Storage etc. then your ping endpoint should call these dependent endpoints and return HTTP 200 if good and HTTP 500 if bad. Here is a simple code that can be used in your SOAP/REST service as the ping method.

Image

Know it. Learn it. Love it.

 

Monitoring Azure deployed services using NewRelic

Windows Azure is becoming a de facto platform to distribute the applications and services using Microsoft stack. With the platform gaining momentum with tons of new features getting added every 3 weeks, one dimension cannot be missed which is maintaining these apps and services. For IT folks – it’s a total paradigm shift. Laying down infrastructure topology, deployment, formulating disaster recovery strategy etc. are few examples. Here we are going to discuss only monitoring aspect.

Application monitoring can be defined as processing and use of related IT tools to detect, diagnose, remediate and report an application’s performance to ensure that it meets or exceeds end-users’ and businesses’ expectations.

There are two basic methods by which application performance can be assessed.

  1. Measuring the resources used by the application
  2. Measuring the response time of applications

The most common aspect from operations perspective is monitoring of all components and services and the ability to troubleshoot if things go wrong. This typically involves monitoring various performance counters, monitoring log files for alerts, monitoring availability of the application and the ability to configure automated alerts for all these monitoring activities.

With Azure deployed applications, applications/services need to be enabled in certain way so that those can be monitored by operations team.  For such applications/services health monitoring, operations team can follow following strategies:

Inside out monitoring

If the application and services has logging, tracing and performance counters embedded directly into the code then we can use this strategy. Code reports all the health and performance counters, which Windows Azure Diagnostics monitor can then collect and report.

This also can be achieved using a health dashboard page which is authorized to be viewed only by the operations team. This page or component makes dummy service calls to all the services and components based upon the scheduled frequency and displays the success/failure for the same. It can ping the database (SQL Azure), external endpoints (E.g.: endpoints exposed on Service Bus, Storage etc.) and publish the response on the page.

Outside-in monitoring

This means that application itself does not take care of reporting the health and performance and it’s the duty of the external component. In this technique – you need a piece of software (an agent) that gets deployed along with the package which works in a low priority demon and reports the health and performance to a remote server. The agent looks at the deployed code and “reflects” the status dynamically.

My vote always will be with outside-in strategy. This gives you more flexibility and achieves what’s popularly known as separation of concerns. Separating the actual product code from monitoring code. E.g. If Ops team want to change let’s say some perf. counter they do not need to touch the actual code for that.

I came across this awesome tool/service in Windows Azure Store called NewRelic. NewRelic can be used to monitor Web Sites, Web/Worker Roles and VMs. The kind of statistics it produces with a large degree of simplicity is just breathtaking. New Relic is the all-in-one web application performance tool that lets you see performance from the end user experience, through servers, and down to the line of app code. As far as I know, New Relic is the only SaaS app monitoring tool that monitors the entire app using a single product and a single UI. It also gives you this beautiful real-time performance dashboard that can be accessed anytime, anywhere.

In the Windows Azure portal if you go to New -> Store -> Choose an Add-on you will see New Relic add-on. This can be used to create a New Relic end point.Image

Image

As you can see New Relic has multiple plans to choose from. The paid plans offer more detailed diagnostics and data. Visit their web site for more details regarding plans. For the tutorial sake – I am sticking to Slandered (Free) which is far more than sufficient. Go ahead, and provide a name to your New Relic endpoint, which creates the New Relic endpoint.Image

If you go on your dashboard it’s going to show as follows. Click on the endpoint name to see more details.Image

Clicking on Connection Info will give you the API/license key. Save it, we are going to need it.Image

In Visual Studio go to your Azure Web Site or Web role. Open Nuget Package Manager window and install NewRelicWindowsAzure package. You will prompted for authorization, provide the API key which we got from the above step.  After the setup is done, deploy the application/website. Image

Go back to New Relic endpoint and click on “Manage” to navigate to the New Relic dashboard for your application. Image

If things are deployed correctly and successfully you see various beautiful graphs like as follows:

Image

Image

ImageImageImageImage

Azure Pack – my 20 cents

Couple years back, Microsoft started talking about the “mystical” Azure Appliance and entire community thought they finally found the Holy Grail solution for running their services/applications securely on cloud. The idea was that Microsoft was to give all Windows Azure goodness in “one box” and enterprises can run their stuff from behind their fire walls using all awesome things that Azure provides. Azure Compliance never happened. Nobody knows why.

Meanwhile Scott Guthrie – The Rockstar performer – was asked to govern and own Windows Azure organization. Scott with his technical brilliance and roots being firmly buried into community completely shifted the momentum and Windows Azure became this cool, “open” technology. Scott Gu quickly realized the pulse and got no. of features onboarded one after another. One of the most important being a huge release they did called Infrastructure As Service (IaaS). Suddenly Microsoft was talking about the Cloud OS. Cloud OS has been Microsoft’s vision wherein they are talking about a simple, uniform cloud platform for everybody which provides clear interface for Public Cloud as well enterprise folks to get quickly onboarded and use the power of cloud. OS no bar. Technology no bar. Just plain cloud power. Well, Windows Azure was always a complete (sorta ;) ) solution for folks who want to deploy their apps/services in public domain. But in the light of fierce completion, price war and the failed attempt to address the enterprise cloud (Appliance) it became very essential for Microsoft to provide a consistent story for enterprises as well. Microsoft always had Hyper V and System Center for enterprise people, but how can this infrastructure use Windows Azure’s appeal and goodness? In comes Azure Pack.

Azure Pack – if I go by the definition – is a technology that runs inside the enterprise datacenters, behind the firewall and provides the self-served, multi-tenant services like Windows Azure. Community starts dreaming again. Private cloud in box? Holy Grail? Azure Appliance reborn?  Well not really, let me explain.

Azure Pack is far from “private cloud in a box”. A better description would be a Windows Azure like management portal, management API in front of the existing Hyper-V/System Center infrastructure. It has absolutely nothing to do with the Azure Appliance. So Azure Pack works as the wrapper on your Hyper V with a better user experience. But it definitely becomes critical piece of Cloud OS jigsaw puzzle, as it enables enterprises, to put a very nice Azure-like portal/interface in front of their private cloud infrastructure.

The fact that using your in-house infrastructure to scale your enterprises with VMs, Web Sites, Service Bus is really exciting. Bunch of same administrators and developers can build and distribute the code very securely without any special training gives me goose bumps. Azure Pack definitely opens tons of new business scenarios, e.g. now you can create and distribute “Well-Defined” VM Templates for your enterprises very seamlessly from the Gallery.

As of now, Azure Pack comes with following stuff:

  • Management portal for tenants

Windows Azure like self-service portal experience for provisioning, monitoring and managing services such as Web Sites, Virtual Machines and Service Bus.

  • Management portal for admins

A portal for admins to configure and manage resources, user accounts, tenants and billing.

  • Service management API

Doing all the management stuff using neatly designed APIs, so anything and everything can be scripted

  • Web Sites
  • VMs
  • Service Bus

Nobody knows how tough it’s going to be to setup an Azure Pack in the data centers. The pricing model is not defined yet as well. So I can’t comment on the success yet. The existing thing for me as the Windows Azure developer is there are more avenues to go and implement stuff now. Hopefully, this release provides opportunity to more and more enterprises (especially in banking, healthcare sector) to go and taste Windows Azure. Amen!

DevOps – My 20 cents

Context

Before going in DevOps details – let’s think how business requirements are delivered these days. While many organizations have understood the importance of delivering features in multiple small sprints instead of doing them in one big monolithic waterfall block – practically every organization is practicing some version of “agile” methodology (XP, CanBan and so on). Which means teams get requirements in the form of user stories that traverse to production from Development team to Test team and Ops team in the end. Nothing wrong with this methodology really, it works ok sprint after sprint. But with the fast paced world we live in, what if the critical business requirements just pour in at a brisk pace which need to be deployed to production immediately. Probably the version of Agile we use might not scale.

We know and exercise “Continuous Integration” which means any piece of code that is being checked in has to be rock solid and is tested automatically every time. Now – organizations (especially the ones who use some sort of virtualized environments to deploy their services/apps) have gone a little ahead and have implemented what is known as “Continuous Delivery”. This means every check-in is not only run against the tests suite on build server, but gets deployed to staging and production as well. Many organizations, do multiple deployments to production on any particular day. This becomes possible when everything your team is doing is very well automated, planned scripted and all your engineering departments collaborate with each other.

E.g. Facebook does multiple deployments/day. The way they have implemented is any feature that Development team commits is tested rigorously against automated test cases locally and staging before getting exposed to the production. Obviously the feature is not enabled to the entire audience directly. Using traffic routing in load balancer partial traffic is routed to the new feature which yields performance numbers and user feedback. Once confident – the feature is made available globally to all users. This concept is known as testing in production which is the topic for my next post. This is obviously done using precise planning, scripting and automation.

What is DevOps?

DevOps – in my mind is the way continuous delivery/deployment is achieved by making Development and Operations/IT folks talk with each other, collaborate with each other and last but not the least trust each other. The way test engineers collaborate with Dev teams and start writing test cases from Day 1; with the amount of deployments that are happening to cloud these days, time has come to invite Ops team in the design reviews and get them working from Day 1.

The applications/services getting deployed to cloud differ from the traditional deployments in many ways. Logging/monitoring, SCOM packs or alerting, security, scalability and availability – are buzz words with cloud deployments. E.g. For logging and monitoring, Dev team cannot afford to say that it’s Ops headache and for me it’s a low priority task. Logging and monitoring has to be baked into the code from Day1.  Your services and apps need to be secure from Day1, because once your feature is checked in, it might be deployed to production straight away. DevOps practice – essentially tears the walls down between Dev and Ops teams. Both teams work collaboratively on planning, preparing environments, configuring security, writing the SCOM packs etc. To implement DevOps – everything including integration tests, BVTs, Smoke Tests, environments, configuration and deployments need to be scripted and automated.

Delivery methodologies that are used by organizations traditionally with classic setup of separate teams for Devs, Test and Ops did not have deep cross-teams integration. DevOps promotes a suite of common processes, methods and tools for communication and collaboration between teams. DevOps – really is a collaborative work relationship between development team and operations team.

Conlusion

Traditionally, Devs target set of goals like architecture/implementation, while Ops team worry about availability and stability. It is designed that way keeping separation of concerns in mind. With DevOps – both these teams do not have 2 separate goals. While Dev team need to care about the availability of the service in production, Ops team need to care about how security has been implemented. Like I said before, it’s about tearing the walls down, if not – at least making holes in walls, while marching towards a common set of goals. In fact, many organizations have started hiring developers with some sort of IT/Ops background. Looking at the Cloud computing burst, that day is not too far when there would not be two separate departments for engineering and IT.

To conclude, I am trying to define DevOps again! In my mind, it’s a relationship that increases the overall delivery efficiency and reduces the production risk associated with frequent releases.