TL;DR using Azure Open AI Service

Premise

No, I am not talking about Chat GPT or Generative AI in this post. Those topics are already well documented. My intention for this post is to talk about Azure Open AI Service – which is a new managed, platform-as-service offered by Microsoft.

In this post, I am showing how to consume Azure Open AI Service from a development point of view. I am demonstrating how Azure Open AI Service-based application can be created, and how extremely simple it is to use!

Let’s Cook!

Azure Open AI Service is invitations-only and not available in many regions yet (only available in East US, South Central US & West Europe, as of now). It is nothing but a REST API wrapper on top of OpenAI’s language models including the GPT-4, GPT-3, Codex, and DALL-E. These models can be easily adapted to the required business domain scenarios that demand content generation, content summarization, search, and natural language to code translation. Users can access the service through REST APIs, SDKs available for various languages, or by using the web-based interface in the Azure OpenAI Studio.

Certain Keywords that developers need to be familiar with while using Azure Open AI Service are as below:

Prompts & Completions	Azure Open AI Service provides a simple but powerful text-in, text-out interface to any of the models. User inputs some text as a prompt, and the model generates a text completion that attempts to match whatever context or pattern. With prompt-based models, the user interacts with the model by entering a text prompt, to which the model responds with a text completion
Deployments	Once you create an Azure OpenAI Resource, you must deploy a model before you can start making API calls and generating text. This action can be done using the Deployment APIs or one could use a GUI available on the Azure Portal. Deployments allow you to specify the model you wish to use.
Tokens	Azure OpenAI processes text by chunking it down into string tokens. Tokens are words. The total number of tokens processed in a given request depends on the length of input, output, and request parameters.

Keywords assocoaited with Azure Open AI service

Various Models available

Azure OpenAI Service model names following a certina naming convention:

[capability]-[family]-[identifier]

Capability : E.g., GPT-3 models uses text, and Codex models use code.

Family : E.g., GPT-3 models include ada, babbage, curie, and davinci.

Identifier : Version identifier

Following are the most popular basic text-based GPT3 models and reasoning which can tell when to use what

Ada	Fastest model. Can be used for: Parsing text, simple classifications, address correction etc.
Babbage	Can be used for moderrate text based classifications, semantic searches.
Curie	Used for language translation, complex classification, text sentiment, summarization.
Davinci	Most sophisticated. Used for complex intent, cause and effect, content summarization.

Various GPT3 models

Let’s play!

I am a C# developer, so I was eagerly waiting to be able to program using Visual Studio, C# (my power tool of choice). And the wait is over. While Azure Open AI can be called as any REST API using HttpClient, my recommendation is to use the SDK/.NET Client created for the API.

Step 1 : Get access to Azure Open AI Service using this link (as of now, it’s by invitations only)

Step 2 : Once you have the access – create a new Azure Open AI Service resource using CLI or Azure Portal. Please note it’s available only in a certain set of regions.

Step 3: Grab the keys/credentials, and endpoint from the Azure Portal. Please take a note that “Azure OpenAI Studio” – which is a web-based tool you can open directly from the Azure portal to play/test and interact with your deployments.

Step 3: Once you have the resource, next thing you want to do is create a depoyment and grab the deployment name. While creating the deployment, you need to chose which model you want to use. Here I am using, text-davinci-03 model.

TL;DR Service

Here I am showcasing a simple, reusable service that gets a large string as an input talking about a certain topic and then use Azure Open AI Service to summarize it using Azure Open AI Service.

While GPT is capable of content generation, content summarization, search, and natural language to code translation etc. here I am showing content summarization aspect.

Here is how the code looks like:

namespace TLDRService
{
    // Encapsulate the setting and configuration 
	public class AzureOpenAIServiceConfiguration
	{
		//Azure Open AI Deployment Name
		public string DeploymentName { get; set; }

		//Azure Open AI Resource Key
		public string Credential { get; set; }

		//Azure Open AI Resource Endpoint
		public string Endpoint { get; set; }
	}

	public class TLDRService
	{
		private readonly AzureOpenAIServiceConfiguration azureOpenAIServiceConfiguration;

		public TLDRService(AzureOpenAIServiceConfiguration azureOpenAIServiceConfiguration) 
		{
			this.azureOpenAIServiceConfiguration = azureOpenAIServiceConfiguration;
		}

		public string GetTLDR(string messageToBeSummarized)
		{
			OpenAIClient client = new(new Uri(this.azureOpenAIServiceConfiguration.Endpoint), new AzureKeyCredential(this.azureOpenAIServiceConfiguration.Credential));

			var completionsOptions = new CompletionsOptions()
			{
				Prompts = { ConvertToBasicPrompt(messageToBeSummarized) },
				MaxTokens = 250 // hard coded for now, can come from config
			};

			var completionsResponse = client.GetCompletions(this.azureOpenAIServiceConfiguration.DeploymentName, completionsOptions);
			return completionsResponse.Value.Choices[0].Text;
		}

		/// <summary>
		/// Create a prompt for given message
		/// </summary>
		/// <param name="message"></param>
		/// <returns></returns>
		private string ConvertToBasicPrompt(string message)
		{
			return @$"
											Summarize the following text.

											Text:
											""""""
											{message}
											""""""

											Summary:";
		}
	}
}

I wrote a tiny Windows app client to test the service and it works like a charm.

Conslusion

Azure Open AI Service is a managed, platform-as-a-service just like any other Azure service with performance, ease of use, security, and scalability. It provides a perfect abstraction layer to use Open AI models. So, as a developer – you just get to focus on the scenario you are implementing and leave all the complexities to Microsoft Azure which we know and love.