This directory contains an example of the Priority Queue pattern.

This example demonstrates how to implement priority queues by using Service Bus topics and subscriptions. A time-triggered Azure Function sends messages to a topic, each with an assigned priority. The consumer Azure Functions read messages from subscriptions that match the corresponding priority.

In the Azure deployment, the PriorityQueueConsumerHigh function can scale out to 200 instances, while the PriorityQueueConsumerLow function can scale out only to 40 instances. This simulates high-priority messages being processed with higher urgency than low-priority messages.

The Azure deployment also demonstrates operational aspects of applications running on Azure. Monitoring tools are essential to understand how the sample behaves. The Azure Functions in this solution must be configured to use diagnostics. Otherwise, the trace information generated by the example will not be visible.

Install the prerequisites and follow the steps to deploy and run this Priority Queue pattern example.

• Permission to create a new resource group and resources in an Azure subscription.
• Git
• Azure CLI
• .NET 10 SDK
• Azure Functions Core Tools v4.x

1. Clone the repository

   Open a terminal, clone the repository, and navigate to the priority-queue directory.

   git clone https://github.com/Azure-Samples/cloud-design-patterns.git
   cd cloud-design-patterns
   cd priority-queue

2. Sign in to Azure and create an empty resource group.

   Create an empty resource group to hold the resources for this example. The location you select in the resource group creation command below is the Azure region that your resources will be deployed in; modify as needed.

   az login
   az account set -s <Name or ID of subscription>
   
   LOCATION=eastus2
   RESOURCE_GROUP_NAME=rg-priority-queue-${LOCATION}
   az group create -n $RESOURCE_GROUP_NAME -l $LOCATION

This Bicep template sets up the core infrastructure for a priority-based message processing system on Azure Functions. It creates a secure Storage Account, an Application Insights instance for monitoring, and a Service Bus namespace to enable communication between the sender and consumer functions. The deployment includes three Azure Function Apps: one sender and two consumers, each with different scaling limits to simulate message prioritization. The funcPriorityQueueConsumerHigh function can scale out to 200 instances, allowing it to process high-priority messages quickly. The funcPriorityQueueConsumerLow function is limited to 40 instances, handling lower-priority messages with less urgency. All function apps use the Flex Consumption plan and are connected to Application Insights for diagnostics and monitoring. Role assignments are configured to grant secure access to Service Bus and Storage by using managed identities. All Azure Function Apps share the same Storage Account and Application Insights instance, which centralizes observability and logging.

   SERVICE_BUS_NAMESPACE_NAME="sbns-priority-queue-$(LC_ALL=C tr -dc 'a-z0-9' < /dev/urandom | fold -w 7 | head -n 1)"
   # This takes about three minutes
   az deployment group create -n deploy-priority-queue-sites -f bicep/main.bicep -g $RESOURCE_GROUP_NAME -p serviceBusNamespaceName=$SERVICE_BUS_NAMESPACE_NAME 

After deploying the infrastructure, publish each Azure Function to its corresponding Function App by using Azure Functions Core Tools:

   # Construct the fully qualified namespace (hostname) for the Service Bus.
   SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE="${SERVICE_BUS_NAMESPACE_NAME}.servicebus.windows.net"

   sed "s|{SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|${SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|g" ./PriorityQueueSender/local.settings.template.json > ./PriorityQueueSender/local.settings.json
   sed "s|{SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|${SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|g" ./PriorityQueueConsumerHigh/local.settings.template.json > ./PriorityQueueConsumerHigh/local.settings.json
   sed "s|{SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|${SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|g" ./PriorityQueueConsumerLow/local.settings.template.json > ./PriorityQueueConsumerLow/local.settings.json

   cd ./PriorityQueueSender
   func azure functionapp publish funcPriorityQueueSender --dotnet-isolated
   cd ..

   cd ./PriorityQueueConsumerLow
   func azure functionapp publish funcPriorityQueueConsumerLow --dotnet-isolated
   cd ..

   cd ./PriorityQueueConsumerHigh
   func azure functionapp publish funcPriorityQueueConsumerHigh --dotnet-isolated
   cd ..

You can view the maximum scaling configuration in the Azure portal. Go to each Function App, and then under Settings, select Scale and concurrency. There, you can see the Maximum instance count setting.

Once the Azure Functions are deployed, you can use Application Insights to monitor activity. In the Azure portal, go to the Application Insights resource and select Logs. Switch to KQL mode and run the following queries to view trace logs for each function:

   // Traces for the High priority consumer
   traces
   | where operation_Name contains "High"

   // Traces for the Low priority consumer
   traces
   | where operation_Name contains "Low"

   // Traces for the Sender function
   traces
   | where operation_Name contains "Sender"

You can also use Application Insights to analyze request-level data for each Azure Function. The following queries help you inspect recent requests and understand how frequently each function is being called:

   // Recent requests with key details
   requests
   | project timestamp, operation_Name, cloud_RoleName, id, success, resultCode, duration, operation_Id
   | order by timestamp desc

   // Count of requests by Function
   requests
   | summarize RequestCount = count() by operation_Name
   | order by RequestCount desc

Be sure to delete Azure resources when not using them. Since all resources were deployed into a new resource group, you can simply delete the resource group.

az group delete -n $RESOURCE_GROUP_NAME -y