Background Processing with Worker Roles and Queues ~ Learn infos

A worker role runs in the background to provide services or execute time related tasks like a service process.
In this exercise, you create a worker role to read work items posted to a queue by the web role front-end. To process the work item, the worker role extracts information about a guest book entry from the message and then retrieves the corresponding entity from table storage. It then fetches the associated image from blob storage and creates its thumbnail, which it also stores as a blob. Finally, to complete the processing, it updates the URL of the generated thumbnail blob in the guest book entry.

Task 1 – Creating a Worker Role to Process Images in the Background

In this task, you add a worker role project to the solution and update it so that it reads items posted by the front-end from the queue and processes them.

If not already open, launch Visual Studio as administrator from Start | All Programs | Microsoft Visual Studio 2010 by right clicking the Microsoft Visual Studio 2010 shortcut and choosing Run as administrator.
In the File menu, choose Open and then Project/Solution. In the Open Project dialog, browse to \Source\Ex2-UsingWorkerRolesAndQueues\Begin, select Begin.sln in the folder for the language of your preference (Visual C# or Visual Basic) and click Open. Alternatively, you may continue with the solution that you obtained after completing the previous exercise.
In Solution Explorer, right-click the Roles node in the GuestBook project, point to Add and then select New Worker Role Project.
In the Add New Role Project dialog, select the Worker Role category and choose the Worker Role template for the language of your choice (Visual C# or Visual Basic). Set the name of the worker role to GuestBook_WorkerRole and click Add.

Figure 23Adding a worker role project to the solution (C#)

Figure 24Adding a worker role project to the solution (Visual Basic)
In the new worker role project, add a reference to the data model project. In Solution Explorer, right-click the GuestBook_WorkerRole project and select Add Reference, switch to the Projects tab, select GuestBook_Data and then click OK.
Next, add a reference to the System.Drawing assembly, only this time, in the Add Reference dialog, switch to the .NET tab instead, select the System.Drawing component and then click OK.
Now, open the WorkerRole.cs file (for Visual C# projects) or WorkerRole.vb file (for Visual Basic projects) of the GuestBook_WorkerRole project and insert the followings namespace declarations.(Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Namespaces – CS)
C#
```
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Drawing.Imaging;
using System.IO;
using GuestBook_Data;
```
(Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Namespaces – VB)
Visual Basic
```
Imports System.Drawing
Imports System.Drawing.Drawing2D
Imports System.Drawing.Imaging
Imports System.IO
Imports GuestBook_Data
```

Add member fields to the WorkerRole class for the blob container and the queue, as shown below.(Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Fields – CS)

public class WorkerRole : RoleEntryPoint


  private CloudQueue queue;
  private CloudBlobContainer container;

...

(Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Fields – VB)

Visual Basic

Public Class WorkerRole

  Inherits RoleEntryPoint





  

  Private queue As CloudQueue
  Private container As CloudBlobContainer




  ...
End Class

Insert the following code into the body of the OnStart method immediately after the line that subscribes the RoleEnvironmentChanging event and before the call to the OnStart method in the base class.(Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole OnStart – CS)

public class WorkerRole : RoleEntryPoint

...

  public override bool OnStart()

    // Set the maximum number of concurrent connections

    ServicePointManager.DefaultConnectionLimit = 12;





  



    // read storage account configuration settings
    CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
    {
      configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));
    });
    var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
    // initialize blob storage
    CloudBlobClient blobStorage = storageAccount.CreateCloudBlobClient();
    container = blobStorage.GetContainerReference("guestbookpics");
    // initialize queue storage 
    CloudQueueClient queueStorage = storageAccount.CreateCloudQueueClient();
    queue = queueStorage.GetQueueReference("guestthumbs");
    Trace.TraceInformation("Creating container and queue...");
    bool storageInitialized = false;
    while (!storageInitialized)
    {
      try
      {
        // create the blob container and allow public access
        container.CreateIfNotExist();
        var permissions = container.GetPermissions();
        permissions.PublicAccess = BlobContainerPublicAccessType.Container;
        container.SetPermissions(permissions);
        // create the message queue(s)
        queue.CreateIfNotExist();
        storageInitialized = true;
      }
      catch (StorageClientException e)
      {
        if (e.ErrorCode == StorageErrorCode.TransportError)
        {
          Trace.TraceError("Storage services initialization failure. "
            + "Check your storage account configuration settings. If running locally, "
            + "ensure that the Development Storage service is running. Message: '{0}'", e.Message);
          System.Threading.Thread.Sleep(5000);
        }
        else
        {
          throw;
        }
      }
    }





    return base.OnStart();
  }
  ...
}

(Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole OnStart – VB)

Visual Basic

Public Class WorkerRole

  Inherits RoleEntryPoint

...

  Public Overrides Function OnStart() As Boolean

    ' Set the maximum number of concurrent connections 
    ServicePointManager.DefaultConnectionLimit = 12




  

    ' read storage account configuration settings
    CloudStorageAccount.SetConfigurationSettingPublisher(Function(configName, configSetter) configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)))
    Dim storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString")
    ' initialize blob storage
    Dim blobStorage = storageAccount.CreateCloudBlobClient()
    container = blobStorage.GetContainerReference("guestbookpics")
    ' initialize queue storage 
    Dim queueStorage = storageAccount.CreateCloudQueueClient()
    queue = queueStorage.GetQueueReference("guestthumbs")
    Trace.TraceInformation("Creating container and queue...")
    Dim storageInitialized = False
    Do While (Not storageInitialized)
      Try
        ' create the blob container and allow public access
        container.CreateIfNotExist()
        Dim permissions = container.GetPermissions()
        permissions.PublicAccess = BlobContainerPublicAccessType.Container
        container.SetPermissions(permissions)
        ' create the message queue(s)
        queue.CreateIfNotExist()
        storageInitialized = True
      Catch e As StorageClientException
        If (e.ErrorCode = StorageErrorCode.TransportError) Then
          Trace.TraceError("Storage services initialization failure. " _
              & "Check your storage account configuration settings. If running locally, " _
              & "ensure that the Development Storage service is running. Message: '{0}'", e.Message)
          System.Threading.Thread.Sleep(5000)
        Else
          Throw
        End If
      End Try
    Loop




    Return MyBase.OnStart()
  End Function
  ...
End Class

Replace the body of the Run method with the code shown below.(Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Run – CS)

public class WorkerRole : RoleEntryPoint

...

  public override void Run()


    Trace.TraceInformation("Listening for queue messages...");
    while (true)
    {
      try
      {
        // retrieve a new message from the queue
        CloudQueueMessage msg = queue.GetMessage();
        if (msg != null)
        {
          // parse message retrieved from queue
          var messageParts = msg.AsString.Split(new char[] { ',' });
          var imageBlobUri = messageParts[0];
          var partitionKey = messageParts[1];
          var rowkey = messageParts[2];
          Trace.TraceInformation("Processing image in blob '{0}'.", imageBlobUri);
          string thumbnailBlobUri = System.Text.RegularExpressions.Regex.Replace(imageBlobUri, "([^\\.]+)(\\.[^\\.]+)?$", "$1-thumb$2");
          CloudBlob inputBlob = container.GetBlobReference(imageBlobUri);
          CloudBlob outputBlob = container.GetBlobReference(thumbnailBlobUri);
          using (BlobStream input = inputBlob.OpenRead())
          using (BlobStream output = outputBlob.OpenWrite())
          {
            ProcessImage(input, output);
            // commit the blob and set its properties
            output.Commit();
            outputBlob.Properties.ContentType = "image/jpeg";
            outputBlob.SetProperties();
            // update the entry in table storage to point to the thumbnail
            GuestBookDataSource ds = new GuestBookDataSource();
            ds.UpdateImageThumbnail(partitionKey, rowkey, thumbnailBlobUri);
            // remove message from queue
            queue.DeleteMessage(msg);
            Trace.TraceInformation("Generated thumbnail in blob '{0}'.", thumbnailBlobUri);
          }
        }
        else
        {
          System.Threading.Thread.Sleep(1000);
        }
      }
      catch (StorageClientException e)
      {
        Trace.TraceError("Exception when processing queue item. Message: '{0}'", e.Message);
        System.Threading.Thread.Sleep(5000);
      }
    }

  }
  ...
}

(Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Run – VB)

Visual Basic

Public Class WorkerRole

  Inherits RoleEntryPoint

...

  Public Overrides Sub Run()





  

    Trace.TraceInformation("Listening for queue messages...")
    Do
      Try
        ' retrieve a new message from the queue
        Dim msg As CloudQueueMessage = queue.GetMessage()
        If msg IsNot Nothing Then
          ' parse message retrieved from queue
          Dim messageParts = msg.AsString.Split(New Char() {","c})
          Dim imageBlobUri = messageParts(0)
          Dim partitionKey = messageParts(1)
          Dim rowKey = messageParts(2)
          Trace.TraceInformation("Processing image in blob '{0}'.", imageBlobUri)
          Dim thumbnailBlobUri As String = System.Text.RegularExpressions.Regex.Replace(imageBlobUri, "([^\\.]+)(\\.[^\\.]+)?$", "$1-thumb$2")
          ' download original image from blob storage
          Dim inputBlob As CloudBlockBlob = container.GetBlockBlobReference(imageBlobUri)
          Dim outputBlob As CloudBlockBlob = container.GetBlockBlobReference(thumbnailBlobUri)
          Using input As BlobStream = inputBlob.OpenRead()
            Using output As BlobStream = outputBlob.OpenWrite()
              ProcessImage(input, output)
              ' commit the blob and set its properties
              output.Commit()
              outputBlob.Properties.ContentType = "image/jpeg"
              outputBlob.SetProperties()
              ' update the entry in table storage to point to the thumbnail
              Dim ds = New GuestBookDataSource()
              ds.UpdateImageThumbnail(partitionKey, rowKey, thumbnailBlobUri)
              ' remove message from queue
              queue.DeleteMessage(msg)
              Trace.TraceInformation("Generated thumbnail in blob '{0}'.", thumbnailBlobUri)
            End Using
          End Using
        Else
          System.Threading.Thread.Sleep(1000)
        End If
      Catch e As StorageClientException
        Trace.TraceError("Exception when processing queue item. Message: '{0}'", e.Message)
        System.Threading.Thread.Sleep(5000)
      End Try
    Loop




  End Sub
  ...
End Class

Finally, add the following method to the WorkerRole class to create thumbnails from a given image.(Code Snippet – Introduction to Windows Azure -Ex2 ProcessImage – CS)

public class WorkerRole : RoleEntryPoint

...


  public void ProcessImage(Stream input, Stream output)
  {
    int width;
    int height;
    var originalImage = new Bitmap(input);
    if (originalImage.Width > originalImage.Height)
    {
      width = 128;
      height = 128 * originalImage.Height / originalImage.Width;
    }
    else
    {
      height = 128;
      width = 128 * originalImage.Width / originalImage.Height;
    }
    var thumbnailImage = new Bitmap(width, height);
    using (Graphics graphics = Graphics.FromImage(thumbnailImage))
    {
      graphics.InterpolationMode = InterpolationMode.HighQualityBicubic;
      graphics.SmoothingMode = SmoothingMode.AntiAlias;
      graphics.PixelOffsetMode = PixelOffsetMode.HighQuality;
      graphics.DrawImage(originalImage, 0, 0, width, height);
    }
    thumbnailImage.Save(output, ImageFormat.Jpeg);
  }

(Code Snippet – Introduction to Windows Azure - Ex2 ProcessImage – VB)

Visual Basic

Public Class WorkerRole

  Inherits RoleEntryPoint

...

  Private Sub ProcessImage(ByVal input As Stream, ByVal output As Stream)
    Dim width As Integer
    Dim height As Integer
    Dim originalImage As New Bitmap(input)
    If originalImage.Width > originalImage.Height Then
      width = 128
      height = 128 * originalImage.Height / originalImage.Width
    Else
      height = 128
      width = 128 * originalImage.Width / originalImage.Height
    End If
    Dim thumbnailImage As New Bitmap(width, height)
    Using graphic = Graphics.FromImage(thumbnailImage)
      graphic.InterpolationMode = InterpolationMode.HighQualityBicubic
      graphic.SmoothingMode = SmoothingMode.AntiAlias
      graphic.PixelOffsetMode = PixelOffsetMode.HighQuality
      graphic.DrawImage(originalImage, 0, 0, width, height)
    End Using
    thumbnailImage.Save(output, ImageFormat.Jpeg)
  End Sub

End Class

Note:

Even though the code shown above uses classes in the System.Drawing namespace for simplicity, you should be aware that the classes in this namespace were designed for use with Windows Forms. They are not supported for use within a Windows or ASP.NET service. You should conduct exhaustive testing if you intend to use these classes in your own Windows Azure applications.

The worker role also uses Windows Azure storage services and you need to configure your storage account settings, just as you did in the case of the web role. To create the storage account setting, in Solution Explorer, expand the Roles node of the GuestBook project, double-click GuestBook_WorkerRole to open the properties for this role and select the Settings tab. Click Add Setting, type “DataConnectionString” in the Name column, change the Type to ConnectionString, and then click the button labeled with an ellipsis. In the Storage Account Connection String dialog, choose the option labeled Use the Windows Azure storage emulator and click OK. Press CTRL + S to save your changes.

Verification

You now launch the updated application in the Windows Azure compute emulator to verify that the worker role can retrieve queued work items and generate the corresponding thumbnails.

Press F5 to launch the service in the local compute emulator.
Switch to Internet Explorer to view the application. Provided you completed the verification section of the previous exercise successfully, you will see the guest book entry that you entered, including the uploaded image displayed in its original size. If you recall, during the last task of that exercise, you updated the web role code to post a work item to a queue for each new entry submitted. These messages remain in the queue even though the web role was subsequently recycled.
Wait a few seconds until the worker role picks up the queued message and processes the image that you are viewing. Once that occurs, it generates a thumbnail for this image and updates the corresponding URL property for the entry in table storage. Eventually, because the page refreshes every few seconds, it will show the thumbnail image instead.

Figure 25Home page showing the thumbnail generated by the worker role
If you are using Visual Studio 2010, in Server Explorer, expand the Blobs node in the Windows Azure Storage node, and then double-click the guestbookpics container. Notice that it now contains an additional blob for the generated thumbnail image.

Figure 26Blob container showing the blob for the generated thumbnail
Add some more guest book entries. Notice that the images update after a few seconds once the worker role processes the thumbnails.
Press SHIFT + F5 to stop the debugger and shut down the deployment in the compute emulator.

source:http://msdn.microsoft.com

Learn infos