A Short Introduction to .Net Multithreading Programming (I)

Dan Sabadis
Team Lead @ SDL

PROGRAMMING

Java and .Net stand for a miniature operating system, handling complex scenarios like memory allocation and cleanup (see Garbage Collector), and concurrent access to resources

Today we will focus on the "difficult" subject of multi-threading in a big enterprise environment. Because my previous experience was anchored in the .Net Framework, we chose to provide the examples from the C# language, but the very same concepts, classes and abstractions described below have an almost word-to-word equivalence in Java.

The most important parts in any presentation are the definitions, so we have to define what a thread is. The answer is surprisingly complex and, as a prerequisite, we need to define what a computer program is and what a process is!

A computer program is a set of instructions to be given to any computing machine. A computing machine can be silicone/metal-based, like a robot or a personal computer. It can equally be a "biological" robot like us humans or animals. Some ordinary examples of computer programs for humans are a cooking receipt, a first-help manual in case of injuries, or a knitting pattern. Among some ordinary examples of computer programs for "normal" silicone-based computers could be a set of instructions to read and display strings from a text file, or a knitting pattern for Japanese sewing machines (made of metal).

A process is the computer in action, executing the aforementioned program. This is why we say that a process is a program in execution state! Many times, a process can be quite complex, composed of steps that must be executed in parallel. For instance, a cooking recipe can instruct humans to pour some spice in the soup and, AT THE SAME TIME, to stir the liquid until it gets a solid red color. In this case, the process is composed of two concurrent sub-processes, or threads of execution (or light processes in LINUX parlance): stirring with the spoon and pouring the spice.

Therefore, it can be said that a thread is an atomic sub-process, a simple thread of execution that cannot be split in other sub-processes, or other sub-threads! In Java or .Net, a thread can have many states (see image below), but for the purpose of the current presentation we will focus only on three of them: Running, Blocked and Frozen (or WaitSleepJoin).

Below we have the following function routine, used to synchronously make a HTTP request to a web site (or any URI):

public static string Synchronous_WebDownload(string siteAddress)
{
 using (WebClient client = 
   new WebClient())
  {
    return client.
    DownloadString(siteAddress);
    }
}

Based on the address that is used as parameter, this synchronous web request can take from a few milliseconds to a much longer period of time (entire seconds or even minutes, examples below).

String googleContent = 
  Synchronous_WebDownload(
  "http://www.google.com");

String yahooContent  = 
  Synchronous_WebDownload(
  "http://www.yahoo.com");

String localWebContent  = 
  Synchronous_WebDownload(
  "http://127.0.0.1");

Moreover, this type of web request will cause the calling thread to enter the "blocked" state until the Google, Yahoo or local servers will fully return all their web site content. For a simple application this is just fine, but for a major web site that serves hundreds of requests per second, calls like the one above can cause serious problems, and here is why.

Let's suppose we have a web site which supports 10 calls per second and inside each call our site makes a single call to the Synchronous_WebDownload method described above. That means those 10 calls, each served by its own dedicated thread, will be blocked, and as more calls arrive to our site - which will need to be served as well - the Common Language Runtime (CLR) will create additional Threads, each thread with its own memory allocation (well over 1 MB of memory for each thread in the .Net CLR). If the new calls are not fast enough, new threads will be created until the memory is full and OutOfMemory exceptions will begin to popup. After a while, our web server will be completely unresponsive. Therefore, the synchronous web download approach described above, though extremely simple, is deceptively simple in case of highly-trafficked web services. This implies the need of a new approach: enter the world of the three rendezvous techniques.

All these three rendezvous techniques (rendezvous = place where the threads meet) are based on the most important interface for thread synchronization in .Net, the IAsyncResult:

public interface IAsyncResult

  public interface IAsyncResult
  {
    bool IsCompleted { get; }
    WaitHandle AsyncWaitHandle { get; }
    object AsyncState { get; }
    bool CompletedSynchronously { get; }
  }

The first technique is called "BeginCall-EndCall".

The method below reads a physical file from the local hard drive and displays its content on the console. As you can see, the read file is not synchronous anymore, as in the previous example, but it is asynchronous: fs.BeginRead! The FileStream class has a corresponding synchronous method simply called Read, but we chose to call the BeginRead. This asynchronous invocation will return right away and the read operation will not be blocked anymore, as when calling the Read function!

public static byte[] 
 BeginEndXXX_Rendezvous_FileRead()
{
  byte[] buffer = new byte[100];
  string filename = String.Concat(
  Environment.SystemDirectory, "ntdll.dll");
  FileStream fs = new FileStream(filename, 
   FileMode.Open, FileAccess.Read, FileShare.Read, 
   1024, FileOptions.Asynchronous);

  IAsyncResult result = fs.BeginRead(buffer, 0, 
    buffer.Length, null, null);
  int numBytes = fs.EndRead(result);

  fs.Close();
  Console.WriteLine(
    $"From {filename} We Read {numBytes}  Bytes:");
  Console.WriteLine(BitConverter.ToString(buffer));

  return buffer;
}

So where is the content of the file? It is not ready, as the read operation was only initiated, but not yet completed. We still have some kind of a reference to the file via the IAsyncResult result variable returned by the BeginRead asynchronous function.

As you can see, immediately after the call of the BeginRead, we call the EndRead method. This call will "magically freeze" the currently calling thread until the file read operation completes. This is the main advantage of this first async approach when compared to the previously synchronous call: the thread is frozen, put to sleep, and automatically awoken, compared to having it blocked.

The second technique is called "Wait until done".

As you can see from the method below, the biggest difference is that, after the call to BeginRead, we don't immediately call the EndRead, thus putting the thread to sleep, but we query the IsCompleted property of the IAsyncResult variable returned by the BeginRead. In this way, we can do some useful operation waiting until the async read is completed. Once we arrive with the execution at the EndRead, the currently running thread won't be put to sleep anymore, because we are ensured the asynchronous read was already completed (when escaped from the while loop).

public static byte[] 
  WaitUntilDone_Rendezvous_FileRead()
{
  byte[] buffer = new byte[1000];
  string filename = String.Concat(
    Environment.SystemDirectory, "ntdll.dll");

  FileStream fs = new FileStream(filename, 
    FileMode.Open, FileAccess.Read, 
    FileShare.Read, 1024, FileOptions.Asynchronous);

  fs.Read(buffer, 0, buffer.Length);
  IAsyncResult result = fs.BeginRead(buffer, 0, 
     buffer.Length, null, null);

  while (!result.IsCompleted)
  {
  // do some work here if the call isn't completed
  Console.Write(".");
  }
  int numBytes = fs.EndRead(result);
  fs.Close();
  Console.WriteLine(
    $"From {filename} We Read {numBytes}  Bytes:");

  Console.WriteLine(BitConverter.ToString(buffer));

  return buffer;
}

The third technique is called "Callback Rendezvous"

Instead of just one method, as in the 2 asynchronous scenarios described above, here we have 2 methods Callback_RendezVous_FileRead and ReadIsDone.

public static void Callback_Rendezvous_FileRead()
{
  string filename = String.Concat(
    Environment.SystemDirectory, "\\ntdll.dll");

  FileStream fs = new FileStream(filename, 
    FileMode.Open, FileAccess.Read, 
    FileShare.Read, 1024, FileOptions.Asynchronous);

// Initiate an asynchronous read operation against /
// the FileStream
// Pass the FileStream (fs) to the callback method 
// (ReadIsDone)
  fs.BeginRead(__buffer, 0, __buffer.Length, 
    ReadIsDone, fs);

  Console.WriteLine($"main thread "+
    "ID={Thread.CurrentThread.ManagedThreadId}");
  Console.ReadLine();
}

private static void ReadIsDone(IAsyncResult ar)
{
  // Show the ID of the thread executing ReadIsDone
  Console.WriteLine($"ReadIsDone thread "+
    "ID={Thread.CurrentThread.ManagedThreadId}");

  // Extract the FileStream (state) out of the 
  // IAsyncResult object
  FileStream fs = (FileStream) ar.AsyncState;

  // Get the result
  Int32 bytesRead = fs.EndRead(ar);
  // No other operations to do, close the file
  fs.Close();

  // Now, it is OK to access the byte array and 
  // show the result.
  Console.WriteLine("Number of bytes read={0}", 
    bytesRead);

  Console.WriteLine(BitConverter.ToString(__buffer, 
    0, __buffer.Length));
}

This third technique is arguably the best of all 3 asynchronous approaches. The first method Callback_RendezVous_FileRead initiates the async file read with a classic BeginRead and returns immediately. If you see the parameters passed to the BeginRead method, the ReadIsDone method is passed in the fourth place! This guarantees that when the I/O read operation is complete, the ReadIsDone method will be called and this function needs to have an IAsyncResult argument! From this argument we can obtain the original FileStream which initiated the read operation and access the full-read content! In terms of performance this is the best approach as we don't wait in either frozen state (as in the first scenario) or a spinning state (as in the second scenario where we waited in a while-loop). The biggest disadvantage of this third approach is code readability and complexity. We don't have a single method as before, but two, and as the scenarios get more complex, the program is more prone to become spaghetti-code.

In order to prevent this splitting (or "spagettisation") of the asynchronous calls, the .Net framework creators implemented a whole new multi-threading library in the 4.0 .Net version, the so-called TPL - Task Parallel Library.

Let's suppose we want to make a call to the Yahoo web site and, once the result is returned, we will use that result to make another call to the Google web site.

With the callback rendezvous pattern we would need to have 3 functions: the first Async_Callback_YahooWebDownload which initiates the call to Yahoo, the second Yahoo_DownloadStringCompleted which is the callback to be invoked once the Yahoo web site returned the result and immediately to invoke the Google web site and the third method would be the second callback Google_DownloadStringCompleted to be automatically invoked when the Google web site returns its result.

public static void Async_Callback_YahooWebDownload()
{
  WebClient client = new WebClient();
  client.DownloadStringAsync(
    new Uri("http://www.yahoo.com"));

  client.DownloadStringCompleted += 
    Yahoo_DownloadStringCompleted;

  Console.ReadLine();
}

private static void 
  Yahoo_DownloadStringCompleted(object sender, 
  DownloadStringCompletedEventArgs e)
{
  string content = e.Result;
  ((WebClient) sender).Dispose();

  Console.WriteLine($"content={content}");

  WebClient client = new WebClient();
  client.DownloadStringAsync(
    new Uri("http://www.google.com"));

  client.DownloadStringCompleted += 
    Google_DownloadStringCompleted;
}

private static void Google_DownloadStringCompleted (object sender, DownloadStringCompletedEventArgs e)
{
  string content = e.Result;
  ((WebClient) sender).Dispose();

  Console.WriteLine($"content={content}");
}

Observe that there is already a minor improvement in readability because the .Net Framework creators provided 2 dedicated methods for the callback scenario on the WebClient class: DownloadStringAsync and DownloadStringCompleted. Anyway, compare the code above with the code below:

var yahooTask = client.DownloadDataTaskAsync(
  new Uri("http://www.yahoo.com"));

var googleTask = yahooTask.
  ContinueWith(initialYahooTask => client.
  DownloadDataTaskAsync(
  new Uri("http://www.google.com")));

googleTask.Wait();
var googleByteResult = googleTask.Result.Result;
Console.WriteLine($"content={
  Encoding.UTF8.GetString(googleByteResult)}");

This task-based code looks more compact, more readable and more synchronous because of the continuation Google task.

With the advent of the 4.5 .Net framework, the simplicity and readability of the asynchronous calls go even further with the async-await pattern:

public static async void 
  Async_Await_Task_Continuation_WebDownload()
{
WebClient client = new WebClient();
var googleResult = await client.
  DownloadDataTaskAsync(
  new Uri("http://www.google.com"));

var stringGoogle = Encoding.UTF8.GetString(googleResult);
Console.WriteLine($"content={stringGoogle}");

var yahooResult = await client.DownloadDataTaskAsync(
  new Uri("http://www.yahoo.com"));

Console.WriteLine($"content={
 System.Text.Encoding.UTF8.GetString(yahooResult)}");
}

This trend of making the I/O asynchronous functions look more readable and synchronous-like is manifesting itself not only in the .Net/Java environments, but in the Javascript world as well. In the Javascript/ECMAscript, the Promises abstraction was made possible in the ECMA 6 implementation and they mimic the Task class presented above. The async-await approach will be fully implemented in the ECMA 7 standard implementation.

In a next article, we will approach the synchronization constructs that make the IAsyncResult implementations possible, namely the kernel-mode WaitHandle classes: ManualResetEvent and AutoResetEvent, as well as Monitor.Enter/Pulse methods. We will also compare the kernel-mode with the user-mode synchronization constructs such as "volatile" and "Interlocked".

Part II of the article