Monthly Archives: October 2011

Thread Pool Synchronization

My latest Windows with C++ column, Thread Pool Synchronization, is now available in the November 2011 issue of MSDN Magazine. From the article:

I’ve said it before: Blocking operations are bad news for concurrency. Often, however, you need to wait for some resource to become available, or perhaps you’re implementing a protocol that stipulates some amount of time needs to elapse before resending a network packet. What do you do then? You could use a critical section, call functions like Sleep and WaitForSingleObject and so on. Of course, that means you’ll have threads just sitting around blocking again. What you need is a way for the thread pool to wait on your behalf without affecting its concurrency limits. The thread pool can then queue a callback once the resource is available or the time has elapsed.

I hope you enjoy it. Next month I wrap up the series on the thread pool by exploring thread pool timers and I/O.

You can find links to more of my articles here.

The road to Windows 8

Windows 8 is Microsoft’s gift to the native C++ developer. After years of pushing the .NET Framework as the “one true path,” Microsoft is returning to its native roots with the most groundbreaking release of Windows since David Cutler’s Windows NT in 1993. Cutler changed Windows from a little operating system that UNIX developers joked about into a formidable competitor and a dominant player both in desktop and server markets. Although no longer in print, Showstopper by G. Pascal Zachary tells the fascinating story of how Bill Gates hired Cutler to design and develop Windows NT and the many challenges and struggles they went through in developing that foundational piece of software.

From COM to the Common Language Runtime

After Sun proved that managed code was indeed a viable option, Microsoft took a detour into the world of managed code and released the first beta of the .NET Framework in 2000. Over the next ten years, this was to have a profound effect on Microsoft’s developer division and even the company as a whole. The message was clear: managed code is in many ways superior to native code. It is safer, more reliable, and can even be faster! That was the theory anyway. In practice, the results proved to be somewhat different.

Microsoft touted Windows Forms as a simpler way to develop Windows applications. Then the Windows Presentation Foundation (WPF) was developed as a way to solve some of the limitations in Windows Forms. Then Silverlight was developed as a lighter and faster WPF. Similarly, they developed ASP.NET as a way to simplify web services and then came Windows Communication Foundation (WCF) as a better way to communicate. These technologies however never quite lived up to the hype. I have written before in this column about how the native Windows Web Services (WWS) API knocks the socks off WCF in terms of throughput and working set and how the native Direct2D API provides far better performance than WPF and with a much smaller footprint.

After developers started playing with the .NET Framework beta in 2000, there were many questions about the direction that Microsoft had taken. Up until then Microsoft had been pushing COM as the preferred technology for developing software components for Windows. Microsoft had published Kraig Brockschmidt’s hugely influential Inside OLE 2 and Don Box summed it up nicely in the more concise Essential COM.

Microsoft Technical Fellow Brian Harry, who was then the development manager for the Common Language Runtime (CLR), sent a now famous email to a .NET development mailing list. It was lovingly dubbed “Brian’s little email” as it was well over five thousand words in length. It helped to clarify some of the rationale behind one of the thornier issues and most fundamental shifts away from COM in terms of resource management. COM at its core was based on an invasive reference-counting model expressed through a C++ virtual function table. This was the IUnknown interface. Two problems with COM’s reference counting model were perceived to be insurmountable. The first was the cost of reference counting itself. Each call to AddRef and Release typically resulted in an interlocked instruction making the simple act of assigning a reference relatively costly. To be fair, COM was designed for use at component boundaries but the intention was to design a universal object model that would be used for all objects within an application. It should be noted that much of the initial design of the CLR was driven by the need for compatibility with Visual Basic, as C# did not yet exist. The second problem was the issue of reference counting cycles. With a tracing garbage collector, cycles are not an issue. Again, there are other ways to solve that problem such as with the use of weak pointers as exemplified by the Standard Template Library.

From .NET to the Windows Runtime

Rather than building on the CLR, the Windows team decided that it was time to return to the proven reliability and performance of native code and the essentials of COM as the basis for the next great API for Windows. The Windows Runtime (WinRT) is rooted in the IUnknown interface and then takes COM in a new direction.

Gone is the need for intermediate language code that is compiled “just-in-time.” Why force your users to compile your code when you can aggressively optimize and compile your application ahead of time and make sure your users get the fastest experience the first time and every time on every supported processor? Gone is the nondeterministic resource management inherent in garbage-collected runtimes. To dispose or not to dispose, that is the question. Gone is the managed security model that was too complicated and costly. Windows NT introduced a perfectly good model for application isolation and security that has stood the test of time.

Of course, there were many lessons learned in the last ten years that have been brought back to COM and now WinRT. Naïve .NET developers, and Java and UNIX developers before them, would love to mock the Windows Registry and offer good old text files as a better alternative. WinRT continues to use the registry, as it is a fast, reliable and even simple way to manage settings. It does however take some ideas from the CLR related to application packaging that makes for a simpler and more manageable environment. WinRT also borrows the CLR’s metadata format to augment COM’s dynamic cast, good old QueryInterface, with a priori knowledge about a components capabilities.

Where does this leave the .NET developer? Well unlike the cold shoulder that the C++ developer received when the .NET Framework was first released, Microsoft has ensured that the .NET developer will continue to get a first-class experience albeit with a slight performance hit simply due to the nature of managed code. Since WinRT is based on COM interfaces, the CLR could easily extend its impressive COM interop facilities to support WinRT. The Visual Studio IDE may continue to provide a better developer experience for .NET developers as it has done in recent years but the C++ developer still has what really matters and that is a first-class compiler and a Windows API without compromise.

From C++/CLI to C++/CX

So is it back to ATL and CComPtr for the C++ developer? Well not exactly. Although you could use ATL if you really wanted, there is a better way. In 2003 the Visual C++ team was experiencing an identity crisis. This whole “managed code” thing appeared to be taking over the world. Maybe they needed to jump in with both feet and make their mark. Therefore, the team resolved to redesign their support for the CLR in the form of a set of language extensions that came to be known as C++/CLI. Notable members of the Visual C++ team such as Herb Sutter and Brandon Bray worked tirelessly to design a beautiful and first-class language that was to be the most powerful language for .NET programming. I remember writing what is most likely the very first article on C++/CLI. I had no compiler and simply wrote the article based on information I had gathered from discussions with Bray and Sutter.

It was an exciting time but that excitement was short lived. Shortly after the experience with C++/CLI the Visual C++ team recovered from their identity crisis and realized that there really is a great need for native code and that the .NET Framework is not for everyone. They realized that they could best serve the developer community by focusing on native code, libraries and tools. We have already seen some of the fruits of this decision in the excellent support that Visual C++ today offers for the new C++11 standard.

Then to everyone’s surprise C++/CLI made a comeback of sorts at the BUILD conference in September. It was not actually C++/CLI and has nothing to do with .NET but it looked remarkably similar. The most notable difference being the use of the “ref new” contextual keyword to create objects instead of C++/CLI’s gcnew keyword.

auto w = ref new Widget;

But what exactly does ref new return? Well I could be more explicit:

Widget ^ w = ref new Widget;

In C++/CLI the ^ operator declared the variable as a handle to a CLR reference type. However, this is not the CLR so it must be something else. Moreover, what happens when w goes out of scope? Well it turns out that C++/CX has baked the semantics of a smart pointer right into the language. A Widget ^ as returned by ref new is the moral equivalent of ComPtr<Widget> but with some fancy sugar coating. The ref new returns a pointer to a COM object, or more specifically a WinRT object, and the handle is responsible for one reference count. Once the handle goes out of scope, the reference is released. This is perhaps easier to understand in code:

void scope()
    auto a = ref new Widget; // RoActivateInstance

    auto b = a; // IUnknown::AddRef

    a = nullptr; // IUnknown::Release for a

}   // IUnknown::Release for b

The RoActivateInstance function is the moral equivalent in WinRT of CoCreateInstance in classic COM. Had you called RoActivateInstance directly it would have returned a pointer for an object whose reference count had already been incremented. When a is assigned to b the compiler makes sure that AddRef is called. Similarly, when a null pointer value is assigned to a and when b goes out of scope the compiler makes sure that Release is called each time.

The key here is that this is not some funny new managed environment. This is good old native code. Again, some code helps to illustrate this reality:

auto a = ref new Widget;

auto raw = reinterpret_cast<IUnknown *>(a);

auto x = raw->AddRef();
assert(2 == x);

auto y = raw->Release();
assert(1 == y);

Nothing says native code like reinterpret_cast! Given the underlying IUnknown interface pointer, you can naturally call all of its members including QueryInterface. Using AddRef and Release is just a simple way of proving to you that this is native since they both return the resulting reference count for diagnostics.

Of course, you could eschew the C++/CX language extensions and just use the core WinRT API. You could call the RoInitialize function to initialize the WinRT concurrency model and then call RoActivateInstance to create, or activate a particular type and use a regular C++ smart pointer to manage the resulting reference-counted interface pointer. There are not however very many good reasons for doing that. As I illustrated above, you can easily break out of the sugar coating if you find that you need a bit more fine-grained control over your code for some algorithm or optimization.

Windows 8, the thread pool and me

I have been engrossed in Windows 8 ever since the BUILD conference. Due to the latency of print publication, my first column focusing on Windows 8 will only appear in January. There is a lot to write about but do not be fooled by the hype. Windows 8 is the evolution of Windows. Everything you have learned about writing native applications on Windows will only help to make you a better Windows 8 developer. What do you think is powering Metro? It is the same stuff I have been writing about for years, from the Windows thread pool to Direct2D to Windows security to COM and a whole lot more.

The October issue of MSDN Magazine was just released and you can read my latest column online now: Thread Pool Cancellation and Cleanup

Cancellation and cleanup are notoriously difficult problems to solve when it comes to multithreaded applications. When is it safe to close a handle? Does it matter which thread cancels an operation? To make matters worse, some multithreaded APIs are not reentrant, potentially improving performance but also adding complexity for the developer.

This is the third in a series on the Windows thread pool API. You can find links to more of my articles here.