The road to Windows 8

Windows 8 is Microsoft’s gift to the native C++ developer. After years of pushing the .NET Framework as the “one true path,” Microsoft is returning to its native roots with the most groundbreaking release of Windows since David Cutler’s Windows NT in 1993. Cutler changed Windows from a little operating system that UNIX developers joked about into a formidable competitor and a dominant player both in desktop and server markets. Although no longer in print, Showstopper by G. Pascal Zachary tells the fascinating story of how Bill Gates hired Cutler to design and develop Windows NT and the many challenges and struggles they went through in developing that foundational piece of software.

From COM to the Common Language Runtime

After Sun proved that managed code was indeed a viable option, Microsoft took a detour into the world of managed code and released the first beta of the .NET Framework in 2000. Over the next ten years, this was to have a profound effect on Microsoft’s developer division and even the company as a whole. The message was clear: managed code is in many ways superior to native code. It is safer, more reliable, and can even be faster! That was the theory anyway. In practice, the results proved to be somewhat different.

Microsoft touted Windows Forms as a simpler way to develop Windows applications. Then the Windows Presentation Foundation (WPF) was developed as a way to solve some of the limitations in Windows Forms. Then Silverlight was developed as a lighter and faster WPF. Similarly, they developed ASP.NET as a way to simplify web services and then came Windows Communication Foundation (WCF) as a better way to communicate. These technologies however never quite lived up to the hype. I have written before in this column about how the native Windows Web Services (WWS) API knocks the socks off WCF in terms of throughput and working set and how the native Direct2D API provides far better performance than WPF and with a much smaller footprint.

After developers started playing with the .NET Framework beta in 2000, there were many questions about the direction that Microsoft had taken. Up until then Microsoft had been pushing COM as the preferred technology for developing software components for Windows. Microsoft had published Kraig Brockschmidt’s hugely influential Inside OLE 2 and Don Box summed it up nicely in the more concise Essential COM.

Microsoft Technical Fellow Brian Harry, who was then the development manager for the Common Language Runtime (CLR), sent a now famous email to a .NET development mailing list. It was lovingly dubbed “Brian’s little email” as it was well over five thousand words in length. It helped to clarify some of the rationale behind one of the thornier issues and most fundamental shifts away from COM in terms of resource management. COM at its core was based on an invasive reference-counting model expressed through a C++ virtual function table. This was the IUnknown interface. Two problems with COM’s reference counting model were perceived to be insurmountable. The first was the cost of reference counting itself. Each call to AddRef and Release typically resulted in an interlocked instruction making the simple act of assigning a reference relatively costly. To be fair, COM was designed for use at component boundaries but the intention was to design a universal object model that would be used for all objects within an application. It should be noted that much of the initial design of the CLR was driven by the need for compatibility with Visual Basic, as C# did not yet exist. The second problem was the issue of reference counting cycles. With a tracing garbage collector, cycles are not an issue. Again, there are other ways to solve that problem such as with the use of weak pointers as exemplified by the Standard Template Library.

From .NET to the Windows Runtime

Rather than building on the CLR, the Windows team decided that it was time to return to the proven reliability and performance of native code and the essentials of COM as the basis for the next great API for Windows. The Windows Runtime (WinRT) is rooted in the IUnknown interface and then takes COM in a new direction.

Gone is the need for intermediate language code that is compiled “just-in-time.” Why force your users to compile your code when you can aggressively optimize and compile your application ahead of time and make sure your users get the fastest experience the first time and every time on every supported processor? Gone is the nondeterministic resource management inherent in garbage-collected runtimes. To dispose or not to dispose, that is the question. Gone is the managed security model that was too complicated and costly. Windows NT introduced a perfectly good model for application isolation and security that has stood the test of time.

Of course, there were many lessons learned in the last ten years that have been brought back to COM and now WinRT. Naïve .NET developers, and Java and UNIX developers before them, would love to mock the Windows Registry and offer good old text files as a better alternative. WinRT continues to use the registry, as it is a fast, reliable and even simple way to manage settings. It does however take some ideas from the CLR related to application packaging that makes for a simpler and more manageable environment. WinRT also borrows the CLR’s metadata format to augment COM’s dynamic cast, good old QueryInterface, with a priori knowledge about a components capabilities.

Where does this leave the .NET developer? Well unlike the cold shoulder that the C++ developer received when the .NET Framework was first released, Microsoft has ensured that the .NET developer will continue to get a first-class experience albeit with a slight performance hit simply due to the nature of managed code. Since WinRT is based on COM interfaces, the CLR could easily extend its impressive COM interop facilities to support WinRT. The Visual Studio IDE may continue to provide a better developer experience for .NET developers as it has done in recent years but the C++ developer still has what really matters and that is a first-class compiler and a Windows API without compromise.

From C++/CLI to C++/CX

So is it back to ATL and CComPtr for the C++ developer? Well not exactly. Although you could use ATL if you really wanted, there is a better way. In 2003 the Visual C++ team was experiencing an identity crisis. This whole “managed code” thing appeared to be taking over the world. Maybe they needed to jump in with both feet and make their mark. Therefore, the team resolved to redesign their support for the CLR in the form of a set of language extensions that came to be known as C++/CLI. Notable members of the Visual C++ team such as Herb Sutter and Brandon Bray worked tirelessly to design a beautiful and first-class language that was to be the most powerful language for .NET programming. I remember writing what is most likely the very first article on C++/CLI. I had no compiler and simply wrote the article based on information I had gathered from discussions with Bray and Sutter.

It was an exciting time but that excitement was short lived. Shortly after the experience with C++/CLI the Visual C++ team recovered from their identity crisis and realized that there really is a great need for native code and that the .NET Framework is not for everyone. They realized that they could best serve the developer community by focusing on native code, libraries and tools. We have already seen some of the fruits of this decision in the excellent support that Visual C++ today offers for the new C++11 standard.

Then to everyone’s surprise C++/CLI made a comeback of sorts at the BUILD conference in September. It was not actually C++/CLI and has nothing to do with .NET but it looked remarkably similar. The most notable difference being the use of the “ref new” contextual keyword to create objects instead of C++/CLI’s gcnew keyword.

auto w = ref new Widget;

But what exactly does ref new return? Well I could be more explicit:

Widget ^ w = ref new Widget;

In C++/CLI the ^ operator declared the variable as a handle to a CLR reference type. However, this is not the CLR so it must be something else. Moreover, what happens when w goes out of scope? Well it turns out that C++/CX has baked the semantics of a smart pointer right into the language. A Widget ^ as returned by ref new is the moral equivalent of ComPtr<Widget> but with some fancy sugar coating. The ref new returns a pointer to a COM object, or more specifically a WinRT object, and the handle is responsible for one reference count. Once the handle goes out of scope, the reference is released. This is perhaps easier to understand in code:

void scope()
    auto a = ref new Widget; // RoActivateInstance

    auto b = a; // IUnknown::AddRef

    a = nullptr; // IUnknown::Release for a

}   // IUnknown::Release for b

The RoActivateInstance function is the moral equivalent in WinRT of CoCreateInstance in classic COM. Had you called RoActivateInstance directly it would have returned a pointer for an object whose reference count had already been incremented. When a is assigned to b the compiler makes sure that AddRef is called. Similarly, when a null pointer value is assigned to a and when b goes out of scope the compiler makes sure that Release is called each time.

The key here is that this is not some funny new managed environment. This is good old native code. Again, some code helps to illustrate this reality:

auto a = ref new Widget;

auto raw = reinterpret_cast<IUnknown *>(a);

auto x = raw->AddRef();
assert(2 == x);

auto y = raw->Release();
assert(1 == y);

Nothing says native code like reinterpret_cast! Given the underlying IUnknown interface pointer, you can naturally call all of its members including QueryInterface. Using AddRef and Release is just a simple way of proving to you that this is native since they both return the resulting reference count for diagnostics.

Of course, you could eschew the C++/CX language extensions and just use the core WinRT API. You could call the RoInitialize function to initialize the WinRT concurrency model and then call RoActivateInstance to create, or activate a particular type and use a regular C++ smart pointer to manage the resulting reference-counted interface pointer. There are not however very many good reasons for doing that. As I illustrated above, you can easily break out of the sugar coating if you find that you need a bit more fine-grained control over your code for some algorithm or optimization.

39 thoughts on “The road to Windows 8

  1. Nish

    Hey Kenny,

    Good to see you blogging on WinRT. Looking forward to more such posts.

    BTW some trivia on the first C++/CLI article : beat you by 3 months. Apr 2004 to your Jul 2004. *grin*

    1. small_mountain

      Hey, Kenny,

      Great start and really looking forward to this article series. I hope you will include information about what if anything from these advances in C++ apply to the desktop world (I’m thinking not much) and also a clear picture of the sandboxed Metro environment – what are the things your desktop app can do that your metro app simply can’t. For example, under what circumstances can COM objects be instantiated, and will there be a JRE available to metro apps.



      1. Kenny Kerr Post author

        Technically, there is nothing stopping you from using WinRT on the desktop. There may be Metro-specific functionality but the core Windows Runtime is not Metro-specific. I had planned to explore the Metro environment in my column. Again, there is no technical reason why your existing code will not run inside a Metro app. There may be security restrictions and app store restrictions. The former I planned to discuss and the latter you will have to ask the lawyers about.

  2. small_mountain

    @kenny – (there was no Reply link in your reply so this may be out of order) – Okay, we don’t have to hash this out in comments, but what you listed seems like two separate things: A WinRT app running in desktop (so in an overlapping window environment) and “no technical reason why your existing code will not run inside a Metro app” – that’s running a Metro app that incorporates existing C++ code. The latter sort of has to work. The former I will be pleasantly surprised if it works. I’m thinking particularly of using XAML that generates native C++ code (and/or instantiating the same objects that the XAML does in purely native C++ code) as part of a desktop application.


    1. Kenny Kerr Post author

      I am sorry if my response was a bit confusing. There are two issues here and I think you brought up a third. The first relates to the use of WinRT, which is not a Metro-specific technology although it is heavily used to make Metro a reality. The point is that you could use WinRT as a component technology completely independent of Metro. The second relates to the use of existing code (C, C++, C#, Java, whatever) inside a Metro app and that will definitely work. The third relates to using some of the Metro-specific UX capabilities outside of Metro. Technically, this works today (e.g. it is not obvious but I have done it) but whether or not this will continue to work is up to the lawyers. Personally, I hope it does and I see no reason that they should prevent it in the beta.

      1. Alex

        It appears that Kenny is wrong (about using new user interface in desktop apps). MS provides an official (I suppose) answer to the similar question here:

        MSDN now has new documentation (unfinished) for DirectComposition and Direct Manipulation APIs, which look very great, but again nothing seems to be done to ease native user interface development for desktop apps.

      2. Kenny Kerr Post author

        In my analysis of Windows 8, I focused only on the technology. Technically, it works on the desktop. Whether Microsoft wants you to use it on the desktop is another story.

    1. ahmed

      No kidding. It was not intended nor advertised as such. As kenny mentioned, it’s a language extension. As kenny also mentioned, you don’t have to use it. But since COM is a MS-specific technology, why not use it?

  3. Richard

    “… they developed ASP.NET as a way to simplify web services …”
    No, they developed ASP.NET to replace the ageing and decrepit “classic” ASP.

    Also, I don’t think WCF could be accused of “simplifying” web services. It was intended as a wrapper for several communication protocols – web services, remoting, etc. – and introduces more than enough complexity to confuse the hell out of anyone who tries to use it! :o)

  4. John

    I think you’re using the ‘auto’ keyword way too much. Sure it’s convenient that the ‘auto’ keyword hides cumbersome to declare type information (C++ template iterators), but what you lose is reading comprehension because now you have to think about what every auto type represents. In general, strict typing of variables is much more readable and useful.

    I shudder to think of seeing future code written entirely in ‘auto’ variables.

    1. Kenny Kerr Post author

      I use auto where I already know the type based on the context. Anything more feels redundant. To some degree it is a matter of style although there are some cases where it would be much harder to write the code without auto such as with metaprogramming.

  5. George

    Hey Kenny
    I was beginning to think Metro was dumping C developers en masse but this article is inspiring.
    I’m an MFC/C++ developer that has developed COM based DirectX apps.
    Metro appears to be dropping MFC off the face of the earth.
    What technologies would you recommend to get the MFC tribe back into the Windows app development game?

      1. George

        Hi Kenny

        At the BUILD conference examples of Metro app development, using SiverLight and XAML were demonstrated.

        As I watched this all I could think of was how completely different from traditional Win32 API programming using message pumps, resource editors and document/view architecture Windows programming has become.

        I have many questions regarding how much of the older software development tools and API software will be applicable to creating future Metro apps. I look forward to reading your expert opinion.

        Will C++ continue to be a preferred programming language for Metro development?

        Will there be Metro APIs like Win32 API to aid in C++ app development?

        Is there, or will there be a C++ application framework to kickstart C++ Metro app development?

        Is Microsoft turning WIndows App Development into a job done exclusively with scripting languages?

        If you can create Metro apps with C++ what does a Metro C++ app look like?

        Will Visual Studio provide support to build Metro apps using C++?

        Is there an alternative to MFC and C++ Class libraries that can be applied to Metro app development?

        Will programmimg for the WIn32 API be useful to Metro app development?

        Thank you

  6. Bartosz Bielecki

    I don’t get something here. “Microsoft’s gift to the native C++ developer” is basically a new non-standard C++ version (C++/CX) together with new COM (WinRT)? Seriously, I have a sense of deja vu. It sounds to me like Microsoft is trying to make us think .NET way once more, but this time instead of bluntly saying “C++ is dead, managed is the king”, it tells us “look, we have shiny new libs – you just have to accept the ref-count-everywhere-through-syntatic-sugar-non-standard-extension”.

    I am not saying I won’t try it out (maybe it’s better than expected), but just to be sure: are there any new things for native sticking-to-standard developers in Win8, or are we will just have to use winapi?

    1. Kenny Kerr Post author

      You can use Standard C++ if you want. You can even use C. I will try to find some time to illustrate this. There is plenty of new functionality for the Standard C++ or C developer to appreciate in Windows 8.

  7. rt8277

    If you are writing a native C++ app of the Chrome variety (Jensen Harris’s expression for non-Metro Windows apps like Microsoft Word or Photoshop), then Visual Studio 10’s MFC has an excellent Ribbon Resource editor that eliminates the non-C++ XAML editing cycle and still lets you develop ribbon-based apps. Generated ribbon code may also be intermixed with method calls to a new set of CMFCRibbon… classes. Although the MFC Ribbon editor code has some problems and needs a lot more documentation, it does work and does allow the developer to write sophisticated Windows apps without any runtime bloat or ever leaving the C++ standard.

    I would expect Microsoft to integrate the MetroUI into Visual Studio into MFC at some point. The elimination of XAML to create ribbon interface code is so new that developers don’t seem to know anything about it. Fortunately for me, my apps are of the Chrome class and I won’t need to worry about MetroUI for awhile. By then the integration into MFC will have been completed and tested.

    MFC has never been a language like .NET etc, but rather a very thin and efficient C++ wrapper for the OS. Therefore no huge runtime modules. Microsoft has no choice at this point but to integrate MetroUI into MFC since mobility devices crave fewer CPUs which means less power consumption, and native C++ is the best language for that according to Sutter and Stroustrup.

    1. Old MFC Guy

      I sure hope you are right!
      MetroUI integrated with MFC in Visual Studio.
      I think I’ll party for a week if Microsoft comes through with that platform.

      1. rt8277

        Right now you can create a native C++ MetroUI app using the C++/Cx API of WinRT. This connects up MFC and MetroUI for the first time. Unfortunately, C++/Cx is C#-like which, being an ‘old MFC guy’ as well, I am not interested in. So I’m holding out for a better API.

        And I found it: Just bear through the first part of the interview until you get to Sridhar’s white-boarding of WRL, Windows Runtime.

        WRL will do (it’s not out yet) what ATL COM did for MFC if that makes any sense. Reading between the lines I heard in the interview that they’ve upgraded COM at the same time which would be great. WRL has an .idl file that the MIDL compiler uses. In the .idl you can write your interfaces, including MetroUI interfaces. It looks like it’s going to reduce a lot of COM coding for us in MFC and let us skip the C#-like stuff.

        By the way, my original posting talked about MFC Ribbon and Visual Studio 10. I’ve done a full dynamic ribbon development cycle using it and posted many comments on the MFC C++ ATL and MFC forum. There were a number of gotchas and a couple of real bugs I had to wade through, but bottom line, I got the app done and it looks great complete with dynamic gallery buttons created mixing Ribbon resource editor code and CMFC… method calls. No XAML.

  8. Em

    C++ goes complex and complex every day , I am programming C++ 10+ years I enjoy it , but I love C# more than C++. because C# has very classic and standard way to program and I just recommend to MS Compiler team to build C# native optimizing compiler like C++ Native Optimizing Compiler , it is obvious that The Native compiler can be optimized better than Virtual Compiler
    the objective of Intermediate language like CIL is to be independent from CPU(as there are so many different CPUs around the world) but The CIL (JIT) Compiler team is very slow to build CIL (JIT) Compiler for even Linux! and …
    after 10 years there is no (good) multi platform compiler for C#.NET? but here I expected even native optimized compiler for C#.NET (Such as Intel C++ Optimizing Compiler for Intel CPUs).
    from this point of view JAVA has more better support and widely accepted.

  9. Tony Teveris

    Kenny love the blog, lots of good info. I was wondering since you are a great advocate of Direct2D could you direct me to a code sample of Direct2D and WPF. I want to have a simple app that draws graphics using Direct2D but I can not figure out how to set it up and have looked everywhere. Hoping you can point me in the right direction. Many thanks.

    1. Kenny Kerr Post author

      Do you mean rendering with Direct2D inside an otherwise WPF application? I don’t know of any code samples. It’s certainly possible, although a fair bit of work depending on your particular requirements. WPF provides an interop class called D3DImage, which is an image source. It gets a bit easier with Direct2D 1.1 but it’s still a jolly pain in the neck and in no way ideal. If XAML is what you’re after then the good news is that Windows 8’s native XAML implementation provides fantastic interop with Direct2D. If I get enough interest in the WPF angle, I can include some material on how to do that in part 2 of my Direct2D Fundamentals course.

      1. Eric Hill

        This is definitely something we are interested in. That “fantastic interop with Direct2D” that Windows 8 XAML has is only for Metro apps. We are a long way from porting to Metro. We have a large desktop application that used to be MFC and is now WPF but still uses GDI/GDI+ for rendering graphics (using InteropBitmap/WriteableBitmap). What we really want is the ability to get at the native back buffer of WPF Image elements like what Microsoft has given to Metro app developers in SurfaceImageSource. We have tried using Direct2D but because we are still rendering the Direct2D to a GDI bitmap and then blitting to InteropBitmap/WriteableBitmap, performance gains have been slight.

      2. Kenny Kerr Post author

        Rendering with Direct2D to a GDI bitmap and then passing that to WPF is definitely going to be slow. You can do better with Direct2D to Direct3D to WPF, but it won’t be quite like having access to the DXGI swap chain back buffer as you do with Window 8’s SwapChainBackgroundPanel. WPF just doesn’t have a very efficient rendering pipeline. The D3DImage is about as close to the SurfaceImageSource as you can get without ripping WPF apart. They let you compose into the back buffer without copying back and forth between GPU and CPU memory, but you need to configure Direct2D correctly for the best performance.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s