A Windows Runtime ABI for Modern C++

Given some Windows Runtime type like Windows.UI.Core’s CoreWindow, you have the option to call its methods in the most natural way thanks to Modern C++ for the Windows Runtime:

using namespace Windows::UI::Core;
CoreWindow window = CoreWindow::GetForCurrentThread();

window.Activate();

The Activate method is a modern C++ projection of the underlying ABI call that the operating system expects, but because Modern is a library-based language projection you can easily call the underlying ABI directly simply by using the -> operator to reach down to the underlying interface pointer. What might be a little surprising is that the following won’t compile:

CoreWindow window = CoreWindow::GetForCurrentThread();

HRESULT hr = window->Activate();
// error C2039: 'Activate': is not a member of ...

That’s because there is no such method on the ABI interface. The method is actually called abi_Activate so you need to write the statement as follows:

HRESULT hr = window->abi_Activate();

Now why is that? That’s certainly not how the Windows SDK defines the ICoreWindow ABI.

The Modern compiler generates its own ABI so that’s not a problem, but why does it do that? Well there’s the actual reason and then there’s another good reason that Martyn Lovell came up with during a recent conversation while I was trying to explain my reason. I’ll start with his. :)

Developers approaching the modern C++ language projection without a good understanding of modern C++ may not realize why object->Method is any different to object.Method. Indeed, given all the prior art thanks to C++/CX they could be forgiven for thinking that object->Method is the preferred approach. So had window->Activate compiled, they may not immediately realize the mistake (and the lack of error handling). On the other hand, forcing the developer to choose between window.Activate and window->abi_Activate makes the choice far more obvious. We might say that this is the design of least surprise.

OK, so that’s a good reason and I hadn’t thought of that. Now here’s the real reason. It is very difficult to project the Windows Runtime efficiently into standard C++ in a compile-time manner. Everything about the Modern language projection depends on compile-time type information to produce the fastest possible implementation that adds no run-time overhead. Now a language projection must deal with two rather disparate sets of abstractions. There are those involving binary COM interfaces that must be projected into C++, such as the ICoreWindow interface that is projected as the CoreWindow class. Then there are those involving C++ constructs like classes or lambdas that must be transported back through the runtime as COM interfaces. It is this latter camp where the trouble arises.

When implementing ABI interfaces in modern C++ it’s rather important that they be implemented to satisfy the vtable while also being projected into modern C++ without any additional overhead. Consider the IFrameworkViewSource interface as defined by the Windows SDK. I might need to implement both the virtual functions to satisfy the ABI as well as the modern implementation in the same class. This works reasonably well and might look something like this:

#include <windows.applicationmodel.core.h>
namespace abi = ABI::Windows::ApplicationModel::Core;

struct IFrameworkView {};

struct FrameworkViewSource : Implements<abi::IFrameworkViewSource>
{
    // ABI implementation ...
    virtual HRESULT __stdcall CreateView(abi::IFrameworkView ** view) noexcept override
    {
        // ...
        *view = detach(CreateView());
        // ...
    }

    // Modern implementation ...
    IFrameworkView CreateView()
    {
        // ...
    }
};

Incidentally, Implements is a variadic class template that implements both IUnknown and IInspectable and is included in the Modern library. Anyway, the compiler is happy and the implementation is clean and efficient. I then move on to the IFrameworkView interface and try to do the same thing and all of a sudden I have a problem:

struct FrameworkView : Implements<abi::IFrameworkView>
{
    // ABI implementation ...
    virtual HRESULT __stdcall Run() noexcept override
    {
        // ...
        Run();
        // ...
    }

    // Modern implementation ...
    void Run()
    {
        // ...
    }
};

The FrameworkView class suddenly has two methods with the same name lacking any parameters and thus the ABI’s virtual function and the modern projection differ only by return type and other factors that don’t affect overload resolution. This won’t compile. I would now have to tear apart the implementation to separate the vtable methods from the modern implementation and this adds additional complexity and run-time cost.

Instead, the Modern compiler produces an ABI for the Windows API that avoids the problem. It goes further and produces class templates employing the curiously recurring template pattern (CRTP) to implement the ABI while leaving a modern implementation up to the developer:

template <typename T>
struct IFrameworkViewT : Implements<IFrameworkView>
{
    // ...

    virtual HRESULT __stdcall abi_Run() noexcept override
    {
        return call([&]
        {
            static_cast<T *>(this)->Run();
        });
    }

    // ...
};

The app developer then doesn’t have to think about virtual functions and ABIs and can simply write a Run method in modern C++:

struct FrameworkView : IFrameworkViewT<FrameworkView>
{
    void Run()
    {
        using namespace Windows::UI::Core;

        CoreWindow window = CoreWindow::GetForCurrentThread();
        window.Activate();

        CoreDispatcher dispatcher = window.Dispatcher();
        dispatcher.ProcessEvents(CoreProcessEventsOption::ProcessUntilQuit);
    }
};

This same technique is used to power Windows Runtime components. The Modern compiler produces a modern projection of a component’s runtime classes that a developer would work on, but it also produces base class templates that implement all of the ABI virtual functions while delegating to a modern implementation.

Incidentally, the Modern compiler produced its own ABI long before I solved this problem with the abi_ preamble. It did so because the ABI provided by the Windows SDK is unnecessarily large due to the way the MIDL compiler produces definitions for both C and C++. The Modern ABI is much smaller and thus precompiles a whole lot quicker.

Now here’s another design question for you to ponder: why do the Modern library’s helper functions, such as call and detach above, start with a lower case letter when the convention for everything else mirrors that of the Windows Runtime? Shouldn’t they also start with an uppercase letter? Until next time, check out Modern C++ for the Windows Runtime!

Windows Runtime Internals

My new course about the internals of the Windows Runtime is now available from Pluralsight!

Windows Runtime Internals

Discover the compilers, tools, mechanics, and abstractions that make up the Windows Runtime’s component architecture.

This course will help you understand how the class-based component abstraction presented by Windows Runtime language projections is actually implemented in terms of traditional COM interfaces. You will learn all about the tools and compilers that are necessary to build rich components entirely with Standard C++ in the most efficient manner. You will discover how abstractions like constructors and static methods are implemented with activation factories. You will learn how delegates and events are actually implemented with COM interfaces and lock-free algorithms. And you will discover the weird and wonderful way that the Windows Runtime uses C++ templates to simulate CLR generics using parameterized interfaces.

This is my first advanced level course for Pluralsight and not for the faint-hearted. Enjoy!

Windows Runtime Components

Welcome to a new series in MSDN Magazine exploring the essentials and internals of the Windows Runtime. You can read the first installment right now in the July 2015 issue:

Windows Runtime Components

Over the next few months I’m going to explore the essentials of the Windows Runtime. The purpose is to break down the higher-level abstractions that developers use in the various language projections and toolchains in order to examine how the Windows Runtime works at the application binary interface (ABI)—the boundary between applications and the binary components they rely on to access OS services.

Many thanks to Larry Osterman for his feedback on this series.

What does it mean to be constexpr complete?

Yesterday Stephan Lavavej announced on the Visual C++ blog that their compiler would be constexpr complete for the Visual Studio 2015 build that is released to manufacturing. While that sounds exciting – and it is – I don’t think many developers really know what that means and may make certain reasonable assumptions about what is achievable at compile time. Stephan alluded to a difference between the original goals and aspirations for constexpr in the earlier C++11 standard compared to what is found in the newer C++14 standard. So what does this mean?

The original notion of a generalized constant expression is careful to limit the definition to sufficiently simple functions. That meant that such functions could not contain local variables, iteration statements, or selection statements. Such restrictions quickly became onerous and were lifted in C++14. I’m not a language lawyer so I’ll just give you an illustration. Imagine I need to count the occurrences of some character within a string. I would naturally like to express this at compile time as follows:

static_assert(Count("a % b % c % d") == 3, "fail");

With Visual Studio 2015, I need to be careful to implement the Count constexpr function to avoid using any of the aforementioned constructs. This is a reasonably simple problem so it doesn’t take a great leap of the imagination to write a simple recursive function:

constexpr unsigned Count(char const * const string)
{
    return *string == '\0' ? 0 : (*string == '%') + Count(string + 1);
}

While this is a recursive function, the recursion happens at compile time. In fact, the compiler may choose to unroll or transform the implementation to avoid recursion. From what I can tell, the Visual C++ compiler does not attempt to transform this so if I have a very large string then it will hit the compiler’s call-depth limit. Fortunately, this can be avoided with a compiler option. On the other hand, recursion is clearly not ideal and writing such contrived expressions quickly becomes error prone to say the least. The C++14 definition of constexpr functions makes it possible to write this function with more of the C++ language at my disposal:

constexpr unsigned Count(char const * string)
{
    unsigned count = 0;

    for (; *string != '\0'; ++string)
    {
        count += *string == '%';
    }

    return count;
}

Notice that I’m able to manually transform the recursive function with an iteration statement and use a local variable to keep track of the count. I am even able to increment the pointer. While this is a simple example and there’s nothing wrong with the recursive example above (unless I have very large strings) it sure is handy to be able to have the expressive power of C++14 constexpr functions at my fingertips.

A second dark age for C++

The first beta of the .NET Framework arrived in 2000. As I look back at the decade following its unveiling, I cannot help but think of it as the first dark age of C++. Following on the success of Java, Microsoft likewise chose to distance itself from its strong roots in C and C++. The mighty developer division embraced managed code and worked tirelessly to build incredible tooling for C#. For a time the Visual C++ team tried to play catch-up with not one but two different language extensions for the Common Language Runtime (CLR), but they quickly realized that they were the unwelcome guest at the party. Thankfully, they returned to their roots in native code and so began the C++ renaissance championed by the likes of Charles Torre and others. This eventually culminated in amazing achievements such as the Concurrency Runtime, C++11 and dramatic improvements to the Visual C++ C Runtime (CRT) and Standard Template Library.

Just over ten years following the arrival of .NET, Microsoft announced Windows 8 and the new Windows Runtime. While .NET had arguably succeeded splendidly on the server, it always had an uneasy relationship to the Windows client mainly due to poor performance and the lack of a universal UI framework. With the advent of smaller devices, the operating system group decided to remedy the client problem by returning to the proven performance of C++ and the essentials of COM. The Windows Runtime would become the universal platform for not only the ultimate XAML implementation but also the entire Windows API.

At this point, any optimistic C++ programmer would be forgiven for thinking that the dark ages have passed and good times are ahead. Unfortunately, while the operating system group has embraced C++, the developer division remains largely unchanged. The CLR was retrofitted with support for the Windows Runtime. This is beyond ironic since .NET was specifically designed as a departure from COM with an incompatible vtable model. Performance isn’t great despite the fact that the Windows Runtime does make a number of concessions for the CLR, but at least most of the heavy lifting is now done by the operating system so C# code gets a pass.

While the operating system group has embraced C++, the developer division continues to champion C# as the tool of choice and the Visual C++ team continues to focus on standards compliance and more recently cross-platform development. Little has been done to support the Windows Runtime, certainly nothing that has captured the hearts and minds of C++ developers. C++/CLI was dusted off – a language extension designed for a garbage-collected runtime – and now offers an unnecessarily complex way to work with the Windows Runtime that makes no sense for COM. C++ is a language for library developers. The library developer’s job may not be a simple one, but you certainly have the tools at your disposal to do some amazing things. So when you hit some feature that isn’t directly supported by the language, then you write a library – you don’t go and invent a new language or write a set of language extensions. Prior to C++11 you could be forgiven for thinking in that way, but not now.

Microsoft desperately needs an infusion of modern C++ for Windows. The Universal Windows Platform built on the Windows Runtime is the ideal environment for C++ to thrive. What is missing is the determination to make it happen. Modern C++ for the Windows Runtime is the jumpstart that the Windows platform needs but it cannot happen without the support of both the operating system group and the developer division.

Jim Radigan has done some incredible work bringing code written for Android and iOS apps to Windows. Now we need to bring modern C++ to Windows as well. The C++ community is stronger than ever and forging ahead with incredible innovations to the C++ language and standard libraries. Will Windows be part of this new C++ renaissance or will this be another dark age for C++ programmers on the Windows platform?