Variadic generators in C++

Coroutines allow us to write generators that yield values such that a function can effectively return multiple values. The coroutine return type can then provide begin and end functions and thus behave like an STL container or range expression. If you have Visual C++ handy, you can follow along using its simple experimental generator:

#include <experimental/generator>
using namespace std::experimental;

We can write a function that will produce a sequence of values as follows:

generator<int> get_values()
{
    co_yield 1;
    co_yield 2;
    co_yield 3;
}

From the caller’s perspective, it behaves much the same as if the get_values function had been written as follows:

std::vector<int> get_values()
{
    return { 1, 2, 3 };
}

The difference is that the coroutine does not require the container to be prepared in advance (or even allocated), but the caller can simply rely on the resulting range expression and write the same range-based for loop regardless:

int main()
{
    for (int value : get_values())
    {
        printf("%d\n", value);
    }
}

As you might expect, the results are staggering:

1
2
3
Press any key to continue . . .

But that’s not very interesting. Let’s imagine we have two (or more) containers with numbers as follows:

std::vector<int> a{ 1, 2, 3 };
std::vector<int> b{ 4, 5, 6 };

And we’d like to use the same simple range-based for loop to print out all their values:

for (int value : get_container_values(a, b))
{
    printf("%d\n", value);
}

How might we achieve that? With variadics of course. First we need to write a generalized get_values function that can effectively turn a parameter pack into a range:

template <typename First, typename ... Rest>
generator<First> get_values(First const& first, Rest const& ... rest)
{
    int ignored[] = { (co_yield first, 0), (co_yield rest, 0) ... }; ignored;
}

We need to separate out the first template argument from the parameter pack so that we can identify the type of the resulting generator. We then simply expand the function arguments and yield up each value in turn.

With this version of get_values, we can now write a more generalized version of the original example as follows:

for (int value : get_values(1, 2, 3))
{
    printf("%d\n", value);
}

No longer does the get_values function need to hardcode the values to yield to the caller and it works with types other than int. Using get_values as a building block, we can then write the get_container_values function that will iterate over its arguments and then iterate over each container’s values, yielding each in turn:

template <typename First, typename ... Rest, typename Value = First::value_type>
generator<Value> get_container_values(First const& first, Rest const& ... rest)
{
    for (First const& current : get_values(first, rest ...))
    {
        for (Value const& value : current)
        {
            co_yield value;
        }
    }
}

Again, the first template argument must be separated to determine the type of the resulting generator. The get_values function is used to produce a range from the get_container_values arguments. Could this be implemented without coroutines? Sure, but not nearly as simply and most likely involving copies of containers and elements. The get_container_values function will not make a copy of a single element.

We can now write the following simple program:

int main()
{
    std::vector<int> a{ 1, 2, 3 };
    std::vector<int> b{ 4, 5, 6 };

    for (int value : get_container_values(a, b))
    {
        printf("%d\n", value);
    }
}

The results are beautifully simple:

1
2
3
4
5
6
Press any key to continue . . .

2 thoughts on “Variadic generators in C++

  1. Chris

    Thanks Kenny. Just curious… what does the error message look like when the containers have incompatible value types?

    Reply
    1. Kenny Kerr Post author

      error C2664: ‘…’: cannot convert argument 1 from ‘…’ to ‘…’

      I would like to think that concepts will help with error messages.

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s