blog

JavaScript Generators

Asynchronous programming in JavaScript seems to be a double edged sword: on one side, you have a program that does not block the I/O, on the other side, you have Callback Hell and promises. People seem to love promises, I absolutely hate them. Yes, I know they simplify Callback Hell, but it they don't do it well - they are still ugly and messy. There. I said it.

So, what exactly is the problem again? The problem is: suppose you have a series of asynchronous functions that you want to execute sequentially:

function asyncFunction1() {
    setTimeout(function () {
        console.log('- 1');
    }, 1000);
}

function asyncFunction2() {
    setTimeout(function () {
        console.log('- 2');
    }, 500);
}

function asyncFunction3() {
    setTimeout(function () {
        console.log('- 3');
    }, 0);
}

function run() {
    asyncFunction1();
    asyncFunction2();
    asyncFunction3();
}

run();

/*
I wanted:
- 1
- 2
- 3
Result:
- 3
- 2
- 1
*/

This didn't work because the last function executed much faster than the first one. Ok, what is the solution? Callbacks:

function asyncFunction1(cb) {
    setTimeout(function () {
        console.log('- 1');
        cb();
    }, 1000);
}

function asyncFunction2(cb) {
    setTimeout(function () {
        console.log('- 2');
        cb();
    }, 500);
}

function asyncFunction3(cb) {
    setTimeout(function () {
        console.log('- 3');
        cb();
    }, 0);
}


function run() { // EEEEWWW -v
    asyncFunction1(() => {
        asyncFunction2(() => {
            asyncFunction3(() => {});
        });
    });
}

run();

/*
- 1
- 2
- 3
*/

It solved the problem, but now the run function looks messy and weird. So how exactly generators can solve this problem?

First, I want to explain what is a generator: it is a function that can be stopped and resumed. Useless, right? This is how it looks like:

// The notation function* () indicates a generator
function* Sequence() {
    // The yield keyword stops the function at that point
    yield 1;
    yield 2;
    yield 3;
}

// Here we are instantiating the generator - it did not run yet!
const sequence = Sequence();

// When we call .next(), the generator resumes
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());

/*
What we get from a .next is an object with 2 memebers: value,
which is the value that was yielded, and done, a boolean
indicating if we are done or not. In this example, we called
.next() 4 times:
{ value: 1, done: false }
{ value: 2, done: false }
{ value: 3, done: false }
{ value: undefined, done: true }
*/

It doesn't seem very useful yet, but we can get some interesting functionality from it. In the case below, the generator will never be completed, but every time you call .next(), it will increment a counter in 1:

function* Sequence() {
    let i = 0;
    while (true) { yield i++; }
}

const sequence = Sequence();

console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());

/*
{ value: 0, done: false }
{ value: 1, done: false }
{ value: 2, done: false }
{ value: 3, done: false }
{ value: 4, done: false }
*/

Or maybe we could use it to square the number, instead of incrementing:

function* Sequence() {
    let i = 2;
    while (true) { yield i, i *= i; }
}

const sequence = Sequence();

console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());

/*
{ value: 2, done: false }
{ value: 4, done: false }
{ value: 16, done: false }
{ value: 256, done: false }
{ value: 65536, done: false }
*/

Another important aspect of a generator is that the 'yield' keyword returns a value: the value is whatever is passed as a parameter for the .next() function. In the next example, I'm using the value passed to .next() to reset my sequence:

function* Sequence() {
    let i = 1;
    while (true) {
        const reset = yield i++;
        if (reset !== undefined) { i = reset; }
    }
}

const sequence = Sequence();

console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next(100));
console.log(sequence.next());
console.log(sequence.next());

/*
{ value: 1, done: false }
{ value: 2, done: false }
{ value: 3, done: false }
{ value: 100, done: false }
{ value: 101, done: false }
{ value: 102, done: false }
*/

It is important to notice that in order to reach a yield, you need to call a .next() first, otherwise the parameter will simply get tossed away. This example will not print the first parameter:

function* Sequence() {
    while (true) { console.log(yield); }
}

const sequence = Sequence();

sequence.next("a");
sequence.next("b");
sequence.next(1);
sequence.next(2);

/*
b
1
2
*/

So, where exactly do generators solve the problem with asynchronous functions? Ok. Let's start easy. Suppose we have a sequence of 3 typical functions, being the one in the middle asynchronous. This is how we would make them execute in sequence:

function asyncFunction(value, cb) {
    setTimeout(function () {
        console.log(value);
        cb();
    }, 1000);
}

function syncFunction(value) {
    console.log(value);
}

// This function executes them in the order we want
function run() {
    syncFunction('- 1');
    asyncFunction('- 2', function () {
        syncFunction('- 3');
    });
}
run();

/*
- 1
- 2
- 3
*/

Notice the pattern: when we have an asynchronous function (asyncFunction), we give it a callback (syncFunction) to execute when it is done. Now here is the magic: if the function "run" was a generator instead of a normal function, we could make it yield right after the asynchronous function call, and instead of the asynchronous function calling the next function when it is done, it would resume the generator!

This is how it could look like:

let run;

function asyncFunction() {
    setTimeout(function () {
        console.log('- 2');
        run.next();
    }, 1000);
}

function syncFunction(value) {
    console.log(value);
}

// Notice how cleaner the block of this function is
function* Run() {
    syncFunction('- 1');
    yield asyncFunction('- 2');
    syncFunction('- 3');
}

run = Run();
run.next();

/*
- 1
- 2
- 3
*/

A bit messy, right? But you probably agree that the function "run" (now the generator "Run") looks a lot nicer. We can make a wrapper in order to make it look much cleaner.

In this case, I made a function called executeGenerator, which takes care of the ugly part. This is how nice a sequence of 5 function (3 of them asynchronous) would look like using this wrapper:

function syncFunction1() {
    console.log('- 1');
}

// Notice how the asynchronous functions still think they are
// using classic callbacks - this means that libraries that
// rely on callbacks would still be compatible
function asyncFunction2(cb) {
    setTimeout(function () {
        console.log('- 2');
        cb();
    }, 1000);
}

function asyncFunction3(cb) {
    setTimeout(function () {
        console.log('- 3');
        cb();
    }, 500);
}

function asyncFunction4(cb) {
    setTimeout(function () {
        console.log('- 4');
        cb();
    }, 0);
}

function syncFunction5() {
    console.log('- 5');
}

function* run() {
    syncFunction1();
    yield asyncFunction2; // <- no function call here, we
    yield asyncFunction3; //    are yielding the asynchronous
    yield asyncFunction4; //    functions themselves*
    syncFunction5();
}

executeGenerator(run);

/*
- 1
- 2
- 3
- 4
- 5
*/
  • If you had to pass parameters in that case, you could just use .bind to prepare the function.

It not only looks a lot simpler and cleaner, it is still compatible with asynchronous functions that use callbacks. This is how my wrapper looks like:

function executeGenerator(Gen) {
    const gen = Gen();

    function cb() {
        executeTask();
    }

    function executeTask() {
        const nextTask = gen.next();
        const nextAsyncFunction = nextTask.value;
        const isDone = nextTask.done;
        if (isDone) { return; }
        nextAsyncFunction(cb);
    }

    executeTask();
}

There are 2 sub-functions there: 1- cb is a fake callback (I'll explain what it does later) 2- executeTask gets the yielded values from the generator (the asynchronous functions) and executes them, sending the fake callback we just made as the callback

We first start the generator, and then resume it to get the first yielded value (the asynchrnous function), we then execute it passing a callback that will execute the task again: it will resume the generator (which will get the next yielded asynchronous function) and execute the next asynchronous function with the fake callback, which will resume the generator, and so on until our generator is done.

There are several libraries that simplify asynchronous programming with generators, like "co". Despite being hard to understand in the beginning, they are a great alternative to classic callbacks and promises (at least until "async" and "await" are implemented, which will probably happen for the next version of JavaScript).