blog

Concurrent functions with Go using channels

I've always been a big fan of new technologies and languages: there is always something new and interesting in them. For the past weeks, I've been experimenting with Go, a free and open-source programming language made by Google. It is imperative, strong typed, and compiled, and with a syntax that remind me of C; just like C, it also has pointers, but it has a garbage collector.

One thing that really caught my attention was how concurrent (asynchronous) functions can be made and synchronized: they are called goroutines, described as "light-weight threads of execution", and can be synchronized using channels - a First In First Out queue. In this post, I want to show how I one example of how these goroutines with channels work.

This example is a little application that simulates a pizzeria: we will have a line that makes the sauce, a line that makes the dough, and a line that prepares the toppings; after all these 3 lines have their ingredients ready, the pizza is assembled and baked, and then a receipt is printed. In normal synchronous programming, first, we would make the sauce, then the dough, then we would prepare the toppings, assemble and bake, and then print the receipt. In asynchronous programming, however, we can fire the functions to make the sauce, the dough and prepare the toppings all at the same time, then, after we have all these 3 steps, we can assemble the pizza and bake it. In asynchronous JavaScript, we would start executing the three first functions, and then, when the last one was finished, it would execute a callback to assemble and bake the pizza, and then, it would execute another callback to print the receipt. In Go, we can use Goroutines and channels for this task.

To keep things simple, let's suppose that every line (the line that makes the dough, for example) can work on several orders at the same time. For example: they can make the pizza dough for 3 clients at the same time.

First, I'll declare the name of my package and make the imports for the modules I need:

package main

import (
    "fmt"
    "math/rand"
    "time"
    "sync"
)

Now I will make a struct (as far as I know, there are not classes in Go, only structs; but you can attach methods to them and turn them into classes) for a Pizza. A pizza will have a client (name of the client, string), some details about it (how the dough was made, how the sauce was made, etc. They will be a channel (I will show you why later) of a string), some boolean values that indicate which steps were completed (they are also channels), and a function that can be called when everything is ready and the pizza is finished.

type Pizza struct {
    client  string
    details struct {
        dough     chan string
        sauce     chan string
        toppings  chan string
        assembled chan string
    }
    completed struct {
        dough     chan bool
        sauce     chan bool
        toppings  chan bool
        assembled chan bool
    }
    Done func()</pre>
}

I also made a little function that will give me a random integer so every step will take a different amount of time to be completed:

func randomTime() time.Duration {
    r := time.Duration(rand.Int31n(9))
    return time.Second * r;
}

Now, the three functions that will be ran at the same time: makeDough, makeSauce and prepareToppings. They are just normal functions, the difference is how they get executed; this is what makeDough looks like:

// This function receives the name of the client, a string channel for it
// to record a message (details), and a bool channel for it to record when the dough
// is ready
func makeDough(client string, message chan<- string, completed chan<- bool) {
    fmt.Print("Starting making pizza dough for #", client, "\n")

    // We take a random amount of time for the function to be completed
    time.Sleep(randomTime())

    fmt.Print("Finished pizza dough for #", client, "\n")

    // Recording the message and "true" in the channels
    // You can imagine the channel as being "cout" from C++
    // and the <- operator being "<<": you are recording
    // something into the channel
    message <- "Pizza Dough"
    completed <- true
}

And here are the other functions:

func makeSauce(client string, message chan<- string, completed chan<- bool) {
    fmt.Print("Starting making pizza sauce for #", client, "\n")

    time.Sleep(randomTime())

    fmt.Print("Finished pizza sauce for #", client, "\n")

    message <- "Pizza Sauce"
    completed <- true
}

func prepareToppings(client string, message chan<- string, completed chan<- bool) {
    fmt.Print("Starting preparing pizza toppings for #", client, "\n")

    time.Sleep(randomTime())

    fmt.Print("Finished preparing pizza toppings for #", client, "\n")

    message <- "Pizza Toppings"
    completed <- true
}

Simple enough, right? Channels are like queues, where you input data, and then you can pop it later. But here is the catch: channels will block the execution of the function until the other "side" is ready; in other words: if you push something in the channel, it will block the function until you pop it - it also works on the opposite: if you try to pop something from an empty channel, it will block the function until there is something there to be popped. This can be used to pause/unpause goroutines.

Now, if you go back to the functions that I described, you can imagine what is going to happen in this case:

func prepareToppings(client string, message chan<- string, completed chan<- bool) {
    fmt.Print("Starting preparing pizza toppings for #", client, "\n")

    time.Sleep(randomTime())

    fmt.Print("Finished preparing pizza toppings for #", client, "\n")

    // The following line will be executed and then the goroutine will stop: it will
    // only continue when we remove the string from the channel
    message <- "Pizza Toppings"

    // This line will only be executed when the message "Pizza toppings" is
    // removed from the channel above
    completed <- true
}

So, to make sure we don't reach a deadlock, we must make sure that the channels are properly emptied and closed: I will show how to extract the data from a channel and how to close them in this next function. This function will listen to the "completed" channels to make sure the sauce, the dough, and the toppings are prepared - we can only assemble and bake the pizza if we have these three parts ready:

func assembleAndBake(pizza Pizza) {

    // Here I am extracting the boolean from the "dough" channel. Since
    // we don't care about the values, we just discard them
    // Notice that the execution will be blocked here until there is a "completed" value for
    // dough that we can pop; in other words: the function will not execute past this
    // until we get a boolean from the "makeDough" function
    <- pizza.completed.dough

    // After we got the message, we can close the channel to prevent any more writing into it
    close(pizza.completed.dough)

    <- pizza.completed.sauce
    close(pizza.completed.sauce)

    <- pizza.completed.toppings
    close(pizza.completed.toppings)

    fmt.Print("Starting assembling and baking pizza for #", pizza.client, "\n")

    time.Sleep(randomTime())

    fmt.Print("Finished assembling and baking pizza for #", pizza.client, "\n")

    // If we reached here, it means that the pizza is now assembled and baked: we
    // record a message and a boolean for this event in the channels
    pizza.details.assembled <- "Assembling and baking"
    pizza.completed.assembled <- true
}

Now I am going to receive the details (message) in my function to print the receipt - I want to print the messages in the receipts. This is how my function look like:

// This function receives the Pizza object
func printReceipt(pizza Pizza) {

    // "defer" tells the function to execute this line only when the function finishes: it will
    // tell the program that this pizza is done and the "chain" is over for this client.
    // I will explain what this part does in more details later - I need to show you the 
    // rest of my script first. For now, just ignore it.
    defer pizza.Done()

    // Here I am taking whatever message we have in the details for the dough and
    // recording it in a variable called 'msg1'.
    msg1 := <- pizza.details.dough
    close(pizza.details.dough)

    msg2 := <- pizza.details.sauce
    close(pizza.details.sauce)

    msg3 := <- pizza.details.toppings
    close(pizza.details.toppings)

    msg4 := <- pizza.details.assembled
    close(pizza.details.assembled)

    // Here I am popping the boolean value from the "completed" field of the pizza. Since
    // I don't really care what the value is, I am not saving it anywhere
    <- pizza.completed.assembled
    close(pizza.completed.assembled)

    // If we reached here, it means that the pizza was assembled and baked - we can now
    // print the receipt
    fmt.Print("--------------------------------------------------\n" +
              "Receipt for #", pizza.client, ":\n" +
              ". ", msg1, "\n" +
              ". ", msg2, "\n" +
              ". ", msg3, "\n" +
              ". ", msg4, "\n" +
              "--------------------------------------------------\n")
}

Alright, these are the functions we need to assemble the pizza, now we just need the main function.

The main function will be responsible for launching the goroutines for three different clients: John, Alan and Paul. However, it also needs to wait for their orders to finish before the process exits - how can we ensure this will happen?

To make sure our process will not exit before everything is done, we can use a WaitGroup: imagine it as an class that you start and can specify how many groups you want to wait for (in this case, three: one for every client), and every time a group is completed, it calls the function waitGroup.Done() - so when all of them were called, the waitGroup is finished.

This is how my main function looks like:

func main() {
    // Seeding a random time
    rand.Seed(time.Now().UTC().UnixNano())

    // Creating a wait group
    var wg sync.WaitGroup

    // Making a list of clients
    clients := []string {
        "John",
        "Alan",
        "Paul",
    }

    // Getting the number of clients
    clientsNo := len(clients)

    // Looping through every client
    for i := 0; i < clientsNo; i++ {

        // For every client, we add one more group in the WaitGroup
        wg.Add(1);

        // Instantiating a new Pizza for the client
        pizza := Pizza{}
        pizza.client = clients[i]
        pizza.details.dough     = make(chan string)
        pizza.details.sauce     = make(chan string)
        pizza.details.toppings  = make(chan string)
        pizza.details.assembled = make(chan string)
        pizza.completed.dough     = make(chan bool)
        pizza.completed.sauce     = make(chan bool)
        pizza.completed.toppings  = make(chan bool)
        pizza.completed.assembled = make(chan bool)

        // This part is important: remember that line that I "deferred" a method call for
        // Done()? This is where it comes from: when the pizza is done, it tells the
        // WaitGroup that there is one less group to wait for
        pizza.Done = wg.Done

        // Here we are launching the asynchronous functions: the "go" prefix specifies
        // that these are not ordinary functions, but goroutines. To these routines, I am
        // passing the channels and other data they need
        go makeDough(pizza.client,       pizza.details.dough,    pizza.completed.dough)
        go makeSauce(pizza.client,       pizza.details.sauce,    pizza.completed.sauce)
        go prepareToppings(pizza.client, pizza.details.toppings, pizza.completed.toppings)
        go assembleAndBake(pizza)
        go printReceipt(pizza)

    }

    // Here we are telling the WaitGroup to wait until all the groups are done
    wg.Wait()

}

These are the outputs:

For only one client (Paul)

Starting preparing pizza toppings for #Paul
Starting making pizza sauce for #Paul
Starting making pizza dough for #Paul
Finished pizza dough for #Paul
Finished pizza sauce for #Paul
Finished preparing pizza toppings for #Paul
Starting assembling and baking pizza for #Paul
Finished assembling and baking pizza for #Paul
--------------------------------------------------
Receipt for #Paul:
. Pizza Dough
. Pizza Sauce
. Pizza Toppings
. Assembling and baking
--------------------------------------------------

For all three clients

Starting making pizza dough for #Alan
Starting preparing pizza toppings for #John
Finished preparing pizza toppings for #John
Starting preparing pizza toppings for #Alan
Starting making pizza sauce for #Alan
Starting making pizza sauce for #Paul
Starting making pizza dough for #Paul
Starting preparing pizza toppings for #Paul
Starting making pizza sauce for #John
Starting making pizza dough for #John
Finished pizza dough for #Alan
Finished pizza dough for #Paul
Finished preparing pizza toppings for #Alan
Finished pizza dough for #John
Finished pizza sauce for #Paul
Finished preparing pizza toppings for #Paul
Starting assembling and baking pizza for #Paul
Finished pizza sauce for #Alan
Starting assembling and baking pizza for #Alan
Finished pizza sauce for #John
Starting assembling and baking pizza for #John
Finished assembling and baking pizza for #Alan
--------------------------------------------------
Receipt for #Alan:
. Pizza Dough
. Pizza Sauce
. Pizza Toppings
. Assembling and baking
--------------------------------------------------
Finished assembling and baking pizza for #John
--------------------------------------------------
Receipt for #John:
. Pizza Dough
. Pizza Sauce
. Pizza Toppings
. Assembling and baking
--------------------------------------------------
Finished assembling and baking pizza for #Paul
--------------------------------------------------
Receipt for #Paul:
. Pizza Dough
. Pizza Sauce
. Pizza Toppings
. Assembling and baking
--------------------------------------------------