Ch 6 — Futures in Rust

Futures in Rust In Chapter 5, we covered one of the most popular ways of modeling concurrency in a programming language: fibers/green threads. Fibers/green threads are an example of stackful coroutines. The other popular way of modeling asynchronous program flow is by using what we call stackless coroutines, and combining Rust’s futures with async/await is an example of that. We will cover this in detail in the next chapters. This first chapter will introduce Rust’s futures to you, and the main goals of this chapter are to do the following:

• Give you a high-level introduction to concurrency in Rust • Explain what Rust provides and not in the language and standard library when working with async code • Get to know why we need a runtime library in Rust • Understand the difference between a leaf future and a non-leaf future • Get insight into how to handle CPU-intensive tasks

To accomplish this, we’ll divide this chapter into the following sections:

What is a future?

Leaf futures

Non-leaf futures

• Runtimes • A mental model of an async runtime • What the Rust language and standard library take care of • I/O vs CPU-intensive tasks • Advantages and disadvantages of Rust’s async model

What is a future?

A future is a representation of some operation that will be completed in the future. Async in Rust uses a poll-based approach in which an asynchronous task will have three phases:

1. The poll phase: A future is polled, which results in the task progressing until a point where it can no longer make progress. We often refer to the part of the runtime that polls a future as an executor. 2. The wait phase: An event source, most often referred to as a reactor, registers that a future is waiting for an event to happen and makes sure that it will wake the future when that event is ready. 3. The wake phase: The event happens and the future is woken up. It’s now up to the executor that polled the future in step 1 to schedule the future to be polled again and make further progress until it completes or reaches a new point where it can’t make further progress and the cycle repeats.

Now, when we talk about futures, I find it useful to make a distinction between non-leaf futures and leaf futures early on because, in practice, they’re pretty different from one another.

      Leaf futures
      Runtimes create leaf futures, which represent a resource such as a socket.
      This is an example of a leaf future:
        let mut stream = tokio::net::TcpStream::connect("127.0.0.1:3000");

      Operations on these resources, such as a reading from a socket, will be non-blocking and return a
      future, which we call a leaf future since it’s the future that we’re actually waiting on.
      It’s unlikely that you’ll implement a leaf future yourself unless you’re writing a runtime, but we’ll go
      through how they’re constructed in this book as well.
      It’s also unlikely that you’ll pass a leaf future to a runtime and run it to completion alone, as you’ll
      understand by reading the next paragraph.

Non-leaf futures

Non-leaf futures are the kind of futures we as users of a runtime write ourselves using the async keyword to create a task that can be run on the executor. The bulk of an async program will consist of non-leaf futures, which are a kind of pause-able computation. This is an important distinction since these futures represent a set of operations. Often, such a task will await a leaf future as one of many operations to complete the task.

This is an example of a non-leaf future:
  let non_leaf = async {
      let mut stream = TcpStream::connect("127.0.0.1:3000").await.unwrap();
      println!("connected!");
      let result = stream.write(b"hello world\n").await;
      println!("message sent!");
      ...
  };

The two highlighted lines indicate points where we pause the execution, yield control to a runtime, and eventually resume. In contrast to leaf futures, these kinds of futures do not themselves represent an I/O resource. When we poll them, they will run until they get to a leaf future that returns Pending and then yields control to the scheduler (which is a part of what we call the runtime).

Runtimes Languages such as C#, JavaScript, Java, Go, and many others come with a runtime for handling concurrency. So, if you’re used to one of those languages, this will seem a bit strange to you. Rust is different from these languages in the sense that Rust doesn’t come with a runtime for handling concurrency, so you need to use a library that provides this for you. Quite a bit of complexity attributed to futures is actually complexity rooted in runtimes; creating an efficient runtime is hard. Learning how to use one correctly requires quite a bit of effort as well, but you’ll see that there are several similarities between this kind of runtime, so learning one makes learning the next much easier. The difference between Rust and other languages is that you have to make an active choice when it comes to picking a runtime. Most often, in other languages, you’ll just use the one provided for you.

A mental model of an async runtime

I find it easier to reason about how futures work by creating a high-level mental model we can use. To do that, I have to introduce the concept of a runtime that will drive our futures to completion.

Note The mental model I create here is not the only way to drive futures to completion, and Rust’s futures do not impose any restrictions on how you actually accomplish this task.

A fully working async system in Rust can be divided into three parts:

         • Reactor (responsible for notifying about I/O events)
         • Executor (scheduler)
         • Future (a task that can stop and resume at specific points)

So, how do these three parts work together? Let’s take a look at a diagram that shows a simplified overview of an async runtime:

Figure 6.1 – Reactor, executor, and waker

      In step 1 of the figure, an executor holds a list of futures. It will try to run the future by polling it (the
      poll phase), and when it does, it hands it a Waker. The future either returns Poll:Ready (which
      means it’s finished) or Poll::Pending (which means it’s not done but can’t get further at the
      moment). When the executor receives one of these results, it knows it can start polling a different
      future. We call these points where control is shifted back to the executor yield points.
      In step 2, the reactor stores a copy of the Waker that the executor passed to the future when it polled
      it. The reactor tracks events on that I/O source, usually through the same type of event queue that we
      learned about in Chapter 4.

In step 3, when the reactor gets a notification that an event has happened on one of the tracked sources,
it locates the Waker associated with that source and calls Waker::wake on it. This will in turn
inform the executor that the future is ready to make progress so it can poll it once more.
If we write a short async program using pseudocode, it will look like this:
  async fn foo() {
      println!("Start!");
      let txt = io::read_to_string().await.unwrap();
      println!("{txt}");
  }

The line where we write await is the one that will return control back to the scheduler. This is often called a yield point since it will return either Poll::Pending or Poll::Ready (most likely it will return Poll::Pending the first time the future is polled). Since the Waker is the same across all executors, reactors can, in theory, be completely oblivious to the type of executor, and vice-versa. Executors and reactors never need to communicate with one another directly. This design is what gives the futures framework its power and flexibility and allows the Rust standard library to provide an ergonomic, zero-cost abstraction for us to use.

Note I introduced the concept of reactors and executors here like it’s something everyone knows about. I know that’s not the case, and don’t worry, we’ll go through this in detail in the next chapter.

What the Rust language and standard library take care of

Rust only provides what’s necessary to model asynchronous operations in the language. Basically, it provides the following:

• A common interface that represents an operation, which will be completed in the future through the Future trait • An ergonomic way of creating tasks (stackless coroutines to be precise) that can be suspended and resumed through the async and await keywords • A defined interface to wake up a suspended task through the Waker type

That’s really what Rust’s standard library does. As you see there is no definition of non-blocking I/O, how these tasks are created, or how they’re run. There is no non-blocking version of the standard library, so to actually run an asynchronous program, you have to either create or decide on a runtime to use.

      I/O vs CPU-intensive tasks
      As you know now, what you normally write are called non-leaf futures. Let’s take a look at this async
      block using pseudo-Rust as an example:
        let non_leaf = async {
            let mut stream = TcpStream::connect("127.0.0.1:3000").await.unwrap();

            // request a large dataset
            let result = stream.write(get_dataset_request).await.unwrap();

            // wait for the dataset
            let mut response = vec![];
            stream.read(&mut response).await.unwrap();

            // do some CPU-intensive analysis on the dataset
            let report = analyzer::analyze_data(response).unwrap();

            // send the results back
            stream.write(report).await.unwrap();
        };

I’ve highlighted the points where we yield control to the runtime executor. It’s important to be aware that the code we write between the yield points runs on the same thread as our executor. That means that while our analyzer is working on the dataset, the executor is busy doing calculations instead of handling new requests. Fortunately, there are a few ways to handle this, and it’s not difficult, but it’s something you must be aware of:

1. We could create a new leaf future, which sends our task to another thread and resolves when the task is finished. We could await this leaf-future like any other future. 2. The runtime could have some kind of supervisor that monitors how much time different tasks take and moves the executor itself to a different thread so it can continue to run even though our analyzer task is blocking the original executor thread. 3. You can create a reactor yourself that is compatible with the runtime, which does the analysis any way you see fit and returns a future that can be awaited.

Now, the first way is the usual way of handling this, but some executors implement the second method as well. The problem with #2 is that if you switch runtime, you need to make sure that it supports this kind of supervision as well or else you will end up blocking the executor.

The third method is more of theoretical importance; normally, you’d be happy to send the task to the
thread pool that most runtimes provide.
Most executors have a way to accomplish #1 using methods such as spawn_blocking.
These methods send the task to a thread pool created by the runtime where you can either perform
CPU-intensive tasks or blocking tasks that are not supported by the runtime.

Summary

So, in this short chapter, we introduced Rust’s futures to you. You should now have a basic idea of what Rust’s async design looks like, what the language provides for you, and what you need to get elsewhere. You should also have an idea of what a leaf future and a non-leaf future are. These aspects are important as they’re design decisions built into the language. You know by now that Rust uses stackless coroutines to model asynchronous operations, but since a coroutine doesn’t do anything in and of itself, it’s important to know that the choice of how to schedule and run these coroutines is left up to you. We’ll get a much better understanding as we start to explain how this all works in detail as we move forward. Now that we’ve seen a high-level overview of Rust’s futures, we’ll start explaining how they work from the ground up. The next chapter will cover the concept of futures and how they’re connected with coroutines and the async/await keywords in Rust. We’ll see for ourselves how they represent tasks that can pause and resume their execution, which is a prerequisite to having multiple tasks be in progress concurrently, and how they differ from the pausable/resumable tasks we implemented as fibers/green threads in Chapter 5.