Tracking Issue for Async I/O migration

Status

Rocket's master branch is currently fully asynchronous and supports async/await!

Pending

(async) Remove or replace typed headers #1067 - Remove or replace typed headers (PR: Fix #1067 by importing typed headers from hyperx #1535)
Fully update documentation for 0.5 #1329 - Documentation fixes, additions, improvements.
Provide a lower-level connection interface #1070 - Expose APIs for customizing the underlying I/O, instead of being restricted to TCP with optional TLS.
Detect blocking I/O in async: it seems the best we can do for now is to document the issues.
A "blocking-in-async" lint in clippy or rustc would be much nicer, if feasible, but would occur upstream...someday.
Miscellaneous small TODOs (tagged as TODO.async) throughout code.
Response bodies are passed back to hyper using a channel.

Done

Async I/O everything!
Harden and Enforce Data Read Limits #1325 - Data reading and connection limits.
Keep-alive and idle client timeouts
Handling of blocking database connections.
Gracefully handle panics with 500. (PR Catch and gracefully handle panics in routes and catchers. #1491)
(async) Add a Write-like interface for responses #1066 - Add a Write-like interface for responses, and consideration of AsyncRead vs Stream<Item=Result<Chunk, Error>>
Clean shutdown? #180, Graceful shutdown timeout #1374 - fixing graceful shutdown when using "infinite" responders
Support for Server Sent Events #33 - Server-Sent Events
Tracking Issue for Asynchronous Database Pool Support #1117 - Asynchronous database pools, or at the very least forward compatibility with them.

Design

The async APIs are based on rust's async/await feature and std::future::Future, running on the tokio (0.2) runtime.

High-level APIs (routes and catchers):
- Routes can now be defined as #[get|post|route(...)] async fn(params...) -> R where R is any Responder type. Routes that do not await anything can remain unchanged. #[catch]ers can also be defined as an async fn.
- Methods on Data now return Futures that must be .awaited. As a consequence, any route that uses Data directly must be async. Routes that use data guards such as Json are not affected by this change.
- Routes that make their own network requests or other I/O should be async, as it will allow other requests to continue while waiting for the operation.
Mid and low-level APIs (FromTransformedData and FromData, Fairing, Responder)
- Several traits are now defined using async-trait, re-exported as #[rocket::async_trait].
- Data is read asynchronously: DataStream implements tokio::io::AsyncRead instead of Read. Thus, FromTransformedData::transform and From(Transformed)Data::from_data are async fn or return Futures.
- Response bodies are streamed to the client asynchronously: set_body and related methods expect tokio::io::AsyncRead instead of Read.
  - Consequently, Fairing::on_response and Responder::respond_to are now async fn. This allows Responders and Fairings to access and/or replace the body data.
The testing APIs have been moved to rocket::local::asynchronous and rocket::local::blocking. The blocking test API is simpler to use, and recommended when possible.
#[rocket::main], #[rocket::launch], and #[rocket::async_test] have been added to make async fn easily accessible for main() and test().
Many long-standing bugs with keep-alive, graceful shutdown (Clean shutdown? #180), performance, and more are fixed or will be easier to fix by using actively-supported versions of libraries.
Some new functionality will be possible to implement, such as SSE and WebSockets. These will likely not make it into Rocket 0.5. At this point, Rocket should not be missing any APIs necessary to implement SSE "by hand". WebSockets is a bit more complicated, and tracked in Native WebSocket support #90 .

@jebrosen

The async_await feature is now required in addition to proc_macro_hygine and decl_macro

Wasn't decl_macro removed on the master branch a while back?

Currently, routes must either be fn -> R where R: Responder (non-async) or async fn -> R where R: Responder (async). Should we try to support (non-async) fn -> impl Future<Output=R> where R: Responder?

I don't see why not. Though it would be relatively trivial to create it async and await the returned value(s).

I'll take a look at uuid tomorrow to see if any changes are necessary. I suspect the answer is no.

@jebrosen Roughly how much of the documentation can be switched over to async? Are we going to primarily document in async, or will we essentially double everything up?

@jebrosen

Wasn't decl_macro removed on the master branch a while back?

Right! I'll fix that.

Currently, routes must either be fn -> R where R: Responder (non-async) or async fn -> R where R: Responder (async). Should we try to support (non-async) fn -> impl Future<Output=R> where R: Responder?

I don't see why not. Though it would be relatively trivial to create it async and await the returned value(s).

The issue is the return type of routes. Currently the codegen assumes that a non-async function returns some type R that implements Responder and an async function returns a Future whose output type is some R implementing Responder. Allowing functions to return either a Responder or a Future is hairy because both are traits and pretty soon we would end up wrestling with specialization. But I might have missed something in my analysis, and I would really like to support it if we can!

@jebrosen Roughly how much of the documentation can be switched over to async? Are we going to primarily document in async, or will we essentially double everything up?

The only place where there is a choice is routes; currently everything else is an async-only API and documentation should reflect that. I was going to wait a bit before pushing hard for docs, as there are likely a few design issues to be resolved with which APIs return which futures with what lifetime bounds.

My admittedly hand-wavy assumption was that we would explain traits like FromData and Responder in terms of async and futures, but use mostly non-async routes. We should definitely have a few examples of async routes, perhaps for HTTP requests or File I/O. For a lot of routes I do expect it to be as easy as changing fn to async fn and using .await in the right places.

Another really important point here is that manual FromData and Responder implementations should be relatively rare and are much more in the domain of API documentation than in the guide. I am personally more worried about the accessibility and understandability of the guide.

The issue is the return type of routes. Currently the codegen assumes that a non-async function returns some type R that implements Responder and an async function returns a Future whose output type is some R implementing Responder. Allowing functions to return either a Responder or a Future is hairy because both are traits and pretty soon we would end up wrestling with specialization. But I might have missed something in my analysis, and I would really like to support it if we can!

That certainly makes sense. Adding support for -> impl Future would be backwards compatible, as it doesn't currently work. So that's a point that can certainly be postponed to see if there's even demand after async lands.

The only place where there is a choice is routes; currently everything else is an async-only API and documentation should reflect that. I was going to wait a bit before pushing hard for docs, as there are likely a few design issues to be resolved with which APIs return which futures with what lifetime bounds.

Probably best to hold off on docs for the time being, then.

compression - Not for the faint of heart; there might be some important APIs that are not yet available. HELP WELCOME

Quick advertisement for https://github.com/rustasync/async-compression, this should allow just drop-in wrappers to transparently compress/decompress AsyncRead/AsyncWrite instances. Not all encodings are supported for the read/write interfaces yet as the first framework it was used in used the impl Stream<Item = io::Result<Bytes>> adaptors instead, but it should be straightforward to expand them (at least once I merge the initial AsyncWrite support).

@jebrosen

@jebrosen this is awesome!

Just as a side note, both tokio and hyper master are working with std::future::Future now. It's still a WIP but in my testing the essentials are all working, so this could already be based on the master branches instead of dealing with the compat layer.

contrib/lib/src/uuid.rs looks good to me as-is. It's just a thin wrapper around the uuid crate, so there's not really any room for making things async, unless we were to go all in and parse it asynchronously.

databases - We are not necessarily pushing for asynchronous database engines, but we should at least ensure they will be usable. postgres is a good candidate for exploring this.

Hi - what would you be looking for in a pull request to explore this? I've been developing some projects using Rocket and have been using both tokio-postgres and the synchronous rust-postgres, as well as their connection pools, and I would like to help if I think I know how.

Hi - what would you be looking for in a pull request to explore this?

I have not tried too hard to test what "ensure they will be usable" entails. There are a few problems I can see ahead of time:

Futures-aware database engines probably need to be run within a specific runtime (i.e. tokio 0.1 or tokio 0.2). So once Rocket uses tokio 0.2 ((async) Use the futures 0.3 - compatible versions of hyper and tokio #1071), you wouldn't be able to use the current version of tokio-postgres. This is a non-obvious compatibility problem that isn't expressed in any type signature.
"Connect" might be an asynchronous operation. This seems to be the case for tokio-postgres, for example. So we should evaluate if making FromRequest return a future would be worth it for making databases (among other things) easier to work with.

At a minimum, I think we should be able to:

Create a connection pool with tokio-postgres and/or bb8 and put it in managed state. This should tell us if on_attach and on_launch need to be async-capable.
Create a request guard type wrapping a connection (or a future that will resolve to a connection). This and the previous bullet point together are more or less what the guide demonstrated for databases in Rocket 0.3 before we had rocket_contrib::databases.
Use said connection via the request guard within a route to query/alter the database. This should work fine, because routes can now be async fn.
Test, figure out, and document cross-runtime compatibility (or lack thereof). That is, a definitive statement in the form of "Rocket uses tokio 0.2, so your database futures have to as well" if that's the case. Alternatively, maybe we could find a proof of concept of running two runtimes - tokio 0.2 (for rocket) alongside tokio 0.1 (for tokio-postgres) with some kind of message passing between them. I leave this intentionally vague because it's an annoying solution to an annoying problem and someone other than I will probably think of a better idea.

I am mostly waiting on #1071 (after a new tokio alpha and hyperium/hyper#1897) so that we don't need to significantly rewrite too much code that directly deals with the low-level bits -- which are pretty different between tokio 0.1 and 0.2. For example, I suspect #1070 will take almost as much work again as it would to implement it now.

At present all public APIs use async/await and futures 0.3, but everything runs on a tokio 0.1 runtime. This means that I/O types from tokio 0.2 cannot be used directly in routes yet, for example.

For anyone interested in kicking the tires pretty early on, I have written a short and possibly incomplete guide to trying projects against the async branch. At this stage the most helpful feedback would be examples of "things I can't do anymore" or that now require expensive allocations, especially in implementations of FromData or Fairing.

Depend on rocket = { git = "https://github.com/SergioBenitez/Rocket", branch = "async" } and rocket_contrib = { git = "https://github.com/SergioBenitez/Rocket", branch = "async" }
~~Add #![feature(async_await)] to your crate.~~ You can remove #[feature(decl_macro)] thanks to Remove use of decl_macro by using a different hack. #964.
Typed headers are removed and a replacement has not been decided on ((async) Remove or replace typed headers #1067), so you will need to build headers manually and/or use raw headers at least for now
Implementations of FromData, Fairing, AdHoc::on_response, and Responder must be converted to use futures. For some cases, wrapping the function body in Box::pin(async move { }) will be enough. For some traits, just like with FromData for rocket_contrib::json::Json, you might need to copy some data out of the request before creating the future.
Some dependencies such as ring and handlebars have been updated, which may consequently break your application. You should get obvious build failures if this affects you.
You can now declare routes as async, and await futures inside of them.

@jebrosen

@jebrosen have you taken a look at async-trait? I know I've mentioned it before, and it reduces some of the overhead associated with async trait fns, like lifetimes and boxing. As a bonus, all we'd have to do after GATs are (some day) implemented is remove the attribute.

I did try async-trait today and ran into some issues that might be able to be worked around. I would prefer to use async-trait on either all affected traits or none of them, and I'm hopeful that it will be usable.

As a bonus, all we'd have to do after GATs are (some day) implemented is remove the attribute.

Unfortunately that would be a breaking change to any public traits, as far as I know there's no way to be forward compatible with GATs in traits. You can be forward compatible with using 'static bound async blocks in traits (once type_alias_impl_trait is stabilized) by using an associated type and setting that to be the pinned-box in implementations, e.g. skimming the changes it looks like FromDataSimple could be defined like this (and maybe FromData since it uses an external lifetime), but Fairing can't as it borrows from its arguments.

Yeah, it would definitely be a breaking change, as APIs would be different. I was referring to the changes in the code itself, which would amount to deleting a few lines.

Update on the attempt to use async-trait everywhere: it's currently not possible. There is a bug in the compiler that prevents async default fn or default async fn, which is necessary where specialization is currently used.

Test, figure out, and document cross-runtime compatibility (or lack thereof). That is, a definitive statement in the form of "Rocket uses tokio 0.2, so your database futures have to as well" if that's the case. Alternatively, maybe we could find a proof of concept of running two runtimes - tokio 0.2 (for rocket) alongside tokio 0.1 (for tokio-postgres) with some kind of message passing between them. I leave this intentionally vague because it's an annoying solution to an annoying problem and someone other than I will probably think of a better idea.

tokio-postgres has a std-futures branch with an in-progress port to tokio 0.2 (and std::future). We could probably just specify that that version is needed for compatibility.

Hi all,

I've switched over fully to the latest Rocket with full async for a couple of my latest projects, but was running into the fact that compression support doesn't exist yet. This was inflating responses to my users and bringing up the cost of my bill for network egress, so I hacked up a placeholder solution that adds back gzip support: Ameobea@eab53cf

The API is exactly the same as before; you just add features = ["compression"] for rocket-contrib and add the Compression::fairing(). The implementation is not optimal in any way; the compression itself isn't async but it really shouldn't be a big deal for almost all workloads and the huge benefit of having gzip compression outweighs any increase in response time by a huge margin.

I figured this may be useful to other people who wanted to gain the benefits of compression and the latest async Rocket without having to re-write any of their code. Once someone gets around to doing the job properly, it should just be a change to Cargo.toml to fix it.

EDIT 2021-07-17:

I've created a library that performs response compression using the async-compression library under the hood:

https://github.com/Ameobea/rocket_async_compression

Note that I'm getting some issues when using a managed state

async fn dependencies(state: State<App>, repo: String) -> String { }

error[E0726]: implicit elided lifetime not allowed here
  --> src/main.rs:43:30
   |
43 | async fn dependencies(state: State<App>, repo: String) -> String {
   |                              ^^^^^^^^^^ help: indicate the anonymous lifetime: `State<'_, App>`

error[E0621]: explicit lifetime required in the type of `__req`
  --> src/main.rs:43:23
   |
42 | #[get("/dependencies/<repo>")]
   | ------------------------------ help: add explicit lifetime `'static` to the type of `__req`: `&'_b rocket::Request<'static>`
43 | async fn dependencies(state: State<App>, repo: String) -> String {
   |                       ^^^^^ lifetime `'static` required

@mimoo

@mimoo from what I've seen, when using async routes, you have to specify the anonymous lifetime like so

async fn dependencies(state: State<App, '_>, repo: String) -> String { String::new() }

@mimoo

@mimoo from what I've seen, when using async routes, you have to specify the anonymous lifetime like so

Note that this is not a Rocket restriction, but rather, a Rust restriction. Also, it's State<'_, T>.

I'm guessing that Responder is still a sync trait in the latest async code. I was hoping to make a request with reqwest in respond_to().

I'm guessing that Responder is still a sync trait in the latest async code. I was hoping to make a request with reqwest in respond_to().

This was a conscious decision, not something that's left to do. In fact, Responder was first made async and then reverted to be a regular trait. The rationale is that, with the exception of the body itself being an async stream, anything async can happen in the handler before the Responder is returned.

Specific to your comment, sending HTTP requests in the responder sounds rather odd. A responder generates a response, while this sounds like more processing, which should occur in the handler.

@SergioBenitez

@SergioBenitez You're correct, that it's not the place for it. I was using okapi for OpenAPI, and part of the API is for an upstream server, I have stub handlers with dummy data for now, but the idea is that I want to send all headers and all of the body to the upstream server, then respond with exactly what it said. What the upstream server said will likely be what the return type of the handler was, but I didn't want to deserialize just to reserialize, so I made a "wrapper" responder that implemented the traits for Rocket and okapi, but it generated the response completely. However, when I added in reqwest, I got a panic, and I also found that I didn't have access to the body, so I switched to using a FromData struct that I'll pass in to that responder, so all of the respond_to() logic is sync. I know that it's not ideal, but I need these routes to be fast.

Is there a better way to do forwarding?

It sounds like you should be implementing a custom Handler.

@SergioBenitez

@SergioBenitez Thanks! Great framework, by the way!

@Type1J

@Type1J I had this exact issue recently, with some messing around you can get a compatible async stream from the reqwest response body to store in a struct implementing Responder: https://github.com/DuBistKomisch/jakebarnes/blob/ba7171c013bd101e049d8b2af5fc7120b6ea6c1c/src/steam.rs#L93-L99

I've implemented async response compression using the async-compression library and created an external library for it:

https://github.com/Ameobea/rocket_async_compression

It currently only works with the git version of Rocket. The internal implementation is extremely simple and I think it's pretty much good to go for including in Rocket itself. Let me know if anyone is interested in making that work.

@Ameobea

@Ameobea might be worth making the compression level customizable via the Rocket.toml file once integrated into Rocket.

With the new rocket_db_pools, I'm happy to close this issue. The remaining issues tracked in the parent post are not specific to async I/O (typed headers, more docs, and connection APIs) and have independent issues or have no viable solutions outside of rustc (lints for async safety).

Navigation Menu

Repository navigation

Tracking Issue for Async I/O migration #1065

Status

Pending

Done

Design

35 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Tracking Issue for Async I/O migration #1065

Description

Status

Pending

Done

Design

Activity

jhpratt commented on Aug 2, 2019

jebrosen commented on Aug 2, 2019

jhpratt commented on Aug 2, 2019

Nemo157 commented on Aug 2, 2019

theduke commented on Aug 2, 2019

jhpratt commented on Aug 2, 2019

alexsherman commented on Aug 5, 2019

jebrosen commented on Aug 6, 2019

jebrosen commented on Aug 18, 2019

jhpratt commented on Aug 18, 2019

jebrosen commented on Aug 18, 2019

Nemo157 commented on Aug 19, 2019

jhpratt commented on Aug 19, 2019

jhpratt commented on Aug 19, 2019

nicoburns commented on Aug 21, 2019

35 remaining items

Ameobea commented on Sep 14, 2020

mimoo commented on Feb 10, 2021

tforgione commented on Feb 10, 2021

SergioBenitez commented on Mar 6, 2021

Type1J commented on Mar 31, 2021

SergioBenitez commented on Mar 31, 2021

Type1J commented on Mar 31, 2021

Type1J commented on Mar 31, 2021

SergioBenitez commented on Mar 31, 2021

Type1J commented on Mar 31, 2021

DuBistKomisch commented on Mar 31, 2021

Ameobea commented on Jul 18, 2021

mainrs commented on Jul 18, 2021

SergioBenitez commented on Jul 18, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions