Return an async function from a function in Rust

Question

Part 1: What should be the signature of a function returning an async function?

pub async fn some_async_func(arg: &str) {}

// What should be sig here?
pub fn higher_order_func(action: &str) -> ???
{
    some_async_func
}

Part 2: What should be the sig, if based on the action parameter, higher_order_func had to return either async_func1 or async_func2.

I am also interested in learning the performance tradeoffs if there are multiple solutions. Please note that I'd like to return the function itself as an fn pointer or an Fn* trait, and not the result of invoking it.

Answer 1

Returning a function

Returning the actual function pointer requires heap allocation and a wrapper:

use std::future::Future;
use std::pin::Pin;

pub async fn some_async_func(arg: &str) {}

pub fn some_async_func_wrapper<'a>(arg: &'a str)
    -> Pin<Box<dyn Future<Output=()> + 'a>>
{
    Box::pin(some_async_func(arg))
}

pub fn higher_order_func<'a>(action: &str)
    -> fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
    some_async_func_wrapper
}

Why boxing? higher_order_func needs to have a concrete return type, which is a function pointer. The pointed function needs to also have a concrete return type, which is impossible for async function since it returns opaque type. In theory, it could be possible to write return type as fn(&'a str) -> impl Future<Output=()> + 'a , but this would require much more guesswork from the compiler and currently is not supported.

If you are OK with Fn instead of fn , you can get rid of the wrapper:

pub async fn some_async_func(arg: &str) {}

pub fn higher_order_func<'a>(action: &str)
    -> impl Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
    |arg: &'a str| {
        Box::pin(some_async_func(arg))
    }
}

To return a different function based on action value, you will need to box the closure itself, which is one more heap allocation:

pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}

pub fn higher_order_func<'a>(action: &str)
    -> Box<dyn Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>>
{
    if action.starts_with("one") {
        Box::new(|arg: &'a str| {
            Box::pin(some_async_func_one(arg))
        })
    } else {
        Box::new(|arg: &'a str| {
            Box::pin(some_async_func_two(arg))
        })
    }
}

Alternative: returning a future

To simplify things, consider returning a future itself instead of a function pointer. This is virtually the same, but much nicer and does not require heap allocation:

pub async fn some_async_func(arg: &str) {}

pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
    -> impl Future<Output=()> + 'a
{
    some_async_func(arg)
}

It might look like, when higher_order_func_future is called, some_async_func is getting executed - but this is not the case. Because of the way async functions work, when you call some_async_func , no user code is getting executed . The function call returns a Future : the actual function body will be executed only when someone awaits the returned future.

You can use the new function almost the same way as the previous function:

// With higher order function returning function pointer
async fn my_function() {
    let action = "one";
    let arg = "hello";
    higher_order_func(action)(arg).await;
}

// With higher order function returning future
async fn my_function() {
    let action = "one";
    let arg = "hello";
    higher_order_func_future(action, arg).await;
}

Notice, once more, that in both cases the actual some_async_func body is executed only when the future is awaited.

If you wanted to be able to call different async functions based on action value, you need boxing again:

pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}

pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
    -> Pin<Box<dyn Future<Output=()> + 'a>>
{
    if action.starts_with("one") {
        Box::pin(some_async_func_one(arg))
    } else {
        Box::pin(some_async_func_two(arg))
    }
}

Still, this is just one heap allocation, so I strongly advise returning a future. The only scenario that I can imagine where the previous solution is better is when you want to save the boxed closure somewhere and use it many times. In this case, excessive allocation happens only once, and you spare some CPU time by dispatching the call based on action only once - when you make the closure.

Answer 2

Ideally, what you'd want is a nested impl trait: -> impl Fn(&str) -> impl Future<Output = ()> . But nested impl trait is not supported. However, you can emulate that using a trait.

The idea is to define a trait that will abstract over the notion of "function returning a future". If our function would take a u32 , for example, it could look like:

trait AsyncFn: Fn(u32) -> Self::Future {
    type Future: Future<Output = ()>;
}
impl<F, Fut> AsyncFn for F
where
    F: Fn(u32) -> Fut,
    Fut: Future<Output = ()>,
{
    type Future = Fut;
}

And then we would take impl AsyncFn . Trying to apply that naively to &str doesn't work :

error[E0308]: mismatched types
  --> src/lib.rs:16:27
   |
16 | fn higher_order_func() -> impl AsyncFn {
   |                           ^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
              found associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`

The error may look very strange, but it arises from the fact that async fn returns a future bound by the lifetime of all of its argument, ie for a signature async fn foo<'a>(arg: &'a str) , the future is not impl Future<Output = ()> but impl Future<Output = ()> + 'a . There is a way to capture this relationship in our trait, we just need to make it generic over the argument and use HRTB :

trait AsyncFn<Arg>: Fn(Arg) -> Self::Future {
    type Future: Future<Output = ()>;
}
impl<Arg, F, Fut> AsyncFn<Arg> for F
where
    F: Fn(Arg) -> Fut,
    Fut: Future<Output = ()>,
{
    type Future = Fut;
}

And then we specify the type as:

fn higher_order_func() -> impl for<'a> AsyncFn<&'a str> {
    some_async_func
}

Answer 3

In addition to the great accepted answer, depending on your use case it's also possible to "fake" the higher order function and avoid any heap allocations by using a simple macro to expand the wrapper code in-place instead:

pub async fn some_async_func(arg: &str) {}

macro_rules! higher_order_func {
    ($action: expr) => {
        some_async_func
    };
}

fn main() {
    let future = higher_order_func!("action")("arg");
}