为测试目的对 function 呼叫计时的最简单方法是什么？

Question

So I'm still kinda green in Rust, but coming from Python I find this scenario very confusing in general.

I like Python because it's very easy if you want to time a block of code or just a function call:

print(timeit('a = "hee hee la le dah"; my_awesome_fn()', number = 1_000, globals=globals()))

Then just call python script.py or better yet just use the green "run" button in the IDE and you can call the script. But I'm having trouble finding functional equivalent in Rust.

I know there is concept in Rust ecosystem called benchmarking and some libs like criterion exist for this purpose. The problem is that I know nothing about advanced math and statistics (can treat me like a clueless moron essentially) and I doubt I can benefit a lot from a framework or harness such as this.

So I am simply just curious how can I use tests in cargo to test a block of code in Rust or better yet even a function call.

For example assume I have similar function in rust that I want to call multiple times and then check how does performance change etc:

pub fn my_awesome_fn() {
    trace!("getting ready to do something cool...");
    std::thread::sleep(std::time::Duration::from_millis(500));
    info!("finished!");
}

how can I simply just time this func my_awesome_fn in rust? I guess i'm looking for an equivalent like timeit in python or something similar. Ideally it should be striaghtforward to use and assume i don't know anything about what I'm doing. I'm curious if there's an existing library or framework I can leverage for this purposes.

Answer 1

Disclaimer: I've never used timeit

A very quick answer solution is to write a function like:

fn timeit<F: Fn() -> T, T>(f: F) -> T {
  let start = SystemTime::now();
  let result = f();
  let end = SystemTime::now();
  let duration = end.duration_since(start).unwrap();
  println!("it took {} seconds", duration.as_secs());
  result
}

which you can use to "wrap" another function call:

fn main() {
  let x = timeit(|| my_expensive_function());
}

However, if you're trying to understand the time a function takes for the purpose of performance optimizations , this approach is likely too crude.

The problem is that I know nothing about advanced math and statistics

That's arguably one of the main advantages of criterion , it "abstracts the maths away", in a sense.

It uses statistical approaches to give you a better insight into whether differences between benchmarking runs are a product of "randomness", or whether there is a meaningful difference between the code on each run.

To the end user, it essentially gives you a report saying either "a significant change was observed" or "no significant change was observed". It does far more than that, but to fully grasp its capabilities, it might be worth reading up on "hypothesis testing".

If you're OK using nightly Rust, you can also use #[bench] tests:

#![feature(test)]
extern crate test;

#[bench]
fn bench_my_func(b: &mut Bencher) {
  b.iter(|| my_func(black_box(100));
}

which you can run with cargo bench . These are a bit easier to set up than criterion , but do less of the interesting stats (ie you'll have to do it yourself), but they're a very "quick and dirty" way to get a feel for the runtime of your code.

A word of warning, benchmarking code is hard. You may be surprised at what is actually going on under the hood, and you may find yourself benchmarking the wrong thing.

Common "gotchas" are:

• rustc can generally identify "useless" code, and simply skip calculating it. The black_box function can be used to hide the meaning of some data from the optimizer, though it is not without its own overhead
• in a similar vein, LLVM does some slightly spooky optimizations relating to polynomials for example . You might find that your function call is being optimized away into a constant/simple arithmetic. In some cases, this is great. You've written your function in such a way that LLVM can reduce it to something trivial, In other cases, you're now just benchmarking the multiplication instruction on your CPU. which is unlikely to be what you want. Use your best judgement
• benchmarking the wrong thing - some things are significantly more expensive than others, in ways that might seem odd to someone with a python background. For example, cloning a String (even a very short one) might be 2-3 orders of magnitude slower than finding the first character. Consider the following:

fn str_len(s: String) -> usize {
  s.len()
}

#[bench]
fn bench_str_len(b: &mut Bencher) {
  let s = String::from("hello");  
  b.iter(|| str_len(s.clone()));
}

Because String::clone involves a heap allocation, but s.len() is just a field access, it will dominate the results. Instead, if str_len took a &str , it would become more representative (though this is a contrived case).

TLDR Be careful what your benchmark code is doing. The Rust Playground's "view assembly" tool (or godbolt.org) are your friends. You don't need to be an assembly expert, but it can help give you some idea what's going on under the hood