Code benchmarking statistics -

Question

As I wrote in my previous topic: Benchmarking code - am I doing it right? I need to find a way to get benchmark statistics, like average, mean, standard deviation, etc. How can I do this using those methods I posted? Notice that I use a solution to benchmark code with time interval, not by calling a function many times. Any ideas?

I came up with just one, dont know if its correct (pseudocode):

buffsize = 1024;
buffer [buffsize];
totalcycles = 0

// arrays
walltimeresults = []
cputimeresults = []

// benchmarking
for i in (0, iterations):
   start = walltime();
   fun2measure(args, buffer);
   end = walltime();
   walltimeresults[i] = end - start;

   start = cputime();
   fun2measure(args, buffer);
   end = cputime();
   cputimeresults[i] = end - start;

   c1 = cyclecount();
   fun2measure(args, buffer);
   c2 = cyclecount();

   cyclesperbyte = c2-c1/(buffsize);
   totalcycles += cyclesperbyte;

for i in range (0, iterations) : sum += walltimeresults[i];
avg_wall_time = sum / iterations;

sum = 0;

for i in range (0, iterations) : sum += cputimeresults[i];
avg_cpu_time = sum / iterations;

avg_cycles = totalcycles / iterations;

Is it correct? How about mean, standard deviation, etc?

Answer 1

Your average looks OK.

Mean (ie average) is

mean = 1/N * sum( x[i] )

Standard deviation is square root of variance:

sigma = sqrt( 1/N * sum( (x[i]-mean)^2 )