Difference between scalability and speedup?

Question

I'm reading the textbook for my parallel system course and I found these two popular notions:

Can someone tell me the difference? For me, Tseq is semantically (and pratically) equal to Tpar(1)

Update after answers:

I think that I understood the difference, but I have still a question: I'm optimizing this code. In order to make my parallel version well-balanced, an approximate result is used. The difference is very small and be considered negligible, but the workload is greatly balanced and scalability greatly improves from that.

In addition, I did some optimizations, like using raw-pointers instead of data structures built upon them (eg cv::Mat ) used in the original code.

So my question is: to measure speedup should I use the original code (the one that I linked) or would be like cheating?

Answer 1

A parallel execution with a parallelism degree of 1 can still incur some overhead compared to sequential execution. Imagine:

gcc code.c
./a.out

vs

gcc code.c -fopenmp
OMP_NUM_THREADS=1 ./a.out

The latter can be slower due to code that needs to be executed on each parallel section, checking for the number of threads that should be spawned.

You might even require distinctly different implementations for a parallel code compared to a sequential one.

Regarding your Edit:

The most important thing is to be clear and honest about what you are comparing! I would think the best way is to separate the speedup of the general optimizations from the parallelization efforts. If you only have unoptimized serial and optimized parallel version, use the optimized parallel version eg compiled without OpenMP as baseline for parallel speedup.

Answer 2

Speedup tells you how much faster your parallel algorithm can be than its sequential counterpart. Scalability tells you how the performance of your parallel algorithm behaves as you add hardware.

Note the differences in the two formulae against what you are comparing the parallel algorithm: in the scalability formula, you are comparing the performance of the parallel algorithm with N CPUs against itself with 1 CPU. In the speedup formula, you are comparing the performance of the parallel algorithm with N CPUs against a different algorithm (namely, the sequential version of the algorithm).