What does the so-called "Fine-Grained Parallelism" exactly mean in the context of this book?

Question

I'm reading Operating System: Internals and Design Principles . In Section 10.1 Multiprocessor Scheduling, the author introduces a table of synchronization granularity as follows:

• Fine: Parallelism inherent in a single instruction stream
• Medium: Parallelism processing or multitasking within a single application
• Coarse: Multiprocessing of concurrent processes in multiprogramming environment
• Very Coarse: Distributed processing across network nodes to form a single computing environment
• Independent: Multiple unrelated processes

and he explains the fine-grained parallelism like this:

Fine-grained parallelism represents a much more complex use of parallelism than is found in the use of threads. Although much work has been done on highly parallel applications, this is so far a specialized and fragmented area, with many different approaches.

I could understand the medium granularity is about thread and other granularity except the so-called fine-grained parallelism. And I didn't get much information by searching it on the Internet. So could you please clarify it for me? Some examples would really help.

Thanks in advance!

Answer 1

It's a reference to the so called instruction level parallelism that is the instructions that can be executed in parallel from the same thread.

For example if you have two instructions such as

a = b*2
c = d+5 

These instructions can be executed in parallel by the processor without any problem as they do not depend on each other in any way.

On the other hand if you have instructions like

a = b*2
c = a+5

In this case second instruction depends on the first and cannot be processed until result of the first is available. This is the limiting factor on the instruction level parallelism as most instructions are dependent on other instructions.

Fine-grained parallelism represents a much more complex use of parallelism than is found in the use of threads.

This is because in order to identify independent instructions in the same thread you must have the ability to look down the instruction pipeline which is not an easy task and requires special hardware and compiler optimizations.

Thread level parallelism on the other hand is easy as you know that two instructions from two different threads can't possibly depend on each other.