Collapsing Loops to Improve Workload Balance

The parallel for directive applies only to the next loop. As always, it is best to apply parallelization at the outermost loop, because this reduces the number of synchronizations necessary. However, a low trip count for the outer loop will limit the maximum number of threads that can be used in parallel. In these cases, it might be appropriate to parallelize the inner loop, since this could have a higher iteration count. Without knowing the trip counts for the two loops, it is not possible to decide which strategy is more appropriate.

However, OpenMP provides a way of avoiding issues with the outermost loop having a low trip count, which is to collapse the inner and outer loops into a single loop. The clause to do this is collapse, which takes the number of loops to collapse as a parameter. Listing 7.61 shows an example of a code where the outer loop has a low trip count, and using the collapse clause enables scaling to higher numbers of threads.