14.2. More on Collective Communication

Unlike point-to-point communication, collective communication involves every process in a communication group. In Chapter 13, you saw two examples of collective communication functions, MPI_Bcast and MPI_Reduce (along with MPI_Allreduce). There are two advantages to collective communication functions. First, they allow you to express a complex operation using simpler semantics. Second, the implementation may be able to optimize the operations in ways not available with simple point-to-point operations.

Collective operations fall into three categories: a barrier synchronization function, global communication or data movement functions (e.g., MPI_Bcast), and global reduction or collective computation functions (e.g., MPI_Reduce). There is only one barrier synchronization function, MPI_Barrier. It serves to synchronize the processes. No data is exchanged. This function is described in Chapter 17. All of the other collective functions are nonsynchronous. That is, a collective function can return as soon as its role in the communication process is complete. Unlike point-to-point operations, nonsynchronous mode is the only mode supported by collective functions.