Figure 2.

Common parallel algorithms. Illustrative examples of three classes of common parallel algorithms implemented using simple operations and an eight-element, integer array. (A) Embarrassingly parallel algorithms are those that can be computed independently and thus simultaneously on the GPU. The given example, adding one to every element of an array, can be done concurrently to all array elements. In GO Fish, calculating new mutation frequencies and adding new mutations to population are both embarrassingly parallel operations. (B) Reduce is a fundamental parallel algorithm in which all the elements of an array are reduced to a single value using a binary operator, such as in the above summation over the example array (Harris 2007a). This algorithm takes advantage of the fact that in each time step half of the sums can be done independently while synchronized communication is necessary to combine the results of previous time steps. In total, log₂(8) = 3 time steps are required to reduce the example array. (C) Compact is a multi-step algorithm that allows one to filter arrays on the GPU (Billeter et al. 2009). In an embarrassingly parallel step, the algorithm presented above first creates a new Boolean array of those elements that passed the filter predicate (e.g., x > 1). Then a “scan” is performed on the Boolean array. Scan is similar in concept to reduce, wherein for each time step half of the binary operations are independent, but it is a more complex parallel algorithm that creates a running sum over the array rather than condensing the array to a single value [see Harris et al. (2007b)]. This running sum determines the new index of each element in the original array being filtered and the size of the new array. Those elements that passed the filter are then scattered to their new indices of the now smaller, filtered array. Compact is used in GO Fish to filter out fixed and lost mutations. GPU, Graphics Processing Unit.