CS6230 – HPC Tools and ApplicationsReductions and Parallel Prefix SumJeffrey S. VetterComputational Science and EngineeringCollege of ComputingGeorgia Institute of Technologyhttp://ft.ornl.gov/~vetter[email protected]
EXAMPLE: REDUCTIONSBorrowed from David Kirk/NVIDIA, Wen‐mei W. Hwu, and John Stratton, 2007‐2009ECE 498AL, University of Illinois, Urbana‐Champaign2
has intentionally blurred sections.
Sign up to view the full version.
3Parallel Reductions•Simple array reductions reduce all of the data in an array to asingle value that contains some information from the entire array.–Sum, maximum element, minimum element, etc.•Used in lots of applications, although not always in parallel form–Matrix Multiplication is essentially performing a sum reduction over theelement‐product of two vectors for each output element: but the sum iscomputed by a single thread•Assumes that the operator used in the reduction is associative–Technically not true for things like addition on floating‐point numbers, butit’s common to pretend that it is