17.1-reductions - Announcements Should be working on your...

Info iconThis preview shows pages 1–10. Sign up to view the full content.

View Full Document Right Arrow Icon
Announcements Should be working on your projects Mid-project status reports will be requested during week of Nov 1 Schedule 26 Oct 10 vetter@computer.org 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
CS6230–HPC Tools and Applications Reductions and Parallel Prefix Sum Jeffrey S. Vetter Computational Science and Engineering College of Computing Georgia Institute of Technology http://ft.ornl.gov/~vetter vetter@computer.org
Background image of page 2
EXAMPLE: REDUCTIONS Borrowed from David Kirk/NVIDIA, Wen mei W. Hwu, and John Stratton, 2007 2009 ECE 498AL, University of Illinois, Urbana Champaign 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
3 Parallel Reductions Simple array reductions reduce all of the data in an array to a single value that contains some information from the entire array. Sum, maximum element, minimum element, etc. Used in lots of applications, although not always in parallel form Matrix Multiplication is essentially performing a sum reduction over the element product of two vectors for each output element: but the sum is computed by a single thread Assumes that the operator used in the reduction is associative Technically not true for things like addition on floating point numbers, but it’s common to pretend that it is
Background image of page 4
4 Parallel Prefix Sum (Scan) Definition: The all prefix sums operation takes a binary associative operator with identity I , and an array of n elements [ a 0 , a 1 , …, a n 1 ] and returns the ordered set [ I , a 0 , ( a 0 a 1 ), …, ( a 0 a 1 a n 2 )] . Example: if is addition, then scan on the set [3 1 7 0 4 1 6 3] returns the set [0 3 4 11 11 15 16 22] (From Blelloch, 1990, “Prefix Sums and Their Applications) Each element is the array reduction of all previous elements
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
5 Relevance of Scan Scan is a simple and useful parallel building block Convert recurrences from sequential : for(j=1;j<n;j++) out[j] = out[j-1] + f(j); into parallel: forall(j) { temp[j] = f(j) }; scan(out, temp); Useful for many parallel algorithms: radix sort quicksort String comparison Lexical analysis Stream compaction Polynomial evaluation Solving recurrences Tree operations Histograms Etc.
Background image of page 6
6 Example: Application of Scan Tid 1 2 3 4 5 6 7 8 Scan 0 3 4 11 15 16 22 C n t31704163 0 1 2 3 4 5 6 7 8 9 1 01 21 31 41 51 61 71 81 92 02 12 22 32 4 Computing indexes into a global array where each thread needs space for a dynamic number of elements Each thread computes the number of elements it will produce The scan of the thread element counts will determine the beginning index for each thread.
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
7 Scan on the CPU Just add each element to the sum of the elements before it Trivial, but sequential Exactly n adds: absolute minimum bound void scan( float* scanned, float* input, int length) { scanned[0] = 0; for(int i = 1; i < length; ++i) { scanned[i] = input[i-1] + scanned[i-1]; } }
Background image of page 8
8 A First Attempt Parallel Scan Algorithm 1. Read from input into a temporary array we can work on in place.
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 10
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 11/04/2010 for the course CSE 6530 taught by Professor Jeffreyvetter during the Fall '10 term at Georgia Institute of Technology.

Page1 / 42

17.1-reductions - Announcements Should be working on your...

This preview shows document pages 1 - 10. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online