Unformatted text preview: be ideal • maximize use of HW without oversubscribing • a CPU centric view of computaCon – # tasks > # cores can be useful… • if algorithm inherently wants to use many disCnct tasks • as a task driven way of doing dynamic load balancing • to hide memory latencies by switching between tasks (?) – If thrashing memory, maybe # tasks < # cores is beqer? CSEP 524: Parallel ComputaCon Winter 2013: Chamberlain 13 Data Parallelism Task vs. Data Parallelism Data Parallelism: – expressed in terms of a data set that drives the parallelism • “data set” = typically an array, data structure, or set of indices e.g., forall i in 1..n do … // for all integers/indices 1 thru n do… e.g., forall a in A do … // for all elements in array A do… – generally more implicit + a simpler concept, easier for programmers to grasp + abstracts details of implementaCon to some lower level SW/HW  not as general as task parallelism • but an important common case to support and opCmize for • can typically be thought of as a special case of “may” parallelism (of course, in pracGce, data parallelism is implemented using tasks; and in pracGce most task parallel programs operate on some sort of data, so the line between the two can be a liJle fuzzy) CSEP 524: Parallel ComputaCon Winter 2013: Chamberlain 15 Example of Task vs. Data Parallelism • ReducCons – collecCve (“members contribute”) == a task parallel reducCon coforall tid in 0..#numTasks {
const myContribution = doSomeWork(…);
const total = sumReduceAll(myContribution);
} – global view (“holisGc”) == a data parallel reducCon const total = + reduce A; CSEP 524: Parallel ComputaCon // sum A’s elements Winter 2013: Chamberlain 16 Speaking of reduc3ons… where were we? 0.1 Task 0 0.2 Task 1 0.3 1.0 0.3 Use a Reduc6on Task 2 0.7 0.4 Task 3 0.5 Task 4 1.1 2.6 0.6 Task 5 0.7 0.8 Task 6 Task 7 1.5 Depth: O(log2#tasks) Conten3on: O(1) What if we used a tree with degree d? 3.6 totTime CSEP 524: Parallel ComputaCon Winter 2013: Chamberlain 18 Two Flavors of Reduc3ons • collecCve (“members contribute”) create tasks…
const myContribution = doSomeWork(…);
const total = sumReduceAll(myContribution);
join tasks… •...
View
Full
Document
This document was uploaded on 04/04/2014.
 Winter '09

Click to edit the document details