01 task 0 02 task 1 03 10 03 use a reduc6on

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: be ideal •  maximize use of HW without oversubscribing •  a CPU- centric view of computaCon –  # tasks > # cores can be useful… •  if algorithm inherently wants to use many disCnct tasks •  as a task- driven way of doing dynamic load balancing •  to hide memory latencies by switching between tasks (?) –  If thrashing memory, maybe # tasks < # cores is beqer? CSEP 524: Parallel ComputaCon Winter 2013: Chamberlain 13 Data Parallelism Task vs. Data Parallelism Data Parallelism: –  expressed in terms of a data set that drives the parallelism •  “data set” = typically an array, data structure, or set of indices e.g., forall i in 1..n do … // for all integers/indices 1 thru n do… e.g., forall a in A do … // for all elements in array A do… –  generally more implicit + a simpler concept, easier for programmers to grasp + abstracts details of implementaCon to some lower level SW/HW -  not as general as task parallelism •  but an important common case to support and opCmize for •  can typically be thought of as a special case of “may” parallelism (of course, in pracGce, data parallelism is implemented using tasks; and in pracGce most task parallel programs operate on some sort of data, so the line between the two can be a liJle fuzzy) CSEP 524: Parallel ComputaCon Winter 2013: Chamberlain 15 Example of Task- vs. Data- Parallelism •  ReducCons –  collecCve (“members contribute”) == a task- parallel reducCon coforall tid in 0..#numTasks { const myContribution = doSomeWork(…); const total = sumReduceAll(myContribution); } –  global- view (“holisGc”) == a data- parallel reducCon const total = + reduce A; CSEP 524: Parallel ComputaCon // sum A’s elements Winter 2013: Chamberlain 16 Speaking of reduc3ons… where were we? 0.1 Task 0 0.2 Task 1 0.3 1.0 0.3 Use a Reduc6on Task 2 0.7 0.4 Task 3 0.5 Task 4 1.1 2.6 0.6 Task 5 0.7 0.8 Task 6 Task 7 1.5 Depth: O(log2#tasks) Conten3on: O(1) What if we used a tree with degree d? 3.6 totTime CSEP 524: Parallel ComputaCon Winter 2013: Chamberlain 18 Two Flavors of Reduc3ons •  collecCve (“members contribute”) create tasks… const myContribution = doSomeWork(…); const total = sumReduceAll(myContribution); join tasks… •...
View Full Document

This document was uploaded on 04/04/2014.

Ask a homework question - tutors are online