Alterna3ves to block and cyclic other distribudons

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview:   Algorithmically-­‐aware distribuDons CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 36 Distribu3on #3: Block-­‐Cyclic Distribu3on •  As the name suggests, a hybrid of Block and Cyclic –  deals blocks of items out cyclically •  parameterized by block size, b •  ideally, b should match or exceed cache line size •  opDmal choice of b osen depends on algorithm, working set size, … –  tradeoffs: + gives tasks chunks of work (good spaDal locality; less false sharing) + like cyclic, results in probabilisDcally-­‐oriented load balancing -  results in slightly more complicated loop nests CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 37 Dynamic Distribu3ons Concept: –  don’t deal work out according to a fixed, a priori schedule –  instead, deal work out to tasks (or have them grab it) as they become idle Goal: –  no task gets stuck with more work than it can handle Challenge: –  what granularity (granulariDes?) to deal out work? •  if too large: tasks may get unlucky and stuck with too much work •  if too small: too much effort coordinaDng, not enough compuDng CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 38 Algorithmically-­‐Aware Distribu3ons Concept: –  For some algorithms, there may be a way to scan the input data in order to compute a good distribuDon •  e.g., dynamically sample the input data set to try and predict trends? •  e.g., examine the placement of zeroes and non-­‐zeroes in a sparse matrix? •  e.g., compute a dependence graph for the computaDon and distribute it using a graph parDDoning algorithm Goal: –  use algorithmic-­‐centric knowledge to improve load balance Challenge: –  Cost:Benefit raDo needs to be taken into account •  since any overhead in compuDng a distribuDon is new work that wouldn’t have been required in a serial version CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 39 Mul3dimensional Distribu3ons •  So far, we’ve looked solely at 1D distribuDons •  DistribuDons can also be mulDdimensional –  one opDon is to apply a 1D distribuDon per dimension CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 40 2D Block x Block (distributed to 2x2 tasks) CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 41 2D Block x Block (distributed to 1x4 tasks) CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 42 2D Block x Block (distributed to 4x1 tasks) CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 43 2D Block-­‐Cyclic x Block-­‐Cyclic (distributed to 2x2 tasks) CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 44 …and so on and so forth •  •  •  •  •  •  •  Cyclic x Cyclic Block x Cyclic Cyclic x Block Block-­‐Cyclic x Block-­‐Cyclic with different block sizes Block-­‐Cyclic x Block Block x Block-­‐Cyclic etc. CSEP 524: Parallel ComputaDon Winter 2013: Chamberlain 45 Q: In a Shared-­‐Memory se]ng, which would you use from the perspec3ve of memory? CSEP 524: Parallel...
View Full Document

Ask a homework question - tutors are online