Lect02 - Parallelprogramming paradigmsandtheir performances...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Parallel programming paradigms and their performances A high level exploration of the HPC world George Bosilca University of Tennessee, Knoxville Innovative Computer Laboratory Overview • Definition of parallel application • Architectures taxonomy • Laws managing the parallel domain • Models in parallel computation • Examples Formal definition Bernstein { I1 O2 = and I2 O1 = and O1 O2 = } General case: P1… Pn are parallel if and only if each for each pair Pi, Pj we have Pi || Pj. 3 limit to the parallel applications: 1. Data dependencies 2. Flow dependencies 3. Resources dependencies P1 I1 O1 P2 I2 O2
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2 Data dependencies I1: A = B + C I2: E = D + A I3: A = F + G I1 I2 I3 Dataflow dependency Anti-dependency Output dependency How to avoid them? Which can be avoided ? Flow dependencies I1: A = B + C I2: if( A ) { I3: D = E + F } I4: G = D + H I1 I2 I3 I4 Flow dependency Dataflow dependency How to avoid ? Resources dependencies I1: A = B + C I2: G = D + H I1 I2 + How to avoid ?
Background image of page 2
3 Flynn Taxonomy 1 Instruction flow > 1 Instruction flow 1 data stream SISD Von Neumann MISD pipeline > 1 data stream SIMD MIMD •Computers classified by instruction delivery mechanism and data stream •4 characters code: 2 for instruction stream and 2 for data stream Flynn Taxonomy: Analogy • SISD: lost people in the desert • SIMD: rowing • MISD: pipeline in the car construction chain • MIMD: airport facility, several desks working at their own pace, synchronizing via a central database. • First law of parallel applications (1967) • Limit the speedup for all parallel applications Amdahl Law s p ( N a a speedup N p s p s speedup + = + + = 1 1 N = number of processors
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
4 Amdahl Law Speedup is bound by 1/a. Amdahl Law • Bad news for parallel applications • 2 interesting facts: – We should limit the sequential part – A parallel computer should be a fast sequential computer to be able to resolve the sequential part quickly • What about increasing the size of the initial problem ? Gustafson Law • Less constraints than the Amdahl law. • In a parallel program the quantity of data to be processed increase, so the sequential part decrease. n P s t / + = n a P * = a s n a s speedup + + = * n speedup a
Background image of page 4
5 Gustafson Law • The limit of Amdahl Law can be transgressed if the quantity of data to be processed increase. s n n speedup ) 1 ( + Rule stating that if the size of most problems is scaled up sufficiently, then any required efficiency can be achieved on any number of processors. Speedup • Superlinear speedup ? Sub-linear Superlinear Sometimes superlinear speedups can be observed! •Memory/cache effects •More processors typically also provide more memory/cache. •Total computation time decreases due to more page/cache hits. •Search anomalies •Parallel search algorithms.
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 23

Lect02 - Parallelprogramming paradigmsandtheir performances...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online