Lect02 - Parallelprogramming paradigmsandtheir performances HPCworld George Bosilca University of Tennessee Knoxville Innovative Computer

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Parallel programming paradigms and their performances A high level exploration of the HPC world George Bosilca University of Tennessee, Knoxville Innovative Computer Laboratory Overview • Definition of parallel application • Architectures taxonomy • Laws managing the parallel domain • Models in parallel computation • Examples Formal definition Bernstein { I1 O2 = and I2 O1 = and O1 O2 = } General case: P1… Pn are parallel if and only if each for each pair Pi, Pj we have Pi || Pj. 3 limit to the parallel applications: 1. Data dependencies 2. Flow dependencies 3. Resources dependencies P1 I1 O1 P2 I2 O2
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Data dependencies I1: A = B + C I2: E = D + A I3: A = F + G I1 I2 I3 Dataflow dependency Anti-dependency Output dependency How to avoid them? Which can be avoided ? Flow dependencies I1: A = B + C I2: if( A ) { I3: D = E + F } I4: G = D + H I1 I2 I3 I4 Flow dependency Dataflow dependency How to avoid ? Resources dependencies I1: A = B + C I2: G = D + H I1 I2 + How to avoid ?
Background image of page 2
3 Flynn Taxonomy 1 Instruction flow > 1 Instruction flow 1 data stream SISD Von Neumann MISD pipeline > 1 data stream SIMD MIMD •Computers classified by instruction delivery mechanism and data stream •4 characters code: 2 for instruction stream and 2 for data stream Flynn Taxonomy: Analogy • SISD: lost people in the desert • SIMD: rowing • MISD: pipeline in the car construction chain • MIMD: airport facility, several desks working at their own pace, synchronizing via a central database. • First law of parallel applications (1967) • Limit the speedup for all parallel applications Amdahl Law s p ( N a a speedup N p s p s speedup + = + + = 1 1 N = number of processors
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 Amdahl Law Speedup is bound by 1/a. Amdahl Law • Bad news for parallel applications • 2 interesting facts: – We should limit the sequential part – A parallel computer should be a fast sequential computer to be able to resolve the sequential part quickly • What about increasing the size of the initial problem ? Gustafson Law • Less constraints than the Amdahl law. • In a parallel program the quantity of data to be processed increase, so the sequential part decrease. n P s t / + = n a P * = a s n a s speedup + + = * n speedup a
Background image of page 4
5 Gustafson Law • The limit of Amdahl Law can be transgressed if the quantity of data to be processed increase. s n n speedup ) 1 ( + Rule stating that if the size of most problems is scaled up sufficiently, then any required efficiency can be achieved on any number of processors. Speedup • Superlinear speedup ? Sub-linear Superlinear Sometimes superlinear speedups can be observed! •Memory/cache effects •More processors typically also provide more memory/cache. •Total computation time decreases due to more page/cache hits. •Search anomalies •Parallel search algorithms.
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 04/01/2010 for the course COMPUTER S cs202 taught by Professor Jiuhui during the Spring '08 term at 東京国際大学.

Page1 / 23

Lect02 - Parallelprogramming paradigmsandtheir performances HPCworld George Bosilca University of Tennessee Knoxville Innovative Computer

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online