10 7 438 010 e 1357 151 15 0088 0

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ad • • • Tuesday, February 14, 12 Example Benchmarking a program on 1, 2, ..., 8 processors produces the following speedups: p ψ 2 1.82 3 2.50 4 3.08 5 3.57 6 4.00 7 4.38 8 4.71 Why is the speedup only 4.71 on 8 processors? p ψ e 2 1.82 0.10 3 2.50 0.10 4 3.08 0.10 5 3.57 0.10 6 4.00 0.10 7 4.38 0.10 e = (1/3.57 - 1/5)/(1-1/5) = (0.08)/.8 = 0. 1 Tuesday, February 14, 12 8 4.71 0.10 Example 2 Benchmarking a program on 1, 2, ..., 8 processors produces the following speedups: p ψ 2 1.87 3 2.61 4 3.23 5 3.73 6 4.14 7 4.46 8 4.71 Why is the speedup only 4.71 on 8 processors? p ψ e 2 3 4 5 6 7 8 1.87 2.61 3.23 3.73 4.14 4.46 4.71 0.07 0.075 0.080 0.085 0.090 0.095 0.1 e is increasing: speedup problem is increasing serial overhead (process startup, communication, algorithmic issues, the architecture of the parallel system, etc. Tuesday, February 14, 12 Which has the efficiency problem? Chart 5 5.00 3.75 2.50 1.25 0 2 3 speedup 1 Tuesday, February 14, 12 4 5 speedup 2 6 7 8 Very easy to see using e Chart 6 0.100 0.075 0.050 0.025 0 2 3 4 e1 Tuesday, February 14, 12 5 6 7 8 e2 Isoefficiency Metric • Parallel system: parallel program executing on a parallel computer • Scalability of a parallel system: measure of its ability to increase performance as number of processors increases • A scalable system maintains efficiency as processors are added • Isoefficiency: way to measure scalability Tuesday, February 14, 12 Isoefficiency Derivation Steps • Begin with speedup formula • Compute total amount of overhead • Assume efficiency remains constant • Determine relation between sequential execution time and overhead Tuesday, February 14, 12 Deriving Isoefficiency Relation Determine overhead Substitute overhead into speedup equation Substitute T(n,1) = σ(n) + ϕ(n). Assume efficiency is constant. Isoefficiency Relation Tuesday, February 14, 12 Scalability Function • Suppose isoefficiency relation is n ≥ f(p) • Let M(n) denote memory required for problem of size n • M(f(p))/p shows how memory usage per processor must increase to maintain same efficiency • We call M(f(p))/p the scalability function Tuesday, February 14, 12 Meaning of Scalability Function • To maintain efficiency when increasing p, we must increase n • Maximum problem size limited by available memory, which is linear in p • Scalability function shows how memory usage per processor must grow to maintain efficiency Tuesday, February 14, 12 Memory needed per processor Interpreting Scalability Function Cplogp Tuesday, February 14, 12 Cannot maintain efficiency Cp Memory Size Can maintain efficiency Clogp C Number of processors Example 1: Reduction • Sequential algorithm complexity T(n,1) = Θ(n) • Parallel algorithm • Computational complexity = Θ(n/p) • Communication complexity = Θ(log p) • Parallel overhead T0(n,p) = Θ(p log p) Tuesday, February 14, 12 Reduction (continued) • Isoefficiency relation: n ≥ C p log p • We ask: To maintain same level of efficiency, how must n increase when p increases? • M(n) = n • The system has good scalability Tuesday, February 14, 12 Example 2: Floyd’s Algorithm • Sequential time complexity: Θ(n3) • Parallel computation time: Θ(n3/p) • Parallel communication time: Θ(n2log p) • Parallel overhead: T0(n,p) = Θ(pn2log p) Tuesday, February 14, 12 Floyd’s Algorithm (continued) • Isoefficiency relation n3 ≥ C(p n2 log p) • Tuesday, February 14, 12 M(n) = n2 n ≥ C p log p Example 3: Finite Difference • Sequential time complexity per iteration: Θ(n2) • Parallel communication complexity per iteration: Θ(n/√p) • Parallel overhead: Θ(n √p) Tuesday, February 14, 12 Finite Difference (continued) • Isoefficiency relation n2 ≥ Cn√p n ≥ C√ p • M(n) = n2 • This algorithm is perfectly scalable Tuesday, February 14, 12 Summary (1/3) • Performance terms • Speedup • Efficiency • Model of speedup • Serial component • Parallel component Tuesday, February 14, 12 Summary (2/3) • What prevents linear speedup? • Serial operations • Communication operations • Process start-up • Imbalanced workloads • Architectural limitations Tuesday, February 14, 12 Summary (3/3) • Analyzing parallel performance • Amdahl’s Law • Gustafson-Barsis’ Law • Karp-Flatt metric • Isoefficiency metric Tuesday, February 14, 12...
View Full Document

This note was uploaded on 02/19/2012 for the course ECE 563 taught by Professor Staff during the Spring '08 term at Purdue University-West Lafayette.

Ask a homework question - tutors are online