speedupBasic

# 10 7 438 010 e 1357 151 15 0088 0

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ad • • • Tuesday, February 14, 12 Example Benchmarking a program on 1, 2, ..., 8 processors produces the following speedups: p ψ 2 1.82 3 2.50 4 3.08 5 3.57 6 4.00 7 4.38 8 4.71 Why is the speedup only 4.71 on 8 processors? p ψ e 2 1.82 0.10 3 2.50 0.10 4 3.08 0.10 5 3.57 0.10 6 4.00 0.10 7 4.38 0.10 e = (1/3.57 - 1/5)/(1-1/5) = (0.08)/.8 = 0. 1 Tuesday, February 14, 12 8 4.71 0.10 Example 2 Benchmarking a program on 1, 2, ..., 8 processors produces the following speedups: p ψ 2 1.87 3 2.61 4 3.23 5 3.73 6 4.14 7 4.46 8 4.71 Why is the speedup only 4.71 on 8 processors? p ψ e 2 3 4 5 6 7 8 1.87 2.61 3.23 3.73 4.14 4.46 4.71 0.07 0.075 0.080 0.085 0.090 0.095 0.1 e is increasing: speedup problem is increasing serial overhead (process startup, communication, algorithmic issues, the architecture of the parallel system, etc. Tuesday, February 14, 12 Which has the efﬁciency problem? Chart 5 5.00 3.75 2.50 1.25 0 2 3 speedup 1 Tuesday, February 14, 12 4 5 speedup 2 6 7 8 Very easy to see using e Chart 6 0.100 0.075 0.050 0.025 0 2 3 4 e1 Tuesday, February 14, 12 5 6 7 8 e2 Isoefﬁciency Metric • Parallel system: parallel program executing on a parallel computer • Scalability of a parallel system: measure of its ability to increase performance as number of processors increases • A scalable system maintains efﬁciency as processors are added • Isoefﬁciency: way to measure scalability Tuesday, February 14, 12 Isoefﬁciency Derivation Steps • Begin with speedup formula • Compute total amount of overhead • Assume efﬁciency remains constant • Determine relation between sequential execution time and overhead Tuesday, February 14, 12 Deriving Isoefﬁciency Relation Determine overhead Substitute overhead into speedup equation Substitute T(n,1) = σ(n) + ϕ(n). Assume efﬁciency is constant. Isoefﬁciency Relation Tuesday, February 14, 12 Scalability Function • Suppose isoefﬁciency relation is n ≥ f(p) • Let M(n) denote memory required for problem of size n • M(f(p))/p shows how memory usage per processor must increase to maintain same efﬁciency • We call M(f(p))/p the scalability function Tuesday, February 14, 12 Meaning of Scalability Function • To maintain efﬁciency when increasing p, we must increase n • Maximum problem size limited by available memory, which is linear in p • Scalability function shows how memory usage per processor must grow to maintain efﬁciency Tuesday, February 14, 12 Memory needed per processor Interpreting Scalability Function Cplogp Tuesday, February 14, 12 Cannot maintain efﬁciency Cp Memory Size Can maintain efﬁciency Clogp C Number of processors Example 1: Reduction • Sequential algorithm complexity T(n,1) = Θ(n) • Parallel algorithm • Computational complexity = Θ(n/p) • Communication complexity = Θ(log p) • Parallel overhead T0(n,p) = Θ(p log p) Tuesday, February 14, 12 Reduction (continued) • Isoefﬁciency relation: n ≥ C p log p • We ask: To maintain same level of efﬁciency, how must n increase when p increases? • M(n) = n • The system has good scalability Tuesday, February 14, 12 Example 2: Floyd’s Algorithm • Sequential time complexity: Θ(n3) • Parallel computation time: Θ(n3/p) • Parallel communication time: Θ(n2log p) • Parallel overhead: T0(n,p) = Θ(pn2log p) Tuesday, February 14, 12 Floyd’s Algorithm (continued) • Isoefﬁciency relation n3 ≥ C(p n2 log p) • Tuesday, February 14, 12 M(n) = n2 n ≥ C p log p Example 3: Finite Difference • Sequential time complexity per iteration: Θ(n2) • Parallel communication complexity per iteration: Θ(n/√p) • Parallel overhead: Θ(n √p) Tuesday, February 14, 12 Finite Difference (continued) • Isoefﬁciency relation n2 ≥ Cn√p n ≥ C√ p • M(n) = n2 • This algorithm is perfectly scalable Tuesday, February 14, 12 Summary (1/3) • Performance terms • Speedup • Efﬁciency • Model of speedup • Serial component • Parallel component Tuesday, February 14, 12 Summary (2/3) • What prevents linear speedup? • Serial operations • Communication operations • Process start-up • Imbalanced workloads • Architectural limitations Tuesday, February 14, 12 Summary (3/3) • Analyzing parallel performance • Amdahl’s Law • Gustafson-Barsis’ Law • Karp-Flatt metric • Isoefﬁciency metric Tuesday, February 14, 12...
View Full Document

## This note was uploaded on 02/19/2012 for the course ECE 563 taught by Professor Staff during the Spring '08 term at Purdue University-West Lafayette.

Ask a homework question - tutors are online