This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ad •
•
• Tuesday, February 14, 12 Example
Benchmarking a program on 1, 2, ..., 8 processors
produces the following speedups:
p
ψ 2
1.82 3
2.50 4
3.08 5
3.57 6
4.00 7
4.38 8
4.71 Why is the speedup only 4.71 on 8 processors?
p
ψ
e 2
1.82
0.10 3
2.50
0.10 4
3.08
0.10 5
3.57
0.10 6
4.00
0.10 7
4.38
0.10 e = (1/3.57  1/5)/(11/5) = (0.08)/.8 = 0. 1
Tuesday, February 14, 12 8
4.71
0.10 Example 2
Benchmarking a program on 1, 2, ..., 8 processors
produces the following speedups:
p
ψ 2
1.87 3
2.61 4
3.23 5
3.73 6
4.14 7
4.46 8
4.71 Why is the speedup only 4.71 on 8 processors?
p
ψ
e 2
3
4
5
6
7
8
1.87 2.61 3.23 3.73 4.14 4.46 4.71
0.07 0.075 0.080 0.085 0.090 0.095 0.1 e is increasing: speedup problem is increasing serial
overhead (process startup, communication, algorithmic
issues, the architecture of the parallel system, etc.
Tuesday, February 14, 12 Which has the
efﬁciency problem?
Chart 5
5.00 3.75 2.50 1.25 0 2 3 speedup 1 Tuesday, February 14, 12 4 5 speedup 2 6 7 8 Very easy to see using e
Chart 6
0.100 0.075 0.050 0.025 0 2 3 4 e1 Tuesday, February 14, 12 5 6 7 8 e2 Isoefﬁciency Metric
• Parallel system: parallel program executing
on a parallel computer • Scalability of a parallel system: measure of its
ability to increase performance as number
of processors increases • A scalable system maintains efﬁciency as
processors are added • Isoefﬁciency: way to measure scalability
Tuesday, February 14, 12 Isoefﬁciency Derivation
Steps
• Begin with speedup formula
• Compute total amount of overhead
• Assume efﬁciency remains constant
• Determine relation between sequential
execution time and overhead Tuesday, February 14, 12 Deriving Isoefﬁciency
Relation
Determine overhead
Substitute overhead into speedup equation
Substitute T(n,1) = σ(n) + ϕ(n).
Assume efﬁciency is constant.
Isoefﬁciency Relation
Tuesday, February 14, 12 Scalability Function
• Suppose isoefﬁciency relation is n ≥ f(p)
• Let M(n) denote memory required for
problem of size n • M(f(p))/p shows how memory usage per processor must increase to maintain same
efﬁciency • We call M(f(p))/p the scalability function
Tuesday, February 14, 12 Meaning of Scalability
Function
• To maintain efﬁciency when increasing p, we
must increase n • Maximum problem size limited by available
memory, which is linear in p • Scalability function shows how memory usage per processor must grow to maintain
efﬁciency Tuesday, February 14, 12 Memory needed per processor Interpreting Scalability
Function Cplogp Tuesday, February 14, 12 Cannot maintain
efﬁciency Cp Memory Size
Can maintain
efﬁciency Clogp
C Number of processors Example 1: Reduction
• Sequential algorithm complexity
T(n,1) = Θ(n) • Parallel algorithm
• Computational complexity = Θ(n/p)
• Communication complexity = Θ(log p)
• Parallel overhead
T0(n,p) = Θ(p log p) Tuesday, February 14, 12 Reduction (continued)
• Isoefﬁciency relation: n ≥ C p log p
• We ask: To maintain same level of efﬁciency,
how must n increase when p increases? • M(n) = n
• The system has good scalability
Tuesday, February 14, 12 Example 2: Floyd’s
Algorithm
• Sequential time complexity: Θ(n3)
• Parallel computation time: Θ(n3/p)
• Parallel communication time: Θ(n2log p)
• Parallel overhead: T0(n,p) = Θ(pn2log p)
Tuesday, February 14, 12 Floyd’s Algorithm
(continued)
• Isoefﬁciency relation
n3 ≥ C(p n2 log p) • Tuesday, February 14, 12 M(n) = n2 n ≥ C p log p Example 3: Finite
Difference
• Sequential time complexity per iteration:
Θ(n2) • Parallel communication complexity per
iteration: Θ(n/√p) • Parallel overhead: Θ(n √p)
Tuesday, February 14, 12 Finite Difference
(continued)
• Isoefﬁciency relation
n2 ≥ Cn√p n ≥ C√ p • M(n) = n2
• This algorithm is perfectly scalable
Tuesday, February 14, 12 Summary (1/3)
• Performance terms
• Speedup
• Efﬁciency
• Model of speedup
• Serial component
• Parallel component
Tuesday, February 14, 12 Summary (2/3)
• What prevents linear speedup?
• Serial operations
• Communication operations
• Process startup
• Imbalanced workloads
• Architectural limitations
Tuesday, February 14, 12 Summary (3/3)
• Analyzing parallel performance
• Amdahl’s Law
• GustafsonBarsis’ Law
• KarpFlatt metric
• Isoefﬁciency metric
Tuesday, February 14, 12...
View
Full
Document
This note was uploaded on 02/19/2012 for the course ECE 563 taught by Professor Staff during the Spring '08 term at Purdue UniversityWest Lafayette.
 Spring '08
 Staff

Click to edit the document details