This preview shows pages 1–8. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Efficiency Scalability Modeling Parallel Numerical Algorithms Chapter 4 – Parallel Performance Prof. Michael T. Heath Department of Computer Science University of Illinois at UrbanaChampaign CSE 512 / CS 554 Michael T. Heath Parallel Numerical Algorithms 1 / 49 Efficiency Scalability Modeling Outline 1 Efficiency Parallel Efficiency Basic Definitions Execution Time and Cost Efficiency and Speedup 2 Scalability Definition Problem Scaling Isoefficiency 3 Modeling Parallel Work Example Michael T. Heath Parallel Numerical Algorithms 2 / 49 Efficiency Scalability Modeling Parallel Efficiency Basic Definitions Execution Time and Cost Efficiency and Speedup Parallel Efficiency Efficiency : effectiveness of parallel algorithm relative to its serial counterpart (more precise definition later) Factors determining efficiency of parallel algorithm Load balance : distribution of work among processors Concurrency : processors working simultaneously Overhead : additional work not present in corresponding serial computation Efficiency is maximized when load imbalance is minimized, concurrency is maximized, and overhead is minimized Michael T. Heath Parallel Numerical Algorithms 3 / 49 Efficiency Scalability Modeling Parallel Efficiency Basic Definitions Execution Time and Cost Efficiency and Speedup Parallel Efficiency (a) (b) (c) (d) (a) perfect load balance and concurrency (b) good initial concurrency but poor load balance (c) good load balance but poor concurrency (d) good load balance and concurrency but additional overhead Michael T. Heath Parallel Numerical Algorithms 4 / 49 Efficiency Scalability Modeling Parallel Efficiency Basic Definitions Execution Time and Cost Efficiency and Speedup Basic Definitions Memory ( M ) — amount of storage required (e.g., words) for given problem Work ( W ) — number of operations (e.g., flops) required for given problem, including loads and stores Velocity ( V ) — number of operations per unit time (e.g., flops/sec) performed by one processor Time ( T ) — elapsed wallclock time (e.g., secs) from beginning to end of computation Cost ( C ) — product of number of processors and execution time (e.g., processorseconds) Michael T. Heath Parallel Numerical Algorithms 5 / 49 Efficiency Scalability Modeling Parallel Efficiency Basic Definitions Execution Time and Cost Efficiency and Speedup Basic Definitions Subscript indicates number of processors used (e.g., T 1 is serial execution time, W p is work using p processors, etc.) We will assume M p ≥ M 1 , and with no replication of data it is reasonable to assume M p = M 1 for p ≥ 1 , in which case we drop subscript and write just M If serial algorithm is optimal and we disregard chance effects, then W p ≥ W 1 , and in general W p > W 1 for p > 1 Parallel overhead : O p ≡ W p W 1 Michael T. Heath Parallel Numerical Algorithms 6 / 49 Efficiency Scalability Modeling Parallel Efficiency Basic Definitions Execution Time and Cost Efficiency and Speedup Basic Definitions...
View
Full
Document
This note was uploaded on 11/06/2011 for the course CSE 494 taught by Professor Staff during the Summer '09 term at CUNY Brooklyn.
 Summer '09
 STAFF
 Algorithms

Click to edit the document details