The problem I just gave you has a significant amount of communication compared to computation ▪Communication costs can dominate a parallel computation, severely limiting speedup
CMU 15-418/618, Fall 2019Course theme 1: Designing and writing parallel programs ... that scale!▪Parallel thinking 1.Decomposing work into pieces that can safely be performed in parallel 2.Assigning work to processors 3.Managing communication/synchronization between the processors so that it does not limit speedup ▪Abstractions/mechanisms for performing the above tasks -Writing code in popular parallel programming languages
CMU 15-418/618, Fall 2019Course theme 2: Parallel computer hardware implementation: how parallel computers work▪Mechanisms used to implement abstractions eﬃciently -Performance characteristics of implementations -Design trade-offs: performance vs. convenience vs. cost ▪Why do I need to know about hardware? -Because the characteristics of the machine really matter (recall speed of communication issues in earlier demos) -Because you care about eﬃciency and performance (you are writing parallel programs after all!)
CMU 15-418/618, Fall 2019Course theme 3: Thinking about eﬃciency▪FAST != EFFICIENT ▪Just because your program runs faster on a parallel computer, it does not mean it is using the hardware eﬃciently -Is 2x speedup on computer with 10 processors a good result? ▪Programmer’s perspective: make use of provided machine capabilities ▪HW designer’s perspective: choosing the right capabilities to put in system (performance/cost, cost = silicon area?, power?, etc.)
CMU 15-418/618, Fall 2019Fundamental Shift in CPU Design PhilosophyBefore 2004: -within the chip area budget, maximize performance -increasingly aggressive speculative execution for ILP After 2004: -area withinthe chip matters (limits # of cores/chip): -maximize performance per area -power consumption is critical (battery life, data centers) -maximize performance per Watt -upshot: major focus on eﬃciencyof cores
CMU 15-418/618, Fall 2019Summary▪Today, single-thread performance is improving very slowly -To run programs significantly faster, programs must utilize multiple processing elements -Which means you need to know how to write parallel code ▪Writing parallel programs can be challenging -Requires problem partitioning, communication, synchronization -Knowledge of machine characteristics is important ▪I suspect you will find that modern computers have tremendously more processing power than you might realize, if you just use it! ▪Welcome to 15-418!
You've reached the end of your free preview.
Want to read all 40 pages?
- Spring '19
- Hassan Kasfy