The problem I just gave you has a signi fi cant amount of communication

The problem i just gave you has a signi fi cant

This preview shows page 35 - 40 out of 40 pages.

The problem I just gave you has a significant amount of communication compared to computation Communication costs can dominate a parallel computation, severely limiting speedup
Background image
CMU 15-418/618, Fall 2019Course theme 1: Designing and writing parallel programs ... that scale!Parallel thinking 1.Decomposing work into pieces that can safely be performed in parallel 2.Assigning work to processors 3.Managing communication/synchronization between the processors so that it does not limit speedup Abstractions/mechanisms for performing the above tasks -Writing code in popular parallel programming languages
Background image
CMU 15-418/618, Fall 2019Course theme 2: Parallel computer hardware implementation: how parallel computers workMechanisms used to implement abstractions eciently -Performance characteristics of implementations -Design trade-offs: performance vs. convenience vs. cost Why do I need to know about hardware? -Because the characteristics of the machine really matter (recall speed of communication issues in earlier demos) -Because you care about eciency and performance (you are writing parallel programs after all!)
Background image
CMU 15-418/618, Fall 2019Course theme 3: Thinking about eciencyFAST != EFFICIENT Just because your program runs faster on a parallel computer, it does not mean it is using the hardware eciently -Is 2x speedup on computer with 10 processors a good result? Programmer’s perspective: make use of provided machine capabilities HW designer’s perspective: choosing the right capabilities to put in system (performance/cost, cost = silicon area?, power?, etc.)
Background image
CMU 15-418/618, Fall 2019Fundamental Shift in CPU Design PhilosophyBefore 2004: -within the chip area budget, maximize performance -increasingly aggressive speculative execution for ILP After 2004: -area withinthe chip matters (limits # of cores/chip): -maximize performance per area -power consumption is critical (battery life, data centers) -maximize performance per Watt -upshot: major focus on eciencyof cores
Background image
CMU 15-418/618, Fall 2019SummaryToday, single-thread performance is improving very slowly -To run programs significantly faster, programs must utilize multiple processing elements -Which means you need to know how to write parallel code Writing parallel programs can be challenging -Requires problem partitioning, communication, synchronization -Knowledge of machine characteristics is important I suspect you will find that modern computers have tremendously more processing power than you might realize, if you just use it! Welcome to 15-418!
Background image

You've reached the end of your free preview.

Want to read all 40 pages?

  • Spring '19
  • Hassan Kasfy

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask You can ask You can ask (will expire )
Answers in as fast as 15 minutes