24.ParallelComputing - Parallel Computing Basics of...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Parallel Computing Basics of Parallel Computers Shared Memory SMP / NUMA Architectures Message Passing Clusters
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Why Parallel Computing No matter how effective ILP/Moore’s Law, more is better Most systems run multiple applications simultaneously Overlapping downloads with other work Web browser (overlaps image retrieval with display) Total cost of ownership favors fewer systems with multiple processors rather than more systems w/fewer processors Price Performance 2P+M 2P+2M P+M Peak performance increases linearly with more processors Adding processor/memory much cheaper than a second complete system
Background image of page 2
3 What about Sequential Code? Sequential programs get no benefit from multiple processors, they must be parallelized. Key property is how much communication per unit of computation. The less communication per unit computation the better the scaling properties of the algorithm. Sometimes, a multi-threaded design is good on uni & multi- processors e.g., throughput for a web server (that uses system multi-threading) Speedup is limited by Amdahl’s Law Speedup <= 1/(seq + (1 – seq)/proc) as proc -> infinity, Speedup is limited to 1/seq Many applications can be (re)designed/coded/compiled to generate cooperating, parallel instruction streams – specifically to enable improved responsiveness/throughput with multiple processors.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 Performance of parallel algorithms is NOT limited by which factor 1. The need to synchronize work done on different processors. 2.
Background image of page 4
Image of page 5
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 03/09/2012.

Page1 / 14

24.ParallelComputing - Parallel Computing Basics of...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online