comp422-Lecture13-CollectiveCommunication

comp422-Lecture13-CollectiveCommunication - Collective...

Info iconThis preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
John Mellor-Crummey Department of Computer Science Rice University johnmc@cs.rice.edu Collective Communication COMP 422 Lecture 13 19 February 2009
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Group Communication Motivation: accelerate interaction patterns among groups Approach: collective communication —whole group works together collectively to realize the pattern —constructed from pairwise point-to-point communications Implementation strategy —standard library of common collective operations —leverage target architecture for efficient implementation Benefits of standard library implementations —reduce development effort and cost for parallel codes —improve performance through efficient implementations —improve software quality
Background image of page 2
3 Topics for Today One-to-all broadcast and all-to-one reduction All-to-all broadcast and reduction All-reduce and prefix-sum operations Scatter and gather All-to-all personalized communication Circular shift Optimizing collective patterns
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 Assumptions Network is bidirectional Communication is single-ported —node can receive only one message per step Communication cost model message of size m, no congestion, time = t s +t w m congestion: model by scaling t w
Background image of page 4
5 One-to-All and All-to-One One-to-all broadcast —a processor has m units of data that it must send to everyone All-to-one reduction —each processor has m units of data —data items must be combined using some associative operator e.g. addition, min, max —result must be available at a target processor
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6 Broadcast and Reduction Example Matrix vector product Problem n x n matrix assigned to an n x n (virtual) processor grid —vector on the first row of processors Steps —one-to-all broadcast of vector element down columns concurrency between columns —processors compute product of vector element & matrix entry —accumulate results of these products to the first column use n concurrent all-to-one sum reductions along rows x
Background image of page 6
7 One-to-All and All-to-One on a Ring
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 34

comp422-Lecture13-CollectiveCommunication - Collective...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online