CommOps - Basic communication operations Possible variants...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
CSE 721, Winter 2011 1 Basic communication operations Possible variants # of nodes nodes involved Point-to-point vs collective operation routing scheme Usually point-to-point implemented in hardware, collective in software Many of the collective have a dual operation the dual can be performed reversing the direction and sequence of messages in the original operation Communication Operations
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
CSE 721, Winter 2011 2 Point-to-point Store-and-forward => t comm t s + lmt w ring l = p /2 t comm = t s + p /2 mt w mesh l = 2 ⎣√ p /2 t comm = t s + 2 ⎣√ p /2 mt w hypercube l = log p t comm = t s + mt w log p Cut-through => t comm = t s + lt h + mt w Small messages: CT S&F t s + lt h Large messages: CT t s + mt w (no dependence from l ) Communication Operations
Background image of page 2
CSE 721, Winter 2011 3 One-to-all broadcast A.k.a single-node broadcast message of size m on source processor at the end of the operation message is replicated on all other procs Dual operation: single-node accumulation (a.k.a reduce operation) initially every processor has message of size m at the end, combination of all messages is on single destination proc combination is through an associative operation (sum, product, max, min) Communication Operations
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
CSE 721, Winter 2011 4 Broadcast over mesh: example Multiplication of 4 x 4 matrix with a 4 x 1 vector Communication Operations
Background image of page 4
CSE 721, Winter 2011 5 Broadcast on ring (S&F) 0 1 2 3 7 6 5 4 1 2 2 3 3 4 4 Number of steps: p/ 2 Latency of communication step: t s + mt w Total duration: T one_to_all = ( t s + mt w ) p/ 2 Communication Operations
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 25

CommOps - Basic communication operations Possible variants...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online