CommOps

# CommOps - Basic communication operations Possible variants...

This preview shows pages 1–6. Sign up to view the full content.

CSE 721, Winter 2011 1 Basic communication operations Possible variants # of nodes nodes involved Point-to-point vs collective operation routing scheme Usually point-to-point implemented in hardware, collective in software Many of the collective have a dual operation the dual can be performed reversing the direction and sequence of messages in the original operation Communication Operations

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
CSE 721, Winter 2011 2 Point-to-point Store-and-forward => t comm t s + lmt w ring l = p /2 t comm = t s + p /2 mt w mesh l = 2 ⎣√ p /2 t comm = t s + 2 ⎣√ p /2 mt w hypercube l = log p t comm = t s + mt w log p Cut-through => t comm = t s + lt h + mt w Small messages: CT S&F t s + lt h Large messages: CT t s + mt w (no dependence from l ) Communication Operations
CSE 721, Winter 2011 3 One-to-all broadcast A.k.a single-node broadcast message of size m on source processor at the end of the operation message is replicated on all other procs Dual operation: single-node accumulation (a.k.a reduce operation) initially every processor has message of size m at the end, combination of all messages is on single destination proc combination is through an associative operation (sum, product, max, min) Communication Operations

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
CSE 721, Winter 2011 4 Broadcast over mesh: example Multiplication of 4 x 4 matrix with a 4 x 1 vector Communication Operations
CSE 721, Winter 2011 5 Broadcast on ring (S&F) 0 1 2 3 7 6 5 4 1 2 2 3 3 4 4 Number of steps: p/ 2 Latency of communication step: t s + mt w Total duration: T one_to_all = ( t s + mt w ) p/ 2 Communication Operations

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 25

CommOps - Basic communication operations Possible variants...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online