This preview shows page 1. Sign up to view the full content.
Unformatted text preview: mbalance!
Time P1 P2 P3 P4 Done! One possible solution to imbalance problem:
Divide work into a larger number of smaller tasks
- Hopefully “long pole” gets shorter relative to overall execution time
- May increase synchronization overhead
- May not be possible (perhaps long task is fundamentally sequential)
CMU 15-418, Spring 2014 Smarter task scheduling
Schedule long task rst to reduce “slop” at end of computation
Time P1 P2 P3 P4 Done! Another solution: smarter scheduling
Schedule long tasks rst
- Thread performing long task performs fewer overall tasks, but approximately the
same amount of work as the other threads.
- Requires some knowledge of workload (some predictability of cost)
CMU 15-418, Spring 2014 Decreasing synchronization overhead
▪ Distributed work queues
- Replicate data to remove synchronization Subproblems
(a.k.a. “tasks”, “work to do”) Set of work queues
(In general, one per worker thread) Steal! Worker threads:
Pull data from OWN work queue
Push new work to OWN work to queue
When localwork queue is empty...
STEAL work from another work queue T1 T2 T3 T4 CMU 15-418, Spring 2014 Distributed work queues
▪ Costly synchron...
View Full Document
- Spring '14