CPEN
161202-ForkJoinParallelism.pdf

# 812 defining speedup and parallelism having defined

• Notes
• 34

This preview shows pages 28–31. Sign up to view the full content.

8.1.2 Defining Speedup and Parallelism Having defined work and span, we can use them to define some other terms more rele- vant to our real goal of reasoning about T P . After all, if we had only one processor then we would not study parallelism and having infinity processors is impossible. We define the speedup on P processors to be T 1 / T P . It is basically the ratio of how much faster the program runs given the extra processors. For example, if T 1 is 20 seconds and T 4 is 8 seconds, then the speedup for P = 4 is 2.5. You might naively expect a speed-up of 4, or more generally P for T P . In practice, such a perfect speedup is rare due to several issues including the overhead of creating threads and communicating answers among them, memory-hierarchy issues, and the inherent computational dependencies related to the span. In the rare case that doubling P cuts the running time in half (i.e., doubles the speedup), we call it perfect linear speedup . In practice, this is not the absolute limit; one can find situations where the speedup is even higher even though our simple computational model does not capture the features that could cause this. It is important to note that reporting only T 1 / T P can be “dishonest” in the sense that it often overstates the advantages of using multiple processors. The reason is that T 1 CPEN 221 – Fall 2016

This preview has intentionally blurred sections. Sign up to view the full version.

Fork-Join Parallelism 29 Figure 4: Example execution dag for summing an array. CPEN 221 – Fall 2016
Fork-Join Parallelism 30 is the time it takes to run the parallel algorithm on one processor, but this algorithm is likely to be much slower than an algorithm designed sequentially. For example, if someone wants to know the benefits of summing an array with parallel fork-join, they probably are most interested in comparing T P to the time for the sequential for-loop. If we call the latter S , then the ratio S / T P is usually the speed-up of interest and will be lower, due to constant factors like the time to create recursive tasks, than the definition of speed-up T 1 / T P . One measure of the overhead of using multiple threads is simply T 1 / S , which is usually greater than 1. As a final definition, we call T 1 / T the parallelism of an algorithm. It is a measure of how much improvement one could possibly hope for since it should be at least as great as the speed-up for any P . For our parallel reductions where the work is Θ( n ) and the span is Θ( log n ) , the parallelism is Θ( n / log n ) . In other words, there is exponential available parallelism ( n grows exponentially faster than log n ), meaning with enough processors we can hope for an exponential speed-up over the sequential version. 8.1.3 The ForkJoin Framework Bound Under some important assumptions we will describe below, algorithms written using the ForkJoin Framework, in particular the divide-and-conquer algorithms in these notes, have the following expected time bound: T P is O ( T 1 / P + T ) The bound is expected because internally the framework uses randomness, so the bound can be violated from “bad luck” but such “bad luck” is exponentially unlikely, so it simply will not occur in practice. Because these notes do not describe the framework’s

This preview has intentionally blurred sections. Sign up to view the full version.

This is the end of the preview. Sign up to access the rest of the document.
• Fall '17
• satish

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern