161202-ForkJoinParallelism.pdf

# The forkjoin framework and similar libraries in other

• Notes
• 34

This preview shows pages 25–27. Sign up to view the full content.

. The ForkJoin framework (and similar libraries in other languages) will give us an optimal expected-time bound for any P . This section explains what that bound is and what it means, but we will not discuss how the framework achieves it. We then turn to discussing Amdahl’s Law, which analyzes the running time of algo- rithms that have both sequential parts and parallel parts. The key and depressing up- shot is that programs with even a small sequential part quickly stop getting much ben- efit from running with more processors. Finally, we discuss Moore’s “Law” in contrast to Amdahl’s Law. While Moore’s Law is also important for understanding the progress of computing power, it is not a mathe- matical theorem like Amdahl’s Law. 8.1 Work and Span 8.1.1 Defining Work and Span We define T P to be the time a program/algorithm takes to run if there are P processors available during its execution. For example, if a program was the only one running on a quad-core machine, we would be particularly interested in T 4 , but we want to think about T P more generally. It turns out we will reason about the general T P in terms of T 1 and T : T 1 is called the work . By definition, this is how long it takes to run on one proces- sor. More intuitively, it is just the total of all the running time of all the pieces of the algorithm: we have to do all the work before we are done, and there is exactly CPEN 221 – Fall 2016

This preview has intentionally blurred sections. Sign up to view the full version.

Fork-Join Parallelism 26 one processor (no parallelism) to do it. In terms of fork-join, we can think of T 1 as doing one side of the fork and then the other, though the total T 1 does not depend on how the work is scheduled. T is called the span , though other common terms are the critical path length or computational depth . By definition, this is how long it takes to run on an un- limited number of processors. Notice this is not necessarily O ( 1 ) or constant time; the algorithm still needs to do the forking and combining of results. For example, under our model of computation — where creating a new thread and adding two numbers are both O ( 1 ) operations — the algorithm we developed is asymptotically optimal with T = Θ( log n ) for an array of length n . We need a more precise way of characterizing the execution of a parallel program so that we can describe and compute the work, T 1 , and the span, T . We will describe a program execution as a directed acyclic graph (dag) where: • Nodes are pieces of work the program performs. Each node will be a constant, i.e., O ( 1 ) , amount of work that is performed sequentially. So T 1 is asymptotically just the number of nodes in the dag. • Edges represent that the source node must complete before the target node be- gins. That is, there is a computational dependency along the edge. This idea lets us visualize T : With unlimited processors, we would immediately start every node as soon as its predecessors in the graph had finished. Therefore T is just the length of the longest path in the dag.
This is the end of the preview. Sign up to access the rest of the document.
• Fall '17
• satish

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern