Dividing the problem involves only simple arithmetic

Info icon This preview shows pages 24–26. Sign up to view the full content.

View Full Document Right Arrow Icon
dividing the problem involves only simple arithmetic (which takes a constant amount of time, or Θ( 1 ) time) on indices, so this works well. While arrays are the most common data structure in parallel programming, balanced trees also support parallel algorithms well. For example, with a binary tree, we can fork to process the left child and right child of each node in parallel. For good sequential cut- offs, it helps to have stored at each tree node the number of descendants of the node, something easy to maintain. However, for trees with guaranteed balance properties, other information — like the height of a subtree — should suffice. Certain tree problems will not run faster with parallelism. For example, searching for an element in a balanced binary search tree takes O ( log n ) time with or without parallelism. However, maps and reduces over balanced trees benefit from parallelism. For example, summing the elements of a binary tree takes O ( n ) time sequentially where n is the number of elements, but with a sufficiently large number of processors, the time is O ( h ) , where h is the height of the tree. Hence, tree balance is even more important with parallel programming: for a balanced tree h = Θ( log n ) compared to the worst case h = Θ( n ) . For the same reason, parallel algorithms over regular linked lists are typically poor. Any problem that requires reading all n elements of a linked list takes time Ω( n ) (or at least time proportional to n , although it may be higher) regardless of how many processors are available. (Fancier list data structures like skip lists are better for exactly this reason — you can get to all the data in O ( log n ) time.) Streams of input data, such as from files, typically have the same limitation: it takes linear time to read the input and this can be the bottleneck for the algorithm. There can still be benefit to parallelism with such “inherently sequential” data struc- tures and input streams. Suppose we had a map operation over a list but each opera- tion was itself an expensive computation (e.g., decrypting a significant piece of data). If each map operation took time O ( x ) and the list had length n , doing each operation in a separate thread (assuming, again, no limit on the number of processors) would pro- CPEN 221 – Fall 2016
Image of page 24

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Fork-Join Parallelism 25 duce an O ( x + n ) algorithm compared to the sequential O ( xn ) algorithm. But for simple operations like summing or finding a maximum element, there would be no benefit. 8 Analyzing Fork-Join Algorithms Much of this section is optional reading but one should read the discussion on Amdahl’s Law and Moore’s “Law” towards the end. As with any algorithm, a fork-join parallel algorithm should be correct and efficient. This section focuses on the latter even though the former should always be one’s first concern. For efficiency, we will focus on asymptotic bounds and analyzing algorithms that are not written in terms of a fixed number of processors. That is, just as the size of the problem n will factor into the asymptotic running time, so will the number of processors P . The ForkJoin framework (and similar libraries in other languages) will
Image of page 25
Image of page 26
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern