This is an example of higher order programming and

Info icon This preview shows pages 23–25. Sign up to view the full content.

This is an example of higher-order programming, and the fork-join framework has sev- eral classes providing this sort of functionality. Higher-order programming has many, many advantages (see the end of this section for a popular one), but when first learn- ing a programming pattern, it is often useful to “code it up yourself” a few times. For that reason, we encourage writing your parallel reductions manually in order to see the parallel divide-and-conquer, even though they all really look the same. Parallel reductions are not the only common pattern in parallel programming. An even simpler one, which we did not start with because it is just so easy, is a parallel map . A map performs an operation on each input element independently; given an array of inputs, it produces an array of outputs of the same length. A simple example would be multiplying every element of an array by 2. An example using two inputs and producing a separate output would be vector addition. Using pseudocode, we could write: int [] add ( int [] arr1, int [] arr2) { assert (arr1. length == arr2. length ); int [] ans = new int [arr1. length ]; forall ( int i= 0 ; i < arr1. length ; i++) ans[i] = arr1[i] + arr2[i]; return ans; } Coding up this algorithm in the ForkJoin framework is straightforward: Have the main thread create the ans array and pass it before starting the parallel divide-and-conquer. Each thread object will have a reference to this array but will assign to different portions of it. Because there are no other results to combine, subclassing RecursiveAction is appropriate. Using a sequential cut-off and creating only one new thread for each recursive subdivision of the problem remain important — these ideas are more general than the particular programming pattern of a map or a reduce. Recognizing problems that are fundamentally maps and/or reduces over large data col- lections is a valuable skill that allows efficient parallelization. In fact, it is one of the key ideas behind Google’s MapReduce framework and the open-source variant Hadoop. In these systems, the programmer just writes the operations that describe how to map data (e.g., “multiply by 2”) and reduce data (e.g., “take the minimum”). The system then does all the parallelization, often using hundreds or thousands of computers to process gigabytes or terabytes of data. For this to work, the programmer must provide operations that have no side effects (since the order they occur is unspecified) and re- duce operations that are associative (as we discussed). As parallel programmers, it is CPEN 221 – Fall 2016
Image of page 23

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Fork-Join Parallelism 24 often enough to “write down the maps and reduces” — leaving it to systems like the ForkJoin framework or Hadoop to do the actual scheduling of the parallelism. 7 Data Structures Besides Arrays So far we have considered only algorithms over one-dimensional arrays. Naturally, one can write parallel algorithms over any data structure, but divide-and-conquer paral- lelism requires that we can efficiently divide the problem into smaller pieces. For arrays,
Image of page 24
Image of page 25
This is the end of the preview. Sign up to access the rest of the document.
  • Fall '17
  • satish
  • fork-join parallelism, SumThread

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern