Because these notes do not describe the frameworks

Info icon This preview shows pages 30–32. Sign up to view the full content.

View Full Document Right Arrow Icon
simply will not occur in practice. Because these notes do not describe the framework’s implementation, we will not see where the randomness arises. Notice that, ignoring constant factors, this bound is optimal: Given only P processors, no framework can expect to do better than T 1 / P or better than T . For small P , the term T 1 / P is likely to be dominant and we can expect roughly linear speed-up. As P grows, the span becomes more relevant and the limit on the run-time is more influenced by T . Constant factors can be relevant, and it is entirely possible that a hand-crafted parallel algorithm in terms of some fixed P could do better than a generic library that has no idea what sort of parallel algorithm it is running. But just like we often use asymptotically optimal data structures even if hand-crafted ones for our task might be a little faster, using a library such as this is often an excellent approach. CPEN 221 – Fall 2016
Image of page 30

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Fork-Join Parallelism 31 Thinking in terms of the program-execution dag, it is rather amazing that a library can achieve this optimal result. While the program is running, it is the framework’s job to choose among all the threads that could run next (they are not blocked waiting for some other thread to finish) and assign P of them to processors. For simple parallel reductions, the choice hardly matters because all paths to the bottom of the dag are about the same length, but for arbitrary dags it seems important to work on the longer paths. Yet it turns out a much greedier algorithm that just picks randomly among the available threads will do only a constant factor worse. But this is all about the library’s internal scheduling algorithm (which is not actually totally random) and we, as library users, can just rely on the provided bound. However, as mentioned above, the bound holds only under a couple assumptions. The first is that all the threads you create to do subproblems do approximately the same amount of work. Otherwise, if a thread with much-more-work-to-do is scheduled very late, other processors will sit idle waiting for this laggard to finish. The second is that all the threads do a small but not tiny amount of work. This again helps with load balancing. The documentation for the library suggests aiming for approximately 5000 basic computational steps (additions, method calls, etc.) and that the exact figure can be off by a factor of 10 or so without problem. In other words, just avoid threads that do millions of operations as well as threads that do dozens. To summarize, as a user of a library like this, your job is to pick a good parallel algorithm, implement it in terms of divide-and-conquer with a reasonable sequential cut-off, and analyze the expected run-time in terms of the provided bound. The library’s job is to give this bound while trying to maintain low constant-factor overheads. While this li- brary is particularly good for this style of programming, this basic division is common: application writers develop good algorithms and rely on some underlying thread sched- uler to deliver reasonable performance.
Image of page 31
Image of page 32
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern