{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

COMP10052-AAAF-09-2010-for-viewing

The mapreduce model 1 18 one approach to ensure

Info icon This preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: y order. Take the expression: (a * b) + (y * z)   Since ‘+’ and ‘* ’ commute, a compiler (or a CPU scheduler) can execute the lef ­ and the right ­hand sides in any order.         This is also true, but harder to automate, at the level of en<re func<ons. Take the expression x = foo(a) + bar(b) If neither foo() nor bar() access shared state, they can be parallelized. The assignment to x becomes the merge/ synchroniza<on point that blocks wai<ng for them to complete. Task ­Level Parallelism 17     Dividing work into reasonably sized par<<ons creates tasks that are the logical units of work for paralleliza<on. Judicious task design eliminates as many synchroniza<on points as possible, but some will always be unavoidable.     Independent tasks can carried out on different physical machines in distributed fashion. The goal of task design, therefore, is to iden<fy kinds of data and classes of computa<on that allow for distributed execu<on. The Map ­Reduce Model (1) 18       One approach to ensure task independence is to limit the expressiveness of the computa<on performed by individual tasks. A prominent example (due to its widespread use in Google, Yahoo, etc.) is referred to as the map ­ reduce computa<onal model. The model has roots in the func<onal programming paradigm.         In (pure) func<onal programs, func<ons are side ­effect free. This means that two func<ons used in the same expression are guaranteed not to share any state. Instead of side ­effec<ng shared data structures, completely new ones are created. Pure func<onal programs are, therefore, inherently parallelizable. The Map ­Reduce Model (2) 19       The map ­reduce model centres around the (classical) map and reduce second ­order func<ons. In its simplest form, map takes a unary func<on f and a collec<on [c1, …, cn] and returns the collec<on [...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern