CS655-Alsaraan-MapReduce

V otherwise wait for an update message from the new b

Info icon This preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: IDLE Slave 3 possible events: v  A node sends an elec�on message to all masters with (1) (a) Internal �meout higher ids, then waits for a response. Discovery Service v  If no response within a �me limit à it wins. v  Otherwise à wait for an update message from the new (b) primary or coordinator. v  If no update message received before a �meout à restart elec�on. (1) masters? Node k: Master Prim. Back. Coord. Tasks (2) Response = No à becomes master. Response = Yes à wait Node i: IDLE Slave Assign tasks (c) Node k: Master Prim. Back. Coord. 57 Become Master Tasks Node i: ACTIVE Slave Tasks Node i: Slave 58 Figure 13 from [3] Implementa�on v  P2P-­‐MR was implemented using the JXTA framework. v  Messages are sent using asynchronous communica�on pipes. v  Each node periodically publishes an adver�sement with its informa�on (role, workload, expira�on �me). v  Each node is composed of 3 so�ware layers. Core Work Hadoop 59 60 10 9/17/13 Commonali�es and Contrasts Commonali�es, Contrasts, Inefficiencies, and Possible Extensions v  Commonality v  Improve MR performance in the clouds environment. v  Contrasts v  The level they operate on. v  LATE requires the usage of the proposed modified MR implementa�on (applica�on level). v  P2P-­‐MR require the usage of the proposed MR middleware (applica�on level). v  MRG Scheduler requires a modifica�on in the OS of the physical machines (patched Xen). (May be a disadvantage.) 61 62 Inefficiencies & Possible Extensions Inefficiencies & Possible Extensions v  LATE Scheduler: v  LATE Scheduler: v  Possible solu�ons: v  Target: a fix in the specula�ve execu�on. v  (1) Dropping the the usage of the progress rate in the v  Problem: ranking procedure. (To make it more robust for heterogeneity) v  The authors used two of the assump�ons that they invalidated: constant progress rate, and that each phase in a reduce task represents 1/3 of its total �me. v  Remaining �me = (1 – progressScore) / progressRate where Progress rate = progressScore / T v  This caused wrong es�mated remaining �mes for tasks à v  Suggested ranking value = T.(1 – progressScore) wrong ranking. v  It is propor�onal to the amount of progress remaining and how long have the task been running. v  The older they are and the less progress they made, the more preferable they are for specula�ve execu�on. 63 64 Inefficiencies & Possible Extensions Inefficiencies & Possible Extensions v  LATE Scheduler: v  MRG Scheduler: v  Problems: v  Possible solu�ons: v  Based on 2 assump�ons: MR jobs are I/O intensive, and all v  (2) Improve the progress score calcula�on. (To adapt to nodes in a group have the same map and reduce func�ons. the different jobs requirements: dynamic assignment instead of sta�c assignment) v  They may not hold. (ineffec�ve à degrada�on???) v  It guarantees propor�onal fairness to each...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern