CS655-Alsaraan-MapReduce

V otherwise wait for an update message from the new b

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: IDLE Slave 3 possible events: v  A node sends an elec�on message to all masters with (1) (a) Internal �meout higher ids, then waits for a response. Discovery Service v  If no response within a �me limit à it wins. v  Otherwise à wait for an update message from the new (b) primary or coordinator. v  If no update message received before a �meout à restart elec�on. (1) masters? Node k: Master Prim. Back. Coord. Tasks (2) Response = No à becomes master. Response = Yes à wait Node i: IDLE Slave Assign tasks (c) Node k: Master Prim. Back. Coord. 57 Become Master Tasks Node i: ACTIVE Slave Tasks Node i: Slave 58 Figure 13 from [3] Implementa�on v  P2P-­‐MR was implemented using the JXTA framework. v  Messages are sent using asynchronous communica�on pipes. v  Each node periodically publishes an adver�sement with its informa�on (role, workload, expira�on �me). v  Each node is composed of 3 so�ware layers. Core Work Hadoop 59 60 10 9/17/13 Commonali�es and Contrasts Commonali�es, Contrasts, Inefficiencies, and Possible Extensions v  Commonality v  Improve MR performance in the clouds environment. v  Contrasts v  The level they operate on. v  LATE requires the usage of the proposed modified MR implementa�on (applica�on level). v  P2P-­‐MR require the usage of the proposed MR middleware (applica�on level). v  MRG Scheduler requires a modifica�on in the OS of the physical machines (patched Xen). (May be a disadvantage.) 61 62 Inefficiencies & Possible Extensions Inefficiencies & Possible Extensions v  LATE Scheduler: v  LATE Scheduler: v  Possible solu�ons: v  Target: a fix in the specula�ve execu�on. v  (1) Dropping the the usage of the progress rate in the v  Problem: ranking procedure. (To make it more robust for heterogeneity) v  The authors used two of the assump�ons that they invalidated: constant progress rate, and that each phase in a reduce task represents 1/3 of its total �me. v  Remaining �me = (1 – progressScore) / progressRate where Progress rate = progressScore / T v  This caused wrong es�mated remaining �mes for tasks à v  Suggested ranking value = T.(1 – progressScore) wrong ranking. v  It is propor�onal to the amount of progress remaining and how long have the task been running. v  The older they are and the less progress they made, the more preferable they are for specula�ve execu�on. 63 64 Inefficiencies & Possible Extensions Inefficiencies & Possible Extensions v  LATE Scheduler: v  MRG Scheduler: v  Problems: v  Possible solu�ons: v  Based on 2 assump�ons: MR jobs are I/O intensive, and all v  (2) Improve the progress score calcula�on. (To adapt to nodes in a group have the same map and reduce func�ons. the different jobs requirements: dynamic assignment instead of sta�c assignment) v  They may not hold. (ineffec�ve à degrada�on???) v  It guarantees propor�onal fairness to each...
View Full Document

Ask a homework question - tutors are online