Unformatted text preview: IDLE Slave 3 possible events: v A node sends an elec�on message to all masters with (1) (a) Internal �meout higher ids, then waits for a response. Discovery Service v If no response within a �me limit à it wins. v Otherwise à wait for an update message from the new (b) primary or coordinator. v If no update message received before a �meout à restart elec�on. (1) masters? Node k: Master Prim. Back. Coord. Tasks (2) Response = No à becomes master. Response = Yes à wait Node i: IDLE Slave Assign tasks (c) Node k: Master Prim. Back. Coord. 57 Become Master Tasks Node i: ACTIVE Slave Tasks Node i: Slave 58 Figure 13 from  Implementa�on v P2P-‐MR was implemented using the JXTA framework. v Messages are sent using asynchronous communica�on pipes. v Each node periodically publishes an adver�sement with its informa�on (role, workload, expira�on �me). v Each node is composed of 3 so�ware layers. Core Work Hadoop 59 60 10 9/17/13 Commonali�es and Contrasts Commonali�es, Contrasts, Ineﬃciencies, and Possible Extensions v Commonality v Improve MR performance in the clouds environment. v Contrasts v The level they operate on. v LATE requires the usage of the proposed modiﬁed MR implementa�on (applica�on level). v P2P-‐MR require the usage of the proposed MR middleware (applica�on level). v MRG Scheduler requires a modiﬁca�on in the OS of the physical machines (patched Xen). (May be a disadvantage.) 61 62 Ineﬃciencies & Possible Extensions Ineﬃciencies & Possible Extensions v LATE Scheduler: v LATE Scheduler: v Possible solu�ons: v Target: a ﬁx in the specula�ve execu�on. v (1) Dropping the the usage of the progress rate in the v Problem: ranking procedure. (To make it more robust for heterogeneity) v The authors used two of the assump�ons that they invalidated: constant progress rate, and that each phase in a reduce task represents 1/3 of its total �me. v Remaining �me = (1 – progressScore) / progressRate where Progress rate = progressScore / T v This caused wrong es�mated remaining �mes for tasks à v Suggested ranking value = T.(1 – progressScore) wrong ranking. v It is propor�onal to the amount of progress remaining and how long have the task been running. v The older they are and the less progress they made, the more preferable they are for specula�ve execu�on. 63 64 Ineﬃciencies & Possible Extensions Ineﬃciencies & Possible Extensions v LATE Scheduler: v MRG Scheduler: v Problems: v Possible solu�ons: v Based on 2 assump�ons: MR jobs are I/O intensive, and all v (2) Improve the progress score calcula�on. (To adapt to nodes in a group have the same map and reduce func�ons. the diﬀerent jobs requirements: dynamic assignment instead of sta�c assignment) v They may not hold. (ineﬀec�ve à degrada�on???) v It guarantees propor�onal fairness to each...
View Full Document
- Fall '13