This preview shows page 1. Sign up to view the full content.
Unformatted text preview: group. v Suggested method: use the user’s knowledge about his/her v Prac�cal, but not perfect. (Trade-‐oﬀ between fairness, own developed map and reduce func�ons. u�liza�on, and context-‐switch overhead.) v When the user submits these codes through the MR v Weights play a strong cri�cal role here. interface, the user can also include an es�mated �me complexity per record read. Then MR uses this informa�on to propor�onally divide the scores for the diﬀerent tasks. v May not be fair to other VMs not belonging to MR jobs. v Possible Solu�ons: ??? 65 66 11 9/17/13 Ineﬃciencies & Possible Extensions Ineﬃciencies & Possible Extensions v P2P–MapReduce: v P2P–MapReduce: v Problem: v Problem: v Role change only happens for idle nodes. v A lot of message passing between the nodes to maintain the v Since load balancing is used to distribute the tasks, there may not system. exist idle nodes but there may be nodes with very light load. v Possible Solu�on: v It gives it the robustness (fault-‐tolerance), so it is more reliable but at the expense of more work-‐load and network traﬃc. v If there is a need to change roles: v The authors preferred management eﬃciency over network v First, check for idle nodes. traﬃc. v If none exists: v Possible Solu�on: ??? v Look for the node with lightest load. v Transfer its load to the second lightest-‐load node and send the necessary update messages. v Change the role of this node. 67 68 Conclusion v Three reviewed approaches address the problem of the MR performance degrada�on in the heterogeneous and dynamic environment of the clouds. Thank you. v They all targeted diﬀerent levels of MR. v They are not conﬂic�ng, but complemen�ng each other. v It is possible to integrate all the three approaches in a single framework in a future work. 69 References (1/2) 70 References (2/2) 1. Dean, J., and Ghemawat, S. “MapReduce: Simpliﬁed 4. Rehman, M., and Sakr, M. “Ini�al Findings for Provisioning 2. Kang,H., Chen, Y., Wong,, J., and Wu, J. “Enhancement 5. Stonebraker, M., Abadi, D., DeWi�, D., Madden, S., Paulson, Varia�on in Cloud Compu�ng”, 2nd IEEE Interna�onal Conference on Cloud Compu�ng Technology and Science, 2010, Pages: 473-‐479. Data Processing on Large Clusters”, Communica�ons of the ACM, Vol. 51, No. 1, 2008, Pages: 107-‐113. E., Pavlo, A., and Rasin, A. “MapReduce and Parallel DBMSs: Friends or Foes?” , Communica�ons of the ACM, Vol. 53, No. 1, 2010, Pages: 64-‐71. of Xen’s Scheduler for MapReduce Workloads”, HPDC’11, San Jose, California, USA, 2011, Pages: 251-‐262. 6. Zaharia, M., Konwinski, A., Joseph, A., Katz, R., and Stoica, I. 3. Marozzo, F., Talia, D., and Trunﬁo, P. “P2P-‐MapReduce: “Improving MapReduce Performance in Heterogeneous Environments”, 8th USENIX Symposium on Opera�ng Systems Design Implementa�on, San Diego, CA, 2008, Pages: 29-‐42. Parallel data processing in dynamic Cloud environments”, Journal of Computer and System Sciences 78, 2012, Pages: 1382–1402. 71 72 12...
View Full Document
- Fall '13