CS655-Alsaraan-MapReduce

Interface the user can also include an esmated me

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: group. v  Suggested method: use the user’s knowledge about his/her v  Prac�cal, but not perfect. (Trade-­‐off between fairness, own developed map and reduce func�ons. u�liza�on, and context-­‐switch overhead.) v  When the user submits these codes through the MR v  Weights play a strong cri�cal role here. interface, the user can also include an es�mated �me complexity per record read. Then MR uses this informa�on to propor�onally divide the scores for the different tasks. v  May not be fair to other VMs not belonging to MR jobs. v  Possible Solu�ons: ??? 65 66 11 9/17/13 Inefficiencies & Possible Extensions Inefficiencies & Possible Extensions v  P2P–MapReduce: v  P2P–MapReduce: v  Problem: v  Problem: v  Role change only happens for idle nodes. v  A lot of message passing between the nodes to maintain the v  Since load balancing is used to distribute the tasks, there may not system. exist idle nodes but there may be nodes with very light load. v  Possible Solu�on: v  It gives it the robustness (fault-­‐tolerance), so it is more reliable but at the expense of more work-­‐load and network traffic. v  If there is a need to change roles: v  The authors preferred management efficiency over network v  First, check for idle nodes. traffic. v  If none exists: v  Possible Solu�on: ??? v  Look for the node with lightest load. v  Transfer its load to the second lightest-­‐load node and send the necessary update messages. v  Change the role of this node. 67 68 Conclusion v  Three reviewed approaches address the problem of the MR performance degrada�on in the heterogeneous and dynamic environment of the clouds. Thank you. v  They all targeted different levels of MR. v  They are not conflic�ng, but complemen�ng each other. v  It is possible to integrate all the three approaches in a single framework in a future work. 69 References (1/2) 70 References (2/2) 1.  Dean, J., and Ghemawat, S. “MapReduce: Simplified 4.  Rehman, M., and Sakr, M. “Ini�al Findings for Provisioning 2.  Kang,H., Chen, Y., Wong,, J., and Wu, J. “Enhancement 5.  Stonebraker, M., Abadi, D., DeWi�, D., Madden, S., Paulson, Varia�on in Cloud Compu�ng”, 2nd IEEE Interna�onal Conference on Cloud Compu�ng Technology and Science, 2010, Pages: 473-­‐479. Data Processing on Large Clusters”, Communica�ons of the ACM, Vol. 51, No. 1, 2008, Pages: 107-­‐113. E., Pavlo, A., and Rasin, A. “MapReduce and Parallel DBMSs: Friends or Foes?” , Communica�ons of the ACM, Vol. 53, No. 1, 2010, Pages: 64-­‐71. of Xen’s Scheduler for MapReduce Workloads”, HPDC’11, San Jose, California, USA, 2011, Pages: 251-­‐262. 6.  Zaharia, M., Konwinski, A., Joseph, A., Katz, R., and Stoica, I. 3.  Marozzo, F., Talia, D., and Trunfio, P. “P2P-­‐MapReduce: “Improving MapReduce Performance in Heterogeneous Environments”, 8th USENIX Symposium on Opera�ng Systems Design Implementa�on, San Diego, CA, 2008, Pages: 29-­‐42. Parallel data processing in dynamic Cloud environments”, Journal of Computer and System Sciences 78, 2012, Pages: 1382–1402. 71 72 12...
View Full Document

Ask a homework question - tutors are online