16-par-intro - Last Time Parallel Cluster Programming CMU...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Parallel Cluster Programming CMU 15-440: Distributed Systems Last Time Questions about Paxos / distributed consensus? If you haven’t picked up your exam yet, it’s with my admin (see web page) Parallelism ubiquitous Even your laptop, if your parents bought you something decent :), has 2 or more cores in it. Each of those cores is actually internally parallel, but that’s not this course. The question of the decade in computing (not exaggerating): How to exploit parallelism. Q1: What’s the workload?? Before we try to solve it, let’s look briefly at the map of problems and solutions... “Trivially parallel” -- e.g., password cracking. Requires little communication, memory, etc. Compute-intensive -- e.g., physical mesh simulations in HPC. Often very memory intensive, need low-latency, high-bw communication Data-intesive -- e.g., computing an index of the web, data- mining, etc. Often disk intensive, bandwidth intensive, but not as latency sensitive.
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Q2: The challenges First: Algorithmic. How do you solve problem X in parallel? Sometimes this is very hard; sometimes it’s straightforward. For our purposes, let’s assume the answer is known. Otherwise, take Guy Blelloch’s course. Second: Systems. How do you practically solve problem X using reasonable amounts of programmer time, efficiently use the parallel resources, etc.? Helping Programmers Use Clusters As systems people, our job is to make stuff work. “Tools to make tools.” What tools can we provide to ease the pain of parallel computing? Step 0: We can make it easy to execute code on lots of machines... and to share those machines between “jobs” (cf Condor) Step 0a: We can provide you with a shared distributed filesystem, or copy your programs onto these machines automatically Step 1: We can give you easy to use mechanisms for communication between processes (RPC, or, in the high-performance domain, MPI.) Step 1a: MPI slightly higher level: library knows which other computers are involved in the processing, handles process spawning (step 0), provides message sending, broadcast, many-to-1, get(), put() abstraction for shared memory regions, synchronization, etc.) Observations: These tools are very general - you can use them to solve any parallel problem These tools are very low-level - they don’t deal with a lot of very hard questions in parallel programming! Some hard questions Psst - you saw a lot of these in lab 1! Dealing with failures (it’s tough - even your professor puts bugs into complex algorithms when explaining them!) Dealing with load balance - efficiently keeping all of the machines busy, splitting up the work into chunks, etc.
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern