Sequential programmers were lucky since every 2 years

Info icon This preview shows pages 2–4. Sign up to view the full content.

Sequential programmers were lucky: since every 2 years or so computers got roughly twice as fast, most programs would get exponentially faster over time without any extra effort. Around 2005, computers stopped getting twice as fast every 2 years. To understand why requires a course in computer architecture. In brief, increasing the clock rate (very roughly and technically inaccurately speaking, how quickly instructions execute) be- came infeasible without generating too much heat. Also, the relative cost of memory accesses can become too high for faster processors to help. Nonetheless, chip manufacturers still plan to make exponentially more powerful chips. Instead of one processor running faster, they will have more processors. The next com- puter you buy will likely have 4 processors (also called cores ) on the same chip and the number of available cores will likely double every few years. What would 256 cores be good for? Well, you can run multiple programs at once — for real, not just with time-slicing. But for an individual program to run any faster than with one core, it will need to do more than one thing at once. This is the reason that multithreaded programming is becoming more important. To be clear, multithreaded programming is not new. It has existed for decades and all the key concepts are just as old. Before there were multiple cores on one chip, you could use multiple chips and/or use time-slicing on one chip — and both remain important techniques today. The move to multiple cores on one chip is “just” having the effect of making multithreading some- thing that more and more software wants to do. CPEN 221 – Fall 2016
Image of page 2

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Introduction to Concurrent and Parallel Programming 3 1.2 Parallelism vs. Concurrency These notes are organized around a fundamental distinction between parallelism and concurrency . Unfortunately, the way we define these terms is not entirely standard, so you should not assume that everyone uses these terms as we will. Nonetheless, most computing experts agree that this distinction is important. Parallel programming is about using additional computational resources to pro- duce an answer faster. As a canonical example, consider the trivial problem of summing up all the numbers in an array. We know no sequential algorithm can do better than Θ( n ) (i.e., proportional to the number of items in the list) time. Suppose instead we had 4 processors. Then hopefully we could produce the result roughly 4 times faster by having each processor add 1/4 of the elements and then we could just add these 4 partial results together with 3 more additions. Θ( n / 4 ) is still Θ( n ) , but constant factors can matter. Moreover, when designing and analyzing a parallel algorithm , we should leave the number of processors as a variable, call it P . Perhaps we can sum the elements of an array in time O ( n / P ) given P processors. As we will see, in fact the best bound under the assumptions we will make is O ( log n + n / P ) .
Image of page 3
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.
  • Fall '17
  • satish
  • Central processing unit, shared memory, Parallel algorithm

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern