dis6

dis6 - CS32 Discussion Sec.on 1B Week 7 TA: Brian...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: CS32 Discussion Sec.on 1B Week 7 TA: Brian Choi Complexity •  By far the most important concept you need to get out of this class. –  “Hey, I came up with this smart algorithm that runs in linear (me.” –  “This algorithm sucks – in the worst case it runs in exponen(al (me.” –  “This algorithm requires sor.ng of items, thus it won’t get beNer than O(n log n)” –  “This problem is intractable… I just proved that there can’t be any polynomial ­(me algorithm for this $#!*.” •  You must to be able to comprehend these conversa.ons – even if you’re an EE major! Complexity? •  In the first week of class I said we will not only build programs that work, but programs that work efficiently. •  How do we assess “efficiency”? •  Another way of saying the same thing is “how do we quan.fy how complex a program/code is”? Size of input vs. Running .me •  A program gets some kind of input, does something meaningful (hopefully), and produces some output. •  Naturally, the size of input determines how long a program runs. –  Sor.ng an array of 1000 items should run a lot longer than sor.ng an array of 10 items. –  But how much longer? •  Some.mes, the size of input doesn’t maNer. –  Figuring out the size of a string – always the same. Big Ques.on •  Given an input of size n, approximately how long does the algorithm take to finish the task, in terms of n? Big ­O nota.on Big ­O •  If your algorithm takes … –  about n steps: O(n) –  about 2n steps: O(n) –  about n2 steps: O(n2) –  about 3n2 + n steps: O(n2) –  about 2n steps: O(2n) •  Which one grows faster in the long term? –  10000n vs. 0.00001n2 Big ­O •  •  •  •  •  O(2n) Takes about n steps: O(n) Takes about 2n steps: O(n) Takes about n2 steps: O(n2) Takes about 3n2 + n steps: O(n2) Takes about 2n steps: O(2n) running time O(n3) O(n2) O(n) O(n) n Efficiency •  Algorithms are considered efficient if it runs reasonably fast even with a large input. •  Algorithms are considered inefficient if it runs slow even with a small input. •  For more “precise” defini.ons, take CS180 and 181. In this class, we’ll use these simple intui.ons to analyze algorithms. Linear Search •  Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n ­1) Best case? { if (arr[i] == v) Average case? return i; } Worst case? return  ­1; } Linear Search •  Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n ­1) Best case? { v is found in the first slot (a[0]) – takes 1 step if (arr[i] == v) Average case? takes n/2 = ½n steps (assuming v can appear at return i; any location in the array with an equal probability) } Worst case? return  ­1; not found – n steps } Linear Search •  Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n ­1) Best case? { v is found in the first slot (a[0]) – O(1) if (arr[i] == v) Average case? O(n) (assuming v can appear at any location return i; in the array with an equal probability) } Worst case? return  ­1; not found – O(n) } Linear Search •  Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n ­1) Best case? { v is found in the first slot (a[0]) – O(1) if (arr[i] == v) Average case? O(n) (assuming v can appear at any location return i; in the array with an equal probability) } Worst case? return  ­1; not found – O(n) } Usually, assessing complexity involves counting nested loops – only one loop means it’s likely to be O(n) All Pairs •  Find all ordered pairs all_pairs( array arr, size n ) { for (i=0 to n ­1) for ( j=0 to n ­1) if (i ≠ j) print “{arr[i] arr[j]}”; } Best case? Average case? Worst case? All Pairs •  Find all ordered pairs all_pairs( array arr, size n ) { for (i=0 to n ­1) for ( j=0 to n ­1) if (i ≠ j) print “{arr[i] arr[j]}”; } Best case? Average case? Worst case? All O(n2) Basic Opera.ons (O(1) opera.ons) • Addi.on, Subtrac.on, Mul.plica.on, Division •  Comparison, Assignment •  Input, Output of a small value (e.g. short string, an integer, etc.) •  If O(1) opera.ons are repeatedly done in a loop for n .mes, then that loop is O(n). •  If this loop is within a loop that repeats n .mes, then this outer loop takes O(n2). Big ­O Arithme.c •  More generally: •  If things happen sequen.ally, we add big ­O’s. –  O(1) opera.on followed by O(1) = O(1) + O(1) = O(1) •  If one thing happens within another, then we mul.ply big ­O’s. –  O(1) opera.on within a O(n) loop = O(1) x O(n) = O(n) •  O(f(n)) + O(g(n)) = O(max(f(n), g(n)) •  O(f(n)) x O(g(n)) = O(f(n) x g(n)) Order of Complexity Big O Name n = 128 O(1) constant 1 O(log n) logarithmic 7 O(n) linear 128 O(n log n) “n log n” 896 O(n2) quadra.c 16192 O(nk), k >= 1 polynomial O(2n) exponen.al 1040 O(n!) factorial 10214 Linear Search •  Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n ­1) O(n) Best case? { v is found in the first slot (a[0]) – O(1) if (arr[i] == v) Average case? O(1) O(n) (assuming v can appear at any location return i; in the array with an equal probability) } Worst case? return  ­1; not found – O(n) } Usually, assessing complexity involves counting nested loops – only one loop means it’s likely to be O(n) All Pairs •  Find all ordered pairs all_pairs( array arr, size n ) { for (i=0 to n ­1) O(n) for ( j=0 to n ­1) O(n) if (i ≠ j) print “{arr[i] arr[j]}”; } O(1) Best case? Average case? Worst case? All O(n2) Binary Search •  Find an item v in a sorted array binary_search( array arr, value v, start index s, end index e ) { if (s > e) return  ­1 Best case? find the middle point i = (s + e) / 2 if (arr[i] == v) return i Average case? else if (arr[i] < v) return binary_search(arr, v, i+1, e) Worst case? else return binary_search(arr, v, s, i ­1) } Binary Search •  Find an item v in a sorted array binary_search( array arr, value v, start index s, end index e ) { if (s > e) return  ­1 Best case? O(1) – by now you can see the best find the middle point I = (s + e) / 2 case analysis doesn’t help much if (arr[i] == v) return i Average case? O(log n) else if (arr[i] < v) return binary_search(arr, v, i+1, e) Worst case? else O(log n) return binary_search(arr, v, s, i ­1) } Binary Search •  At every itera.on, we divide the search space in half. •  You keep dividing the size by 2 un.l it becomes 1. •  It takes ~ log2 n steps to get to 1. •  log10 n = (1 / log10 2) log2 n •  So the base doesn’t maNer. Sor.ng Algorithms •  We now switch gears and discuss some well known sor.ng algorithms. Selec.on Sort 4 3 1 5 2 1 3 4 5 2 1 2 4 5 3 1 2 3 5 4 1 2 3 4 5 •  Find the smallest item in the unsorted por.on, and place it in front. •  What is the running .me (complexity) of this algorithm? Inser.on Sort 4 3 1 5 2 3 4 1 5 2 1 3 4 5 2 1 3 4 5 2 1 2 3 4 5 •  Pick one from the unsorted part, and place it in the “right” posi.on in the sorted part. •  Best case? •  Avg. case? •  Worst case? Inser.on Sort 4 3 1 5 2 3 4 1 5 2 1 3 4 5 2 1 3 4 5 2 1 2 3 4 5 •  Pick one from the unsorted part, and place it in the “right” posi.on in the sorted part. •  Best case? O(n) •  Avg. case? O(n2) •  Worst case? O(n2) Merge Sort 3 7 6 5 8 2 1 4 3 7 6 5 3 7 3 7 6 5 6 5 8 2 1 4 8 2 8 2 1 4 1 4 Keep splitting Merge Sort 3 7 3 7 6 5 5 6 3 5 6 7 8 2 2 8 1 4 1 4 1 2 4 8 1 2 3 4 5 6 7 8 Merge Merge Sort: Running Time? 3 7 6 5 8 2 1 4 O(n) 3 7 5 6 2 8 1 4 O(n) 3 5 6 7 1 2 4 8 O(n) 1 2 3 4 5 6 7 8 O(log n) O(n)O(log n) = O(n log n) General Sor.ng: Running Time •  O(n log n) is faster than O(n2) – merge sort is more efficient than selec.on sort or inser.on sort. •  O(n log n) is the best average complexity that a general (comparison) sor.ng algorithm can get (assuming you know nothing about the data set). •  If more informa.on about the data set is provided, it’s possible to sort things almost linearly. Quick Sort 4 3 1 5 2 3 1 2 4 5 3 1 2 1 2 3 1 3 5 •  Pick a pivot, and move numbers that are less than the pivot to front, and ones that are greater than the pivot to end. •  On average, O(n log n) •  Depending on how you pick your pivots, it can be as bad as O(n2) Quick Ques.ons •  Given an unsorted array of n items, what is the best you can do to search an item, if you are to run this search only once? •  Given an unsorted array of n items, what is the best you can do to search an item, if you are to run this search 100 .mes? (assume: n >> 100) •  Given an unsorted array of n items, what is the best you can do to search an item, if you are to run this search n .mes? ...
View Full Document

This note was uploaded on 02/09/2012 for the course CS 32 taught by Professor Davidsmallberg during the Spring '08 term at UCLA.

Ask a homework question - tutors are online