Unformatted text preview: CS32 Discussion Sec.on 1B Week 7 TA: Brian Choi Complexity
• By far the most important concept you need to get out of this class. – “Hey, I came up with this smart algorithm that runs in linear (me.” – “This algorithm sucks – in the worst case it runs in exponen(al (me.” – “This algorithm requires sor.ng of items, thus it won’t get beNer than O(n log n)” – “This problem is intractable… I just proved that there can’t be any polynomial
(me algorithm for this $#!*.” • You must to be able to comprehend these conversa.ons – even if you’re an EE major! Complexity?
• In the ﬁrst week of class I said we will not only build programs that work, but programs that work eﬃciently. • How do we assess “eﬃciency”? • Another way of saying the same thing is “how do we quan.fy how complex a program/code is”? Size of input vs. Running .me
• A program gets some kind of input, does something meaningful (hopefully), and produces some output. • Naturally, the size of input determines how long a program runs. – Sor.ng an array of 1000 items should run a lot longer than sor.ng an array of 10 items. – But how much longer? • Some.mes, the size of input doesn’t maNer. – Figuring out the size of a string – always the same. Big Ques.on
• Given an input of size n, approximately how long does the algorithm take to ﬁnish the task, in terms of n? Big
O nota.on Big
O
• If your algorithm takes … – about n steps: O(n) – about 2n steps: O(n) – about n2 steps: O(n2) – about 3n2 + n steps: O(n2) – about 2n steps: O(2n) • Which one grows faster in the long term? – 10000n vs. 0.00001n2 Big
O
•
•
•
•
• O(2n) Takes about n steps: O(n) Takes about 2n steps: O(n) Takes about n2 steps: O(n2) Takes about 3n2 + n steps: O(n2) Takes about 2n steps: O(2n) running
time O(n3) O(n2)
O(n) O(n) n Eﬃciency
• Algorithms are considered eﬃcient if it runs reasonably fast even with a large input. • Algorithms are considered ineﬃcient if it runs slow even with a small input. • For more “precise” deﬁni.ons, take CS180 and 181. In this class, we’ll use these simple intui.ons to analyze algorithms. Linear Search
• Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n
1) Best case?
{ if (arr[i] == v) Average case?
return i; } Worst case?
return
1; } Linear Search
• Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n
1) Best case?
{ v is found in the first slot (a[0]) – takes 1 step
if (arr[i] == v) Average case?
takes n/2 = ½n steps (assuming v can appear at
return i; any location in the array with an equal probability)
} Worst case?
return
1; not found – n steps
} Linear Search
• Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n
1) Best case?
{ v is found in the first slot (a[0]) – O(1)
if (arr[i] == v) Average case?
O(n) (assuming v can appear at any location
return i; in the array with an equal probability)
} Worst case?
return
1; not found – O(n)
} Linear Search
• Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n
1) Best case?
{ v is found in the first slot (a[0]) – O(1)
if (arr[i] == v) Average case?
O(n) (assuming v can appear at any location
return i; in the array with an equal probability)
} Worst case?
return
1; not found – O(n)
} Usually, assessing complexity involves counting nested loops
– only one loop means it’s likely to be O(n) All Pairs
• Find all ordered pairs all_pairs( array arr, size n ) { for (i=0 to n
1) for ( j=0 to n
1) if (i ≠ j) print “{arr[i] arr[j]}”; } Best case?
Average case?
Worst case? All Pairs
• Find all ordered pairs all_pairs( array arr, size n ) { for (i=0 to n
1) for ( j=0 to n
1) if (i ≠ j) print “{arr[i] arr[j]}”; } Best case?
Average case?
Worst case? All O(n2) Basic Opera.ons (O(1) opera.ons)
• Addi.on, Subtrac.on, Mul.plica.on, Division • Comparison, Assignment • Input, Output of a small value (e.g. short string, an integer, etc.) • If O(1) opera.ons are repeatedly done in a loop for n .mes, then that loop is O(n). • If this loop is within a loop that repeats n .mes, then this outer loop takes O(n2). Big
O Arithme.c
• More generally: • If things happen sequen.ally, we add big
O’s. – O(1) opera.on followed by O(1) = O(1) + O(1) = O(1) • If one thing happens within another, then we mul.ply big
O’s. – O(1) opera.on within a O(n) loop = O(1) x O(n) = O(n) • O(f(n)) + O(g(n)) = O(max(f(n), g(n)) • O(f(n)) x O(g(n)) = O(f(n) x g(n)) Order of Complexity
Big O Name n = 128 O(1) constant 1 O(log n) logarithmic 7 O(n) linear 128 O(n log n) “n log n” 896 O(n2) quadra.c 16192 O(nk), k >= 1 polynomial O(2n) exponen.al 1040 O(n!) factorial 10214 Linear Search
• Unsorted array – look for an item linear_search( array arr, size n, value v ) { for (i=0 to n
1) O(n)
Best case?
{ v is found in the first slot (a[0]) – O(1)
if (arr[i] == v) Average case?
O(1)
O(n) (assuming v can appear at any location
return i; in the array with an equal probability)
} Worst case?
return
1; not found – O(n)
} Usually, assessing complexity involves counting nested loops
– only one loop means it’s likely to be O(n) All Pairs
• Find all ordered pairs all_pairs( array arr, size n ) { for (i=0 to n
1) O(n)
for ( j=0 to n
1) O(n)
if (i ≠ j) print “{arr[i] arr[j]}”; } O(1) Best case?
Average case?
Worst case? All O(n2) Binary Search
• Find an item v in a sorted array binary_search( array arr, value v, start index s, end index e ) { if (s > e) return
1 Best case?
ﬁnd the middle point i = (s + e) / 2 if (arr[i] == v) return i Average case?
else if (arr[i] < v) return binary_search(arr, v, i+1, e) Worst case?
else return binary_search(arr, v, s, i
1) } Binary Search
• Find an item v in a sorted array binary_search( array arr, value v, start index s, end index e ) { if (s > e) return
1 Best case?
O(1) – by now you can see the best
ﬁnd the middle point I = (s + e) / 2 case analysis doesn’t help much
if (arr[i] == v) return i Average case?
O(log n)
else if (arr[i] < v) return binary_search(arr, v, i+1, e) Worst case?
else O(log n)
return binary_search(arr, v, s, i
1) } Binary Search
• At every itera.on, we divide the search space in half. • You keep dividing the size by 2 un.l it becomes 1. • It takes ~ log2 n steps to get to 1. • log10 n = (1 / log10 2) log2 n • So the base doesn’t maNer. Sor.ng Algorithms
• We now switch gears and discuss some well known sor.ng algorithms. Selec.on Sort
4 3 1 5 2 1 3 4 5 2 1 2 4 5 3 1 2 3 5 4 1 2 3 4 5 • Find the smallest item in the unsorted por.on, and place it in front. • What is the running .me (complexity) of this algorithm? Inser.on Sort
4 3 1 5 2 3 4 1 5 2 1 3 4 5 2 1 3 4 5 2 1 2 3 4 5 • Pick one from the unsorted part, and place it in the “right” posi.on in the sorted part. • Best case? • Avg. case? • Worst case? Inser.on Sort
4 3 1 5 2 3 4 1 5 2 1 3 4 5 2 1 3 4 5 2 1 2 3 4 5 • Pick one from the unsorted part, and place it in the “right” posi.on in the sorted part. • Best case? O(n) • Avg. case? O(n2) • Worst case? O(n2) Merge Sort
3 7 6 5 8 2 1 4 3 7 6 5 3 7 3 7 6 5 6 5 8 2 1 4 8 2 8 2 1 4 1 4 Keep splitting Merge Sort
3 7 3 7 6 5 5 6 3 5 6 7 8 2 2 8 1 4 1 4 1 2 4 8 1 2 3 4 5 6 7 8 Merge Merge Sort: Running Time?
3 7 6 5 8 2 1 4 O(n) 3 7 5 6 2 8 1 4 O(n) 3 5 6 7 1 2 4 8 O(n) 1 2 3 4 5 6 7 8 O(log n) O(n)O(log n) = O(n log n) General Sor.ng: Running Time
• O(n log n) is faster than O(n2) – merge sort is more eﬃcient than selec.on sort or inser.on sort. • O(n log n) is the best average complexity that a general (comparison) sor.ng algorithm can get (assuming you know nothing about the data set). • If more informa.on about the data set is provided, it’s possible to sort things almost linearly. Quick Sort
4 3 1 5 2 3 1 2 4 5 3 1 2 1 2 3 1 3 5 • Pick a pivot, and move numbers that are less than the pivot to front, and ones that are greater than the pivot to end. • On average, O(n log n) • Depending on how you pick your pivots, it can be as bad as O(n2) Quick Ques.ons
• Given an unsorted array of n items, what is the best you can do to search an item, if you are to run this search only once? • Given an unsorted array of n items, what is the best you can do to search an item, if you are to run this search 100 .mes? (assume: n >> 100) • Given an unsorted array of n items, what is the best you can do to search an item, if you are to run this search n .mes? ...
View
Full Document
 Spring '08
 DavidSmallBerg
 Computational complexity theory, Best, worst and average case, best case, the00, average case, to00

Click to edit the document details