0. Chapter Introduction
0.1 The Definition of Algorithms (cont.)
For a given problem, there are many algorithms for it, e.g., there are a number of algorithms for the sorting problem. 1. insertion sort
Algorithms Algorithm Analysis Algorithm Design
2. bubble sort 3. mergesort 4. quicksort 5. shellsort 6. heapsort 7. bucket sort
COMP3600/6466: Lecture 1
2008
2
COMP3600/6466: Lecture 1
2008
4
Algorithms
0.1 What is an Algorithm?
An algorithm is any well-defined computational procedure that takes some value or a set of values as input, and produces some value or a set of values as output. Example one: Sort n integers in increasing order. Input: a sequence of n numbers, a1, a2, . . . , an Output: a permutation (rearrangement) of the numbers as a , a , . . . , a , such that 1 2 n a1 a2 an. Example two: Given an integer N , determine whether it is a prime number. Input: N Output: YES if N is a prime number, otherwise, NO. Example three: Given an integer N , Print out prime numbers no greater than N . Input: N Output: a sequence of prime numbers, and no number is larger than N .
Question: Given 8 identical coins, there is a fake coin, which may be either heavier or lighter. Assume that there is a balancing-scale available, to identify that fake coin,
how many rounds of comparisons are needed? how many rounds of comparisons are required, if the number of coins is not 8 but n instead?
COMP3600/6466: Lecture 1
2008
1
COMP3600/6466: Lecture 1
2008
3
0.2 Algorithm Analysis (continued)
Insertion Sorting Important concepts used in the analysis of algorithms:
0.2 Algorithm Analysis (continued)
Input size: If the input to an algorithm is "5413133", is the input size 1, 7, 23 or
5413133?
Running time (worst-case and average-case): The running time of an algorithm
on a particular input is the number of primitive operations or steps executed. However there is more than one idea about what a primitive operation is.
Order of growth: To simplify the analysis of algorithms, we are interested in the
growth rate of the running time, i.e., we only consider the leading term of a time formula. e.g., the leading term is n2 in the expression n2 + 100n + 50000.
Algorithm Insertion(A) 0 n length[A] 1 for j 2 to n do 2 key A[ j] 3 /* Insert A[ j] into sorted sequence A[1 . . . j - 1] */ 4 i j-1 5 while i > 0 and A[i] > key do 6 A[i + 1] A[i] 7 i i-1 8 A[i + 1] key
E.g., analyze insertion sort algorithm: time and space. See page 24.
COMP3600/6466: Lecture 1
2008
6
COMP3600/6466: Lecture 1
2008
8
0.2 Algorithm Analysis
Analysis of an algorithm is predict to the resources that the algorithm requires. Resources are measured in relation to a model of computation, which is a mathematical abstraction of a computer on which the algorithm is implemented. In the following several popular computational models are listed.
0.2 Algorithm Analysis (continued)
Most analysis of algorithms of the type we will encounter in this course is done using the following assumptions.
RAM (random access machine) model with unlimited memory. Operations such as arithmetic with small integers, boolean operations, fetching
and storing small quantities, take unit time.
RAM: time and space (traditional serial computers) PRAM: parallel time, number of processors, and read-and-write restrictions
(SIMD type of parallel computers)
Worst case performance is emphasised, but average and amortised
performance is also important. Many other conventions exist (example: bit complexity) but we will not encounter them very much.
Message Passing Model: communication cost (number of messages), and
computational cost (usually the cost for local computation is ignored) (Distributed computing, peer-to-peer networking, MIMD type of machine)
Turing Machine: time and space (abstract theoretical machine)
COMP3600/6466: Lecture 1 2008 5 COMP3600/6466: Lecture 1 2008 7
Divide-and-Conquer (D&C) Paradigm
Given a problem A , if the D&C strategy is applicable, the problem is then solved using the strategy. The detailed steps are as follows. 1. Divide A into a number of subproblems with the same size roughly. 2. Conquer the subproblems by solving them recursively. For small enough subproblems, solve them directly without dividing them further. 3. Combine the solutions of the subproblems into the solution for the original problem.
Cormen, p33
COMP3600/6466: Lecture 1
2008
10
COMP3600/6466: Lecture 1
2008
12
0.3 Algorithms Design
For a given problem, there are many ways to design algorithms for it, (e.g. Insertion sort is an incremental approach). The following is a list of several popular design approaches.
Divide-and-Conquer (D&C) Paradigm (cont.)
Divide-and-Conquer (D&C) Greedy Approach Dynamic Programming Linear Programming Branch-and-Bound Pruning
COMP3600/6466: Lecture 1 2008 9
An Example of D&C Strategy: Consider the merge-sort algorithm.
Divide: divide the n-element sequence into two subsequences of n/2 elements each (or as close as possible if n is odd) Conquer: sort the two subsequences recursively using the merge sort Combine: merge the two sorted subsequences into a sorted sequence
COMP3600/6466: Lecture 1
2008
11