Lectures on Graph Algorithms: searching, testing and sorting COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski Graph Algorithms 1 Overview Previous lectures: Recursive method (searching, sorting, ) This lecture: algorithms on graphs, in particular Representation of graphs Breadth-First Search (BFS) Depth-First Search (DFS) Testing bipartiteness Testing for (directed) cycles Ordering nodes in acyclic directed graphs Graph Algorithms 2 How to represent graphs? Given graph G = (V,E) , how to represent it? 1. Adjacency matrix: ith node is represented as ith row and ith column, edge between ith and jth nodes is represented as 1 in row i and column j, and vice versa (0 if there is no edge between i and j) 2. Adjacency list: nodes are arranged as array/list, each node record has the list of its neighbors attached 1 2 3 4 1 2 3 4 1 0 1 1 1 2 1 0 1 0 3 1 1 0 1 4 1 0 1 0 Adjacency matrix Graph Algorithms 1 2 3 4 2 1 1 1 3 4 3 2 4 3 Adjacency list 3 Adjacency matrix Advantages: Check in constant time if an edge belongs to the graph Representation takes memory O(n2) - versus O(m+n) Examining all neighbors of a given node requires time O(n) - versus O(m/n) in average 1 2 3 4 1 0 1 1 1 2 1 0 1 0 3 1 1 0 1 4 1 0 1 0 Adjacency matrix Graph Algorithms Disadvantages: Disadvantages especially for sparse graphs! 1 2 3 4 1 2 3 4 2 1 1 1 3 4 3 2 4 3 4 Adjacency list Adjacency list Advantages: Representation takes memory O(m+n) - versus O(n2) Examining all neighbors of a given node requires time O(m/n) in average - versus O(n) Check if an edge belongs to the graph requires time O(m/n) in average - versus O(1) 1 2 3 4 1 0 1 1 1 2 1 0 1 0 3 1 1 0 1 4 1 0 1 0 Adjacency matrix Graph Algorithms Disadvantages: Advantages especially for sparse graphs! 1 2 3 4 1 2 3 4 2 1 1 1 3 4 3 2 4 3 5 Adjacency list Breadth-First Search (BFS) Given graph G = (V,E) of diameter D and a root node Goal: find a spanning tree such that the distances between nodes and the root are the same as in graph G Idea of the algorithm: For layers i = 0,1, ,D while there is a node in layer i + 1 not added to the partial BFS tree : add the node and any edge connecting it with layer i root layer 0 layer 2 layer 1 Graph Algorithms 6 Implementing BFS Structures: Adjacency list Lists L0,L1, ,LD Array Discovered[1 n] Algorithm: Set L0 = {root} For layers i = 0,1, ,D Initialize empty list Li+1 For each node v in Li take next edge adjacent to v and if its second end w is not marked as Discovered then add w to Li+1 and {v,w} to partial BFS root layer 0 layer 2 layer 1 Graph Algorithms 7 Analysis of BFS Correctness: By induction on layer number: each node in layer i is in the list Li Each node w in layer i has its predecessor on the distance path in layer i-1, consider the first of such predecessors v in list Li-1: w is added to list Li by the step where it is considered as the neighbor of v Complexity: Time: O(m+n) - each edge is considered at most twice - since it occurs twice on the adjacency list Memory: O(m+n) - adjacency list takes O(m+n), lists L0,L1, ,LD have at most n elements in total, array Discovered has n elements root layer 0 layer 2 layer 1 Graph Algorithms 8 Depth-First Search (DFS) Given graph G = (V,E) of diameter D and a root node Goal: find a spanning tree such that each edge in graph G corresponds to the ancestor relation in the tree Recursive idea of the algorithm: Repeat until no neighbor of the root is free Select a free neighbor v of the root and add the edge from root to v to partial DFS tree Recursively find a DFS* in graph G restricted to free nodes and node v as the root* and add it to partial DFS tree root* root Graph Algorithms root** 9 Implementing DFS Data structures: root Adjacency list List (stack) S Array Discovered[1 n] root* Algorithm: Set S = {root} Repeat until S is empty Consider the top element v in S root** if v.next is not Discovered then put v.next into the stack S and set v := v.next and v.next to be the first neighbour of v if v.next is Discovered then set v.next to be the next neighbour of v if such neighbour exists otherwise remove v from the stack, add edge {v,z} to partial DFS, where z is the current top element in S Remark: after considering the neighbor of node v we remove this neighbor from adjacency list to avoid considering it many times! Graph Algorithms 10 Analysis of DFS Correctness: By induction on the number of recursive calls: Consider edge {v,w} in G : suppose v joints DFS before w. Then w is free and since it is in a connected component with v it will be in DFS* with root* (which is v), so v is the ancestor of w Complexity: Time: O(m+n) - each edge is considered at most twice - while adding or removing from the stack Memory: O(m+n) - adjacency list consumes O(m+n), stack S and array Discovered have at most n elements each root root* Graph Algorithms root** 11 Textbook and Exercises READING: Chapter 3 Graphs , Sections 3.2 and 3.3 EXERCISES: Prove that obtained DFS and BFS trees are spanning trees Prove that if DFS and BFS trees in graph G are the same, than G is also the same like they (does not contain any other edge). Prove that if G has n nodes, where n is even, and each node has at least n/2 neighbours then G is connected. Graph Algorithms 12 Testing graph properties Testing bipartiteness Testing for directed cycles Graph Algorithms 13 How to represent graphs? Given graph G = (V,E) , how to represent it? 1. Adjacency matrix: ith node is represented as ith row and ith column, edge between ith and jth nodes is represented as 1 in row i and column j, and vice versa (0 if there is no edge between i and j) 2. Adjacency list: nodes are arranged as array/list, each node record has the list of its neighbours attached 1 2 3 4 1 2 3 4 1 0 1 1 1 2 1 0 1 0 3 1 1 0 1 4 1 0 1 0 Adjacency matrix Graph Algorithms 1 2 3 4 2 1 1 1 3 4 3 2 4 3 14 Adjacency list Adjacency list Advantages: Representation takes memory O(m+n) - versus O(n2) Examining all neighbours of a given node requires time O(m/n) in average - versus O(n) Check if an edge belongs to the graph requires time O(m/n) in average - versus O(1) 1 2 3 4 1 0 1 1 1 2 1 0 1 0 3 1 1 0 1 4 1 0 1 0 Adjacency matrix Graph Algorithms Disadvantages: Advantages especially for sparse graphs! 1 2 3 4 1 2 3 4 2 1 1 1 3 4 3 2 4 3 15 Adjacency list Bipartiteness Graph G = (V,E) is bipartite iff it can be partitioned into two sets of nodes A and B such that each edge has one end in A and the other end in B Alternatively: Graph G = (V,E) is bipartite iff all its cycles have even length Graph G = (V,E) is bipartite iff nodes can be coloured using two colours Question: given a graph represented as an adjacency list, how to test if the graph is bipartite? Note: graphs without cycles (trees) are bipartite bipartite: non bipartite Graph Algorithms 16 Testing bipartiteness Method: use BFS search tree Recall: BFS is a rooted spanning tree having the same layers as the original graph G (each node has the same distance from the root in BFS tree and in graph G) Algorithm: Run BFS search and colour all nodes in odd layers red, others blue Go through all edges in adjacency list and check if each of them has two different colours at its ends - if so then G is bipartite, otherwise it is not We use the following alternative definitions in the analysis: Graph G = (V,E) is bipartite iff all its cycles have even length, or Graph G = (V,E) is bipartite iff it has no odd cycle non bipartite bipartite Graph Algorithms 17 Testing bipartiteness - why it works Property of layers: Every edge is either between two consecutive layers or in a single layer (two ends are in the same layer) - it follows from BFS property 1. Suppose that graph G is not bipartite. Hence there is an odd cycle. This cycle must have an edge in one layer, and so the ends of this edge have the same colour. Thus the algorithm answers not bipartite correctly. 2. Suppose that graph G is bipartite. Hence all its cycles have even length. Suppose, to the contrary, that the algorithm answers incorrectly that G is not bipartite. Hence it found a monochromatic edge. This edge must be in one layer. Consider one of such edges which is in the smallest possible layer. Consider a cycle containing this edge and the BFS path between its two ends. The length of this cycle is odd, thus G is not bipartite. Contradiction! bipartite not bipartite Graph Algorithms 18 Complexity Time: O(m+n) BFS search: O(m+n) Checking colours of all edges: O(m+n) Memory: O(m+n) Adjacency list with colour labels: O(m+n) Array Active and lists Li for BFS algorithm: O(n) Graph Algorithms 19 Testing for directed cycles Directed graph G (edges have direction - one end is the beginning, the other one is the end of the edge) Reversed graph Gr has the same nodes but each edge is a reversed edge from graph G Representing directed graph: adjacency list - each node has two lists: of in-coming and out-coming edges/neighbours Problem: does the graph have a directed cycle? Graph Algorithms 20 How to represent directed graphs? Given directed graph G = (V,E) , how to represent it? 1. Adjacency matrix: ith node is represented as ith row and ith column, edge from ith to jth node is represented as 1 in row i and column j, (0 if there is no directed edge from i to j) 2. Adjacency list: nodes are arranged as array/list, each node record has two lists: one stores in-neighbours, the other stores out-neighbours 1 2 3 4 1 2 3 4 4 1 2 3 4 1 0 1 1 1 1 2 3 2 0 0 1 0 4 2 1 3 3 0 0 0 0 1 4 1 3 4 1 0 1 0 Adjacency matrix Adjacency list Graph Algorithms 21 Testing cycles Technique: use directed DFS tree for graph G Algorithm: Find a node with no incoming edges - if not found answer cycled Build a directed DFS tree - consider only edges in proper direction. If during building the DFS tree two nodes which are already in the stack are considered then answer cycled, otherwise answer acyclic General remark: if graph G is not connected then do the same until no free node remains Graph Algorithms 22 Testing cycles - analysis Property of acyclic graph: There is a node with no incoming edges 1. Suppose that graph G is acyclic. Hence there is no directed cycle and then while building a directed DFS tree we never try to explore the already visited node. Answer acyclic is then proper. 2. Suppose that graph G is not acyclic. It follows that there is a cycle in it. Let v be the first node in a cycle visited by DFS search. By the property of DFS, all nodes from this cycle will be reached during the search rooted in v, and so the in-neighbour of v from this cycle will attempt to visit v, which causes that the algorithm stops with the correct answer cycled. Graph Algorithms 23 Complexity Time: O(m+n) Finding a node with no incoming edges: O(m+n) DFS search with checking cycle condition: O(m+n) Memory: O(m+n) Adjacency list: O(m+n) Array Active and stack S for DFS algorithm: O(n) Graph Algorithms 24 Textbook and Exercises READING: Chapter 3, Sections 3.4 and 3.5 EXERCISES: Prove the following property of layers: every edge is either between two consecutive layers or in a single layer (two ends are in the same layer). Prove the following property of acyclic graphs: there is a node with no incoming edges. Is this property true for out-coming edges? Design and analyse time and memory taken by BFS for directed graphs. What happens if you try to use a different method of searching to check bipartiteness or acyclicity? Graph Algorithms 25 Ordering nodes in acyclic graphs (topological order) Graph Algorithms 26 Directed Acyclic Graphs (DAG) Directed graph G (edges have directions - one end is the beginning, the other one is the end of the edge) Reversed graph Gr has the same nodes but each edge is a reversed edge from graph G Test if a given direct graph is acyclic (DAG): last lecture, directed DFS approach in time and memory O(m+n) Problem: is it possible to order all nodes topologically, which means that if (v,w) is a directed edge in G then v is before w? Note: there may be many topological orders graph G Graph Algorithms reversed graph Gr 27 Opposite property Fact: If graph G has a topological order then it is DAG. Proof: Suppose, to the contrary, that G has a topological order and it also has a cycle. Let v be the smallest element in the cycle according to the topological order. It follows that its predecessor in the cycle must be...