ADT
A Dictionary dictionary is an ADT consisting of a collection of items with operations Insert, Delete and Search (this operation may also be named Find). We search for items by the value of their keys. Typically, items may contain additional satellite data. Usually, we will assume that all keys are different, and there is an ordering (denoted by <) defined on keys.
80 / 102
Simple Implementations
We can implement the dictionary ADT using an unordered array, an ordered array or a binary search tree. Suppose the dictionary contains n items. The various operations have differing (worst-case) complexities depending on the data structure used in the implementation, as summarized in the following table: implementation unordered array ordered array binary search tree Insert (1) (n) (height) Search (n) (log n) (height) Delete (1) (n) (height)
In the case of Delete in an unordered array, we assume that we have previously performed a successful search for the item to be deleted.
81 / 102
Operations in Binary Search Trees (review)
For any node X in a binary search tree and for any descendant Y of X, we have Y.key < X.key if Y is in the left subtree of X, and Y.key > X.key if Y is in the right subtree of X. Search is basically a binary search: start at the root and follow the appropriate path down the tree until the desired key value is found or we reach an empty subtree. To Insert an item with a new key, we perform an (unsuccessful) Search, and then insert the new node at the leaf node where the search terminated. To Delete an item with a specified key, we first use Search to find the node X containing the given item. One of three possible cases then occurs.
82 / 102
Deletion in Binary Search Trees
X is a leaf node. Delete X from the tree. X has only one child. Attach the nonempty subtree of X to the parent of X, and then delete X. X has two children. Find the successor of X, say Y , which is the leftmost node in the right subtree of X. Y has no left child. Attach the right subtree of Y as the left subtree of the parent of Y . Then replace X by Y . Alternatively, find the predecessor of X, say Y , which is the rightmost node in the left subtree of X. Y has no right child. Attach the left subtree of Y as the right subtree of the parent of Y . Then replace X by Y .
83 / 102
Height of a Binary Search Tree
Suppose we build a binary search tree by performing n Insert operations, where the n items have n random (but distinct keys). It can be shown that the expected height of this random binary search tree is (log n). (A related result is shown on the next slide.) However, in the worst case, a binary search tree can have height (n). Therefore, Search, Insert and Delete all have worst-case complexity (n). Our goal is for each of these three operations to have worst-case complexity (log n). This is typically done by keeping the tree sufficiently balanced.
84 / 102
Average Internal Path Length of a Random BST
For every node X in a BST T having n nodes, define T (X) to be the number of edges in the path from the root node to X. Then define aT = X T (X). Note that aT is the total internal path length of T . The average internal path length of T is aT /n. We now consider the average (i.e., expected) value of aT over all BSTs built on a fixed set of n distinct keys by performing n Insert operations. Denote this value by an . We have the following recurrence: an = n - 1 + 1 n
n-1
(ai + an-1-i ).
i=0
We know from the analysis of QuickSort that an (n log n), so the average internal path length of a random BST is (log n).
85 / 102
AVL Trees
A node in a binary tree is balanced if the heights of its two subtrees differ by at most 1 (an empty subtree is defined to have height equal to -1). A binary search tree is an AVL tree if every node is balanced. We will label each node X of an AVL tree with a condition code as follows:
"/" means that
height(LeftSubtree(X)) = height(RightSubtree(X)) + 1
"=" means that
height(LeftSubtree(X)) = height(RightSubtree(X))
"\" means that
height(LeftSubtree(X)) = height(RightSubtree(X)) - 1
86 / 102
Height of an AVL Tree
Let ni denote the smallest number of nodes in an AVL tree of height i . It is easy to compute some small values: n0 = 1, n1 = 2, n2 = 4, n3 = 7, n4 = 12, n5 = 20. Observe that if we add 1 to each term in the sequence n1 , n2 , . . . , we get the Fibonacci sequence 2, 3, 5, 8, 13, . . . . To prove this, we observe that the following recurrence relation holds: ni = 1 + ni-1 + ni-2 . If we let gi = ni + 1, then we have gi - 1 = 1 + gi-1 - 1 + gi-2 - 1, so gi = gi-1 + gi-2 . This is the Fibonacci recurrence!
87 / 102
Height of an AVL Tree (cont.)
In fact, gi = Fi+3 , so ni = Fi+3 - 1 = i+3 - (1 - )i+3 - 1, 5
where = (1 + 5)/2. From this, it follows that ni (i ). In any AVL tree of height i on n nodes, we have n ni , so n (i ) and therefore i O(log n) = O(log n).
88 / 102
Rotations in an AVL Tree
In order to maintain the "balance" property of an AVL tree, it will be necessary to perform rotations. Suppose a node X in a tree T has left child Y and right subtree C. Also, Y has left subtree A and right subtree B. A right rotation of the node X will create a tree T in which Y has left subtree A and right child X. Furthermore, X has left subtree B and right subtree C. A left rotation of the node Y in the tree T yields the original tree T.
89 / 102
A Right Rotation in an AVL Tree
X Y
Y X
C A B
A B C
90 / 102
Double Rotations in an AVL Tree
Suppose a node X in a tree T has left child Y and right subtree D. Also, Y has left subtree A and right child Z, and Z has left subtree B and right subtree C. A right double rotation of the node X will create a tree T in which Z has left child Y and right child X. Y has left subtree A and right subtree B; Z has has left subtree C and right subtree D. A right double rotation of X is the same as a left of rotation Y followed by a right rotation of X.
91 / 102
A Right Double Rotation in an AVL Tree
X Y Y
Z X
Z
D A B C D
A
B
C
92 / 102
Insertion in an AVL Tree
1. Insert the new item as in a binary search tree. 2. Follow the path P from the newly inserted node back towards the root, updating the condition codes. 3. Suppose a node X becomes unbalanced. Let Y and Z be the child and grandchild of X (resp.) on the path P . Then:
If LeftChild(X) = Y and LeftChild(Y ) = Z, then
RightRotate(X).
If LeftChild(X) = Y and RightChild(Y ) = Z, then
RightDoubleRotate(X).
If RightChild(X) = Y and RightChild(Y ) = Z, then
LeftRotate(X).
If RightChild(X) = Y and LeftChild(Y ) = Z, then
LeftDoubleRotate(X).
93 / 102
Insertion and Deletion in an AVL Tree
It is possible to show that at most one rotation or double rotation is required per Insert operation (i.e., step (3) is executed for at most one node on the path P ). The entire Insert operation takes O(log n) time. A Delete operation is similar, but note that multiple nodes on P may require rotations and/or double rotations. Still, a Delete operation also takes O(log n) time.
94 / 102
B-trees
A B-tree of minimum degree t is a tree T that satisfies the following properties: 1. The number of keys stored in any non-leaf node (excluding the root node) is at least t - 1 and at most 2t - 1. The root node contains at least 1 and at most 2t - 1 keys. Within any node, these keys are stored in increasing order . 2. Suppose a non-leaf node X has j keys, denoted X.keys[1], . . . , X.keys[j]. Then X has exactly j + 1 children. For 2 i j, all keys in the i th subtree of X are greater than X.keys[i - 1], and for 1 i j - 1, all keys in the i th subtree of X are less than X.keys[i ]. 3. All leaf nodes are at the same level of T .
95 / 102
2-3-4 Trees
We will mainly study 2-3-4 trees, which are B-trees of minimum degree 2. Non-leaf nodes in a 2-3-4 tree contain 1, 2 or 3 keys. The algorithms we describe generalize in a sraightforward manner to B-trees of any specified minimum degree t. A 2-3-4 tree containing n keys has height h log(n + 1) - 1. This means that Search, Input and Delete can all be implemented to have complexity O(log n).
96 / 102
Searching in 2-3-4 Trees
Search is similar to a standard binary search, except that there are more options at each node. Suppose we are searching for a key whose value is k. We begin at the root node and follow an appropriate path through the tree. When we are examining a particular node X, we do search through the keys in X. If we find k then we stop (success). If we do not find k and X is a leaf node, then we stop (failure). Otherwise, we move to the appropriate child of X and continue.
97 / 102
Insertion in a 2-3-4 Tree
As in the case of a binary search tree, we first find the leaf node X where the new key k belongs. This is done using the Search operation. If X contains 1 or 2 keys, then we insert k into the node X. However, if X contains 3 keys, then it is "full" so we cannot insert k into K. In this case, we need to split X into two nodes. This is done as follows: 1. Temporarily insert k into X, creating a node with 4 keys. 2. Promote the median key in X to X's parent, say Y . 3. Split X into two nodes, X1 and X2 , each of which has one or two keys.
98 / 102
Insertion in a 2-3-4 Tree (cont.)
When we promote a key to the node Y , this node now has one more key than before. If it was already full, then we will need to split Y , using the same process. In this way, we follow a path back up the tree towards the root node, splitting all nodes that are full. If we backtrack all the way the to root node and it is also full, then we split the root node and create a new root node, increasing the height of the tree by 1. There is an alternative approach that avoids the above-described splitting by backtracking. Namely, we do pre-emptive splitting while we travel down the tree: split every full node on the way down, beginning at the root.
99 / 102
Deletion from a 2-3-4 Tree
We might want to delete a key from a non-leaf node; first, we discuss how to delete a key k from a leaf node X. 1. If X contains 2 or 3 keys, there is no problem. 2. If X contains only 1 key, we have underflow . Check if an adjacent sibling of X contains 2 or 3 keys.
2.1 If so, then we perform a transfer operation: transfer a key from the sibling of X to the parent of X; then transfer a key from the parent of X to X; and then delete k. 2.2 If not, then we perform a fusion operation: merge an adjacent sibling of X with X; transfer a key from the parent of X to X; and then delete k. Now the parent of X has one fewer child than before. If we have underflow at the parent, step 2 must be repeated.
100 / 102
Deletion from a 2-3-4 Tree (cont.)
Underflow might propagate all the way up to the root node. If there is underflow at the root node, then delete the root node. If we want to delete a key k from a non-leaf node X, we do the following: 1. Find the predecessor of k, say k , in the tree. Note that k occurs in a leaf node (it is the rightmost node in the appropriate subtree of X). 2. Exchange k and k . 3. Delete k. The Delete operation can be modified to avoid backtracking (we do not discuss how this can be done).
101 / 102
Dictionaries in External Memory
Very large dictionaries may be stored in external memory (e.g., hard disks). A disk access will load a fixed amount of information (called a disk block or a page, which could contain 4K16K bytes, for example) into main memory. Disk accesses are relatively slow. A B-tree with large minimum degree t is very well suited to implement such a dictionary, because the tree will have quite a small height (this minimizes the number of disk accesses required). In practice, a value of t in the range 1001000 might be used. A B-tree of height 2 having minimum degree t = 1000 can store more than one billion keys!
102 / 102