CSC 325 Chapter 03, Spring 2012

Binary Search Trees 1 Chapter 3 Search Trees 6 9 2 4 1 8 < > = Binary Search Trees 2 Binary Search (§3.1.1) Binary search performs operation findElement(k) on a dictionary implemented by means of an array-based sequence, sorted by key  at each step, the number of candidate items is halved  terminates after O(log n) steps Example: findElement(7) 1 3 4 5 7 8 9 11 14 16 18 19 1 3 4 5 7 8 9 11 14 16 18 19 1 3 4 5 7 8 9 11 14 16 18 19 1 3 4 5 7 8 9 11 14 16 18 19 0 0 0 0 m l h m l h m l h l=m =h Binary Search Trees 3 Binary Search Tree (§3.1.2) A binary search tree is a binary tree storing keys (or key-element pairs) at its internal nodes and satisfying the following property:  Let u, v, and w be three nodes such that u is in the left subtree of v and w is in the right subtree of v. We have key(u)  key(v)  key(w) External nodes do not store items An inorder traversal of a binary search trees visits the keys in increasing order 6 9 2 4 1 8 Binary Search Trees 4 Search (§3.1.3) To search for a key k, we trace a downward path starting at the root The next node visited depends on the outcome of the comparison of k with the key of the current node If we reach a leaf, the key is not found and we return NO_SUCH_KEY Example: findElement(4) Algorithm findElement(k, v) if T.isExternal (v) return NO_SUCH_KEY if k < key(v) return findElement(k, T.leftChild(v)) else if k = key(v) return element(v) else { k > key(v) } return findElement(k, T.rightChild(v)) 6 9 2 4 1 8 < > = Binary Search Trees 5 Insertion (§3.1.4) To perform operation insertItem(k, o), we search for key k Assume k is not already in the tree, and let let w be the leaf reached by the search We insert k at node w and expand w into an internal node Example: insert 5 6 9 2 4 1 8 6 9 2 4 1 8 5 < > > w w Binary Search Trees 6 Deletion (§3.1.5) To perform operation removeElement(k), we search for key k Assume key k is in the tree, and let let v be the node storing k If node v has a leaf child w, we remove v and w from the tree with operation removeAboveExternal(w) Example: remove 4 6 9 2 4 1 8 5 v w 6 9 2 5 1 8 < > Binary Search Trees 7 Deletion (cont.) We consider the case where the key k to be removed is stored at a node v whose children are both internal  we find the internal node w that follows v in an inorder traversal  we copy key(w) into node v  we remove node w and its left child z (which must be a leaf) by means of operation removeAboveExternal(z) Example: remove 3 3 1 8 6 9 5 v w z 2 5 1 8 6 9 v 2 Binary Search Trees 8 Performance (§3.1.6) Consider a dictionary with n items implemented by means of a binary search tree of height h  the space used is O(n)  methods findElement , insertItem and removeElement take O(h) time The height h is O(n) in the worst case and O(log n) in the best case Binary Search Trees 9 AVL Trees 6 3 8 4 v z Binary Search Trees 10 AVL Tree Definition AVL trees are balanced. An AVL Tree is a binary search tree such that for every internal node v of T, the heights of the children of v can differ by at most 1. Fact: The height of an AVL tree storing n keys is O(log n). 88 44 17 78 32 50 48 62 2 4 1 1 2 3 1 1 An example of an AVL tree where the heights are shown next to the nodes: Binary Search Trees 11 Insertion in an AVL Tree Insertion is as in a binary search tree Always done by expanding an external node. Example: 44 17 78 32 50 88 48 62 54 w b=x a=y c=z 44 17 78 32 50 88 48 62 before insertion after insertion Binary Search Trees 12 Trinode Restructuring let (a,b,c) be an inorder listing of x, y, z perform the rotations needed to make b the topmost node of the three b=y a=z c=x T0 T1 T2 T3 b=y a=z c=x T0 T1 T2 T3 c=y b=x a=z T0 T1 T2 T3 b=x c=y a=z T0 T1 T2 T3 case 1: single rotation (a left rotation about a) case 2: double rotation (a right rotation about c, then a left rotation about a) (other two cases are symmetrical) Binary Search Trees 13 Insertion Example, continued 88 44 17 78 32 50 48 62 2 5 1 1 3 4 2 1 54 1 T0 T2 T3 x y z 2 3 4 5 6 7 1 88 44 17 78 32 50 48 62 2 4 1 1 2 2 3 1 54 1 T0 T1 T2 T3 x y z unbalanced... ...balanced 1 2 3 4 5 6 7 T1 Binary Search Trees 14 Restructuring (as Single Rotations) Single Rotations: T0 T1 T2 T3 c = x b = y a = z T0 T1 T2 T3 c = x b = y a = z single rotation T3 T2 T1 T0 a = x b = y c = z T0 T1 T2 T3 a = x b = y c = z single rotation Binary Search Trees 15 Restructuring (as Double Rotations) double rotations: double rotation a = z b = x c = y T0 T2 T1 T3 T0 T2 T3 T1 a = z b = x c = y double rotation c = z b = x a = y T0 T2 T1 T3 T0 T2 T3 T1 c = z b = x a = y Binary Search Trees 16 Removal in an AVL Tree Removal begins as in a binary search tree, which means the node removed will become an empty external node. Its parent, w, may cause an imbalance. Example: 44 17 78 32 50 88 48 62 54 44 17 78 50 88 48 62 54 before deletion of 32 after deletion Binary Search Trees 17 Rebalancing after a Removal Let z be the first unbalanced node encountered while travelling up the tree from w. Also, let y be the child of z with the larger height, and let x be the child of y with the larger height. We perform restructure(x) to restore balance at z. As this restructuring may upset the balance of another node higher in the tree, we must continue checking for balance until the root of T is reached 44 17 78 50 88 48 62 54 w c=x b=y a=z 44 17 78 50 88 48 62 54 Binary Search Trees 18 Running Times for AVL Trees a single restructure is O(1)  using a linked-structure binary tree find is O(log n)  height of tree is O(log n), no restructures needed insert is O(log n)  initial find is O(log n)  Restructuring up the tree, maintaining heights is O(log n) remove is O(log n)  initial find is O(log n)  Restructuring up the tree, maintaining heights is O(log n) Binary Search Trees 19 (2,4) Trees 9 10 14 2 5 7 Binary Search Trees 20 Outline and Reading Multi-way search tree (§3.3.1)  Definition  Search (2,4) tree (§3.3.2)  Definition  Search  Insertion  Deletion Comparison of dictionary implementations Binary Search Trees 21 Multi-Way Search Tree A multi-way search tree is an ordered tree such that  Each internal node has at least two children and stores d -1 key-element items (ki, oi), where d is the number of children  For a node with children v1 v2 … vd storing keys k1 k2 … kd-1  keys in the subtree of v1 are less than k1  keys in the subtree of vi are between ki-1 and ki (i = 2, …, d - 1)  keys in the subtree of vd are greater than kd-1  The leaves store no items and serve as placeholders 11 24 2 6 8 15 30 27 32 Binary Search Trees 22 Multi-Way Inorder Traversal We can extend the notion of inorder traversal from binary trees to multi-way search trees Namely, we visit item (ki, oi) of node v between the recursive traversals of the subtrees of v rooted at children vi and vi + 1 An inorder traversal of a multi-way search tree visits the keys in increasing order 11 24 2 6 8 15 30 27 32 1 3 5 7 9 11 13 19 15 17 2 4 6 14 18 8 12 10 16 Binary Search Trees 23 Multi-Way Searching Similar to search in a binary search tree A each internal node with children v1 v2 … vd and keys k1 k2 … kd-1  k = ki (i = 1, …, d - 1): the search terminates successfully  k < k1: we continue the search in child v1  ki-1 < k < ki (i = 2, …, d - 1): we continue the search in child vi  k > kd-1: we continue the search in child vd Reaching an external node terminates the search unsuccessfully Example: search for 30 11 24 2 6 8 15 30 27 32 Binary Search Trees 24 (2,4) Tree A (2,4) tree (also called 2-4 tree or 2-3-4 tree) is a multi-way search with the following properties  Node-Size Property: every internal node has at most four children  Depth Property: all the external nodes have the same depth Depending on the number of children, an internal node of a (2,4) tree is called a 2-node, 3-node or 4-node 10 15 24 2 8 12 27 32 18 Binary Search Trees 25 Height of a (2,4) Tree Theorem: A (2,4) tree storing n items has height O(log n) Proof:  Let h be the height of a (2,4) tree with n items  Since there are at least 2i items at depth i = 0, … , h - 1 and no items at depth h, we have n  1 + 2 + 4 + … + 2h-1 = 2h - 1  Thus, h  log (n + 1) Searching in a (2,4) tree with n items takes O(log n) time 1 2 2h-1 0 items 0 1 h-1 h depth Binary Search Trees 26 Insertion We insert a new item (k, o) at the parent v of the leaf reached by searching for k  We preserve the depth property but  We may cause an overflow (i.e., node v may become a 5-node) Example: inserting key 30 causes an overflow 27 32 35 10 15 24 2 8 12 18 10 15 24 2 8 12 27 30 32 35 18 v v Binary Search Trees 27 Overflow and Split We handle an overflow at a 5-node v with a split operation:  let v1 … v5 be the children of v and k1 … k4 be the keys of v  node v is replaced nodes v' and v"  v' is a 3-node with keys k1 k2 and children v1 v2 v3  v" is a 2-node with key k4 and children v4 v5  key k3 is inserted into the parent u of v (a new root may be created) The overflow may propagate to the parent node u 15 24 12 27 30 32 35 18 v u v1 v2 v3 v4 v5 15 24 32 12 27 30 18 v' u v1 v2 v3 v4 v5 35 v" Binary Search Trees 28 Analysis of Insertion Algorithm insertItem(k, o) 1. We search for key k to locate the insertion node v 2. We add the new item (k, o) at node v 3. while overflow(v) if isRoot(v) create a new empty root above v v  split(v) Let T be a (2,4) tree with n items  Tree T has O(log n) height  Step 1 takes O(log n) time because we visit O(log n) nodes  Step 2 takes O(1) time  Step 3 takes O(log n) time because each split takes O(1) time and we perform O(log n) splits Thus, an insertion in a (2,4) tree takes O(log n) time Binary Search Trees 29 Deletion We reduce deletion of an item to the case where the item is at the node with leaf children Otherwise, we replace the item with its inorder successor (or, equivalently, with its inorder predecessor) and delete the latter item Example: to delete key 24, we replace it with 27 (inorder successor) 27 32 35 10 15 24 2 8 12 18 32 35 10 15 27 2 8 12 18 Binary Search Trees 30 Underflow and Fusion Deleting an item from a node v may cause an underflow, where node v becomes a 1-node with one child and no keys To handle an underflow at node v with parent u, we consider two cases Case 1: the adjacent siblings of v are 2-nodes  Fusion operation: we merge v with an adjacent sibling w and move an item from u to the merged node v'  After a fusion, the underflow may propagate to the parent u 9 14 2 5 7 10 u v 9 10 14 u v' w 2 5 7 Binary Search Trees 31 Underflow and Transfer To handle an underflow at node v with parent u, we consider two cases Case 2: an adjacent sibling w of v is a 3-node or a 4-node  Transfer operation: 1. we move a child of w to v 2. we move an item from u to v 3. we move an item from w to u  After a transfer, no underflow occurs 4 9 6 8 2 u v w 4 8 6 2 9 u v w Binary Search Trees 32 Analysis of Deletion Let T be a (2,4) tree with n items  Tree T has O(log n) height In a deletion operation  We visit O(log n) nodes to locate the node from which to delete the item  We handle an underflow with a series of O(log n) fusions, followed by at most one transfer  Each fusion and transfer takes O(1) time Thus, deleting an item from a (2,4) tree takes O(log n) time Binary Search Trees 33 Red-Black Trees 6 3 8 4 v z Binary Search Trees 34 Outline and Reading From (2,4) trees to red-black trees (§3.3.3) Red-black tree (§ 3.3.3)  Definition  Height  Insertion  restructuring  recoloring  Deletion  restructuring  recoloring  adjustment Binary Search Trees 35 From (2,4) to Red-Black Trees A red-black tree is a representation of a (2,4) tree by means of a binary tree whose nodes are colored red or black In comparison with its associated (2,4) tree, a red-black tree has  same logarithmic time performance  simpler implementation with a single node type 2 6 7 3 5 4 4 6 2 7 5 3 3 5 OR Binary Search Trees 36 Red-Black Tree A red-black tree can also be defined as a binary search tree that satisfies the following properties:  Root Property: the root is black  External Property: every leaf is black  Internal Property: the children of a red node are black  Depth Property: all the leaves have the same black depth 9 15 4 6 2 12 7 21 Binary Search Trees 37 Height of a Red-Black Tree Theorem: A red-black tree storing n items has height O(log n) Proof:  The height of a red-black tree is at most twice the height of its associated (2,4) tree, which is O(log n) The search algorithm for a binary search tree is the same as that for a binary search tree By the above theorem, searching in a red-black tree takes O(log n) time Binary Search Trees 38 Insertion To perform operation insertItem(k, o), we execute the insertion algorithm for binary search trees and color red the newly inserted node z unless it is the root  We preserve the root, external, and depth properties  If the parent v of z is black, we also preserve the internal property and we are done  Else (v is red ) we have a double red (i.e., a violation of the internal property), which requires a reorganization of the tree Example where the insertion of 4 causes a double red: 6 3 8 6 3 8 4 z v v z Binary Search Trees 39 Remedying a Double Red Consider a double red with child z and parent v, and let w be the sibling of v 4 6 7 z v w 2 4 6 7 .. 2 .. Case 1: w is black  The double red is an incorrect replacement of a 4- node  Restructuring: we change the 4-node replacement Case 2: w is red  The double red corresponds to an overflow  Recoloring: we perform the equivalent of a split 4 6 7 z v 2 4 6 7 2 w Binary Search Trees 40 Restructuring A restructuring remedies a child-parent double red when the parent red node has a black sibling It is equivalent to restoring the correct replacement of a 4- node The internal property is restored and the other properties are preserved 4 6 7 z v w 2 4 6 7 .. 2 .. 4 6 7 z v w 2 4 6 7 .. 2 .. Binary Search Trees 41 Restructuring (cont.) There are four restructuring configurations depending on whether the double red nodes are left or right children 2 4 6 6 2 4 6 4 2 2 6 4 2 6 4 Binary Search Trees 42 Recoloring A recoloring remedies a child-parent double red when the parent red node has a red sibling The parent v and its sibling w become black and the grandparent u becomes red, unless it is the root It is equivalent to performing a split on a 5-node The double red violation may propagate to the grandparent u 4 6 7 z v 2 4 6 7 2 w 4 6 7 z v 6 7 2 w … 4 … 2 Binary Search Trees 43 Analysis of Insertion Recall that a red-black tree has O(log n) height Step 1 takes O(log n) time because we visit O(log n) nodes Step 2 takes O(1) time Step 3 takes O(log n) time because we perform  O(log n) recolorings, each taking O(1) time, and  at most one restructuring taking O(1) time Thus, an insertion in a red- black tree takes O(log n) time Algorithm insertItem(k, o) 1. We search for key k to locate the insertion node z 2. We add the new item (k, o) at node z and color z red 3. while doubleRed(z) if isBlack(sibling(parent(z))) z  restructure(z) return else { sibling(parent(z) is red } z  recolor(z) Binary Search Trees 44 Deletion To perform operation remove(k), we first execute the deletion algorithm for binary search trees Let v be the internal node removed, w the external node removed, and r the sibling of w  If either v of r was red, we color r black and we are done  Else (v and r were both black) we color r double black, which is a violation of the internal property requiring a reorganization of the tree Example where the deletion of 8 causes a double black: 6 3 8 4 v r w 6 3 4 r Binary Search Trees 45 Remedying a Double Black The algorithm for remedying a double black node w with sibling y considers three cases Case 1: y is black and has a red child  We perform a restructuring, equivalent to a transfer , and we are done Case 2: y is black and its children are both black  We perform a recoloring, equivalent to a fusion, which may propagate up the double black violation Case 3: y is red  We perform an adjustment, equivalent to choosing a different representation of a 3-node, after which either Case 1 or Case 2 applies Deletion in a red-black tree takes O(log n) time Binary Search Trees 46 Red-Black Tree Reorganization Insertion remedy double red Red-black tree action (2,4) tree action result restructuring change of 4-node representation double red removed recoloring split double red removed or propagated up Deletion remedy double black Red-black tree action (2,4) tree action result restructuring transfer double black removed recoloring fusion double black removed or propagated up h f d