# CORMEN 2ND EDITION PDF

Introduction to algorithms / Thomas H. Cormen [et al.]nd ed. p. cm. Includes bibliographical references and index. ISBN (hc.: alk. paper, MIT. Contribute to xiayan/Coursera_classes development by creating an account on GitHub. This chapter and Chapter 20 present data structures known as mergeable heaps, which support the following five operations. MAKE-HEAP() creates and returns.

Author: | WILLODEAN KENTER |

Language: | English, German, Arabic |

Country: | Paraguay |

Genre: | Politics & Laws |

Pages: | 547 |

Published (Last): | 18.02.2015 |

ISBN: | 603-3-50892-681-9 |

ePub File Size: | 26.50 MB |

PDF File Size: | 10.15 MB |

Distribution: | Free* [*Sign up for free] |

Downloads: | 35223 |

Uploaded by: | LAVON |

Instructor's Manual by Thomas H. Cormen Clara Lee Erica Lin to Accompany Introduction to Algorithms Second Edition by Thomas H. Cormen Charles E. Solution Manual for: Introduction to ALGORITHMS (Second Edition) by T. Cormen, C. Leiserson, and R. Rivest. John L. Weatherwax∗. May 2. Introduction to algorithms / Thomas H. Cormen [et al.].—3rd ed. p. cm. Includes bibliographical references and index. ISBN ( hardcover.

Simple case: Case 3: See below. Cannot use the master method. Section 4. Recurrences Solution to Exercise 4. Solution to Exercise 4. This recurrence can be similarly solved. Solutions for Chapter 4: Solution to Problem Note: In each of these parts, f n has the form nk. Recurrences c. Solution to Problem [This problem is solved only for parts a, c, e, f, g, h, and i. If we were to do a straight substitution proof, it would be rather involved.

Lecture Notes for Chapter 5: Probabilistic Analysis and Randomized Algorithms [This chapter introduces probabilistic analysis and randomized algorithms. It as- sumes that the student is familiar with the basic probability material in Appendix C. These notes omit the technique of permuting an array by sorting, and they omit the starred Section 5.

Determine what the price of this strategy will be. Probabilistic Analysis and Randomized Algorithms Pseudocode to model this scenario: Worst-case analysis In the worst case, we hire all n candidates. This happens if each one is better than all who came before. In other words, if the candidates appear in increasing order of quality.

Probabilistic analysis In general, we have no control over the order in which candidates appear. We could assume that they come in a random order: Essential idea of probabilistic analysis: We must use knowledge of, or make as- sumptions about, the distribution of inputs.

Probabilistic Analysis and Randomized Algorithms Randomized algorithms We might not know the distribution of inputs, or we might not be able to model it computationally. Instead, we use randomization within the algorithm in order to impose a distribu- tion on the inputs. For the hiring problem: Change the scenario: What makes an algorithm randomized: An algorithm is randomized if its behav- ior is determined in part by values produced by a random-number generator.

Indicator random variables A simple yet powerful technique for computing the expected value of a random variable. Helpful in situations in which there may be dependence. Probabilistic Analysis and Randomized Algorithms Simple example: Slightly more complicated example: In fact, this is what the book does in equation C.

We have only the individual expectations E [X 1 ] , E [X 2 ] ,. Linearity of expectation says that the expectation of the sum equals the sum of the expectations.

The hat-check problem of Exercise 5. See the solution on page of this manual. Probabilistic Analysis and Randomized Algorithms Useful properties: Now compute E [X ]: Thus, the expected hiring cost is O ch ln n , which is much better than the worst- case cost of O nch. Randomized algorithms Instead of assuming a distribution of the inputs, we impose a distribution.

The hiring problem For the hiring problem, the algorithm is deterministic: Then only the best candidate is hired. Each time we run the algorithm, we can get a different hiring cost. Pseudocode for randomized hiring problem: Randomly permuting an array [The book considers two methods of randomly permuting an n -element array.

We omit this method from these notes. The second method is better: We present and analyze the second method in these notes. Produce a uniform random permutation each of the n! See Exercise 5. The following procedure permutes the array A[1. Probabilistic Analysis and Randomized Algorithms Idea: Given a set of n elements, a k-permutation is a sequence containing k of the n elements.

There are n! Proof Use a loop invariant: Loop invariant says that for each possible 0-permutation, subarray A[1. So, A[1. Will show that after the ith iteration, each possible i-permutation appears in A[1.

Incrementing i for the next iteration then maintains the invariant. Let E 2 be the event that the ith iteration puts xi into A[i]. Equation C. Thus, the probability that a given iteration returns 0 equals the probability that it returns 1. The number of trials until a success occurs is given by the geometric distribution, and by equation C. Solutions for Chapter 5: Solution to Exercise 5. Candidate 1 is always hired. The best candidate, i. If the best candidate is candidate 1, then that is the only candidate hired.

Letting j denote the position in the interview order of the best candidate, let F be the event in which candidates 2, 3,. Noting that the events E1 , E 2 ,. One could enumerate all n! This would be a painstaking process, and the answer would turn out to be 1. We can use indicator random variables, however, to arrive at the same answer much more easily.

Note that this is a situation in which the indicator random variables are not inde- pendent. Thus, we can use the technique of indicator random variables even in the presence of dependence. By Lemma 5. The maintenance and termination parts remain the same. The initialization part is for the subarray A[1. Since there are 3! The subtraction and addition of 1 in the index calculation is due to the 1-origin indexing. Thus, once offset is determined, so is the entire permutation.

This procedure does not produce a uniform random permutation, however, since it can produce only n different permutations. We view a toss as a success if it misses bin i and as a failure if it lands in bin i.

**Other books:**

*PRINCIPLES OF MACROECONOMICS 7TH EDITION PDF*

In order for bin i to be empty, we need n successes in n trials. Now we determine the expected number of bins with exactly one ball. We want to compute E [Vn ].

Both ways condition on the value held in the counter, but only the second way incorporates the conditioning into the expression for E[X j ].

The X j are pairwise independent, and so by equation C. Thus, by equation C. Lecture Notes for Chapter 6: Here, we bypass these attributes and use parameter values instead.

Heapsort Example: Similar argument for min-heaps. In general, heaps can be k-ary tree instead of binary. It is used to maintain the max-heap property. Heapsort [Parameter n replaces attribute heap-size[A].

If we hit a leaf, then the subtree rooted at the leaf is trivially a max-heap. In this case, the max-heap is a leaf. Building a heap The following procedure, given an unordered array, will produce a max-heap. By Exercise 6. Children of node i are indexed higher than i, so by the loop invari- ant, they are both roots of max-heaps. Decrementing i reestablishes the loop invariant at each iteration.

By the loop invariant, each node, notably node 1, is the root of a max-heap. A good approach to analysis in general is to start by proving easy bound, then try to tighten it. The heapsort algorithm Given an input array, the heapsort algorithm acts as follows: Sort an example heap on the board. O lg n Total time: Though heapsort is a great algorithm, a well-implemented quicksort usually beats it in practice.

These notes will deal with max- priority queues implemented with max-heaps. Min-priority queues are imple- mented with min-heaps similarly. A heap gives a good compromise between fast insertion but slow extraction and vice versa. Both operations take O lg n time. Actual implementations often have a handle in each heap element that allows access to an object in the application, and objects in the application often have a handle likely an array index to access the heap element.

Will examine how to implement max-priority queue operations. Finding the maximum element Getting the maximum element is easy: Extracting max element Given the array A: Heapsort Analysis: Increasing key value Given set S, element x, and new key value k: Upward path from node i has length O lg n in an n-element heap.

Exchange keys of nodes 4 and 9, then of nodes 2 and 4. Inserting into the heap Given a key k to insert into the heap: Min-priority queue operations are implemented similarly with min-heaps.

Solutions for Chapter 6: Heapsort Solution to Exercise 6. Solution to Exercise 6. Then the maximum element is somewhere else in the sub- tree, possibly even at more than one location. Let m be the index at which the maximum appears the lowest such index if the maximum appears more than once.

Since the maximum is not at the root of the subtree, node m has a parent. So our assumption is false, and the claim is true. Two subtleties to beware of: But the proof for an incomplete tree is tricky and is not derived from the proof for a complete tree. Proof By induction on h. Let x be the number of nodes at depth H —that is, the number of nodes in the bottom possibly incomplete level. Thus if n is odd, x is even, and if n is even, x is odd.

To prove the base case, we must consider separately the case in which n is even x is odd and the case in which n is odd x is even. Here are two ways to do this: First method of proving the base case: Thus see Exercise B. The latter equality holds because n is odd. Observe that we would also increase the number of leaves by 1, since we added a node to a parent that already had a child. The latter equality holds because n is even. Second method of proving the base case: Inductive step: Let n h be the number of nodes at height h in the n-node tree T.

Consider the following counterexample. Input array A: A d-ary heap can be represented in a 1-dimensional array as follows. Heapsort d. Since only parent pointers are followed, the number of children a node has is irrelevant. Increas- ing an element may make it larger than its parent, in which case it must be moved higher up in the tree.

This can be done just as for insertion, travers- ing a path from the increased node toward the root. Using Lomuto partitioning helps simplify the analysis, which uses indicator random variables in the second edition.

Description of quicksort Quicksort is based on the three-step process of divide-and-conquer. Partition A[ p. No work is needed to combine the subarrays, because they are sorted in place. Partitioning Partition subarray A[ p. All entries in A[ p. On an 8-element subarray. Lecture Notes for Chapter 7: Time for partitioning: Quicksort Performance of quicksort The running time of quicksort depends on the partitioning of the subarrays: Instead, randomly pick an element from the subarray that is being sorted.

We add this randomization by not always using A[r] as the pivot, but instead ran- domly picking an element from the subarray that is being sorted. Worst-case analysis We will prove that a worst-case split at every level produces a worst-case running time of O n 2. Second derivative with respect to q is positive.

We will now compute a bound on the overall number of comparisons. For ease of analysis: Each pair of elements is compared at most once, because elements are compared only to the pivot element, and then the pivot element is never in any later call to PARTITION. Solutions for Chapter 7: Quicksort Solution to Exercise 7. Solution to Exercise 7. Similarly, maximum depth corresponds to always taking the larger part of the par- tition, i. What randomization can do is make the chance of encountering a worst-case scenario small.

Quicksort c. Lecture Notes for Chapter 8: Sorting in Linear Time Chapter 8 overview How fast can we sort? We will prove a lower bound, then beat it by playing a different game.

Sorting in Linear Time For insertion sort on 3 elements: Each leaf is labeled by the permutation of orders that the algorithm determines. What is the length of the longest path from root to leaf?

In other words: Why is this useful? Tree is just one node, which is a leaf. Each leaf becomes parent to two new leaves. Sorting in linear time Non-comparison sorts. Counting sort Depends on a key assumption: Array A and values n and k are given as parameters.

B is assumed to be already allocated and is given as a parameter. Auxiliary storage: How big a k is practical? Probably not. Maybe, depending on n. Probably unless n is really small. Counting sort will be used in radix sort. Radix sort How IBM made its money. Card sorters, worked on one column at a time. The human operator was part of the algorithm!

Key idea: To sort d digits: Sorting in Linear Time Correctness: The stable sort on digit i leaves them in the right order. Assume that we use counting sort as the intermediate sort. How to break each key into digits? Compare radix sort to merge sort and quicksort: How does radix sort violate the ground rules for a comparison sort?

Sorting in Linear Time Bucket sort Assumes the input is generated by a random process that distributes elements uni- formly over [0, 1. Auxiliary array: Consider A[i], A[ j ]. So A[i] is placed into the same bucket as A[ j ] or into a bucket with a lower index. Sorting in Linear Time Take expectations of both sides: Used a function of key values to index into an array. Solutions for Chapter 8: Sorting in Linear Time Solution to Exercise 8.

Use the same argument as in the proof of Theorem 8. In particular, n! Proof First notice that, as pointed out in the hint, we cannot prove the lower bound by multiplying together the lower bounds for sorting each subsequence.

That would only prove that there is no faster algorithm that sorts the subsequences independently. This was not what we are asked to prove; we cannot introduce any extra assumptions. Sorting in Linear Time Now, consider the decision tree of height h for any comparison sort for S.

Since the elements of each subsequence can be in any order, any of the k! Thus, any decision tree for sorting S must have at least k! The third line comes from k! We implicitly assume here that k is even. Solution to Exercise 8. The algorithm is correct no matter what order is used! The original algorithm was stable because an element taken from A later started out with a lower index than one taken earlier.

The number of integers in the range [a. When inserting A[ j ] into the sorted sequence A[1. Heapsort and quicksort are not stable. When comparing two elements, compare them by their values and break ties by their indices. Additional space requirements: For n elements, their indices are 1.

Each can be written in lg n bits, so together they take O n lg n additional space. Additional time requirements: The worst case is when all elements are equal. The asymptotic time does not change because we add a constant amount of work to each comparison.

Radix sort sorts separately on each digit, starting from digit 1. Thus, radix sort of d digits, which sorts on digits 1,. The sort on digit d will order the elements by their dth digit.

Consider two ele- ments, a and b, with dth digits ad and bd respectively. If the intermediate sort were not stable, it might rearrange elements whose dth digits were equal—elements that were in the right order after the sort on their lower-order digits.

Sort these 2-digit numbers with radix sort. A simple change that will preserve the linear expected running time and make the worst-case running time O n lg n is to use a worst-case O n lg n -time algorithm like merge sort instead of insertion sort when sorting the buckets. For a comparison algorithm A to sort, no two input permutations can reach the same leaf of the decision tree, so there must be at least n!

Since A is a deterministic algorithm, it must always reach the same leaf when given a particular permutation as input, so at most n! Therefore exactly n! These n! Any remaining leaves will have probability 0, since they are not reached for any input. That is, we can assume that TA consists of only the n!

D T A is the sum of the decision-tree path lengths for sorting all input per- mutations, and the path lengths are proportional to the run time. Since the n! At each randomized node, pick the child with the smallest subtree the subtree with the smallest average number of comparisons on a path to a leaf. Delete all the other children of the randomized node and splice out the randomized node itself.

The randomized algorithm thus takes at least as much time on average as the corresponding deterministic one. The usual, unadorned radix sort algorithm will not solve this problem in the required time bound. The number of passes, d, would have to be the number of digits in the largest integer. We assume that the range of a single digit is constant.

Let us assume without loss of generality that all the integers are positive and have no leading zeros. If there are negative integers or 0, deal with the positive numbers, negative numbers, and 0 separately. Under this assumption, we can observe that integers with more digits are always greater than integers with fewer digits.

One way to solve this problem is by a radix sort from right to left. Since the strings have varying lengths, however, we have to pad out all strings that are shorter than the longest string. Unfortunately, this scheme does not always run in the required time bound.

Suppose that there are m strings and that the longest string has d characters. The correctness of this algorithm is straightforward. Analyzing the running time is a bit trickier. Let us count the number of times that each string is sorted by a call of counting sort. Suppose that the ith string, si , has length li. The string a is sorted its length, 1, time plus one more time. Compare each red jug with each blue jug. To solve the problem, an algorithm has to perform a series of comparisons until it has enough information to determine the matching.

We can view the computation of the algorithm in terms of a decision tree. Every internal node is labeled with two jugs one red, one blue which we compare, and has three outgoing edges red jug smaller, same size, or larger than the blue jug.

The leaves are labeled with a unique matching of jugs. Sorting in Linear Time The height of the decision tree is equal to the worst-case number of comparisons the algorithm has to make to determine the matching. Now we can bound the height h of our decision tree. Every tree with a branch- ing factor of 3 every inner node has at most three children has at most 3h leaves.

Since the decison tree must have at least n! Assume that the red jugs are labeled with numbers 1, 2,. The numbers are arbitrary and do not correspond to the volumes of jugs, but are just used to refer to the jugs in the algorithm description. Moreover, the output of the algorithm will consist of n distinct pairs i, j , where the red jug i and the blue jug j have the same volume.

Termination is also easy to see: It eventually must reach 0 or 1, in which case the recursion terminates.

What about the running time? The analysis of the expected number of com- parisons is similar to that of the quicksort algorithm in Section 7. Let us order the jugs as r1 ,. As in quicksort, a given pair ri and b j is compared at most once.

Still following the quicksort analysis, until a jug from Ri j is chosen, the entire set Ri j is together. The remainder of the analysis is the same as the quicksort analysis, and we arrive at the solution of O n lg n comparisons. Just like in quicksort, in the worst case we always choose the largest or small- est jug to partition the sets, which reduces the set sizes by only 1.

Lecture Notes for Chapter 9: In other words, the ith smallest element of A. The selection problem can be solved in O n lg n time. There are faster algorithms, however. Simultaneous minimum and maximum Some applications need both the minimum and maximum of a set of elements. This leads to only 3 comparisons for every 2 elements. Setting up the initial values for the min and max depends on whether n is odd or even.

Then process the rest of the elements in pairs. Medians and Order Statistics Analysis Worst-case running time: Expected running time: Because it is randomized, no particular input brings out the worst-case behavior consis- tently. We obtain an upper bound on E[T n ] as follows: It depends on whether the ith smallest element is less than, equal to, or greater than the pivot element A[q]. Therefore, we can determine any order statistic in linear time on average.

Guarantee a good split when the array is partitioned. It executes the following steps: Divide the n elements into groups of 5. Now there are three possibilities: Medians and Order Statistics Analysis Start by getting a lower bound on the number of elements that are greater than the partitioning element x: Each white circle is the median of a group, as found in step 2.

Arrows go from larger elements to smaller elements, based on what we know after step 4. Elements in the region on the lower right are known to be greater than x. Solve this recurrence by substitution: We could have used any integer strictly greater than Solutions for Chapter 9: Medians and Order Statistics Solution to Exercise 9. Compare all the numbers in pairs.

To show this more formally, draw a binary tree of the comparisons the algorithm does. The n numbers are the leaves, and each number that came out smaller in a comparison is the parent of the two numbers that were compared. In the search for the smallest number, the second smallest number must have come out smallest in every comparison made with it until it was eventually compared with the smallest.

So the second smallest is among the elements that were com- pared with the smallest during the tournament. Solution to Exercise 9. For groups of 3, however, the algorithm no longer works in linear time. You can also see that T n is nonlinear by noticing that each level of the recursion tree sums to n. S ELECT takes an array A, the bounds p and r of the subarray in A, and the rank i of an order statistic, and in time linear in the size of the subarray A[ p.

Now, if the median is in X but is not in X[k], then the above condition will not hold. Since each binary search takes O lg n time, we spend a total of O lg n time. Medians and Order Statistics Proof We examine various cases. In each case, we will start out with the pipeline at a particular y-coordinate and see what happens when we move it. We start with the case in which n is even. Let us start with the pipeline somewhere on or between the two oil wells whose y-coordinates are the lower and upper me- dians.

Now suppose that the pipeline goes through the oil well whose y-coordinate is the upper median. We conclude that moving the pipeline up from the oil well at the upper median increases the total spur length.

A symmetric argument shows that if we start with the pipeline going through the oil well whose y-coordinate is the lower median and move it down, then the total spur length increases. We see, therefore, that when n is even, an optimal placement of the pipeline is anywhere on or between the two medians. Now we consider the case when n is odd. A symmetric argument shows that moving the pipeline down from the median also increases the total spur length, and so the optimal placement of the pipeline is on the median.

Solution to Problem We assume that the numbers start out in an array. Total worst-case running time: Implement the priority queue as a heap. Note that method c is always asymptotically at least as good as the other two methods, and that method b is asymptotically at least as good as a.

Com- paring c to b is easy, but it is less obvious how to compare c and b to a. The sum of two things that are O n lg n is also O n lg n. The median x of the elements x1 , x2 ,.

The sorting phase can be done in O n lg n worst-case time using merge sort or heapsort , and the scanning phase takes O n time. The total running time in the worst case, therefore, is O n lg n.

The weighted-median algorithm works as follows. Otherwise, we proceed as follows. We then compute the total weights of the two halves. Let the n points be denoted by their coordinates x1 , x2 ,. Let y be any point real number other than x. We are given n 2-dimensional points p1 , p2 ,. Divide the input as follows. If n is even, divide the input into two parts: If n is odd, divide the input into three parts: The number of comparisons in each step is as follows: Medians and Order Statistics 1.

No comparisons. Simple algebraic manipulations gives the following sequence of equivalent con- ditions: Lecture Notes for Chapter A hash table is effective for implementing a dictionary. A hash table is a generalization of an ordinary array.

This is called direct addressing. We use a hash table when we do not want to or cannot allocate an array with one position per possible key. We call this function a hash function. Hash Tables Direct-address tables Scenario: Represent by a direct-address table, or array, T [0. T 0 key satellite data U 1 universe of keys 2 0 6 2 9 3 4 7 3 4 1 2 5 K 5 actual 3 6 keys 5 8 7 8 8 9 Dictionary operations are trivial and take O 1 time each: Often, the set K of keys actually stored is small, compared to U , so that most of the space allocated for T is wasted.

Instead of storing an element with key k in slot k, use a function h and store the element in slot h k. When two or more keys hash to the same slot. Collision resolution by chaining Put all elements that hash to the same slot into a linked list. Hash Tables How to implement dictionary operations with chaining: We focus on average-case performance of hashing with chaining. We consider two cases: To search unsuccessfully for any key k, need to search to the end of the list T [h k ]. Successful search: Proof Assume that the element x being searched for is equally likely to be any of the n elements stored in the table.

These are the elements inserted after x was inserted because we insert at the head of the list. Alternative analysis, using indicator random variables even more: Since insertion takes O 1 worst-case time and deletion takes O 1 worst-case time when the lists are doubly linked, all dictionary operations take O 1 time on average.

Hash functions We discuss some issues regarding hash-function design and present schemes for hash function creation. What makes a good hash function?

Interpret a character string as an integer expressed in some radix notation. Suppose the string is CLRS: Fast, since requires just one division operation. Have to avoid certain values of m: Good choice for m: A prime not too close to an exact power of 2. Multiplication method 1. Multiply key k by A.

Extract the fractional part of k A. Multiply the fractional part by m. Slower than division method. Value of m is not critical. Relatively easy implementation: So we can just take these bits after having formed r0 by multiplying k by s. How to choose A: Universal hashing [We just touch on universal hashing in these notes. See the book for a full treat- ment. Then he could choose keys that all hash to the same slot, giving worst-case behavior. Hash Tables One way to defeat the adversary is to use a different hash function each time.

You choose one at random at the beginning of your program. What we want is to randomly choose a single hash function from a set of good candidates. Why are universal hash functions good? Theorem Using chaining and universal hashing on key k: Examining a slot is known as a probe. If this slot contains NIL, the search is unsuccessful. We compute the index of some other slot, based on k and on which probe count from 0: Pseudocode for searching: Pseudocode for insertion: Cannot just put NIL into the slot containing the key we want to delete.

How to compute probe sequences The ideal situation is uniform hashing: This generalizes simple uniform hashing for a hash function that produces a whole probe sequence rather than just a single number. None of these techniques can produce all m! Linear probing: Linear probing suffers from primary clustering: Result is that the average search and insertion times increase.

Quadratic probing: Un- like linear probing, it jumps around in the table according to a quadratic function of the probe number: Problem explores one way to implement quadratic probing. Can get secondary clustering: Hash Tables Double hashing: Use two auxiliary hash functions, h1 and h 2.

Analysis of open-address hashing Assumptions: Proof Since the search is unsuccessful, every probe is to an occupied slot, except for the last probe, which is to an empty slot. By Exercise C.

Boundary case: Proof Since there is no deletion, insertion uses the same probe sequence as an unsuccessful search. Theorem 1 1 The expected number of probes in a successful search is at most ln. We need to average over all n keys: Simplify by using the technique of bounding a summation by an integral: Hash Tables Solution to Exercise The stack has an attribute top[S], so that only entries S[1. The idea of this scheme is that entries of T and S validate each other.

If key k is actually stored in T , then T [k] contains the index, say j , of a valid entry in S, and S[ j ] contains the value k. Assuming that we also need to store pointers to objects in our direct-address table, we can store them in an array that is parallel to either T or S. The operations on the dictionary work as follows: Given key k, we check whether we have a validating cycle, i.

To delete object x with key k, assuming that this object is in the dictionary, we need to break the validating cycle. That is, we execute the following sequence of assignments: Solutions for Chapter Solution to Exercise The slot thus contains two pointers.

Of course, that pointer points to another slot in the table. The free list must be doubly linked in order for this deletion to run in O 1 time. To do so, allocate a free slot e. Then insert the new element in the now-empty slot as usual. Let j be the slot the element x to be deleted hashes to. Check the slot the key hashes to, and if that is not the desired element, follow the chain of pointers from the slot.

All the operations take expected O 1 times for the same reason they do with the version in the book: If the free list were singly linked, then operations that involved removing an arbitrary slot from the free list would not run in O 1 time.

One can prove this property formally, but infor- mally, consider that both heapsort and quicksort work by interchanging pairs of elements and that they have to be able to produce any permutation of their input array. Suppose that x and y are identical strings of n characters except that the characters in positions a and b are interchanged: By equation A. We now show that each such move increases the number of collisions, so that all the moves together must increase the number of collisions.

Suppose that we move an element from an underloaded value j to an overloaded value k, and we leave all other elements alone. We have the following sequence of equivalent inequalities: Thus, each move increases the number of collisions. Since we assume uniform hashing, we can use the same observation as is used in Corollary As in the proof of Theorem Hash Tables b.

We start by showing two facts. Second, n! Taking logarithms of both sides gives an equivalent condition: Work out the various cases in which i and j are even and odd. We will cover binary search trees, tree walks, and operations on binary search trees.

Binary search trees Binary search trees are an important data structure for dynamic sets. Draw sample tree. Show that the binary-search-tree property holds. Elements are printed in monotonically increasing order. Follows by induction directly from the binary-search-tree property.

Binary Search Trees Example: Search for values D and C in the example tree from above. The algorithm recurses, visiting nodes on a downward path from the root.

Thus, running time is O h , where h is the height of the tree. The above recursive procedure is more straightforward, however.

Traverse the appropriate pointers left or right until NIL is reached. Both procedures visit nodes that form a downward path from the root to a leaf. Both procedures run in O h time, where h is the height of the tree.

## Introduction to Algorithms

No key comparisons are necessary. There are two cases: If node x has an empty right subtree, notice that: Key value 4 Time: Insertion and deletion Insertion and deletion allows the dynamic set represented by a binary search tree to change. The binary-search-tree property must hold after the change. Insertion is more straightforward than deletion.

On a tree of height h, procedure takes O h time. See Exercise We can demonstrate on the above sample tree. O h , on a tree of height h. In later chapters, by varying the properties of binary search trees, we will be able to analyze running time in terms of n. Restructure the tree if necessary. Nothing special is required for querying, but there may be extra work when changing the structure of the tree inserting or deleting. Worst-case analysis In the worst case, we hire all n candidates.

This happens if each one is better than all who came before. In other words, if the candidates appear in increasing order of quality. Probabilistic analysis In general, we have no control over the order in which candidates appear. We could assume that they come in a random order: Essential idea of probabilistic analysis: We must use knowledge of, or make as- sumptions about, the distribution of inputs. Probabilistic Analysis and Randomized Algorithms Randomized algorithms We might not know the distribution of inputs, or we might not be able to model it computationally.

Instead, we use randomization within the algorithm in order to impose a distribu- tion on the inputs. For the hiring problem: Change the scenario: What makes an algorithm randomized: An algorithm is randomized if its behav- ior is determined in part by values produced by a random-number generator.

Indicator random variables A simple yet powerful technique for computing the expected value of a random variable. Helpful in situations in which there may be dependence. Probabilistic Analysis and Randomized Algorithms Simple example: Slightly more complicated example: In fact, this is what the book does in equation C.

We have only the individual expectations E [X 1 ] , E [X 2 ] ,. Linearity of expectation says that the expectation of the sum equals the sum of the expectations.

The hat-check problem of Exercise 5. See the solution on page of this manual. Probabilistic Analysis and Randomized Algorithms Useful properties: Now compute E [X ]: Thus, the expected hiring cost is O ch ln n , which is much better than the worst- case cost of O nch.

Randomized algorithms Instead of assuming a distribution of the inputs, we impose a distribution. The hiring problem For the hiring problem, the algorithm is deterministic: Then only the best candidate is hired.

Each time we run the algorithm, we can get a different hiring cost. Pseudocode for randomized hiring problem: Randomly permuting an array [The book considers two methods of randomly permuting an n -element array. We omit this method from these notes. The second method is better: We present and analyze the second method in these notes. Produce a uniform random permutation each of the n! See Exercise 5. The following procedure permutes the array A[1. Probabilistic Analysis and Randomized Algorithms Idea: Given a set of n elements, a k-permutation is a sequence containing k of the n elements.

There are n! Proof Use a loop invariant: Loop invariant says that for each possible 0-permutation, subarray A[1. So, A[1. Will show that after the ith iteration, each possible i-permutation appears in A[1. Incrementing i for the next iteration then maintains the invariant. Let E 2 be the event that the ith iteration puts xi into A[i].

Equation C. Thus, the probability that a given iteration returns 0 equals the probability that it returns 1. The number of trials until a success occurs is given by the geometric distribution, and by equation C. Solutions for Chapter 5: Solution to Exercise 5. Candidate 1 is always hired. The best candidate, i. If the best candidate is candidate 1, then that is the only candidate hired. Letting j denote the position in the interview order of the best candidate, let F be the event in which candidates 2, 3,.

Noting that the events E1 , E 2 ,. One could enumerate all n! This would be a painstaking process, and the answer would turn out to be 1. We can use indicator random variables, however, to arrive at the same answer much more easily. Note that this is a situation in which the indicator random variables are not inde- pendent. Thus, we can use the technique of indicator random variables even in the presence of dependence. By Lemma 5. The maintenance and termination parts remain the same.

The initialization part is for the subarray A[1. Since there are 3! The subtraction and addition of 1 in the index calculation is due to the 1-origin indexing. Thus, once offset is determined, so is the entire permutation. This procedure does not produce a uniform random permutation, however, since it can produce only n different permutations. We view a toss as a success if it misses bin i and as a failure if it lands in bin i. In order for bin i to be empty, we need n successes in n trials.

Now we determine the expected number of bins with exactly one ball. We want to compute E [Vn ]. Both ways condition on the value held in the counter, but only the second way incorporates the conditioning into the expression for E[X j ]. The X j are pairwise independent, and so by equation C. Thus, by equation C. Lecture Notes for Chapter 6: Here, we bypass these attributes and use parameter values instead. Heapsort Example: Similar argument for min-heaps.

In general, heaps can be k-ary tree instead of binary. It is used to maintain the max-heap property. Heapsort [Parameter n replaces attribute heap-size[A]. If we hit a leaf, then the subtree rooted at the leaf is trivially a max-heap. In this case, the max-heap is a leaf. Building a heap The following procedure, given an unordered array, will produce a max-heap.

By Exercise 6. Children of node i are indexed higher than i, so by the loop invari- ant, they are both roots of max-heaps. Decrementing i reestablishes the loop invariant at each iteration. By the loop invariant, each node, notably node 1, is the root of a max-heap. A good approach to analysis in general is to start by proving easy bound, then try to tighten it.

The heapsort algorithm Given an input array, the heapsort algorithm acts as follows: Sort an example heap on the board. O lg n Total time: Though heapsort is a great algorithm, a well-implemented quicksort usually beats it in practice. These notes will deal with max- priority queues implemented with max-heaps.

Min-priority queues are imple- mented with min-heaps similarly. A heap gives a good compromise between fast insertion but slow extraction and vice versa. Both operations take O lg n time. Actual implementations often have a handle in each heap element that allows access to an object in the application, and objects in the application often have a handle likely an array index to access the heap element.

Will examine how to implement max-priority queue operations. Finding the maximum element Getting the maximum element is easy: Extracting max element Given the array A: Heapsort Analysis: Increasing key value Given set S, element x, and new key value k: Upward path from node i has length O lg n in an n-element heap.

Exchange keys of nodes 4 and 9, then of nodes 2 and 4. Inserting into the heap Given a key k to insert into the heap: Min-priority queue operations are implemented similarly with min-heaps.

Solutions for Chapter 6: Heapsort Solution to Exercise 6. Solution to Exercise 6. Then the maximum element is somewhere else in the sub- tree, possibly even at more than one location. Let m be the index at which the maximum appears the lowest such index if the maximum appears more than once. Since the maximum is not at the root of the subtree, node m has a parent.

So our assumption is false, and the claim is true. Two subtleties to beware of: But the proof for an incomplete tree is tricky and is not derived from the proof for a complete tree. Proof By induction on h. Let x be the number of nodes at depth H —that is, the number of nodes in the bottom possibly incomplete level.

Thus if n is odd, x is even, and if n is even, x is odd. To prove the base case, we must consider separately the case in which n is even x is odd and the case in which n is odd x is even. Here are two ways to do this: First method of proving the base case: Thus see Exercise B. The latter equality holds because n is odd. Observe that we would also increase the number of leaves by 1, since we added a node to a parent that already had a child.

The latter equality holds because n is even. Second method of proving the base case: Inductive step: Let n h be the number of nodes at height h in the n-node tree T.

Consider the following counterexample. Input array A: A d-ary heap can be represented in a 1-dimensional array as follows. Heapsort d. Since only parent pointers are followed, the number of children a node has is irrelevant. Increas- ing an element may make it larger than its parent, in which case it must be moved higher up in the tree.

This can be done just as for insertion, travers- ing a path from the increased node toward the root. Using Lomuto partitioning helps simplify the analysis, which uses indicator random variables in the second edition.

## Introduction to Algorithms, Second Edition Solution Manual

Description of quicksort Quicksort is based on the three-step process of divide-and-conquer. Partition A[ p. No work is needed to combine the subarrays, because they are sorted in place.

Partitioning Partition subarray A[ p. All entries in A[ p. On an 8-element subarray. Lecture Notes for Chapter 7: Time for partitioning: Quicksort Performance of quicksort The running time of quicksort depends on the partitioning of the subarrays: Instead, randomly pick an element from the subarray that is being sorted. We add this randomization by not always using A[r] as the pivot, but instead ran- domly picking an element from the subarray that is being sorted. Worst-case analysis We will prove that a worst-case split at every level produces a worst-case running time of O n 2.

Second derivative with respect to q is positive. We will now compute a bound on the overall number of comparisons. For ease of analysis: Each pair of elements is compared at most once, because elements are compared only to the pivot element, and then the pivot element is never in any later call to PARTITION. Solutions for Chapter 7: Quicksort Solution to Exercise 7.

Solution to Exercise 7. Similarly, maximum depth corresponds to always taking the larger part of the par- tition, i. What randomization can do is make the chance of encountering a worst-case scenario small. Quicksort c. Lecture Notes for Chapter 8: Sorting in Linear Time Chapter 8 overview How fast can we sort? We will prove a lower bound, then beat it by playing a different game.

Sorting in Linear Time For insertion sort on 3 elements: Each leaf is labeled by the permutation of orders that the algorithm determines. What is the length of the longest path from root to leaf? In other words: Why is this useful? Tree is just one node, which is a leaf. Each leaf becomes parent to two new leaves.

## Breadcrumb

Sorting in linear time Non-comparison sorts. Counting sort Depends on a key assumption: Array A and values n and k are given as parameters. B is assumed to be already allocated and is given as a parameter. Auxiliary storage: How big a k is practical? Probably not. Maybe, depending on n. Probably unless n is really small. Counting sort will be used in radix sort. Radix sort How IBM made its money.

Card sorters, worked on one column at a time. The human operator was part of the algorithm! Key idea: To sort d digits: Sorting in Linear Time Correctness: The stable sort on digit i leaves them in the right order.

Assume that we use counting sort as the intermediate sort. How to break each key into digits? Compare radix sort to merge sort and quicksort: How does radix sort violate the ground rules for a comparison sort? Sorting in Linear Time Bucket sort Assumes the input is generated by a random process that distributes elements uni- formly over [0, 1.

Auxiliary array: Consider A[i], A[ j ]. So A[i] is placed into the same bucket as A[ j ] or into a bucket with a lower index. Sorting in Linear Time Take expectations of both sides: Used a function of key values to index into an array. Solutions for Chapter 8: Sorting in Linear Time Solution to Exercise 8.

Use the same argument as in the proof of Theorem 8. In particular, n! Proof First notice that, as pointed out in the hint, we cannot prove the lower bound by multiplying together the lower bounds for sorting each subsequence.

That would only prove that there is no faster algorithm that sorts the subsequences independently. This was not what we are asked to prove; we cannot introduce any extra assumptions. Sorting in Linear Time Now, consider the decision tree of height h for any comparison sort for S.

Since the elements of each subsequence can be in any order, any of the k! Thus, any decision tree for sorting S must have at least k! The third line comes from k!

We implicitly assume here that k is even. Solution to Exercise 8. The algorithm is correct no matter what order is used! The original algorithm was stable because an element taken from A later started out with a lower index than one taken earlier. The number of integers in the range [a. When inserting A[ j ] into the sorted sequence A[1. Heapsort and quicksort are not stable.

When comparing two elements, compare them by their values and break ties by their indices. Additional space requirements: For n elements, their indices are 1. Each can be written in lg n bits, so together they take O n lg n additional space. Additional time requirements: The worst case is when all elements are equal. The asymptotic time does not change because we add a constant amount of work to each comparison. Radix sort sorts separately on each digit, starting from digit 1.

Thus, radix sort of d digits, which sorts on digits 1,. The sort on digit d will order the elements by their dth digit. Consider two ele- ments, a and b, with dth digits ad and bd respectively. If the intermediate sort were not stable, it might rearrange elements whose dth digits were equal—elements that were in the right order after the sort on their lower-order digits.

Sort these 2-digit numbers with radix sort. A simple change that will preserve the linear expected running time and make the worst-case running time O n lg n is to use a worst-case O n lg n -time algorithm like merge sort instead of insertion sort when sorting the buckets. For a comparison algorithm A to sort, no two input permutations can reach the same leaf of the decision tree, so there must be at least n! Since A is a deterministic algorithm, it must always reach the same leaf when given a particular permutation as input, so at most n!

Therefore exactly n! These n! Any remaining leaves will have probability 0, since they are not reached for any input. That is, we can assume that TA consists of only the n!

D T A is the sum of the decision-tree path lengths for sorting all input per- mutations, and the path lengths are proportional to the run time. Since the n! At each randomized node, pick the child with the smallest subtree the subtree with the smallest average number of comparisons on a path to a leaf.

Delete all the other children of the randomized node and splice out the randomized node itself. The randomized algorithm thus takes at least as much time on average as the corresponding deterministic one. The usual, unadorned radix sort algorithm will not solve this problem in the required time bound. The number of passes, d, would have to be the number of digits in the largest integer. We assume that the range of a single digit is constant.

Let us assume without loss of generality that all the integers are positive and have no leading zeros. If there are negative integers or 0, deal with the positive numbers, negative numbers, and 0 separately. Under this assumption, we can observe that integers with more digits are always greater than integers with fewer digits.

One way to solve this problem is by a radix sort from right to left. Since the strings have varying lengths, however, we have to pad out all strings that are shorter than the longest string. Unfortunately, this scheme does not always run in the required time bound. Suppose that there are m strings and that the longest string has d characters. The correctness of this algorithm is straightforward. Analyzing the running time is a bit trickier.

Let us count the number of times that each string is sorted by a call of counting sort.

Suppose that the ith string, si , has length li. The string a is sorted its length, 1, time plus one more time. Compare each red jug with each blue jug. To solve the problem, an algorithm has to perform a series of comparisons until it has enough information to determine the matching.

We can view the computation of the algorithm in terms of a decision tree. Every internal node is labeled with two jugs one red, one blue which we compare, and has three outgoing edges red jug smaller, same size, or larger than the blue jug. The leaves are labeled with a unique matching of jugs. Sorting in Linear Time The height of the decision tree is equal to the worst-case number of comparisons the algorithm has to make to determine the matching. Now we can bound the height h of our decision tree.

Every tree with a branch- ing factor of 3 every inner node has at most three children has at most 3h leaves. Since the decison tree must have at least n! Assume that the red jugs are labeled with numbers 1, 2,. The numbers are arbitrary and do not correspond to the volumes of jugs, but are just used to refer to the jugs in the algorithm description. Moreover, the output of the algorithm will consist of n distinct pairs i, j , where the red jug i and the blue jug j have the same volume.

Termination is also easy to see: It eventually must reach 0 or 1, in which case the recursion terminates. What about the running time?

The analysis of the expected number of com- parisons is similar to that of the quicksort algorithm in Section 7. Let us order the jugs as r1 ,. As in quicksort, a given pair ri and b j is compared at most once. Still following the quicksort analysis, until a jug from Ri j is chosen, the entire set Ri j is together. The remainder of the analysis is the same as the quicksort analysis, and we arrive at the solution of O n lg n comparisons.

Just like in quicksort, in the worst case we always choose the largest or small- est jug to partition the sets, which reduces the set sizes by only 1.

Lecture Notes for Chapter 9: In other words, the ith smallest element of A. The selection problem can be solved in O n lg n time. There are faster algorithms, however. Simultaneous minimum and maximum Some applications need both the minimum and maximum of a set of elements. This leads to only 3 comparisons for every 2 elements. Setting up the initial values for the min and max depends on whether n is odd or even. Then process the rest of the elements in pairs.

Medians and Order Statistics Analysis Worst-case running time: Expected running time: Because it is randomized, no particular input brings out the worst-case behavior consis- tently.

We obtain an upper bound on E[T n ] as follows: It depends on whether the ith smallest element is less than, equal to, or greater than the pivot element A[q]. Therefore, we can determine any order statistic in linear time on average.

Guarantee a good split when the array is partitioned. It executes the following steps: Divide the n elements into groups of 5. Now there are three possibilities: Medians and Order Statistics Analysis Start by getting a lower bound on the number of elements that are greater than the partitioning element x: Each white circle is the median of a group, as found in step 2. Arrows go from larger elements to smaller elements, based on what we know after step 4.

Elements in the region on the lower right are known to be greater than x. Solve this recurrence by substitution: We could have used any integer strictly greater than Solutions for Chapter 9: Medians and Order Statistics Solution to Exercise 9. Compare all the numbers in pairs. To show this more formally, draw a binary tree of the comparisons the algorithm does.

The n numbers are the leaves, and each number that came out smaller in a comparison is the parent of the two numbers that were compared. In the search for the smallest number, the second smallest number must have come out smallest in every comparison made with it until it was eventually compared with the smallest.

So the second smallest is among the elements that were com- pared with the smallest during the tournament. Solution to Exercise 9. For groups of 3, however, the algorithm no longer works in linear time. You can also see that T n is nonlinear by noticing that each level of the recursion tree sums to n. S ELECT takes an array A, the bounds p and r of the subarray in A, and the rank i of an order statistic, and in time linear in the size of the subarray A[ p. Now, if the median is in X but is not in X[k], then the above condition will not hold.

Since each binary search takes O lg n time, we spend a total of O lg n time. Medians and Order Statistics Proof We examine various cases.

In each case, we will start out with the pipeline at a particular y-coordinate and see what happens when we move it. We start with the case in which n is even. Let us start with the pipeline somewhere on or between the two oil wells whose y-coordinates are the lower and upper me- dians. Now suppose that the pipeline goes through the oil well whose y-coordinate is the upper median.

We conclude that moving the pipeline up from the oil well at the upper median increases the total spur length. A symmetric argument shows that if we start with the pipeline going through the oil well whose y-coordinate is the lower median and move it down, then the total spur length increases.

We see, therefore, that when n is even, an optimal placement of the pipeline is anywhere on or between the two medians. Now we consider the case when n is odd. A symmetric argument shows that moving the pipeline down from the median also increases the total spur length, and so the optimal placement of the pipeline is on the median. Solution to Problem We assume that the numbers start out in an array. Total worst-case running time: Implement the priority queue as a heap.

Note that method c is always asymptotically at least as good as the other two methods, and that method b is asymptotically at least as good as a.

Com- paring c to b is easy, but it is less obvious how to compare c and b to a. The sum of two things that are O n lg n is also O n lg n. The median x of the elements x1 , x2 ,. The sorting phase can be done in O n lg n worst-case time using merge sort or heapsort , and the scanning phase takes O n time. The total running time in the worst case, therefore, is O n lg n. The weighted-median algorithm works as follows.

Otherwise, we proceed as follows. We then compute the total weights of the two halves. Let the n points be denoted by their coordinates x1 , x2 ,. Let y be any point real number other than x. We are given n 2-dimensional points p1 , p2 ,. Divide the input as follows. If n is even, divide the input into two parts: If n is odd, divide the input into three parts: The number of comparisons in each step is as follows: Medians and Order Statistics 1.

No comparisons. Simple algebraic manipulations gives the following sequence of equivalent con- ditions: Lecture Notes for Chapter A hash table is effective for implementing a dictionary. A hash table is a generalization of an ordinary array. This is called direct addressing.

We use a hash table when we do not want to or cannot allocate an array with one position per possible key. We call this function a hash function. Hash Tables Direct-address tables Scenario: Represent by a direct-address table, or array, T [0. T 0 key satellite data U 1 universe of keys 2 0 6 2 9 3 4 7 3 4 1 2 5 K 5 actual 3 6 keys 5 8 7 8 8 9 Dictionary operations are trivial and take O 1 time each: Often, the set K of keys actually stored is small, compared to U , so that most of the space allocated for T is wasted.

Instead of storing an element with key k in slot k, use a function h and store the element in slot h k. When two or more keys hash to the same slot. Collision resolution by chaining Put all elements that hash to the same slot into a linked list. Hash Tables How to implement dictionary operations with chaining: We focus on average-case performance of hashing with chaining.

We consider two cases: To search unsuccessfully for any key k, need to search to the end of the list T [h k ]. Successful search: Proof Assume that the element x being searched for is equally likely to be any of the n elements stored in the table. These are the elements inserted after x was inserted because we insert at the head of the list.

Alternative analysis, using indicator random variables even more: Since insertion takes O 1 worst-case time and deletion takes O 1 worst-case time when the lists are doubly linked, all dictionary operations take O 1 time on average. Hash functions We discuss some issues regarding hash-function design and present schemes for hash function creation. What makes a good hash function?

Interpret a character string as an integer expressed in some radix notation. Suppose the string is CLRS: Fast, since requires just one division operation. Have to avoid certain values of m: Good choice for m: A prime not too close to an exact power of 2. Multiplication method 1. Multiply key k by A. Extract the fractional part of k A. Multiply the fractional part by m.

Slower than division method. Value of m is not critical. Relatively easy implementation: So we can just take these bits after having formed r0 by multiplying k by s. How to choose A: Universal hashing [We just touch on universal hashing in these notes.

See the book for a full treat- ment. Then he could choose keys that all hash to the same slot, giving worst-case behavior. Hash Tables One way to defeat the adversary is to use a different hash function each time. You choose one at random at the beginning of your program. What we want is to randomly choose a single hash function from a set of good candidates.

Why are universal hash functions good? Theorem Using chaining and universal hashing on key k: Examining a slot is known as a probe. If this slot contains NIL, the search is unsuccessful.

We compute the index of some other slot, based on k and on which probe count from 0: Pseudocode for searching: Pseudocode for insertion: Cannot just put NIL into the slot containing the key we want to delete. How to compute probe sequences The ideal situation is uniform hashing: This generalizes simple uniform hashing for a hash function that produces a whole probe sequence rather than just a single number.

None of these techniques can produce all m! Linear probing: Linear probing suffers from primary clustering: Result is that the average search and insertion times increase. Quadratic probing: Un- like linear probing, it jumps around in the table according to a quadratic function of the probe number: Problem explores one way to implement quadratic probing.

Can get secondary clustering: Hash Tables Double hashing: Use two auxiliary hash functions, h1 and h 2. Analysis of open-address hashing Assumptions: Proof Since the search is unsuccessful, every probe is to an occupied slot, except for the last probe, which is to an empty slot.

By Exercise C. Boundary case: Proof Since there is no deletion, insertion uses the same probe sequence as an unsuccessful search. Theorem 1 1 The expected number of probes in a successful search is at most ln. We need to average over all n keys: Simplify by using the technique of bounding a summation by an integral: Hash Tables Solution to Exercise The stack has an attribute top[S], so that only entries S[1.

The idea of this scheme is that entries of T and S validate each other. If key k is actually stored in T , then T [k] contains the index, say j , of a valid entry in S, and S[ j ] contains the value k. Assuming that we also need to store pointers to objects in our direct-address table, we can store them in an array that is parallel to either T or S. The operations on the dictionary work as follows: Given key k, we check whether we have a validating cycle, i.

To delete object x with key k, assuming that this object is in the dictionary, we need to break the validating cycle. That is, we execute the following sequence of assignments: Solutions for Chapter Solution to Exercise The slot thus contains two pointers. Of course, that pointer points to another slot in the table. The free list must be doubly linked in order for this deletion to run in O 1 time.

To do so, allocate a free slot e. Then insert the new element in the now-empty slot as usual. Let j be the slot the element x to be deleted hashes to. Check the slot the key hashes to, and if that is not the desired element, follow the chain of pointers from the slot. All the operations take expected O 1 times for the same reason they do with the version in the book: If the free list were singly linked, then operations that involved removing an arbitrary slot from the free list would not run in O 1 time.

One can prove this property formally, but infor- mally, consider that both heapsort and quicksort work by interchanging pairs of elements and that they have to be able to produce any permutation of their input array. Suppose that x and y are identical strings of n characters except that the characters in positions a and b are interchanged: By equation A. We now show that each such move increases the number of collisions, so that all the moves together must increase the number of collisions.

Suppose that we move an element from an underloaded value j to an overloaded value k, and we leave all other elements alone. We have the following sequence of equivalent inequalities: Thus, each move increases the number of collisions. Since we assume uniform hashing, we can use the same observation as is used in Corollary As in the proof of Theorem Hash Tables b. We start by showing two facts.

Second, n! Taking logarithms of both sides gives an equivalent condition: Work out the various cases in which i and j are even and odd. We will cover binary search trees, tree walks, and operations on binary search trees.

Binary search trees Binary search trees are an important data structure for dynamic sets. Draw sample tree. Show that the binary-search-tree property holds. Elements are printed in monotonically increasing order.

Follows by induction directly from the binary-search-tree property.

## Introduction to Algorithms, Third Edition

Binary Search Trees Example: Search for values D and C in the example tree from above. The algorithm recurses, visiting nodes on a downward path from the root. Thus, running time is O h , where h is the height of the tree.

The above recursive procedure is more straightforward, however. Traverse the appropriate pointers left or right until NIL is reached. Both procedures visit nodes that form a downward path from the root to a leaf. Both procedures run in O h time, where h is the height of the tree. No key comparisons are necessary.

There are two cases: If node x has an empty right subtree, notice that: Key value 4 Time: Insertion and deletion Insertion and deletion allows the dynamic set represented by a binary search tree to change. The binary-search-tree property must hold after the change.

Insertion is more straightforward than deletion. On a tree of height h, procedure takes O h time. See Exercise We can demonstrate on the above sample tree. O h , on a tree of height h.

In later chapters, by varying the properties of binary search trees, we will be able to analyze running time in terms of n. Restructure the tree if necessary. Nothing special is required for querying, but there may be extra work when changing the structure of the tree inserting or deleting. Red-black trees are covered in detail in Chapter Expected height of a randomly built binary search tree [These are notes on a starred section in the book.

I covered this material in an optional lecture. Insert them in random order into an initially empty binary search tree. Will get 5 different binary search trees. When we look at the binary search trees resulting from each of the 3! We will show that the expected height of a randomly built binary search tree is O lg n. Base cases: Exactly one Z n,i is 1, and all others are 0. This is Exercise C. Binary Search Trees We solve the recurrence by induction on n.

Binary Search Trees Solution to Exercise In a heap, the largest element smaller than the node could be in either subtree. Note that if the heap property could be used to print the keys in sorted order in O n time, we would have an O n -time algorithm for sorting, because building the heap takes only O n time. Then s cannot have a left child, for a left child of s would come between x and s in the inorder walk.

To see that each edge is traversed at most twice once going down the tree and once going up , consider the edge between any node u and either of its children, node v. By starting at the root, we must traverse u, v downward from u to v, before traversing it upward from v to u.

Hence, no edge is traversed twice in the same direction. Binary Search Trees Worst case: Then there exists a path from the root to a node at depth h, and the depths of the nodes on this path are 0, 1,.If any value in S appears more than once, remove all but one instance. You may choose to allow recursion trees as proofs in your course, in which case some of the substitution proofs in the solutions for this chapter become recursion trees. Lecture Notes for Chapter 3: Exchange keys of nodes 4 and 9, then of nodes 2 and 4.

To use a loop invariant to prove correctness, we must show three things about it: Chapter The explanations have been kept elementary without sacrificing depth of coverage or mathematical rigor. Proof Since f [x] depends only on x and its children, when we alter information in x, changes propagate only upward to p[x], p[ p[x]],. We now show that each such move increases the number of collisions, so that all the moves together must increase the number of collisions.

We will prove a lower bound, then beat it by playing a different game.