Transcript lecture09b

CSE 326: Data Structures
Priority Queues (Heaps)
Lecture 9: Monday, Jan 27, 2003
1
Not Quite Queues
• Consider applications
– ordering CPU jobs
– searching for the exit in a maze
– emergency room admission processing
• Problems?
– short jobs should go first
– most promising nodes should be searched first
– most urgent cases should go first
2
Priority Queue ADT
• Priority Queue operations
create :  heap
insert : heap  value  heap
findMin : heap  value
deleteMin : heap  heap
is_empty : heap  boolean
insert
G(9)
F(7) E(5)
D(100) A(4)
B(6)
deleteMin
C(3)
• Priority Queue property: for two elements in the
queue, x and y, if x has a lower priority value than
y, x will be deleted before y
3
Applications of the Priority Q
• Hold jobs for a printer in order of length
• Store packets on network routers in order of
urgency
• Simulate events
• Anything greedy
4
Discrete Event Simulation
• An event is a pair (x,t) where x describes
the event and t is time it should occur
• A discrete event simulator (DES) maintains
a set S of events which it intends to simulate
in time order
repeat {
Find and remove (x0,t0) from S such that t0 is minimum;
Do whatever x0 says to do;
in the process new events (x1,t1)…(xk,tk) may be generated;
Insert the new events into S;
}
5
Emergency Room Simulation
• Patient arrive at time t with injury of criticality C
– If no patients waiting and a free doctor, assign them to
doctor and create a future departure event; else put patient
in the Criticality priority queue
• Patient departs at time t
– If someone in Criticality queue, pull out most critical and
assign to doctor; create a future departure event
arrive(t,c)
patient
generator
time
queue
depart(t)
arrive(t,c)
depart(t)
assign
patient to
doctor
criticality
(triage)
queue
6
Naïve Priority Queue Data
Structures
• Unsorted list:
– insert:
– findMin:
– deleteMin:
• Sorted list:
– insert:
– findMin:
– deleteMin:
7
BST Tree Priority Queue Data
Structure
•Regular BST:
8
–insert:
–findMin:
–deleteMin:
•AVL Tree:
–insert:
–findMin:
–deleteMin:
5
2
11
6
4
10
7
9
12
13 14
Can we do better?
8
Binary Heap Priority Q Data
Structure
• Heap-order property
– parent’s key is less than
children’s keys
– result: minimum is always
at the top
2
4
• Structure property
– complete tree with fringe
nodes packed to the left
– result: depth is always
O(log n); next open location
always known
7
11
5
6
9
10
8
12 14 20
How do we find the minimum?
9
Nifty Storage Trick
• Calculations:
– child:
2
1
2
– parent:
3
4
5
4
– root:
5
7
– next free:
6
8
11
9
9
7
6
10
8
12 14 20
10
12
11
0
1
2
3
4
5
6
7
8
9
10
11
12
12
2
4
5
7
6
10
8
11
9
12 14 20
10
Nifty Storage Trick
• Calculations:
– child: left = 2*node
right=2*node+1
– parent: floor(node/2)
2
1
2
3
4
5
4
– root: 1
5
7
– next free: length+1
6
8
11
9
9
7
6
10
8
12 14 20
10
12
11
0
1
2
3
4
5
6
7
8
9
10
11
12
12
2
4
5
7
6
10
8
11
9
12 14 20
11
findMin
pqueue.findMin()
2
4
5
2
7
11
6
9
10
8
12 14 20
Time = O(1)
12
DeleteMin
pqueue.deleteMin()
2
2
20
4
7
11
5
6
9
10
12 14 20
4
8
7
11
5
6
9
10
8
12 14 20
13
Percolate Down
20
4
4
7
5
6
10
20
8
7
11 9 12 14
5
6
10
11 9 12 14
4
4
6
7
8
5
20
11 9 12 14
10
6
8
7
5
12
11 9 20 14
10
8
14
DeleteMin Code
Comparable deleteMin(){
x = A[1];
A[1]=A[size--];
percolateDown(1);
return x;
}
Trick to avoid
repeatedly copying
the value at A[1]
Time = O(log n)
(why ?)
percolateDown(int hole) {
tmp=A[hole];
while (2*hole <= size) {
left = 2*hole;
right = left + 1;
if (right <= size &&
A[right] < A[left])
target = right;
else
target = left;
if (A[target] < tmp) {
A[hole] = A[target];
hole = target;
}
else
break;
Move down
}
A[hole] = tmp;
}
15
Insert
pqueue.insert(3)
2
2
4
7
11
5
6
9
10
12 14 20
4
8
7
11
5
6
9
10
12 14 20
8
3
16
Percolate Up
2
2
4
7
5
6
10
4
8
7
11 9 12 14 20 3
5
6
3
8
11 9 12 14 20 10
2
4
7
3
6
5
11 9 12 14 20 10
8
17
Insert Code
void insert(Comparable x) {
// Efficiency hack: we won’t actually put x
// into the heap until we’ve located the position
// it goes in. This avoids having to copy it
// repeatedly during the percolate up.
int hole = ++size;
// Percolate up
for( ; hole>1 && x < A[hole/2] ; hole = hole/2)
A[hole] = A[hole/2];
A[hole] = x;
}
Time = O(log n)
(why ?)
18
Performance of Binary Heap
Binary
Binary
heap
heap avg
worst case case
AVL tree BST tree
worst case avg case
Insert
O(log n)
O(1)
percolates
1.6 levels
O(log n)
O(log n)
Delete
Min
O(log n)
O(log n)
O(log n)
O(log n)
• In practice: binary heaps much simpler to code,
lower constant factor overhead
19
Changing Priorities
• In many applications the priority of an object in a
priority queue may change over time
– if a job has been sitting in the printer queue for a long
time increase its priority
– unix “renice”
• Must have some (separate) way of find the
position in the queue of the object to change (e.g.
a hash table)
20
Other Priority Queue Operations
• decreaseKey
– Given the position of an object in the queue, increase its
priority (lower its key). Fix heap property by:
• increaseKey
– given the position of an an object in the queue, decrease
its priority (increase its key). Fix heap property by:
• remove
– given the position of an an object in the queue, remove
it. Do increaseKey to infinity then …
21
BuildHeap
• Task: Given a set of n keys, build a heap all
at once
• Approach 1: Repeatedly perform
Insert(key)
• Complexity:
22
BuildHeap
Floyd’s Method
12
5
11
3
10
6
9
4
8
1
7
2
pretend it’s a heap and fix the heap-order property!
12
buildHeap(){
for (i=size/2; i>0; i--)
5
percolateDown(i);
}
11
3
4
10
8
1
6
7
9
2
23
Build(this)Heap
12
12
5
11
3
4
10
8
1
2
7
5
9
11
3
6
4
1
2
8 10 7
6
12
12
5
3
4
2
1
8 10 7
6
11
9
1
9
3
4
2
5
8 10 7
6
11
9
24
Finally…
1
3
4
2
5
12 8 10 7
6
9
11
25
Complexity of Build Heap
• Note: size of a perfect binary tree doubles (+1)
with each additional layer
• At most n/4 percolate down 1 level
at most n/8 percolate down 2 levels
at most n/16 percolate down 3 levels…
logn
logn
n
n
n
n
n
i n
n
1  2  3  ...   i i  2   i  2 
4
8
16
4 i 1 2 4
2
i 1 2
O(n)
26
Thinking about Heaps
• Observations
–
–
–
–
finding a child/parent index is a multiply/divide by two
operations jump widely through the heap
each operation looks at only two new nodes
inserts are at least as common as deleteMins
• Realities
– division and multiplication by powers of two are fast
– looking at one new piece of data terrible in a cache line
– with huge data sets, disk accesses dominate
27
Solution: d-Heaps
• Each node has d children
• Still representable by array
• Good choices for d:
1
3
7
2
– optimize performance based
4 8 5 12 11 10 6 9
on # of inserts/removes
– choose a power of two for
12 1 3 7 2 4 8 5 12 11 10 6 9
efficiency
– fit one set of children in a
cache line
– fit one set of children on a
memory page/disk block
28
New Operation: Merge
Merge(H1,H2): Merge two heaps H1 and H2 of size O(N).
– E.g. Combine queues from two different sources to run on one
CPU.
1. Can do O(N) Insert operations: O(N log N) time
2. Better: Copy H2 at the end of H1 (assuming array
implementation) and use Floyd’s Method for
BuildHeap.
Running Time: O(N)
Can we do even better? (i.e. Merge in O(log N) time?)
29
Binomial Queues
insert : heap  value  heap
findMin : heap  value
deleteMin : heap  heap
merge : heap  heap  heap
• All in O(log n) time
• Recursive Definition of Binomial Tree Bk of height k:
– B0 = single root node
– Bk = Attach Bk-1 to root of another Bk-1
• Idea: a binomial heap H is a forest of binomial trees:
H = B0 B1 B2 ... Bk
where each Bi may be present, or may be empty
30
Building a Binomial Tree
•
•
To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
Binomial tree of height k has exactly 2k nodes (by induction)
B0
B1
B2
B3
31
Building a Binomial Tree
•
•
To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
Binomial tree of height k has exactly 2k nodes (by induction)
B0
B1
B2
B3
32
Building a Binomial Tree
•
•
To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
Binomial tree of height k has exactly 2k nodes (by induction)
B0
B1
B2
B3
33
Building a Binomial Tree
•
•
To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
Binomial tree of height k has exactly 2k nodes (by induction)
B0
B1
B2
B3
34
Building a Binomial Tree
•
•
To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
Binomial tree of height k has exactly 2k nodes (by induction)
B0
B1
B2
B3
35
Building a Binomial Tree
•
•
To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
Binomial tree of height k has exactly 2k nodes (by induction)
B0
B1
B2
B3
36
Building a Binomial Tree
•
•
To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
Binomial tree of height k has exactly 2k nodes (by induction)
B0
B1
B2
B3
Recall: a binomial heap may have any subset of these trees
37
Why Binomial?
• Why are these trees called binomial?
– Hint: how many nodes at depth d?
B0
B1
B2
B3
38
Why Binomial?
• Why are these trees called binomial?
– Hint: how many nodes at depth d?
Number of nodes at different depths d for Bk =
[1], [1 1], [1 2 1], [1 3 3 1], …
Binomial coefficients of (a + b)k : k!/((k-d)!d!)
B0
B1
B2
B3
39
Binomial Queue Properties
Suppose you are given a binomial queue of N nodes
1. There is a unique set of binomial trees for N nodes
2. What is the maximum number of trees that can be in
an N-node queue?
– 1 node  1 tree B0; 2 nodes  1 tree B1; 3 nodes  2 trees
B0 and B1; 7 nodes  3 trees B0, B1 and B2 …
–
–
–
Trees B0, B1, …, Bk can store up to 20 + 21 + … + 2k =
2k+1 – 1 nodes = N.
Maximum is when all trees are used. So, solve for (k+1).
Number of trees is  log(N+1) = O(log N)
40
Definition of Binomial Queues
Binomial Queue = “forest” of heap-ordered binomial trees
B0 B2
B0
B1
B3
3
21
1
-1
5
7
9 6
7
Binomial queue H1
5 elements = 101 base 2
 B2 B0
2
3
1
8
11 5
Binomial queue H2
11 elements = 1011 base 2
 B3 B1 B0
6
41
findMin()
• In each Bi, the minimum key is at the root
• So scan sequentially B1, B2, ..., Bk, compute the
smallest of their keys:
• Time: O(log n) (why ?)
B0
B1
B2
B3
42
Binomial Queues: Merge
• Main Idea: Merge two binomial queues by merging
individual binomial trees
– Since Bk+1 is just two Bk’s attached together, merging trees
is easy
• Steps for creating new queue by merging:
1. Start with Bk for smallest k in either queue.
2. If only one Bk, add Bk to new queue and go to next k.
3. Merge two Bk’s to get new Bk+1 by making larger root
the child of smaller root. Go to step 2 with k = k + 1.
43
Example: Binomial Queue Merge
H1:
21
H2:
3
-1
1
7
2
3
1
8
5
9 6
11 5
7
6
44
Example: Binomial Queue Merge
H1:
H2:
3
-1
1
7
2
3
1
8
5
21
11 5
9 6
7
6
45
Example: Binomial Queue Merge
H1:
H2:
-1
1
7
5
2
3
21
3
1
8
9 6
11 5
7
6
46
Example: Binomial Queue Merge
H1:
H2:
-1
1
7
5
3
21
2
9 6
3
1
8
7
11 5
6
47
Example: Binomial Queue Merge
H1:
H2:
-1
2
1
3
1
8
7
11 5
6
5
3
21
9 6
7
48
Example: Binomial Queue Merge
H1:
H2:
-1
2
1
3
1
8
7
11 5
6
5
3
21
9 6
7
49
Binomial Queues: Merge and
Insert
• What is the run time for Merge of two O(N)
queues?
• How would you insert a new item into the
queue?
50
Binomial Queues: Merge and
Insert
• What is the run time for Merge of two O(N)
queues?
– O(number of trees) = O(log N)
• How would you insert a new item into the queue?
– Create a single node queue B0 with new item and
merge with existing queue
– Again, O(log N) time
• Example: Insert 1, 2, 3, …,7 into an empty
binomial queue
51
Insert 1,2,…,7
1
52
Insert 1,2,…,7
1
2
53
Insert 1,2,…,7
3
1
2
54
Insert 1,2,…,7
3
1
2
4
55
Insert 1,2,…,7
1
2
3
4
56
Insert 1,2,…,7
1
5
2
3
4
57
Insert 1,2,…,7
1
5
2
3
6
4
58
Insert 1,2,…,7
1
5
2
3
7
6
4
59
Binomial Queues: DeleteMin
• Steps:
1. Find tree Bk with the smallest root
2. Remove Bk from the queue
3. Delete root of Bk (return this value); You now have a
new queue made up of the forest B0, B1, …, Bk-1
4. Merge this queue with remainder of the original (from
step 2)
• Run time analysis: Step 1 is O(log N), step 2 and 3 are
O(1), and step 4 is O(log N). Total time = O(log N)
• Example: Insert 1, 2, …, 7 into empty queue and
DeleteMin
60
Insert 1,2,…,7
1
5
2
3
7
6
4
61
DeleteMin
5
2
3
7
6
4
62
Merge
5
2
3
7
6
4
63
Merge
5
2
3
7
6
4
64
Merge
5
2
6
7
3
4
65
Merge
5
2
6
7
3
4
DONE!
66
Implementation of Binomial Queues
• Need to be able to scan through all trees, and given
two binomial queues find trees that are same size
– Use array of pointers to root nodes, sorted by size
– Since is only of length log(N), don’t have to worry
about cost of copying this array
– At each node, keep track of the size of the (sub) tree
rooted at that node
• Want to merge by just setting pointers
– Need pointer-based implementation of heaps
• DeleteMin requires fast access to all subtrees of root
– Use First-Child/Next-Sibling representation of trees
67
Efficient BuildHeap for Binomial
Queues
• Insert one at a time - O(n log n)
• Better algorithm:
–
–
–
–
Start with each element as a singleton tree
Merge trees of size 1
Merge trees of size 2
Merge trees of size 4
• Complexity:  n 
n
n
(1  log1) 
(1  log 2) 
(1  log 4)  ...
 
 2
 
 4
 
8
 log n  i  
 n(1  i ) 
 
  O  n   i    O (n)
i
2 
i 1 
 i 1  2  
log n
 i 

 i 2
i 1  2 
log n
because
68