Chapter 1: Introduction
Download
Report
Transcript Chapter 1: Introduction
Chapter 1
Introduction
Levitin: Introduction to The Design and Analysis of Algorithms
Why study algorithms?
Theoretical importance
• the core of computer science
Practical importance
• A practitioner’s toolkit of known algorithms
• Framework for designing and analyzing algorithms for
new problems
1-1
Two main issues related to algorithms
How to design algorithms
How to analyze algorithm efficiency
1-2
What is an algorithm?
An algorithm is a sequence of unambiguous instructions
for solving a problem, i.e., for obtaining a required
output for any legitimate input in a finite amount of
time.
problem
algorithm
input
“computer”
output
1-3
What is an algorithm?
1.
Recipe, process, method, technique, procedure, routine,…
with following requirements:
Finiteness
terminates after a finite number of steps
2.
Definiteness
rigorously and unambiguously specified
3.
Input
valid inputs are clearly specified
4.
Output
can be proved to produce the correct output given a valid input
5.
Effectiveness
steps are sufficiently simple and basic
1-4
Alternative definition
An algorithm is a well-ordered collection of unambiguous
and effectively computable operations that, when executed,
produces a result and halts in a finite amount of time.
An algorithm is a sequence of unambiguous instructions for
solving a problem, i.e., for obtaining a required output for
any legitimate input in a finite amount of time.
Muhammad ibn Musa al-Khwarizmi – 9th century
mathematician
www.lib.virginia.edu/science/parshall/khwariz.html
1-5
Historical Perspective
Euclid’s algorithm for finding the greatest common divisor
• third century BC
What are alternative ways we can compute the gcd of two
integers?
1-6
Euclid’s Algorithm
Problem: Find gcd(m,n), the greatest common divisor of two
nonnegative, not both zero integers m and n
Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?
Euclid’s algorithm is based on repeated application of equality
gcd(m,n) = gcd(n, m mod n)
until the second number becomes 0, which makes the problem
trivial.
Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
1-7
Two descriptions of Euclid’s algorithm
Step 1 If n = 0, return m and stop; otherwise go to Step 2
Step 2 Divide m by n and assign the value fo the remainder to r
Step 3 Assign the value of n to m and the value of r to n. Go to
Step 1.
while n ≠ 0 do
r ← m mod n
m← n
n←r
return m
1-8
Other methods for computing gcd(m,n)
Consecutive integer checking algorithm
Step 1 Assign the value of min{m,n} to t
Step 2 Divide m by t. If the remainder is 0, go to Step 3;
otherwise, go to Step 4
Step 3 Divide n by t. If the remainder is 0, return t and stop;
otherwise, go to Step 4
Step 4 Decrease t by 1 and go to Step 2
1-9
Other methods for gcd(m,n) [cont.]
Middle-school procedure
Step 1
Step 2
Step 3
Step 4
Find the prime factorization of m
Find the prime factorization of n
Find all the common prime factors
Compute the product of all the common prime factors
and return it as gcd(m,n)
Is this an algorithm?
1-10
Sieve of Eratosthenes
Input: Integer n ≥ 2
Output: List of primes less than or equal to n
for p ← 2 to n do A[p] ← p
for p ← 2 to n do
if A[p] 0 //p hasn’t been previously eliminated from the list
j ← p* p
while j ≤ n do
A[j] ← 0 //mark element as eliminated
j←j+p
Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
1-11
Basic Issues Related to Algorithms
How to design algorithms
How to express algorithms
Proving correctness
Efficiency
• Theoretical analysis
• Empirical analysis
Optimality
1-12
1.2 Fundamentals of Algorithmic Problem
Solving
Algorithms are procedural solutions to problems
Steps for designing and analyzing an algorithm
• Understand the problem
• Ascertain the capabilities of a computational device
• Choose between exact and approximate problem solving
• Decide on appropriate data structures
1-13
Fundamentals of Algorithmic Problem Solving, continued:
Algorithm design techniques/strategies
Brute force
Greedy approach
Divide and conquer
Dynamic programming
Decrease and conquer
Iterative improvement
Transform and conquer
Backtracking
Space and time tradeoffs
Branch and bound
1-14
Fundamentals of Algorithmic Problem Solving continued
Methods of specifying an algorithm
• Present: pseudocode
• Earlier: flowchart
Proving an algorithm’s correctness
•
•
•
•
Mathematical induction for recursion
Modus ponens
E.g. algorithm stops since number gets smaller on each iteration
Approximation algorithms are more difficult
1-15
Fundamentals of Algorithmic Problem Solving, continued:
Analysis of Algorithms and Coding
How good is the algorithm?
• Correctness
• Time efficiency
• Space efficiency
Does there exist a better algorithm?
• Lower bounds
• Optimality
Algorithm ultimately implemented as a computer program
• Success depends on above considerations
1-16
1.3 Important problem types
sorting
searching
string processing
graph problems
combinatorial problems
geometric problems
numerical problems
1-17
Example of computational problem: sorting
Statement of problem:
• Input: A sequence of n numbers <a1, a2, …, an>
• Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that
a´i ≤ a´j whenever i < j
Instance: The sequence <5, 3, 2, 8, 3>
Algorithms:
•
•
•
•
Selection sort
Insertion sort
Merge sort
(many others)
1-18
Selection Sort
Input: array a[1],…,a[n]
Output: array a sorted in non-decreasing order
Algorithm:
for i=1 to n
swap a[i] with smallest of a[i],…a[n]
• see also pseudocode, section 3.1
1-19
Fundamental data structures
list
graph
• array
tree
• linked list
set and dictionary
• string
stack
queue
priority queue
1-20
Data Structure
Definition:
• “A particular scheme of organizing related data items.”
• Example:
– Array
– String
– List
• Classified into
–
–
–
–
Linear
Graphs
Tree
Abstract
1-21
Linear Structures
Array
String (Type of Mathematical Sequence)
• Character, Binary, Bit…
Linked List
• Nodes, Head, Tail
• Singly Linked, Doubly Linked,
Stack
• LIFO, Top, Push, Pop
Queue
• FIFO, Front, Rear/End, Enqueue, Dequeue
1-22
Graphs
Graph –
• Def: “A graph G = <V,E> is defined by a pair of two sets: a finite set
V of items called vertices and a set E of pairs (2-tuples) of vertices
that represent the edges between the vertices.
• Undirected, Directed (digraph)
• Loops
• Complete, Sparse, Dense
1-23
Representing Graphs
Graphs –
• Inherently abstract
• Directed or Undirected
• Two common representations:
– Adjacency matrix int P[][] = new int P[][];
– Adjacency linked list
– LinkedList<Edge> Q = new LinkedList<Edge>( );
A
B
C
A
0
1
1
B
1
1
0
C
0
0
1
{ (A,B),(A,C),(B,A),(B,B),(C,C) }
24
Weighted Graphs
Weighted Graph –
• A graph G, with the added property that the edges between nodes is
assigned a “weight”
• Edge weight may represent: cost, distance, time…
A
B
A
0
1.5 2.3
B
1.3 2.0 0
C
0
0
C
1.1
{
(A,B,1.5),(A,C,2.3),(B,A,1.3),
(B,B,2.0),(C,C,1.1)
}
25
Graph Terms (1/2)
Path –
• A path from vertex u and v of a graph G exists if there exists a set
adjacent edges that starts with u and ends with v
Simple –
• The path contains only unique edges (no duplicates)
Length of a path –
• The number of edges in the aforementioned set
26
Graph Terms (2/2)
Connected –
• iff, for every pair of its vertices, <u,v>, there is a path from u to v.
Connected Components –
• If a graph is not connected, it is made of two or more connected
components.
• The “maximal” connected subgraph of a given graph.
Cycle –
• A simple path, of non-zero length, that starts & ends at the same
vertex.
Acyclic Graph –
• A graph with no cycles
27
Trees
Tree
• A connected acyclic graph
• Properties
– |E| = |V|-1
– For every two vertices, <u,v>, there exists exactly one simple path from
u to v.
Forest
• A set of trees
• An acyclic graph that is not necessarily connected
28
Rooted Trees
Rooted Tree
• A tree, with a “root” node
• Root nodes are selected arbitrarily (any vertex)
Tree genealogy
•
•
•
•
•
•
Ancestors
Children
Siblings
Parents
Descendents
Leaf nodes
29
Example
Convert the graph to a rooted tree
• Identify the root node
• Find an ancestor, child, sibling, parent, descendent, and leaf
nodes
D
A
G
B
E
C
H
F
30
Height & Depth of Trees
Depth
• Depth of vertex v is the length of the simple path from the root to v
• Count the edges
Height
• Height of a tree is the length of the longest simple path from the
root to the deepest leaf nodes
31
Binary Tree
Binary Tree
• Simplest tree structure
• Every node has the property: vi {0,1,2} children.
• “Left child” and “Right child”
32
Representing Trees
Linked Lists (ala LISP)
(A (B X) (C X (D E))))
Arrays (for defined n-ary) trees
ABCBXCXXXXXDE
class treeNode<Type> {
Type data;
LinkedList<TreeNode<Type>> children;
}
33
Graph Algorithm
Determine if a graph contains a “universal sink” • A graph with at least one node with an in-degree of |V|-1, and an
out-degree of 0.
34