Transcript Lecture 6: Algorithms: Sorting

ME 171
Computer Programming
Language
Partha Kumar Das
Lecturer,
Department of Mechanical Engineering, BUET
Lecture 6
Algorithms: Sorting
Algorithms
• In mathematics and computer science, an algorithm is a self-contained step-by-step set
of operations to be performed for a given problem.
• When an algorithm is developed to certain problem, then source code is written based
on the algorithm.
Algorithms
• A particular problem can be solved using different types of
algorithms.
• The most efficient algorithms can be defined based on the
following factors:
1. Execution Time
2. Number of statements in the source code
3. Complexity in understanding and
4. Conformation of the solution
I have chosen a particular number from the sorted list below.
Guess the number by making guesses until you find the number.
1
2
3
4
5
6
7
8
9
• A novice one would guess the first number(1) and then
second and repeat it until he gets the chosen number (3).
• But a good one would choose mid number at each turn so
that he will need less turns to select the guessed number.
Sorting
• Suppose you are a librarian whose task is to maintain the books of the same category in
one row. Then you have to sort the books of the same categories, give them codes and
again sort each book in ascending or descending order so that reader could easily find
desired book.
• Several algorithms have been developed to sort an array of numbers.
• From the birth of computer knowledge,
the algorithm for sorting achieved a
great appeal due to its simplicity but
potential and complexity to find out the
most efficient algorithm to solve.
• Some most common are:
Bubble Sort (Most inefficient)
Selection Sort
Insertion Sort
Shell Sort
Quick Sort
Bucket Sort
Merge Sort
Binary Sort
Tim Sort, etc.
Source; Wikipedia
Bubble Sort
• The most ancient sorting algorithm developed in 1956.
• It is the most simple and inefficient and merely used for sorting an approximately fully
sorted array in which some elements have accidentally changed position.
• The algorithm works by comparing each item in the list with the item next to it, and
swapping them if required.
• In other words, the largest element has bubbled to the top of the array.
• The algorithm repeats this process until it makes a pass all the way through the list
without swapping any items.
Example: Sorting of 7, 5, 2, 4, 3, 9 using Bubble sort algorithm:
1st Pass
7, 5, 2, 4, 3, 9
5, 7, 2, 4, 3, 9
5, 2, 7, 4, 3, 9
5, 2, 4, 7, 3, 9
5, 2, 4, 3, 7, 9
5, 2, 4, 3, 7, 9
2nd Pass
5, 2, 4, 3, 7, 9
2, 5, 4, 3, 7, 9
2, 4, 5, 3, 7, 9
2, 4, 3, 5, 7, 9
2, 4, 3, 5, 7, 9
2, 4, 3, 5, 7, 9
3rd Pass
2, 4, 3, 5, 7, 9
2, 4, 3, 5, 7, 9
2, 3, 4, 5, 7, 9
2, 3, 4, 5, 7, 9
2, 3, 4, 5, 7, 9
2, 3, 4, 5, 7, 9
4th Pass
2, 3, 4, 5, 7, 9
2, 3, 4, 5, 7, 9
2, 3, 4, 5, 7, 9
2, 3, 4, 5, 7, 9
2, 3, 4, 5, 7, 9
#include <stdio.h>
int main()
{
int i,j,temp,array[5];
printf("Input an array of 5 elements:\n");
for(i=0;i<5;i++)
scanf("%d",&array[i]);
for(i=0;i<5;i++)
// O(i*i)
{
for(j=0;j<=i;j++)
// O(i)
• The worst-case runtime complexity is O(n2).
{
if(array[j]>array[i]) // O(1)
{
temp=array[j];
array[j]=array[i];
array[i]=temp;
}
}
}
printf("The sorted array is:\n");
for(i=0;i<5;i++)
printf("%d\t",array[i]);
return 0;}
Insertion Sort
• One of the simplest methods to sort an array is an insertion sort.
• An example of an insertion sort occurs in everyday life while playing cards. To sort the
cards in your hand you extract a card, shift the remaining cards, and then insert the
extracted card in the correct place.
• This process is repeated until all the cards are in the correct sequence.
Example: Sorting 26, 20, 73, 34, 64 using insertion sort algorithm:
29, 20, 73, 34, 64
29, 20, 73, 34, 64
20, 29, 73, 34, 64
20, 29, 73, 34, 64
20, 29, 34, 73, 64
20, 29, 34, 64, 73
1st turn
2nd turn
3rd turn
4th turn
5th turn
6th turn
• we need 0 comparisons to insert the first element
we need 1 comparison to insert the second element
we need 2 comparisons to insert the third element
...
we need (N-1) comparisons (at most) to insert the last element.
• Totally,1 + 2 + 3 + ... + (N-1) = O(n2)
• The worst-case runtim ecomplexity is O(n2).
• The best-case runtime complexity is O(n).
Shell Sort
• Shell sort, developed by Donald L. Shell improves on the efficiency of insertion sort by
quickly shifting values to their destination.
• Average sort time is O(n^7/6), while worst-case time is O(n^4/3)
Insertion
Sort
Shell Sort
• Total number of shift in insertion sort: 2+2+1=5
• Total number of shift in shell sort: 1+1+1=3
Shell Sort
• Shell sort begins by an insertion sort using a spacing of two. In the first frame we examine
numbers 3-1. Extracting 1, we shift 3 down one slot for a shift count of 1. Next we
examine numbers 5-2. We extract 2, shift 5 down, and then insert 2. After sorting with a
spacing of two, a final pass is made with a spacing of one. This is simply the traditional
insertion sort. The total shift count using shell sort is 1+1+1 = 3. By using an initial
spacing larger than one, we were able to quickly shift values to their proper destination.
Insertion
Sort
Shell Sort
• Total number of shift in insertion sort: 2+2+1=5
• Total number of shift in shell sort: 1+1+1=3
Shell Sort
• Various spacings may be used to implement a shell sort.
• Typically the array is sorted with a large spacing, the spacing reduced, and the array sorted again.
•
•
•
•
•
•
On the final sort, spacing is one.
Although the shell sort is easy to comprehend, formal analysis is difficult.
The question of deciding which spacing to use is difficult.
Every spacing that contains 1 yields a correct sort.
There numerous formulas of spacings published so far.
Knuth recommends a technique, due to Sedgewick, that determines spacing h based on the
following formulas:
hs = 9.2𝑠 - 9.2𝑠/2 + 1, if s is even
hs = 8.2𝑠 - 6.2(𝑠+1)/2 + 1, if s is odd
These calculations result in values (h0, h1, h2,…) = (1, 5, 19, 41, 109, 209,…).
Calculate h until 3ht >=N, where N is the number of elements in the array.
Then choose ht-1 for a starting value.
For example, to sort 150 items, ht = 109 (cause, 3*109 >= 150), so the first spacing is ht-1, or
41. The second spacing is 19, then 5, and finally 1.
•
Another example of shell sort using Papernov & Stasevich spacing formula (1, 3, 5, 9, ……) is below:
a1
a2
a3
a4
a5
a6
a7
a8
a9
a10
a11
a12
Input data
62
83
18
53
07
17
95
86
47
69
25
28
After 5-spacing
17
28
18
47
07
25
83
86
53
69
62
95
After 3-spacing
17
07
18
47
28
25
69
62
53
83
86
95
After 1-spacing
07
17
18
25
28
47
53
62
69
83
86
95
Quick Sort
• Quick sort is an efficient sorting algorithm, serving as a systematic method for placing the
elements of an array in order.
• Developed by Tony Hoare in 1959, with his work published in 1961, it is still a commonly
used algorithm for sorting.
• Mathematical analysis of quicksort shows that, on average, the algorithm takes O(n log n)
comparisons to sort n items.
• In the worst case, it makes O(n^2) comparisons, though this behavior is rare.
ALGORITHM:
1. Pick an element, called a pivot, from the array. Picking this pivot element is the most
important decision to be made in quick sort. Always try to pick the mid element as pivot.
Picking corner element will result in worst case scenario on already sorted array.
2. Partitioning: Reorder the array so that all elements with values less than the pivot come
before the pivot, while all elements with values greater than the pivot come after it (equal
values can go either way). After this partitioning, the pivot is in its final position. This is
called the partition operation.
3. Recursively apply the above steps to the sub-array of elements with smaller values of the
previous pivot and to the sub-array of elements with greater values of the previous pivot
separately.
Selection Sort
• Selection sort is performs worse than the similar insertion sort.
• Selection sort is noted for its simplicity, and it has performance advantages over more
complicated algorithms in certain situations, particularly where auxiliary memory is limited.
ALGORITHM:
1.Find the smallest number. Swap it with the first place number.
2.Find the second-smallest number. Swap it with the second place number.
3.Find the third-smallest number. Swap it with the third place number.
4.Repeat finding the next-smallest number, and swapping it into the correct position until the
array is sorted.
When you sort exam sheets after examination, what sorting algorithm do you use, Selection or
Insertion? Write the C source code of your algorithm.
References:
• Introduction to Algorithms, by Thomas H. Cormen, Charles E. Leiserson, Ronald L.
Rivest and Clifford Stein.
• https://visualgo.net/sorting
• https://www.khanacademy.org/computing/computer-science/algorithms
• https://www.cs.cmu.edu/~adamchik/15-121/lectures/Sorting%20Algorithms/sorting.html
• Sorting and searching algorithms, by Thomas Niemann.
• https://en.wikipedia.org/wiki/Sorting_algorithm
• The Art of Computer Programming vol.: 1, 2, 3, by Donald E. Knuth.