Basic Sorts (Analysis)

Basic Sorts (Analysis)#

Internships & Jobs#

Worst-case, Average-case, Best-case#

Warm-up 1

Analyze this code

unsigned int argmin (const std::vector<int> &values) { 
    unsigned int length = values.size();
    assert(length > 0);
    unsigned int idx = 0;
    int current = values[0];
    for (unsigned int i = 1; i < length; i++) {
        if (values[i] < current) {
            current = values[i];
            idx = i;
        }
    }
    return idx;
}

Warm-up 2

Analyze this code

bool argk (const std::vector<int> &values, 
                                    int k, 
                        unsigned int &idx) 
{  
    unsigned int length = values.size();
    for (unsigned int i = 0; i < length; i++) {
        if (values[i] == k) {
            idx = i;
            return true;
        }
    }
    return false;
}

Analysis types#

Note

While asymptotic analysis describes \(T(n) \Rightarrow \infty\)…

asymptotic notation: big-O, big-\(\Omega\), \(\Theta\)

Case analysis looks into the different input types

best-case, worst-case, average-case

\[Both\ analysis\ types\ are\ orthogonal\ to\ each\ other\]

Examples#

Factorial of a number (iterative algorithm)

// pseudocode

function factorial(n):
    if n <= 1:
        return 1
    result = 1
    for i in range(1, n+1):
        result *= i
    return result

Sequential search (return first occurrence)

// pseudocode

function find_first(items, target):
    index = 0
    while index < len(items):
        if items[index] == target:
            return index
        index += 1
    return -1

Sequential search (return last occurrence)

// pseudocode

function find_last(items, target):
    index = len(items) - 1
    while index >= 0:
        if items[index] == target:
            return index
        index -= 1
    return -1

Basic Sorting Algorithms#

Sorting#

Given \(n\) elements that can be compared according to a total order relation

we want to rearrange them in non-increasing/non-decreasing order
for example (non-decreasing):
- input: sequence of items \(\ \ \ \ \ \ \ \ \ \ \ A = [k_0, k_1, \dots, k_{n - 1}]\)
- output: permutation of A \(\ \ \ \ \ \ \ \ \ \ \ B\ |\ B[0]\ \le\ B[1]\ \le \dots B[n - 1]\)

\[\begin{split}\\ \\ \\ Central\ problem\ in\ computer\ science\end{split}\]

Insertion Sort #

Defined as…

Array is divided into sorted and unsorted parts

algorithm scans array from left to right

Invariants

elements in sorted are in ascending order
elements in unsorted have not been seen

void insertion_sort(int* items, int size) {
    //grows the left part (sorted)
    for (int i = 1; i < size; i++) {
        int j = i;
        //inserts j in sorted part
        while (j > 0 && items[j-1] > items[j]) {
            swap(items[j-1], items[j]);
            j--;
        }
    }
}

\[Number\ of\ comparisons?\ |\ Number\ of\ exchanges?\]

Analysis — Insertion Sort(comparisons)#

Running time depends on the input

Worst-case?#: input is already sorted in descending order

\[\begin{split}\begin {align} \sum_{i = 1}^{N - 1} i &= 1 + 2 + 3 + \dots + (N - 1) \\ &= \frac{N(N - 1)}{2} \\ &= O(N^2) \\ \end {align}\end{split}\]

Average-case?#

each element \(\approx\) halfway order

expect every element to move \(O(\frac{n}{2})\) times

\[\begin{split}\begin {align} \sum_{i = 1}^{N - 1} \frac{i}{2} &= \frac{1}{2} (1 + 2 + 3 + \dots + (N - 1)) \\ &= \frac{N(N - 1)}{4} \\ &= O(N^2) \\ \end {align}\end{split}\]

Best-case?#: input is already sorted in ascending order

\[\begin{split}\begin {align} \sum_{i = 1}^{N - 1} 1 &= (N - 1) \\ &= O(N) \\ \end {align}\end{split}\]

{90, 80, 70, 60, 50, 40, 30, 20}

{60, 50, 30, 90, 20, 70, 80, 40}

{20, 30, 40, 50, 60, 70, 80, 90}

Partially sorted arrays#

inversion

A pair of keys that are out of order

“array is partially sorted if the number of pairs that are out-of-order is \(O(n)\)”

For partially-sorted arrays, insertion sort runs in linear time.

\[Θ(n)\]

Selection Sort #

Defined as

Array is divided into sorted and unsorted parts

algorithm scans array from left to right

Invariants

elements in sorted are fixed and in ascending order
no element in unsorted is smaller than any element in sorted

void selectionsort (int *A, unsigned int n) {
    int temp;
    unsigned int i, j, min;
    // grows the left part (sorted) 
    for (i = 0; i < n; i++) {
        min = i;
        // find min in unsorted part
        for (j = i + 1; j < n; j++) {
            if (A[j] < A[min]) {
                min = j;
            }
        }
        // swap A[i] and A[min]
        temp = A[i];  
        A[i] = A[min];
        A[min] = temp;
    }
}

Summary#

Incomplete

	Best-Case	Average-Case	Worst-Case
Selection Sort
Insertion Sort

Complete

	Best-Case	Average-Case	Worst-Case
Selection Sort	\(O(n^2)\)	\(O(n^2)\)	\(O(n^2)\)
Insertion Sort	\(O(n)\)	\(O(n^2)\)	\(O(n^2)\)