Computational Cost

Computational Cost#

Analyzing running time#

Empirical Model

Run algorithm
Measure actual time

Mathematical Model

Analyze algorithm
Develop Model

Theoretical Models#

Mathematical Model

High-level analysis
- no need to implement
Independent of hardware / software
Based on counts of basic instructions
- additions, multiplications, comparisons,etc
- exact definition not important but must be relevant to the problem

Basic assumptions

In order to use a formal framework, we will make certain assumptions
- count basic instructions: additions, multiplications, comparisons, assignments
- each instruction takes one time unit
- instructions are executed sequentially
- infinite memory
Focus on analyzing running time

Examples#

Model

What to count?

only relevant instructions?
all instructions?

for (int i = 0; i < n; i++) {
    std::cout << (2 * i);
}

Time Complexity & Why…

The time complexity of this code is \(O(n)\), because the operation within the loop is done n times.

Application

#include <iostream>
using namespace std;

int main()
{
  int i, n = 8;
  for (i = 1; i <= n; i++) {
    cout << "Hello World !!!\n";
  }
  return 0;
}

Output

Hello World !!!
Hello World !!!
Hello World !!!
Hello World !!!
Hello World !!!
Hello World !!!
Hello World !!!
Hello World !!!

Time Complexity: In the above code “Hello World !!!” is printed only n times on the screen, as the value of n can change. So, the time complexity is linear: \(O(n)\) i.e. every time, a linear amount of time is required to execute code.
Auxiliary Space: \(O(1)\)

Example 1

for (int i = 0; i < n; i++) {
    for (int j = 0; j < n; j++) {
        std::cout << (i * j);
    }
}

Time Complexity & Why…

This code has a time complexity of \(O(n^2)\). This is because it has nested loops that are both iterating over n elements, so the total number of operations increases quadratically with the size of the input.

Example 2

for (int i = 0; i < n; i++) {
    for (int j = 0; j < n * n; j++) {
        std::cout << (i * j);
    }
}

Time Complexity & Why…

The time complexity of this code is \(O(n^3)\). This is because there are two nested for loops, each of which has a time complexity of \(O(n)\), resulting in a time complexity of \(O(n^2)\). The total time complexity is \(O(n^3)\) because the inner loop is iterating \(n * n\) times.

Example 3

for (int i = 0; i < n; i++) {
    for (int j = 0; j < i; j++) {
        sum = sum * j;
    }
}

Time Complexity & Why…

This code has a time complexity of \(O(n^2)\). This is because there is an outer loop that runs n times and an inner loop that runs i times, where i is the value of the outer loop’s iterator. Since i starts at 0 and increases each iteration of the loop, the inner loop will run 0 times on the first iteration, 1 time on the second iteration, 2 times on the third iteration, and so on, up to n times on the last iteration. This results in a total of \(n(n+1)/2\) iterations of the inner loop, which is equivalent to \(O(n^2)\).

\[\begin{split}\begin {align} n + n-1 + n-2 + ... + 1 & = n(n+1)/2 \\ & = O(n^2) \\ \end {align}\end{split}\]

Example 4

for (int i = 0; i < n; i++) {
    for (int j = 0; j < n; j++) {
        for (int k = 0; k < n; k++) {
            //count 1 instruction
        }
    }
}

Time Complexity & Why…

The time complexity of this code is \(O(n^3)\) because there are three nested loops, each of which runs n times. This results in \(n^3\) total iterations of the innermost loop.

Example 5

for (int i = 0; i < n; i++) {
    for (int j = 0; j < i * i; j++) {
        for (int k = 0; k < j; k++) {
            // count 1 instruction
        }
    }
}

Time Complexity & Why…

This code has a time complexity of \(O(n^4)\). This is because the first loop runs in \(O(n)\) time, the second loop runs in \(O(n^2)\) time, and the third loop runs in \(O(n)\) time. Therefore, the total time complexity is the product of all of these which is \(O(n^4)\).

Some rules…#

Single loops

essentially, \(total\ iterations\ \times total\ instructions\ performed\ per\ iteration\)

Nested loops

count instructions inside out
careful with the range of the loop
when possible, multiplications can be used for counts from each loop

Consecutive statements

just add the counts

Conditionals

consider the branch with the highest count