Algorithm Analysis¶

Algorithm:

1. Input \((\geq 0)\)
2. Output \((\geq 1)\)
3. Definiteness
4. Finiteness
5. Effectiveness

for (i = 0; i < n; i++){
    Examine list[i] to list[n-1] and suppose that the smallest is at list[min];
    Interchange list[i] and list[min];
}

What to Analyze¶

• Time & space complexity: Machine & compiler independent

Assumptions:

1. instructions are executed sequentially
2. each instruction is simple, and takes exactly one time unit

3. integer size is fixed and we have infinite memory

4. \(T_{\text{avg}}(N)\) & \(T_{\text{worst}}(N)\)

the average and worst-case time complexity as a function of the input size \(N\)

void add(int a[][MAX_SIZE],
         int b[][MAX_SIZE],
         int c[][MAX_SIZE],
         int rows, int cols)
{
    int i, j;
    for (i = 0; i < rows; i++) /* rows+1 times (the last time is for the exit condition) */
        for (j = 0; j < cols; j++) /* rows·(cols +1) */
            c[i][j] = a[i][j] + b[i][j]; /* rows·cols */
}

float sum(float list[], int n)
{
    float sum = 0; /* 1 */
    int i;
    for (i = 0; i < n; i++) /* n+1 times */
        sum += list[i]; /* n */
    return sum; /* 1 */
}

float rsum(float list[], int n)
{
    if (n) /* n+1 */
        return list[n-1] + rsum(list, n-1); /* n */
    return 0; /* 1 */
}

Asymptotic Notation ( \(\mathcal{O}\) )¶

The point of counting the steps is to predict the growth in run time as \(N\) changes, so we only care about the asymptotic behavior.

Definition¶

• \(T(N) = \mathcal{O}(f(N))\) if there are positive constants \(c\) and \(N_0\) such that \(T(N) \leq c \cdot f(N)\) for all \(N \geq N_0\). (Worst case)
• \(T(N) = \Omega(g(N))\) if there are positive constants \(c\) and \(N_0\) such that \(c \cdot g(N) \leq T(N)\) for all \(N \geq N_0\). (Best case)
• \(T(N) = \Theta(h(N))\) if and only if \(T(N) = \mathcal{O}(h(N))\) and \(T(N) = \Omega(h(N))\).
• \(T(N) = o(p(N))\) if \(T(N) = \mathcal{O}(p(N))\) and \(T(N) \neq \Theta(p(N))\).

Notes:

• We shall always take the smallest \(f(N)\) (smallest upper bound)
• We shall always take the largest \(g(N)\) (largest lower bound)

Rules¶

If \(T_1(N) = \mathcal{O}(f(N))\) and \(T_2(N) = \mathcal{O}(g(N))\), then:

1. \(T_1(N) + T_2(N) = \max(\mathcal{O}(f(N)), \mathcal{O}(g(N)))\)

2. \(T_1(N) \cdot T_2(N) = \mathcal{O}(f(N) \cdot g(N))\)

3. If \(T(N)\) is a polynomial of degree \(k\), then \(T(N) = \Theta(N^k)\).

4. \(\log^k N = \mathcal{O}(N^{\epsilon})\) for any \(\epsilon > 0\).

int fib(int n)
{
    if (n <= 1) // O(1)
        return 1; // O(1)
    else
        return fib(n-1) + fib(n-2); // T(n-1) + T(n-2)
}

Compare the Algorithms¶

- $T(N) = \mathcal{O}(N^2)$

分治算法：

例：`[4,-3,5,-2,-1,2,6,-2]`

把序列分为两半，最大子串可能在左半部分、右半部分、或者跨越中间，共 3 种情况

$$
\begin{aligned}
T(N) &= 2T(N/2) + cN, \quad T(1) = \mathcal{O}(1) \\
&= 2(2T(N/4) + cN/2) + cN \\
&... \\
&= 2^k T(N/2^k) + k \cdot cN \\
&\Rightarrow N/2^k = 1 \Rightarrow k = \log N \\
&\Rightarrow T(N) = \mathcal{O}(N \log N)
\end{aligned}
$$

```c {.line-numbers} title="Algorithm 2: Divide and Conquer"
/*  */

Logarithms in the Running Time¶

int BinarySearch(const int A[], int N, int X)
{
    int low, mid, high;
    low = 0;
    high = N - 1;
    while (low <= high)
    {
        mid = (low + high) / 2;
        if (A[mid] < X)
            low = mid + 1;
        else if (A[mid] > X)
            high = mid - 1;
        else
            return mid;
    }
    return -1;
}

Checking Your Analysis¶

• Method 1
  • When \(T(N) = \mathcal{O}(N)\), check if \(T(2N)/T(N) \approx 2\)