DAA Sadaf Riaz

Design and Analysis of Algorithms

Lecture 2

Instructor : Sadaf Riaz

Class: CS 6th A & B

Algorithm Analysis

❖ Analysis determines the runtime of an algorithm

based on the input size

❖ Gives you the relative rate of growth but not

specific time/space of execution

❖ Allows for comparison of algorithms without

implementation

❖ Not hardware specific

❖ Describes the number of operations for a given

What is “Input Size”?

❖ Depends on the problem being analyzed

❖ The number of items in the input is often used

Number of values in a sorting algorithm

❖ May use more than one parameter

Asymptotic Analysis of Algorithms

(Growth of function)

❖ The word Asymptotic means approaching a value or curve arbitrarily closely (i.e., as some sort of limit is taken).

❖ Execution time of an algorithm depends on the instruction set, processor speed, disk I/O speed, etc. Hence, we

estimate the efficiency of an algorithm asymptotically.

❖ It is a technique of representing limiting behavior.

❖ It can be used to analyze the performance of an algorithm for some large input data set.

❖ Asymptotic notations are used to write fastest and slowest possible running time for an algorithm('best case' and

'worst case' scenarios respectively).

Asymptotic Analysis of Algorithms

(Growth of function)

❖ Examples:

❖ Let f (n) = an2+bn+c

(n2 dominates the function that is when n gets sufficiently large)

❖ ƒ (n) = n2+3n

(The term 3n becomes insignificant compared to n2 when n is very large.)

❖ The function "ƒ (n) is said to be asymptotically equivalent to n2 as n → ∞“

Types of Asymptotic Notations

❖ Following asymptotic notations are used to

calculate the running time complexity of an algorithm.

1) O − Big Oh (Asymptotic Upper Bound)

2) Ω − Big omega (Asymptotic Lower Bound)

3) θ − Big theta (Asymptotic Tight Bound)

4) o − Little Oh

1) O − Big Oh (Asymptotic Upper Bound)

❖ Big-oh is the formal method of expressing the

upper bound of an algorithm's running time.

❖ It is the measure of the longest amount of time.

❖ The function f (x) = O (g (x)) [read as "f of x is

big-oh of g of x"] if and only if exist positive constant C and k such that

|f(x)| ≤ C|g(x)| whenever x > k in all cases

❖ function g(x) is an upper bound for function f(x),

1) O − Big Oh (Asymptotic Upper Bound)

❖ Big-O provides an upper bound on a function (to

within a constant factor)

❖ O(g(x)) is a set of functions

❖ Commonly used notation

f(x) = O(g(x))

❖ Correct notation

f(x) ∈ O(g(x))

❖ Meaningless statement

O(g(x)) = f(x)

We will use both

1) O − Big Oh (Asymptotic Upper Bound)

Properties of O − Big Oh

❖ If “f(x) is O(g(x))” then g(x) grows at least as fast

as f(x) (reflexive)

❖ “f(x) is O(g(x))” iff O(f(x)) ⊆ O(g(x))

❖ If “f(x) is O(g(x))” and “g(x) is O(f(x))” then

O(f(x))=O(g(x)) (symetric)

❖ If “f(x) is O(g(x))” and “h(x) is O(g(x))” then

1) O − Big Oh (Asymptotic Upper Bound)

Properties of O − Big Oh

❖ If a is a scalar and “f(x) is O(g(x))”, then

a*f(x) is O(g(x)) If “f(x) is O(g(x))” and “g(x) is O(h(x))”, then

f(x) is O(h(x)) (transitive property)

❖ f(x) = O(g(x))

Common Complexity Functions

Complexity

Term

O(1) constant

O(log n) logarithmic

O(n) linear

O(n log n) n log n

O(nb) polynomial

O(bn) b > 1 exponential

Important Big-O Result

❖ Let f(x)=a_nxn+a_n-1xn-1+…+a₁x+a₀, where a₀, a₁,

…,a_n-1, an are real numbers.

Then f(x) is O(xn).

❖ 3n+2=O(n) as 3n+2≤4n for all n≥2

❖ 3n+3=O(n) as 3n+3≤4n for all n≥3

❖ f(n)=4.n3+10.n2+5.n+1

Considering g(n)=n3, f(n)⩽5.g(n) for all the

values of n>2 so the complexity of f(n) can be

Big-O Estimates

❖ What is the big-O estimate for n!?

n! = 1 • 2 • 3 • … • n

⇒ _{n! ≤ n • n • n • … • n}

⇒ n! ≤ nn n! is O(nn)

⇒ _{log n! ≤ log n}n

⇒ _{log n! ≤ n log n} log n! is O(n log n)

❖ What is the big-O estimate for log(n!)?

2) Ω − Big omega (Asymptotic Lower

Bound)

❖ Big-Omega is the formal method of expressing

lower bound for a function to within a constant factor.

❖ Let f and g be functions from N or R to R. We

say that “f(x) is Ω(g(x))” if there are positive constants C and k such that

|f(x)| ≥ C |g(x)| whenever x > k.

3) θ − Big theta (Asymptotic Tight

Bound)

❖ f(x)=O(g(x)) only provides an upper bound in

terms of g(x) for f(x).

❖ f(x)= Ω(g(x)) only provides a lower bound in

terms of g(x) for f(x).

❖ Big-Theta provides both an upper bound and a

3) θ − Big theta (Asymptotic Tight

Bound)

❖ Let f and g be functions from N or R to R. We

say that “f(x) is θ(g(x))” if “f(x) is O(g(x))” and “f(x) is Ω(g(x))”, i.e., there are positive constants

C₁, C₂, and k such that

C₁|g(x)| ≤ |f(x)| ≤ C₂|g(x)| whenever x>k.

❖ Read as: “f(x) is big-Theta of g(x)”

The “sets” and their use – big Oh

❖ Big “oh” - asymptotic upper bound on the growth

of an algorithm

❖ When do we use Big Oh?

1. Theory of NP-completeness

2. To provide information on the maximum number of operations that an algorithm performs

– Insertion sort is O(n2) in the worst case

• This means that in the worst case it performs at most cn2 operations – Insertion sort is also O(n6) in the worst case since it also performs at most

The “sets” and their use – Omega

Omega - asymptotic lower bound on the growth of an algorithm or a

problem*

When do we use Omega?

1. To provide information on the minimum number of operations that an algorithm performs

– Insertion sort is Ω(n) in the best case

• This means that in the best case its instruction count is at least cn,

– It is Ω(n2) in the worst case

The “sets” and their use – Omega

2. To provide information on a class of algorithms that solve a problem

– Sort algorithms based on comparison of keys are Ω(nlgn) in the worst case

• This means that all sort algorithms based only on

comparison of keys have to do at least cnlgn operations – Any algorithm based only on comparison of keys to find the

maximum of n elements is Ω(n) in every case