Lecture-5-Measuring-an-Algorithm-efficiency-2011-1.pdf

(1)

Measuring an Algorithm efficiency: Big Oh

Notation

• Program design = algorithm + data

Program design = algorithm + data

• Algorithm: is a finite set of clear meaning

steps that can be performed in a finite

amount of time.

Al

ith

i

l

i

l

f t

• Algorithm: is a logical sequence of steps

that describes a complete solution to a

i

bl

C

t bl i

fi it

(2)

Analysis of algorithms

• Analysis of algorithms: is used to predict

Analysis of algorithms: is used to predict

the efficiency and select the best one.

• Analysis can be done on the :

– Space (size)

Ti

(

d)

– Time (speed)

(3)

Running time

• Running time of algorithms typically

Running time of algorithms typically

depends on the input set, and its size (n).

So we describe the running time using

functions of n.

• One of general methodology used to

measure the running time is Big-Oh

notation

(4)

Big-Oh

Big Oh

• Big-Oh notation: is a notation that expresses

Big Oh notation: is a notation that expresses

computing time as the term in a function of the

size.

• Example: T(n)= 5 n

2

O(n

2

)

T(n)= 10 + 3 n

( )

3

O(n

(

3

)

• In other words we can measure the complexity

of an algorithm by counting the number of

(5)

Examples

• Write an algorithm for adding up the integers

g

g p

g

from 1 to 100.

• In order to solve the problem we have the

following three algorithms

following three algorithms.

• Algorithm 1: 0 + 1 + 2 + 3 ……

– Function sum (n), where n=100.(input size)

– Declaration int i, sum=0; 1 – Loop for(i=1;i<=n;i++) n

• sum +=i; n;

– Return sum 1

– T(n)= 1 + n + n + 1 (number of operations)

– = 2 + 2n O(n)

(6)

Examples

time

n

time

n

10 tc

10

100 tc

100

100 tc

100

1000 tc

1000

(7)

Examples

• Algorithm 2: 0 +(1)+ (1+1)+(1+1+1)

Algorithm 2: 0 +(1)+ (1+1)+(1+1+1)…..

– Function sum (n), where n=100.(input size)

Declaration int i j sum=0;

1

– Declaration int i,j, sum=0; 1

– Loop for(i=1;i<=n;i++) n

• Loop for (j=1; j< i; j++) n

– sum +=1; n

– Return sum

Return sum 1

1

– T(n)= 1 + n + n*n + 1

(8)

Examples

time

n

time

n

100 tc

10

10000 tc

100

10000 tc

100

(9)

Examples

• Algorithm3: which is the Gauss solution

g

( n + 1) *n / 2

• 1 2 3 4 5

96 97 98 99 100

• 1 2 3 4 5 . . . 96 97 98 99 100

101 101

101

•Function sum (n)

(10)

• T(n)= 1 +1 + 1

T(n) 1 1 1

n

time

•

= 3

O(1)

time

n

1 tc

10

1 tc

100

1 tc

1000

(11)

(12)

Examples

• Example : use Big-Oh notation to analyze the time p g y efficiency of the following fragment

• K=n

while (k >1) while (k >1)

{ x= 2* y z= x * x k= k/2 }

Suppose n 64 then k 64 32 16 8 4 2

• Suppose n=64 then k= 64 32 16 8 4 2

(13)

Algorithm’s efficiency

Algorithm s efficiency

•

As a programmers we are interested to

notice the effect of an inefficient Alg.

not when the size of the Problem is

small but when the problem is large.

•n

n +n+1 behaves like n when n is large

2

+n+1 behaves like n

2

when n is large

(14)

growth rate function

• Note: till now, we cannot compute the actual time

i t f Al Wh ? (b h t t

requirement of an Alg. Why? (because we have not yet implemented the algorithm in JAVA). but we found a Function depends on the size of the problem, which behaves like the algorithm's actual time requirement behaves like the algorithm s actual time requirement. • If the time requirement increases, the value of the

function increases.

The value of the function is directly proportional to the • The value of the function is directly proportional to the

time requirement

• Such function is called growth rate function: it measures how the time requirement grows as the problem size

how the time requirement grows as the problem size grow (asymptotic growth rate or Order)

(15)

Formal definition of Big Oh

• Formal definition of Big Oh

Formal definition of Big Oh

• An algorithm's time requirement f(n) is of

order at most g(n) where

order at most g(n) ,where

– f (n) = O( g( n))

F

iti

l

b

d

iti

(16)

(17)

(18)

(19)

Some Rules of Big-Oh

Some Rules of Big Oh

• If f(n) is a polynomial of degree k, Then f(n) is

O(n

k

),

_i.e.,

1

D l d t

1.

Drop lower-order terms 2. Drop constant factors

• Use the smallest possible class of functions - Say

“2n

is

0(n)”

instead of

“2n

is

O(n

2

)”

• Use the simplest expression of the classUse the simplest expression of the class

(20)

The time efficiencies of the ADT list operations for two

i l t ti d i Bi Oh t ti