MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency

Design and Analysis of Algorithm

A n a l y si si s o f A l g o r i t h m Efficiency c

Defini Definiti tion on of of Analy nalysis sis of Algorithm orithm

c

Time andSpace Effi Efficiency ciency

c

Asymptotic ptotic Order rderof Growth

c

AlgorithmCaseEff Effiicienci ciencies es

c

Asymptotic ptotic Notations Notations

c

Basic Effici Efficien ency cyClasses

c

Mathem Mathematica aticall Analy nalysis sis of Recursi Recursiv ve Algorithms

c

Mathem Mathematica aticall Analy nalysis sis of NonNon-Recursive Algorithms

c

Empirica piricall Analy nalysis sis ofAlgorithm

Analysis of Algorithm efficiency

* Property of STI

Page Page 1 of 37


A n a l y si si s F r a m e w o r k

c

Analy nalysis sis of alg algorithm orithm q

c

used in in a narrower narrower technical sense sense to mean mean an investi nvestig gation of alg algorithms orithms effic efficiency iency with respect to to the two two resources resources running running tim time and memoryspace pace

main ainconsiderations: considerations: q q

sim si mplici plicity ty,, generali generality ty and effi efficiency ciencyof alg algorithm speed and memory of computer nowadays


* Property of STI

Page Page 2 of 37


A n a l y si si s F r a m e w o r k c

Space Space effici efficien ency cy q

c

Time effi efficiency ciency q

c

indicates indicates how fast fast an algorithm algorithm is processed processed in the machi machine ne

Measuring easuringanInp Input’s ut’s Size Size q

c

amount of memory required by an algorithm to run the application

exam examiine an alg algorithm’s orithm’s effi efficiency ciency as a function of some parameter n indicating the algorithm algorithm’s ’s input nput size size

Techni echniques ques in in Measuri easuring ngRunning RunningTime q q

Using standard unit of time measurement Counti Counting ng the number number of tim times each algorithm’s operations is executed


* Property of STI

Page Page 3 of 37



Disadv Disadvantages of using usingtechnique techniques s in in analy analysis sis ofal algorithm orithm: c

q

Dependence Dependence of the computer’s computer’s speed speed

q

Dependence on the quality of the program impl plem ementi enting ngthe computer

q

Depende Dependence nce on the compil compiler er used used in in generating enerating the machi machine ne code

q

Diffi fficulty of clocking clocking the the actual running running time time of the program


* Property of STI

Page Page 4 of 37



cOrder of Growth q

how do time or space requirements grow as the probl problem si size ze increases

Approximate value of different important functions for algorithm analysis


* Property of STI

Page Page 5 of 37



c

Worst orst-case -caseeffici efficiency ency

c

Best Best-case effi effici cien ency cy

c

Average-ca erage-case se effi efficiency ciency

c

Amortized effi efficiency ciency


* Property of STI

Page Page 6 of 37


A sy sy m p t o t i c N o t a t i o n c

Asymptotic ptotic com complex plexity q

c

way of expressing the main component of the cost of an alg algorithm using using idealized dealized units of computati computational onal work

Notation Notation used in in compari comparing ng and ranking ranking the order of growth rowth q

Big-0h Notation (0 )

q

Big Bi g-0meg 0mega a Notation Notation (? )

q

Big-Theta Notation (T)


* Property of STI

Page Page 7 of 37



Big Big-0 Notation Notation (0) q q

c

Big-O Big-Omeans eans “onthe order of of” ” q

c

introduced in 1854 by Paul Bachmann most commonly used notation for specifying asymptotic complexity

formal method of expressing the upper bound of an algorithm's running time.

(g(n )) if there exis Write f (n) = O(g(n existt constants c > 0 , n 0 > 0 such that 0 = f(n) = cg(n) cg(n) for all all n = n 0


* Property of STI

Page Page 8 of 37



Prac Practic tical al Example: q

O (n ) • printing printing a list list of n item items to the the scree screen, n, looking looking at each item once

q

O (log n ) • takin taking g a list of item items, cutt cutting ing it in half half repeatedly until there's only one item left

q

O (n2 ) • taking a list list of n items, items, and com comparing paring every every item to every other item


* Property of STI

Page Page 9 of 37



TheoryExamp Example: q

Given: f(n) = 2 n + 8 , and g( g (n) = n2

q

Find c , so that 2n + 8 <= n 2


* Property of STI

Page Page 10 of 37



TheBi Big-Onotationcan canbederiv derivedfromf(n) using usingthe foll followingsteps: q

set the coeffici coefficient ent of the term to 1

q

keep the largest term in the function and discard the others

q

terms terms are ranked from from lowest owest to hig highest

2 , 3 . . . . k n l o g n 2 n ! 2 n, n, nlog 2 n, n n


* Property of STI

Page Page 11 of 37



Example#1:

q

remov remove e all all coefficients coefficients • n2 + n

q

remov remove e the smaller smaller factors factors • n 2

q

BigBi g-O Onotation: notation: • o(f(n) o(f(n) = o(n2 )


* Property of STI

Page Page 12 of 37



Example#2: k k -1 2 f(n) = a n + a n + … + a n j j-1 j -1 2 + a 1 n + a 0

q

remov remove e all coeffi coefficients cients • f(n) = n k + n k -1 + …+ n 2 +n+1

q

the order of a polynomial expression is: k • o(f(n) = o(n )


* Property of STI

Page Page 13 of 37



Big Big-0 Measur Measures es of Effici Efficien ency cy


* Property of STI

Page Page 14 of 37



Big Big-0megaNotat Notation(? (? ) q

q

t(n) t(n) ? (g(n (g(n)) )) 2 Sample formal proof where n 3 ? (n )

• B 3 = n 2 for all n = 0

q

select c =1 and n 0 = 0


* Property of STI

Page Page 15 of 37



Big Big-Theta -Theta Notation Notation (T) (T) q q

applied applied in tig tight bound bound function t(n) is said to be in T(g(n)), denoted t(n) T(g(n)) • if t(n) is bounde bounded d both both abov above e and and b below elow bysome some positiv positive constant multi multiples ples of g(n) for all large n • If there there ex exist some some positiv positive e constan constantt c1 and and c2 and and some nonnegative integer n0 such that:

c (n) = t (n) = c 1 g(n) g(n) for all n = n 0 2 g (n)


* Property of STI

Page Page 16 of 37



For example, let us prove that right inequality (the upper bound):

q

First, we prove the right inequality (the upper bounds)


* Property of STI

Page Page 17 of 37



Continuation: q

Second, we prove the left inequality (the lower bounds).

q

Hence:


* Property of STI

Page Page 18 of 37


B a si si c Ef Ef f i c i e n c y Cl C l a ss ss e s c

Algorithm’s effi efficiency ciency q

a function of the number of elements to be processed

q

general format: ormat:

f (n) = efficiency


* Property of STI

Page Page 19 of 37



Linear LinearLoops Loops q

Example: • identify howmanytimes times the body bodyof the loop loop is repeated:

1 x =0 2 loop (x >=1000) 1 app appllicati ication on code 2 x =x pl plus us 1 3 end loop q

the number number of iterations iterations is directly directly proportional to the loop factor

q

the hig higher the factor, factor, the hig higher the number number of loops effici efficiency ency is directly directly proportional proportional to the number number of iterati iterations ons

q

f (n) = n


* Property of STI

Page Page 20 of 37



Linear LinearLoops Loops q

Example:

1 x =0 2 loop (x >=1000) 1 app appllicati ication on code 2 x =x pl plus us 2 3 end loop

q

the num number of iterations iterations is is half half the loop factor

q

Effici Efficiency encyis propo proportional rtional to half the factor

f (n) = n/ n/2 2


* Property of STI

Page Page 21 of 37



Logarithm arithmic Loops q

a loop in which the controlling variable is mul ulti tipli plied ed or divided in each loop

q

determine how many many tim times the body body of the loops is repeated:

1 x =1 2 loop (x >=1000) 1 app appllication code 2 x =x mul ulttipl ply y2 3 end loop

1 x =1000 2 loo oop p (x >=1) 1 app appllicati ication on code 2 x =x di divi vide de 2 3 end loop


* Property of STI

Page Page 22 of 37



Logarithm arithmic Loops

Analysis of Multiplication and Division Loops

f (n) = [log2n]


* Property of STI

Page Page 23 of 37



NestedLoop q

loop within a loop

Iterations = outer loop iterations x inner loop iterations

q

includes ncludes Linear Linear Logari Logarithm thmiic, Depende Dependent nt Quadratic, uadratic, and Quadrati Quadratic c


* Property of STI

Page Page 24 of 37



Outer loop

c

Linear Linear logarithm arithmic: 1 x=1 2 loop (x<=10) 1 k=1 2 loop(k <=10) 1 application code 2 k = k multiply 2 3 end loop 3 end loop

Inner loop

number of iteration in the inner loop is [log210]

10 x [log210]

f(n) = [nlog2n]


* Property of STI

Page Page 25 of 37



Dependentquadratic quadratic : 1 x = 10 2 loop (x <= 10) 1k=1 2 loop (k <= x) 1 application code 2 k = k +1 3 end loop 3 end loop

c

inner loop depends on the outer loop for one of its factors

c

the num number of iteration iteration is 55


* Property of STI

Page Page 26 of 37



averag average e is 5.5


* Property of STI

Page Page 27 of 37



Quadratic: 1 x = 10 2 loop ( x <= 10) 1 k=1 2 loop ( k <= 10) 1 application code 2 k=k+1 3 end loop 4 x=x+1 3 end loop

f(n) = n 2


* Property of STI

Page Page 28 of 37


SEATWORK

c

Algorithmeffi efficiency ciencyappl application: cation:State whether the statem statement ent below below is Trueor False. Defend your answ answer. 1.

20n2 +2n+5 =O(n2)

2.

100n +5 =O(n2)


* Property of STI

Page Page 29 of 37


M a t h e m a t i c a l A n a l y si si s o f R e c u r si v e A l g o r i t h m s c

General General plan of analy analyzing zingTime Effi Effici ciency encyof recursiv recursive alg algorithm 1.

Decide on a parameter parameter (or param parameters eters)) indicating indicating an input’s size size..

2.

Identi dentify the algori algorithm’s thm’s basic basic operation.

3.

Check Check whether number number of tim times the the basic basic operation is executed can vary on different inputs of the same same size size

4.

Set up recurrence relation, relation, wi with appropriate appropriate initial initial condition, condition, for the number number of of tim times the basic basic operation is is executed executed..

5.

Solv Solve the recurrence recurrence or at least least ascertain ascertain the order of growth of its solution.


* Property of STI

Page Page 30 of 37


M a t h e m a t i c a l A n a l y si si s o f R e c u r si v e A l g o r i t h m s c

Compute the factorial function f(n) = n! for an arbitrary arbitrary nonnegati nonnegativ ve integer nteger n. Si Since: nce: n ! = 1 , ….(n….(n-1 1 ) . n = (n-1 (n-1 )! . n for n = 1

c

With 0 != 1 by definition, we can compute F(n) = F(n-1) . n with the following recursive algorithm.

1 if n = 0 return 1 2 else 3 return F(n-1) * n

c

F(n) = F(n-1 F(n-1 ) . n for every everyn > 0

c

M(n) = M (n( n-1 1 ) + 1 for everyn > 0


* Property of STI

Page Page 31 of 37


M a t h e m a t i c a l A n a l y si si s o f N o n - R e c u r si v e A l g o r i t h m s c

General General Plan Plan for Analy nalyze Ti Time Effi Effici ciency ency 1.

Decide on a parameter parameter (or param parameters eters)) indicating indicating an input’s size size..

2.

Identi dentify the alg algorithms orithms basic basic operation. Check whether the number of times the basic operation is executed executed depends only only on the size of an input. Set up a sum express expressiing the number number of tim times the algorithm’s algorithm’s basic basic operation is is executed executed..

3.

4.

5.

Usi sing ng standard form formulas ulas and rules of sum manipulati anipulation, on, either find find a close close-form -form formula formula for the count, or the very very least, least, establis establish h its its order of growth.


* Property of STI

Page Page 32 of 37


M a t h e m a t i c a l A n a l y si si s o f N o n - R e c u r si v e A l g o r i t h m s c

Example: maxValue maxVal ue = A[0] A[0] loop index = n if A[index ] > maxValue return maxValue

c

find a formula in expressing C as a function of size size n


* Property of STI

Page Page 33 of 37


Em p i r i c a l A n a l y s i s o f Algorithm c

Empirical iricalan analy alysis q

c

alternative alternative of of the mathem mathemati atical cal analysi analysis s of algorithm algorithms s efficiency efficiency

General General Plan Plan for Empirica piricall Analy nalysis sis of Algorithm orithmTime Effi Effici ciency ency 1.

Unders Understand tand the experi experim ments purpose.

2.

Deci Decide de on effi efficiency ciency metric etric M to be measure measure and the measur measurem ement ent unit unit (an operation’s operation’s count vs. vs. a tim time unit). unit).

3.

Decide on characteristics of the input sample (its range, range, size size and so on). Prepare a program program impl plem ementi enting ng the alg algorithm orithm ( or alg algorithm orithms) s) for for the experimentation. Generate a sample sample of inputs.

4.

5. 6.

7.

Run algorithms on the sample inputs and record the data observed. observed. Analy nalyzed the data obtained. obtained.


* Property of STI

Page Page 34 of 37


Em p i r i c a l A n a l y s i s o f Algorithm c

Limitations of Empirical Analysis q

q

q

Quality of implementation Extraneous factors Requires Requires a worki working ng program program


* Property of STI

Page Page 35 of 37


SEATWORK

c

Calcul alculate ate the run-ti run-tim me effi efficiency ciencyof the foll ollowi owing pseudocode pseudocode for program segm segments.(10 ents.(10 points each)

x=0 loop( x <= n) print x x=x+1 end loop

sum = 0; for( i = 0; i < n; i++) for( j = 0; j < n; j++) sum++;


* Property of STI

Page Page 36 of 37


A SSI G N M EN T

c

c

Research for the foll followi owing topics:

1.

Defi Definiti nition on of brute force force algori algorithm thm technique desig design technique technique

2.

Applicati pplications ons of brute force alg algorithm desig design technique

Note: The The research work work should should be in in a printed copy and compil compiled ed in in a folder folder wi with a title page.


* Property of STI

Page Page 37 of 37

MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency

Recommend Documents