Recurrences(CLRS4.1-4.2)Lasttimewediscusseddivide-and-conqueralgorith - PDF document

Recurrences(CLRS4.1-4.2)Lasttimewediscusseddivide-and-conqueralgorithmsDivideandConquerToSolveP:1.DividePintosmallerproblemsP1;P2;P3:::::Pk.2.Conquerbysolvingthe(smaller)subproblemsrecursively.3.CombinesolutionstoP1;P2;:::PkintosolutionforP.Analysisofdivide-and-conqueralgorithmsandingeneralofrecursivealgorithmsleadstorecurrences.Merge-sortleadtotherecurrenceT(n)=2T(n=2)+n{orrather,T(n)=((1)Ifn=1T(dn2e)+T(bn2c)+(n)Ifn�1{butwewilloftencheatandjustsolvethesimpleformula(equivalenttoassumingthatn=2kforsomeconstantk,andleavingoutbasecaseandconstantin).Methodsforsolvingrecurrences1.Substitutionmethod2.IterationmethodRecursion-treemethod(Mastermethod)1 1SolvingRecurrenceswiththeSubstitutionMethodIdea:Makeaguessfortheformofthesolutionandprovebyinduction.CanbeusedtoprovebothupperboundsO()andlowerbounds\n().Let'ssolveT(n)=2T(n=2)+nusingsubstitution{GuessT(n)cnlognforsomeconstantc(thatis,T(n)=O(nlogn)){Proof:Basecase:weneedtoshowthatourguessholdsforsomebasecase(notnecessarilyn=1,somesmallnisok).Ok,sincefunctionconstantforsmallconstantn.Assumeholdsforn=2:T(n=2)cn2logn2(Question:Whynotn1?)Provethatholdsforn:T(n)cnlognT(n)=2T(n=2)+n2(cn2logn2)+n=cnlogn2+n=cnlogncnlog2+n=cnlogncn+nSookifc1SimilarlyitcanbeshownthatT(n)=\n(nlogn)Exercise!SimilarlyitcanbeshownthatT(n)=T(bn2c)+T(dn2e)+nis(nlgn).Exercise!Thehardpartofthesubstitutionmethodisoftentomakeagoodguess.Howdowemakeagood(i.e.tight)guess???Unfortunately,there'sno\recipe"forthisone.TryiterativelyO(n3);\n(n3);O(n2);\n(n2)andsoon.Trysolvingbyiterationtogetafeelingofthegrowth.2 2SolvingRecurrenceswiththeIteration/Recursion-treeMethodIntheiterationmethodweiteratively\unfold"therecurrenceuntilwe\seethepattern".Theiterationmethoddoesnotrequiremakingagoodguesslikethesubstitutionmethod(butitisoftenmoreinvolvedthanusinginduction).Example:SolveT(n)=8T(n=2)+n2(T(1)=1)T(n)=n2+8T(n=2)=n2+8(8T(n22)+(n2)2)=n2+82T(n22)+8(n24))=n2+2n2+82T(n22)=n2+2n2+82(8T(n23)+(n22)2)=n2+2n2+83T(n23)+82(n242))=n2+2n2+22n2+83T(n23)=:::=n2+2n2+22n2+23n2+24n2+:::{Recursiondepth:Howlong(howmanyiterations)ittakesuntilthesubproblemhasconstantsize?itimeswheren2i=1)i=logn{Whatisthelastterm?8iT(1)=8lognT(n)=n2+2n2+22n2+23n2+24n2+:::+2logn1n2+8logn=logn1Xk=02kn2+8logn=n2logn1Xk=02k+(23)lognNowPlogn1k=02kisageometricsumsowehavePlogn1k=02k=(2logn1)=(n)(23)logn=(2logn)3=n3T(n)=n2
(n)+n3=(n3)3 2.1RecursiontreeAdierentwaytolookattheiterationmethod:istherecursion-tree,discussedinthebook(4.2).wedrawouttherecursiontreewithcostofsinglecallineachnode|runningtimeissumofcostsinallnodesifyouarecarefuldrawingtherecursiontreeandsummingupthecosts,therecursiontreeisadirectproofforthesolutionoftherecurrence,justlikeiterationandsubstitutionExample:T(n)=8T(n=2)+n2(T(1)=1)T(n/4)n2T(n/4)T(n)1)T(n/2)T(n/2)T(1)T(1)2222T(n)=n2+2n2+22n2+23n2+24n2+:::+2logn1n2+8logn4 3MatrixMultiplicationLetXandYbennmatricesX=8�����&#x-2.4;䆘&#x-2.4;䆘&#x-2.4;䆘&#x-2.4;䆘&#x-2.4;䆘:x11x12


x1nx21x22


x1nx31x32


x1n











xn1xn2


xnn9&#x-2.4;䆘&#x-2.4;䆘&#x-2.4;䆘&#x-2.4;䆘&#x-2.4;䆘=&#x-2.4;䆘&#x-2.4;䆘&#x-2.4;䆘&#x-2.4;䆘&#x-2.4;䆘;WewanttocomputeZ=X
Y{zij=Pnk=1Xik
YkjNaivemethoduses)n2
n=(n3)operationsDivide-and-conquersolution:Z=(ABCD)
(EFGH)=((A
E+B
G)(A
F+B
H)(C
E+D
G)(C
F+D
H)){Theabovenaturallyleadstodivide-and-conquersolution:DivideXandYinto8sub-matricesA,B,C,andD.Do8matrixmultiplicationsrecursively.ComputeZbycombiningresults(doing4matrixadditions).{Letsassumen=2cforsomeconstantcandletA,B,CandDben=2n=2matricesRunningtimeofalgorithmisT(n)=8T(n=2)+(n2))T(n)=(n3){Butwealreadydiscusseda(simpler/naive)O(n3)algorithm!Canwedobetter?3.1Strassen'sAlgorithmStrassenobservedthefollowing:Z=(ABCD)
(EFGH)=((S1+S2S4+S6)(S4+S5)(S6+S7)(S2+S3+S5S7))whereS1=(BD)
(G+H)S2=(A+D)
(E+H)S3=(AC)
(E+F)S4=(A+B)
HS5=A
(FH)S6=D
(GE)S7=(C+D)
E5 {LetstestthatS6+S7isreallyC
E+D
GS6+S7=D
(GE)+(C+D)
E=DGDE+CE+DE=DG+CEThisleadstoadivide-and-conqueralgorithmwithrunningtimeT(n)=7T(n=2)+(n2){Weonlyneedtoperform7multiplicationsrecursively.{Division/Combinationcanstillbeperformedin(n2)time.LetssolvetherecurrenceusingtheiterationmethodT(n)=7T(n=2)+n2=n2+7(7T(n22)+(n2)2)=n2+(722)n2+72T(n22)=n2+(722)n2+72(7T(n23)+(n22)2)=n2+(722)n2+(722)2
n2+73T(n23)=n2+(722)n2+(722)2n2+(722)3n2::::+(722)logn1n2+7logn=logn1Xi=0(722)in2+7logn=n2
((722)logn1)+7logn=n2
(7logn(22)logn)+7logn=n2
(7lognn2)+7logn=(7logn){Nowwehavethefollowing:7logn=7log7nlog72=(7log7n)(1=log72)=n(1=log72)=nlog27log22=nlog7{Oringeneral:alogkn=nlogka6 SothesolutionisT(n)=(nlog7)=(n2:81:::)Note:{Weare'hiding'amuchbiggerconstantin()thanbefore.{CurrentlybestknownboundisO(n2:376::)(anothermethod).{Lowerboundis(trivially)\n(n2).4MasterMethodWehavesolvedseveralrecurrencesusingsubstitutionanditeration.wesolvedseveralrecurrencesoftheformT(n)=aT(n=b)+nc(T(1)=1).{Strassen'salgorithm)T(n)=7T(n=2)+n2(a=7;b=2,andc=2){Merge-sort)T(n)=2T(n=2)+n(a=2;b=2,andc=1).ItwouldbenicetohaveageneralsolutiontotherecurrenceT(n)=aT(n=b)+nc.Wedo!T(n)=aTnb
+nca1;b1;c�0+T(n)=8�&#x-2.4;䆘:(nlogba)a&#x-2.4;䆘bc(nclogbn)a=bc(nc)abcProof(Iterationmethod)T(n)=aTnb
+nc=nc+anb
c+aTnb2=nc+abc
nc+a2Tnb2=nc+abc
nc+a2nb2c+aTnb3=nc+abc
nc+abc
2nc+a3Tnb3=:::=nc+abc
nc+abc
2nc+abc
3nc+abc
4nc+:::+abc
logbn1nc+alogbnT(1)=ncPlogbn1k=0abc
k+alogbn=ncPlogbn1k=0abc
k+nlogbaRecallgeometricsumPnk=0xk=xn+11x1=(xn)7 abcabc,abc1)Plogbn1k=0abc
kP+1k=0abc
k=11(abc)=(1)abc,logbalogbbc=cT(n)=ncPlogbn1k=0abc
k+nlogba=nc
(1)+nlogba=(nc)a=bca=bc,abc=1)Plogbn1k=0abc
k=Plogbn1k=01=(logbn)a=bc,logba=logbbc=cT(n)=Plogbn1k=0abc
k+nlogba=nc(logbn)+nlogba=(nclogbn)a&#x-3.2;⅔bca&#x-3.2;⅔bc,abc&#x-3.2;⅔1)Plogbn1k=0abc
k=abc
logbn=alogbn(bc)logbn=alogbnncT(n)=nc
alogbnnc+nlogba=(nlogba)+nlogba=(nlogba)Note:Bookstatesandprovestheresultslightlydierently(don'treadit).5ChangingvariablesSometimesreucurrencescanbereducedtosimpleronesbychangingvariablesExample:SolveT(n)=2T(pn)+lognLetm=logn)2m=n)pn=2m=2T(n)=2T(pn)+logn)T(2m)=2T(2m=2)+mLetS(m)=T(2m)T(2m)=2T(2m=2)+m)S(m)=2S(m=2)+m)S(m)=O(mlogm))T(n)=T(2m)=S(m)=O(mlogm)=O(lognloglogn)8 6OtherrecurrencesSomeimportant/typicalboundsonrecurrencesnotcoveredbymastermethod:Logarithmic:(logn){Recurrence:T(n)=1+T(n=2){Typicalexample:Recurseonhalftheinput(andthrowhalfaway){Variations:T(n)=1+T(99n=100)Linear:(N){Recurrence:T(n)=1+T(n1){Typicalexample:Singleloop{Variations:T(n)=1+2T(n=2);T(n)=n+T(n=2);T(n)=T(n=5)+T(7n=10+6)+nQuadratic:(n2){Recurrence:T(n)=n+T(n1){Typicalexample:NestedloopsExponential:(2n){Recurrence:T(n)=2T(n1)9