Keywordsconjugategradientmethodtransparenciesagonizingpain 4224664224123122221628Sample2dlinearsystemoftheform3226 nn 28 nn r 42024664202405010015042 ID: 226925
Download Pdf The PPT/PDF document "ClassroomFiguresfortheConjugateGradientM..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
ClassroomFiguresfortheConjugateGradientMethodWithouttheAgonizingPainEdition114JonathanRichardShewchukAugust4,1994SchoolofComputerScienceCarnegieMellonUniversityPittsburgh,PA15213AbstractThisreportcontainsasetoffull-page®guresdesignedtobeusedasclassroomtransparenciesforteachingfromthearticleªAnIntroductiontotheConjugateGradientMethodWithouttheAgonizingPainº.SupportedinpartbytheNaturalSciencesandEngineeringResearchCouncilofCanadaundera1967ScienceandEngineeringScholarshipandbytheNationalScienceFoundationunderGrantASC-9318163.Theviewsandconclusionscontainedinthisdocumentarethoseoftheauthorandshouldnotbeinterpretedasrepresentingtheof®cialpolicies,eitherexpressorimplied,ofNSERC,NSF,ortheU.S.Government. Keywords:conjugategradientmethod,transparencies,agonizingpain -4-2246-6-4-224 1 23 12 222 16 28Sample2-dlinearsystemoftheform :3226 \n\n 28 \n\n \r -4-20246-6-4-2024050100150-4-20246 1 2 1Graphofquadraticform 12 .Theminimumpointofthissurfaceisthesolutionto .-4-2246-6-4-224 1 2Contoursofthequadraticform.Eachellipsoidalcurvehasconstant . -4-2246-8-6-4-2246 1 2Gradient ofthequadraticform.Forevery ,thegradientpointsinthedirectionofsteepestincreaseof ,andisorthogonaltothecontourlines. (c) 1 2 1(d) 1 2 1(a) 1 2 1(b) 1 2 1(a)Quadraticformforapositive-de®nitematrix.(b)Foranegative-de®nitematrix.(c)Forasingular(andpositive-inde®nite)matrix.Alinethatrunsthroughthebottomofthevalleyisthesetofsolutions.(d)Foraninde®nitematrix. 0.20.40.620406080100120140-4-2246-6-4-224-4-2246-6-4-224-2.502.55-5-2.502.5050100150-2.502.55 (c) ! #" %$& ' #"( 1(d) 2 ) 1" 1(a) 2 0"* 0" (b) 1 2 1ThemethodofSteepestDescent. -2-1123-3-2-112 1 2Solidarrows:Gradients.Dottedarrows:Slopealongsearchline. -4-2246-6-4-224 1 2 ) 0" ThemethodofSteepestDescent. B vB vv23Bv+isaneigenvectorof,withacorrespondingeigenvalueof0\r5.As-increases,, +convergestozero.BvvB vB v23Here,+hasacorrespondingeigenvalueof2.As-increases,, +divergestoin®nity.B xB x23xBxvv12 +1+2.Oneeigenvectordiverges,so alsodiverges. -4-2246-6-4-22472 1 2Theeigenvectorsofaredirectedalongtheaxesoftheparaboloidde®nedbythequadraticform .Eacheigen-vectorislabeledwithitsassociatedeigenvalue. -4-2246-6-4-224 1(a) 20\r470\r47-4-2246-6-4-224 1(b) 2 ) 0" -4-2246-6-424 1(c) 2* 0"-4-2246-6-4-224 1(d) 2* 1"-4-2246-6-4-224 1(e) 2* 2"-4-2246-6-4-224 1(f) 2ConvergenceoftheJacobiMethod.In(a),theeigenvectorsof,areshownwiththeircorre-spondingeigenvalues.TheseeigenvectorsareNOTtheaxesoftheparaboloid. -4-2246-6-4-224 1 2SteepestDescentconvergestotheexactsolutiononthe®rstiterationiftheerrortermisaneigenvector. -4-2246-4-2246 1 2SteepestDescentconvergestotheexactsolutiononthe®rstiterationiftheeigenvaluesareallequal. -6-4-22-22468 1 2Theenergynormofthesetwovectorsisequal. 0510152012040608010000.20.40.60.805101520./0.Convergence0ofSteepestDescent..istheslopeof* ! #"withrespecttotheeigenvectoraxes./istheconditionnumberof.Convergenceisworstwhen.1/. -4-224-4-2246-4-224-4-2246-4-224-4-2246-4-224-4-2246+1(c)+2+1(d)+2+1(a)+2+1(b)+2(a)Large/,small..(b)Anexampleofpoorconvergence./and.arebothlarge.(c)Small/and..(d)Small/,large.. -4-2246-6-4-224 1 2 ) 0"Solidlines:WorststartingpointsforSteepestDescent.Dashedlines:Stepstowardconvergence.Greyarrows:Eigenvectoraxes.Here,/3\r5. 2040608010000.20.40.60.81/0ConvergenceofSteepestDescent(periteration)worsensastheconditionnumberofthematrixincreases. -4-2246-6-4-224 1 2 ) 0" 1"* 1"2 0"TheMethodofOrthogonalDirections. -4-224-4-224 1 2Thesepairsofvectorsare-orthogonal\r3\r3\r-4-224-4-224 1 2\r3\r3\rbecausethesepairsofvectorsareorthogonal. -4-2246-6-4-224 1 2 0" 1"* 1"2 0"-4-2246-6-4-224 1 2* 0"ThemethodofConjugateDirectionsconvergesin4steps.* 1"mustbe-orthogonalto2 0". dduuuu+*d051(0)(0)(1)Gram-Schmidtconjugationoftwovectors. -4-2246-6-4-224 1 2ThemethodofConjugateDirectionsusingtheaxialunitvectors,alsoknownasGauûianelimination. dd(0)(1)e(2)e(0)(1)e0Theshadedareais* 0"6* 0"span72 0"982 1";:.Theellipsoidisacontouronwhichtheenergynormisconstant.Aftertwosteps,CG®nds* 2",thepointon* 0"6thatminimizes*�. 1" $ 1"2 0"2 1"* 1" ) 0" ) 0"61(a) $ 1" ) 1" 0"2 0"2 1"* 1" 0"61(b) 1" ) 2"2 0" 0"62 0"$& 1"2 1" 0"61(c) ) 2"2 1" 1"2 0" 0"$? 1" 0"62 ) 0"61(d)(a)2Dproblem.(b)Stretched2Dproblem.(c)3Dproblem.(d)Stretched3Dproblem. dddr(0)(1)(2)(2)uu10u2e(2)2 0"and2 1"spanthesamesubspaceas@08@1(thegray-coloredplane62).* 2"is-orthogonalto62.$ 2"isorthogonalto62.2 2"isconstructed(from@2)tobe-orthogonalto62. dddr(0)(1)(2)(2)rr(0)(1)e(2)2 0"and2 1"spanthesamesubspaceas$? 0"8$? 1"(thegray-coloredplane62).* 2"is-orthogonalto62.$? 2"isorthogonalto62.2 2"isconstructed(from$ 2")tobe-orthogonalto62. -4-2246-6-4-224 1 2 ) 0" ThemethodofConjugateGradients. 27-1-0.75-0.5-0.250.250.50.75127-1-0.75-0.5-0.250.250.50.75127-1-0.75-0.5-0.250.250.50.75127-1-0.75-0.5-0.250.250.50.751A(c)B2AA(d)B2AA(a)B0AA(b)B1ATheconvergenceofCGafter-iterationsdependsonhowcloseapolynomialB ofdegree-canbetozerooneacheigenvalue,giventheconstraintthatB 01. -1-0.50.51-2-1.5-1-0.50.511.52-1-0.50.51-2-1.5-1-0.50.511.52-1-0.50.51-2-1.5-1-0.50.511.52-1-0.50.51-2-1.5-1-0.50.511.520C100D0C490E0C20E0C50EChebyshevpolynomialsofdegree2,5,10,and49. 12345678-1-0.75-0.5-0.250.250.50.751AB2ATheoptimalpolynomialB2AforAGF IH2andAGF JLK7inthegeneralcase.*=isreducedbyafactorofatleast0.183aftertwoiterationsofCG. 2040608010000.20.40.60.81/0ConvergenceofConjugateGradients(periteration)asafunctionofconditionnumber.20040060080010000510152025303540/0NumberofiterationsofSteepestDescentrequiredtomatchoneiterationofCG. -4-2246-8-6-4-2 1 2Contourlinesofthequadraticformofthediagonallypre-conditionedsampleproblem.Theconditionnumberhasimprovedfrom3\r5toroughly2\r8. -4-20246-20246-2500250500-4-20246(a) 1 2 1-4-2246-2246 1(b) 2 0"-0.04-0.020.020.04-200200400600 (c) ! #" 2 ! #"(-4-2246-2246 1(d) 2 0"ThenonlinearConjugateGradientMethod.(b)Fletcher-ReevesCG.(c)Cross-sectionofthesurfacecorrespondingtothe®rststepofFletcher-Reeves.(d)Polak-RibiÁereCG. -4-2246-2246 1 2 0"NonlinearCGcanbemoreeffectivewithperiodicrestarts. -1-0.50.511.52-1-0.75-0.5-0.250.250.50.751 MTheNewton-Raphsonmethod.Solidcurve:Thefunctiontominimize.Dashedcurve:Parabolicapproximationtothefunction,basedon®rstandsecondderivativesat .Mischosenatthebaseoftheparabola. -11234-1.5-1-0.50.51 MTheSecantmethod.Solidcurve:Thefunctiontominimize.Dashedcurve:Parabolicapproximationtothefunction,basedon®rstderivativesat 0and 2.Mischosenatthebaseoftheparabola. -4-2246-2246 1 2 0"ThepreconditionednonlinearConjugateGradientMethod.Polak-RibiÁereformulaandadiagonalpreconditioner.Thespacehasbeenªstretchedºtoshowtheimprovementincircularityofthecontourlinesaroundtheminimum.