brPage 2br Intr oduction The singular alue decomposition SVD is just as amazing as the LU and QR decompositions It is closely related to the diagonal form of symmetric matrix What happens if the matrix is not symmetric It turns out that we can actor ID: 27533
Download Pdf The PPT/PDF document "Singular alue Decomposition Notes on Lin..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
SingularValueDecompositionNotesonLinearAlgebraChia-PingChenDepartmentofComputerScienceandEngineeringNationalSunYat-SenUniversityKaohsiung,TaiwanROCSingularValueDecompositionp.1 IntroductionThesingularvaluedecomposition,SVD,isjustasamazingastheLUandQRdecompositions.ItiscloselyrelatedtothediagonalformA=QQTofasymmetricmatrix.Whathappensifthematrixisnotsymmetric?ItturnsoutthatwecanfactorizeAbyQ1QT,whereQ1;Q2areorthogonalandisnonnegativeanddiagonal-like.Thediagonalentriesofarecalledthesingularvalues.SingularValueDecompositionp.2 SVDTheoremAnymnrealmatrixAcanbefactoredintoA=Q1QT=(orthogonal)(diagonal)(orthogonal):Thematricesareconstructedasfollows:ThecolumnsofQ1(mm)aretheeigenvectorsofAAT,andthecolumnsofQ2(nn)aretheeigenvectorsofATA.Thersingularvaluesonthediagonalof(mn)arethesquarerootsofthenonzeroeigenvaluesofbothAATandATA.SingularValueDecompositionp.3 ProofofSVDTheoremThematrixATAisrealsymmetricsoithasacompletesetoforthonormaleigenvectors:ATAxj=jxj,andxTATAxj=jxTxj=jij:Forpositivej's(sayj=1;:::;r),wedenej=pjandqj=Axjj.ThenqTiqj=ij.Extendtheqi'stoabasisforRm.Putx'sinQ2andq'sinQ1,then(QTAQ2)ij=qTiAxj=0ifj�r;jqTiqj=jijifjr:Thatis,QTAQ2=.SoA=Q1QT2.SingularValueDecompositionp.4 RemarksForpositivedenitematrices,SVDisidenticaltoQQT.Forindenitematrices,anynegativeeigenvaluesinbecomepositivein.ThecolumnsofQ1;Q2giveorthonormalbasesforthefundamentalsubspacesofA.(RecallthatthenullspaceofATAisthesameasA).AQ2=Q1,meaningthatAmultipliedbyacolumnofQ2producesamultipleofcolumnofQ1.AAT=Q1TQTandATA=Q2TQT2,whichmeanthatQ1mustbetheeigenvectormatrixofAATandQ2mustbetheeigenvectormatrixofATA.SingularValueDecompositionp.5 ApplicationsofSVDThroughSVD,wecanexpandamatrixtobeasumofrank-onematricesA=Q1QT=u11vT1++urrvTr:Supposewehavea10001000matrix,foratotalof106entries.Supposeweusetheaboveexpansionandkeeponlythe50mostmostsignicantterms.Thiswouldrequire50(1+1000+1000)numbers,asaveofspaceofalmost90%.Thisisusedinimageprocessingandinformationretrieval(e.g.Google).SingularValueDecompositionp.6 SVDforImageApictureisamatrixofgraylevels.ThismatrixcanbeapproximatedbyasmallnumberoftermsinSVD.SingularValueDecompositionp.7 PseudoinverseSupposeA=Q1QTistheSVDofanmnmatrixA.ThepseudoinverseofAisdenedbyA+=Q2+QT;where+isnmwithdiagonals11;:::;1r.ThepseudoinverseofA+isA,orA++=A.Theminimum-lengthleast-squaresolutiontoAx=bisx+=A+b.Thisisageneralizationoftheleast-squareproblemwhenthecolumnsofAarenotrequiredtobeindependent.SingularValueDecompositionp.8 ProofofMinimumLengthMultiplicationbyQTleavesthelengthunchanged,sojAx bj=jQ1QTx bj=jQT2x QT1bj=jy QT1bj;wherey=QTx=Q 12x.Sinceisadiagonalmatrix,weknowtheminimum-lengthleast-squaresolutionisy+=+QTb.Sincejyj=jxj,theminimum-lengthleast-squaresolutionforxisx+=Q2y+=Q2QTb=A+b:SingularValueDecompositionp.9