Singular V alue Decomposition SVD T rucco Appendix A Singular V alue Decomposition SVD T rucco Appendix A

Singular V alue Decomposition SVD T rucco Appendix A - PDF document

danika-pritchard . @danika-pritchard
Uploaded On 2014-12-27

Singular V alue Decomposition SVD T rucco Appendix A - PPT Presentation

6 De64257nition A ny r eal matrix can be decomposed uniquely as UDV is and column orthogonal its columns are eigen ve ctors of AA AA UDV VDU UD is and orthogonal its columns are eigen ve ctors of VDU UDV VD is diagonal nonne ga ti ve r eal v alues ca ID: 30370





Download Presentation from below link

Download Pdf The PPT/PDF document "Singular V alue Decomposition SVD T rucc..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript

Singular Value Decomposition (SVD)(Trucco, Appendix A.6)ƒDeŒnition-Any realmxnmatrixAcan be decomposed uniquely asA=UDVTUismxnand column orthogonal (its columns are eigenvectors ofAAT)(AAT=UDVTVDUT=UD2UT)Visnxnand orthogonal (its columns are eigenvectors ofATA)(ATA=VDUTUDVT=VD2VT)Disnxndiagonal (non-negative real values calledsingularvalues)D=diag(1,2,...,n)ordered so that1³2³...³n(ifis a singular value ofA,it'ssquare is an eigenvalue ofATA)-IfU=(u1u2...un)andV=(v1v2...vn), thenA=ni=1SiuivTi(actually,the sum goes from 1 torwhereris the rank ofA)ƒAn exampleA=éêêë121232121ùúúû,thenAAT=ATA=éêêë61061017106106ùúúûThe eigenvalues ofAAT,ATAare:éêêë123ùúúû=éêêë28. 860. 140ùúúûThe eigenvectors ofAAT,ATAare:u1=v1=éêêë0. 4540. 7660. 454ùúúû,u2=v2=éêêë0. 542-0. 6430. 542ùúúû,u3=v3=éêêë-0. 7070-0. 707ùúúû -2-The expansion ofAisA=2i=1SiuivTiImportant:note that the second eigenvalue is much smaller than the Œrst; if weneglect it from the above summation, we can representAby introducing rela-tively small errors only:A=éêêë1. 111. 871. 111. 873. 151. 871. 111. 871. 11ùúúûƒComputing the rank using SVD-The rank of a matrix is equal to the number of non-zero singular values.ƒComputing the inverse of a matrix using SVD-Asquare matrixAis nonsingulariffi¹0for alli-IfAis anxnnonsingular matrix, then its inverse is givenbyA=UDVTorA-1=VD-1UTwhereD-1=diag(11,12,...,1n)-IfAis singular or ill-conditioned, then we can use SVD to approximate itsinverse by the following matrix:A-1=(UDVT)-1»VD-10UTD-10=ìíî1/i0ifi�totherwise(wheretis a small threshold) -3-ƒThe condition of a matrix-Consider the system of linear equationsAx=bIf small changes inbcan lead to relatively large changes in the solutionx,thenwe callAill-conditioned.-The ratio givenbelowisrelated to theconditionofAand measures the degreeof singularity ofA(the larger this value is, the closerAis to being singular)1/n(largest oversmallest singular values)ƒLeast Squares Solutions ofmxnSystems-Consider theover-determinedsystem of linear equationsAx=b,(Aismxnwithm�n)-Letrbe the residual vector for somex:r=Ax-b-The vectorx*which yields the smallest possible residual is called aleast-squaressolution (it is an approximate solution).||r||=||Ax*-b||£||Ax-b|| for allxÎRn-Although a least-squares solution always exist, it might not be unique !-The least-squares solutionxwith the smallest norm ||x|| is unique and it isgivenby:ATAx=ATborx=(ATA)-1ATb=A+bExample:éêêë-112223-1ùúúûéêëx1x2ùúû=éêêë075ùúúû -4-x=A+b=éêë-.ùúûéêêë075ùúúû=éêë2. 4920. 787ùúûƒComputingA+using SVD-IfATAis ill-conditioned or singular,wecan use SVD to obtain a least squaressolution as follows:x=A+b»VD-10UTbD-10=ìíî1/i0ifi�totherwise(wheretis a small threshold)ƒLeast Squares Solutions ofnxnSystems-IfAis ill-conditioned or singular,SVD can give usaworkable solution in thiscase too:x=A-1b»VD-10UTbƒHomogeneous Systems-Supposeb=0, then the linear system is called homogeneous:Ax=0(assumeAismxnandA=UDVT)-The minimum-norm solution in this case isx=0 (trivial solution).-For homogeneous linear systems, the meaning of a least-squares solution ismodiŒed byimposing the constraint:||x||=1-This is a "constrained" optimization problem:min||x||=1||Ax|| -5--The minimum-norm solution for homogeneous systems is not always unique.Special case:rank(A)=n-1(m³n-1,n=0)solution isx=avn(ais a constant)(vnis the last column ofV-- corresponds to the smallest)General case:rank(A)=n-k(m³n-k,n-k+1=...=n=0)solution isx=a1vn-k+a2vn-k-1+...+akvn(aisisaconstant)witha21+a22+...+a2k=1