Diagonal and low bank matrix decomposistions correlation matrices and ellipsoid fittings - PDF document

SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKYMuchoftheliteratureondiagonalandlow-rankmatrixdecompositionsisinoneoftwoveins.Anearlyapproach[1]thathasseenrecentrenewedinterest[11]isanal-gebraicone,wheretheprincipalaimistogiveacharacterizationofthevanishingidealofthesetofsymmetricmatricesthatdecomposeasthesumofadiagonalmatrixandarankmatrix.Suchacharacterizationhasonlybeenobtainedfortheborder=1,1(duetoKalman[17])andtherecentlyresolved=2case(duetoBrouwerandDraisma[3]followingaconjecturebyDrton,Sturmfels,andSullivan[11]).Thisapproachdoesnot(yet)oerscalablealgorithmsforperform-ingdecompositions,renderingitunsuitableformanyapplications,includingthoseinhigh-dimensionalstatistics,optics[12],andsignalprocessing[24].Theothermainapproachtofactoranalysisisviaheuristiclocaloptimizationtechniques,oftenbasedontheexpectationmaximizationalgorithm[9].Thisapproach,whilecomputationallytractable,typicallyoersnoprovableperformanceguarantees.Athirdwayisoeredbyconvexoptimization-basedmethodsfordiagonalandlow-rankdecompositionssuchasminimumtracefactoranalysis(MTFA),theideaandinitialanalysisofwhichdatesatleasttoLedermanns1940work[21].MTFAiscomputationallytractable,beingbasedonasemide“niteprogram(seesection2),andyetoersthepossibilityofprovableperformanceguarantees.InthispaperweprovideanewanalysisofMTFAthatisparticularlysuitableforhigh-dimensionalproblems.Semide“niteprogrammingdualitytheoryprovidesalinkbetweenthismatrixdecompositionheuristicandthefacialstructureofthesetofcorrelationmatricespositivesemide“nitematriceswithunitdiagonal„alsoknownastheelliptope[19].Thissetisoneofthesimplestofspectrahedra„anesectionsofthepositivesemidef-initecone.Spectrahedraareofparticularinterestfortworeasons.First,spectrahedraarearichclassofconvexsetsthathavemanyniceproperties(suchasbeingfaciallyexposed).Second,therearewell-developedalgorithms,ecientbothintheoryandinpractice,foroptimizinglinearfunctionalsoverspectrahedra.Theseoptimizationproblemsareknownassemide“niteprograms[30].Theelliptopearisesinsemide“niteprogramming-basedrelaxationsofproblemsinareassuchascombinatorialoptimization(e.g.,theproblem[14])andstatisticalmechanics(e.g.,the-vectorspinglassproblem[2]).Inaddition,theproblemofprojectingontothesetof(possiblylow-rank)correlationmatriceshasenjoyedconsiderableinterestinmathematical“nanceandnumericalanalysisinrecentyears[16].Ineachoftheseapplicationsthestructureofthesetoflow-rankcorrelationmatrices,i.e.,thefacialstructureofthisconvexbody,playsanimportantrole.Understandingthefacesoftheelliptopeturnsouttoberelatedtothefollowingellipsoid“ttingproblem:givenpointsin),underwhatconditionsonthepointsisthereanellipsoidcenteredattheoriginthatpassesexactlythroughthesepoints?Whilethereisconsiderableliteratureonmanyellipsoid-relatedproblems,wearenotawareofanyprevioussystematicinvestigationofthisparticularproblem.1.1.Illustrativeapplication:Directionofarrivalestimation.Directionofarrivalestimationisaclassicalprobleminsignalprocessingwhere(block-)diagonalandlow-rankdecompositionproblemsarisenaturally.Inthissectionwebrie”ydiscusssomestylizedmodelsofthedirectionofarrivalestimationproblemthatcanbereducedtomatrixdecompositionproblemsofthetypeconsideredinthispaper.Supposewehavesensorsatlocations(thatarepassivelylisteningŽforwaves(electromagneticoracoustic)ataknownfrequencyfromsourcesinthefar“eld(sothatthewavesareapproximatelyplanewaveswhentheyreachthesensors).Theaimistoestimatethenumberofsourcesandtheir DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONS 1 2 Fig.1.1Planewavesfromdirectionsandarrivingatanarrayofsensorsequallyspacedonacircle(auniformcirculararray).directionsofarrival)givensensormeasurementsandknowledgeofthesensorlocations(seeFigure1.1).Astandardmathematicalmodelforthisproblem(see[18]foraderivation)istomodelthevectorofsensormeasurementsattime(1.1)whereisthevectorofbasebandsignalwaveformsfromthesources,isthevectorofsensormeasurementnoise,and)isthematrixwithcomplexentries[ cos(sin(apositiveconstantrelatedtothefrequencyofthewavesbeingsensed.Thecolumnspaceof)containsalltheinformationaboutthedirectionsofarrival.Assuch,subspace-basedapproachestodirectionofarrivalestimationaimtoestimatethecolumnspaceof)(fromwhichanumberofstandardtechniquescanbeemployedtoestimateTypically)and)aremodeledaszero-meanstationarywhiteGaussianpro-cesseswithcovariancesariancess(t)s(t)H]=Panddn(t)n(t)H]=Q,respectively(wheredenotestheHermitiantransposeofandd·]theexpectation).Inthesimplestsetting,)and)areassumedtobeuncorrelatedsothatthecovarianceofthesensormeasurementsatanytimeisThe“rsttermisHermitianpositivesemide“nitewithrank,i.e.,thenumberofsources.Undertheassumptionthatspatiallywell-separatedsensors(suchasinasensornetwork)haveuncorrelatedmeasurementnoise,isdiagonal.Inthiscasethecovarianceofthesensormeasurementsdecomposesasasumofapositivesemide“nitematrixofrankandadiagonalmatrix.Givenanapproximationof(e.g.,asamplecovariance)approximatelyperformingthisdiagonalandlow-rankmatrixdecompositionallowstheestimationofthecolumnspaceof)andinturnthedirectionsofarrival. SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKYAvariationonthisproblemoccursiftherearemultiplesensorsateachlocation,sensing,forexample,wavesatdierentfrequencies.Againundertheassumptionthatwell-separatedsensorshaveuncorrelatedmeasurementnoise,andsensorsatthesamelocationhavecorrelatedmeasurementnoise,thesensornoisecovariancematrixwouldbeblock-diagonal.Assuchthecovarianceofallthesensormeasurementswoulddecomposeasthesumofalow-rankmatrix(withrankequaltothetotalnumberofsourcesoverallmeasuredfrequencies)andablock-diagonalmatrix.Ablock-diagonalandlow-rankdecompositionproblemalsoarisesifthesecond-orderstatisticsofthenoisehavecertainsymmetries.Thismightoccurincaseswherethesensorsthemselvesarearrangedinasymmetricway(suchasintheuniformcirculararrayshowninFigure1.1).Inthiscasethereisaunitarymatrix(dependingonlyonthesymmetrygroupofthearray)suchthatTQTblock-diagonal[25].Thenthecovarianceofthesensormeasurements,whenwrittenincoordinateswithrespectto,isTQTwhichhasadecompositionasthesumofablock-diagonalmatrixandarankHer-mitianpositivesemide“nitematrix(asconjugationbydoesnotchangetherankofthisterm).NotethatthematrixdecompositionproblemsdiscussedinthissectioninvolveHermitianmatriceswithcomplexentries,ratherthanthesymmetricmatriceswithrealentriesconsideredelsewhereinthispaper.InAppendixBwebrie”ydiscusshowresultsforthecomplexcasecanbeobtainedfromourresultsfortheblock-diagonalandlow-rankdecompositionproblemoverthereals.1.2.Contributions.RelatingMTFA,correlationmatrices,andellipsoid“tting.WeintroduceandmakeexplicitthelinksbetweentheanalysisofMTFA,thefacialstructureoftheelliptope,andtheellipsoid“ttingproblem,showingthattheseproblemsare,inaprecisesense,equivalent(seeProposition3.1).Assuch,werelateabasicprobleminstatisticalmodeling(tractablediagonalandlow-rankmatrixdecompositions),abasicprobleminconvexalgebraicgeometry(understandingthefacialstructureofperhapsthesimplestofspectrahedra),andabasicgeometricproblem.Asucientconditionforthethreeproblems.Themainresultofthepaperistoestablishanew,simple,sucientconditiononasubspacethatensuresthatMTFAcorrectlydecomposesmatricesoftheform,whereisthecolumnspaceof.Theconditionisstatedintermsofameasureofcoherenceofasubspace(madepreciseinDe“nition4.1).Informally,thecoherenceofasubspaceisarealnumberbetweenzeroandonethatmeasureshowclosethesubspaceistocontaininganyoftheelementaryunitvectors.ThisresultcanbetranslatedintonewresultsfortheothertwoproblemsunderconsiderationbasedontherelationshipbetweentheanalysisofMTFA,thefacesoftheelliptope,andellipsoid“tting.Block-diagonalandlow-rankdecompositions.Insection5weturnourattentiontotheblock-diagonalandlow-rankdecompositionproblem,showinghowourresultsgeneralizetothatsetting.Ourargumentscombineourresultsforthediagonalandlow-rankdecompositioncasewithanunderstandingofthesymmetriesoftheblock-diagonalandlow-rankdecompositionproblem.1.3.Outline.Theremainderofthepaperisorganizedasfollows.Wedescribenotation,givesomebackgroundonsemide“niteprogramming,andprovideprecise DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONSproblemstatementsinsection2.Insection3wepresentour“rstcontributionbyestablishingrelationshipsbetweenthesuccessofMTFA,thefacesoftheelliptope,andellipsoid“tting.Wethenillustratetheseconnectionsbynotingtheequivalenceofaknownresultaboutthefacesoftheelliptope,andaknownresultaboutMTFA,andtranslatingtheseintothecontextofellipsoid“tting.Section4isfocusedonestablishingandinterpretingourmainresult:asucientconditionforthethreeproblemsbasedonacoherenceinequality.Finallyinsection5wegeneralizeourresultstotheanalogoustractableblock-diagonalandlow-rankdecompositionproblem.2.Backgroundandproblemstatements.2.1.Notation.x,ywedenotebyx,ythestandardEuclideaninnerproductandbyx,xthecorrespondingEuclideannorm.Wewrite0and0toindicatethatisentrywisenonnegativeandstrictlypositive,respectively.Correspondingly,ifX,Y,thesetofsymmetricmatrices,thenwedenotebyX,Y=tr()thetraceinnerproductandbyX,XtheFrobeniusnorm.Wewrite0and0toindicatethatpositivesemide“niteandstrictlypositivede“nite,respectively.Wewritefortheconeofpositivesemide“nitematrices.Thecolumnspaceofamatrixisdenoted)andthenullspaceisdenoted).Ifisanmatrix,thendiag(isthediagonalof.Ifthendiagisthediagonalmatrixwith[diagfor,...,nisasubspaceof,thendenotestheorthogonalprojectoronto,thatis,theself-adjointlinearmapsuchthatandtr()=dim(Weusethenotationforthevectorwithaoneinthethpositionandzeroselsewhereandthenotationtodenotethevectorallentriesofwhichareone.Weusetheshorthand[]fortheset,...,n.Thesetofcorrelationmatrices,i.e.,positivesemide“nitematriceswithunitdiagonal,isdenoted.Forbrevitywetypicallyrefertoastheelliptopeandtheelementsofascorrelationmatrices.2.2.Semide“niteprogramming.Thetermsemide“niteprogramming[30]referstoconvexoptimizationproblemsoftheform(2.1)minimizeC,Xsubjecttowhereandsymmetricmatrices,,andisalinearmap.Thedualsemide“niteprogramis(2.2)maximizey,Sb,ysubjecttowhereistheadjointofGeneralsemide“niteprogramscanbesolvedecientlyusinginteriorpointmeth-ods[30].Whileourfocusinthispaperisnotonalgorithms,weremarkthatforthestructuredsemide“niteprogramsdiscussedinthispaper,manydierentspecial-purposemethodshavebeendevised.Themainresultaboutsemide“niteprogrammingthatweuseisthefollowingoptimalitycondition(see[30],forexample).Theorem2.1.Suppose(2.1)and(2.2)arestrictlyfeasible.Thenandareoptimalfortheprimal(2.1)anddual(2.2),respectively,ifandonlyisprimalfeasible,isdualfeasible,and SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKY2.3.Tractablediagonalandlow-rankmatrixdecompositions.Todecom-poseintoadiagonalpartandapositivesemide“nitelow-rankpart,wemaytrytosolvethefollowingrankminimizationproblem:D,Lrank()subjecttodiagonal.Sincetherankfunctionisnonconvexanddiscontinuous,itisnotclearhowtosolvethisoptimizationproblemdirectly.Oneapproachthathasbeensuccessfulforotherrankminimizationproblems(forexample,thosein[22,23])istoreplacetherankfunctionwiththetracefunctionintheobjective.Thiscanbeviewedasaconvexi“cationoftheproblemasthetracefunctionistheconvexenvelopeoftherankfunctionwhenrestrictedtopositivesemide“nitematriceswithspectralnormatmostone.Performingthisconvexi“cationleadstothesemide“niteprogramMTFA:(2.3)minimizeD,Ltr()subjecttodiagonal.IthasbeenshownbyDellaRicciaandShapiro[7]thatifMTFAisfeasibleithasauniqueoptimalsolution.Onecentralconcernofthispaperistounderstandwhenthediagonalandlow-rankdecompositionofamatrixgivenbyMTFAiscorrectŽinthefollowingsense.RecoveryproblemSupposeisamatrixoftheform,whereisdiagonalandispositivesemide“nite.Whatconditionson()ensurethat)istheoptimumofMTFAwithinputWeestablishinsection3thatwhether()istheoptimumofMTFAwithdependsonlyonthecolumnspaceof,motivatingthefollowingde“nition.Definition2.2.AsubspacerecoverablebyMTFAifforeverydi-agonalandeverypositivesemide“nitewithcolumnspaceistheoptimumofMTFAwithinputIntheseterms,wecanrestatetherecoveryproblemsuccinctlyasfollows.RecoveryproblemDeterminewhichsubspacesofarerecoverablebyMTFA.MuchofthebasicanalysisofMTFA,includingoptimalityconditionsandrelationsbetweenminimumrankfactoranalysisandminimumtracefactoranalysis,wascar-riedoutinasequenceofpapersbyShapiro[26,27,28]andDellaRicciaandShapiro[7].Morerecently,Chandrasekaranetal.[6]andCand`esetal.[4]consideredconvexoptimizationmethodsfordecomposingamatrixasasumofasparseandlow-rankmatrix.Sinceadiagonalmatrixiscertainlysparse,theanalysisin[6]canbespe-cializedtogivefairlyconservativesucientconditionsforthesuccessoftheirconvexprogramsinperformingdiagonalandlow-rankdecompositions(seesection4.1).Thediagonalandlow-rankdecompositionproblemcanalsobeinterpretedasalow-rankmatrixcompletionproblem,wherewearegivenalltheentriesofalow-rankmatrixexceptthediagonalandaimtocorrectlyreconstructthediagonalentries.Assuch,thispaperiscloselyrelatedtotheideasandtechniquesusedintheworkofCand`esandRecht[5]andanumberofsubsequentpapersonthistopic.Wewouldliketoemphasizeakeypointofdistinctionbetweenthatlineofworkandthepresentpaper.Therecentlow-rankmatrixcompletionliteraturelargelyfocusesondeter-miningtheproportionofrandomlyselectedentriesofalow-rankmatrixthatneed DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONStoberevealedtobeabletoreconstructthatlow-rankmatrixusingatractablealgo-rithm.Theresultsofthispaper,ontheotherhand,canbeinterpretedasattemptingtounderstandwhichlow-rankmatricescanbereconstructedfroma“xedandquitecanonicalpatternofrevealedentries.2.4.Facesoftheelliptope.Thefacesoftheconeofpositivesemide“nitematricesarealloftheform(2.4)U}whereisasubspaceof[19].Conversely,givenanysubspaceisafaceof.Asaconsequence,thefacesofarealloftheform(2.5)diag(whereisasubspaceof[19].Itisnotthecase,however,thatforeverysubspacethereisacorrelationmatrixwithnullspacecontaining,motivatingthefollowingde“nition.Definition2.3(see[19]AsubspacerealizableifthereisancorrelationmatrixsuchthatTheproblemofunderstandingthefacialstructureofthesetofcorrelationmatricescanberestatedasfollows.Facialstructureproblem.Determinewhichsubspacesofarerealizable.Muchisalreadyknownaboutthefacesoftheelliptope.Forexample,allpossibledimensionsoffacesaswellaspolyhedralfacesareknown[20].Characterizationsoftherealizablesubspacesofofdimension1,2,and1aregivenin[8]andimplicitlyin[19]and[20].Nevertheless,littleisknownaboutwhich-dimensionalsubspacesofarerealizableforgeneraland2.5.Ellipsoid“tting.Throughout,anellipsoidisasetoftheform,where0.NotethatthisincludesdegenerateŽellipsoids.Ellipsoid“ttingproblemWhatconditionsonacollectionofpointsinensurethatthereisacenteredellipsoidpassingexactlythroughallthosepoints?Letusconsidersomebasicpropertiesofthisproblem.Numberofpoints.wecanalways“tanellipsoidtothepoints.Indeedifisthematrixwithcolumns,...,v,thentheimageoftheunitsphereinisacenteredellipsoidpassingthrough.Ifandthepointsaregeneric,Žthenwecannot“tacenteredellipsoidtothem.Thisisbecauseifwerepresenttheellipsoidbyasymmetricmatrix,theconditionthatitpassesthroughthepoints(ignoringthepositivityconditionon)meansthatmustsatisfylinearlyindependentequations.Invariances.)isaninvertiblelinearmap,thenthereisanellip-soidpassingthrough,...,vifandonlyifthereisanellipsoidpassingthrough,Tv,...,Tv.Thismeansthatwhetherthereisanellipsoidpassingthroughpointsindependsnotontheactualsetofpointsbutonasubspaceofrelatedtothepoints.Wesummarizethisobservationinthefollowinglemma.Lemma2.4.Supposeisamatrixwithrowspace.Ifthereisacenteredellipsoidinpassingthroughthecolumnsof,thenthereisacenteredellipsoidpassingthroughthecolumnsofanymatrixwithrowspaceLemma2.4assertsthatwhetheritispossibleto“tanellipsoidtodependsonlyontherowspaceofthematrixwithcolumnsgivenbythe,motivatingthefollowingde“nition. SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKYDefinition2.5.-dimensionalsubspacehastheellipsoid“ttingpropertyifthereisamatrixwithrowspaceandacenteredellipsoidinthatpassesthrougheachcolumnofAssuchwecanrestatetheellipsoid“ttingproblemasfollows.Ellipsoid“ttingproblemDeterminewhichsubspacesofhavetheellipsoid“ttingproperty.3.Relatingellipsoid“tting,diagonalandlow-rankdecompositions,andcorrelationmatrices.Inthissectionweshowthattheellipsoid“ttingprob-lem,therecoveryproblem,andthefacialstructureproblemareequivalentinthefollowingsense.Proposition3.1.Letbeasubspaceof.Thenthefollowingareequivalent:isrecoverablebyMTFA.isrealizable.hastheellipsoid“ttingproperty.Proof.Letdim(.Toseethat2implies3,letbeamatrixwithnullspaceandletdenotethethcolumnof.Ifisrealizablethereisacorrelationmatrixwithnullspacecontaining.Hencethereissome0suchthatand=1forrn].Sincehasnullspace,ithasrow.Hencethesubspacehastheellipsoid“ttingproperty.Byreversingtheargumentweseethattheconversealsoholds.Theequivalenceof1and2arisesfromsemide“niteprogrammingduality.Follow-ingaslightreformulation,MTFA(2.3)canbeexpressedas(3.1)maximizesubjectto=diaganditsdualas(3.2)minimizeX,Ysubjecttodiag(whichisclearlyjusttheoptimizationofthelinearfunctionalde“nedbyovertheelliptope.Wenotethat(3.1)isexactlyinthestandarddualform(2.2)forsemide“niteprogrammingandcorrespondinglythat(3.2)isinthestandardprimalform(2.1)forsemide“niteprogramming.SupposeisrecoverablebyMTFA.Fixadiagonalmatrixandapositivesemide“nitematrixwithcolumnspaceandlet.Since(3.1)and(3.2)arestrictlyfeasible,byTheorem2.1(optimalityconditionsforsemide“niteprogramming),thepair(diag()isanoptimumof(3.1)ifandonlyifthereissomecorrelationmatrixsuchthat=0.Sincethisimpliesthatisrealizable.Conversely,ifisrealizable,thereissomesuchthatforeverywithcolumnspace,showingthatisrecoverablebyMTFA. Remark.WenotethatintheproofofProposition3.1weestablishedthatthetwoversionsoftherecoveryproblemstatedinsection2.3areactuallyequivalent.Inparticular,whether()istheoptimumofMTFAwithinputdependsonlyonthecolumnspaceof3.1.Certi“catesoffailure.Wecancertifythatasubspaceisrealizablebyconstructingacorrelationmatrixwithnullspacecontaining.Wecanalsoestablishthatasubspaceisnotrealizablebyconstructingamatrixthatcerti“esthisfact.Geometrically,asubspaceisrealizableifandonlyifthesubspace DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONSU}ofsymmetricmatricesintersectswiththeelliptope.Soacerti“catethatisnotrealizableisahyperplaneinthespaceofsymmetricmatricesthatstrictlyseparatestheelliptopefrom.Thefollowinglemmadescribesthestructureoftheseseparatinghyperplanes.Lemma3.2.Asubspacenotrealizableifandonlyifthereisadiagonalmatrixsuchthattr(andforallProof.ByProposition3.1,isnotrealizableifandonlyifdoesnothavetheellipsoid“ttingproperty.Letdim(andletbeamatrixwithrowspace.Thendoesnothavetheellipsoid“ttingpropertyifandonlyifwecannot“ndanellipsoidpassingthroughthecolumnsof,i.e.,thesemide“niteprogram(3.3)minimizesubjecttodiag(isinfeasible.Thesemide“niteprogrammingdualof(3.3)is(3.4)maximizesubjecttodiagBoththeprimalanddualproblemsarestrictlyfeasible,sostrongdualityholds.Since(3.4)isclearlyalwaysfeasible,(3.3)isinfeasibleifandonlyif(3.4)isunbounded(bystrongduality).Thisoccursifandonlyifthereissome0andyetdiag0.Then=diag)hasthepropertiesinthestatementofthelemma. 3.2.Exploitingconnections:Resultsforone-dimensionalsubspaces.1940,Ledermann[21]characterizedtheone-dimensionalsubspacesthatarerecover-ablebyMTFA.In1990,Grone,Pierce,andWatkins[15]gaveanecessaryconditionforasubspacetoberealizable.In1993,independentlyofLedermannswork,DelormeandPoljak[8]showedthatthisconditionisalsosucientforone-dimensionalsub-spaces.SincewehaveestablishedthatasubspaceisrecoverablebyMTFAifandonlyifitisrealizable,LedermannsresultandDelormeandPoljaksresultsareequivalent.Inthissectionwetranslatetheseequivalentresultsintothecontextoftheellipsoid“ttingproblem,givingageometriccharacterizationofwhenitispossibleto“tacenteredellipsoidto+1pointsinDelormeandPoljakstatetheirresultintermsofthefollowingde“nition.Definition3.3(DelormeandPoljak[8]Avectorbalanced(3.5)forallln].Iftheinequalityisstrictwesaythatstrictlybalanced.Asubspace(strictly)balancedifeveryis(strictly)balanced.Inthefollowing,thenecessaryconditionisduetoGroneetal.[15]andthesucientconditionisduetoLedermann[21](inthecontextoftheanalysisofMTFA)andDelormeandPoljak[8](inthecontextofthefacialstructureoftheelliptope).Westatetheresultonlyintermsofrealizabilityofasubspace.Theorem3.4.Ifasubspaceisrealizable,thenitisbalanced.Ifasubspaceisbalancedand)=1,thenitisrealizable.FromProposition3.1weknowthatwhetherasubspaceisrealizablecanbedeter-minedbydecidingwhetherwecan“tanellipsoidtoaparticularcollectionofpoints.Wenextdeveloptheanalogousgeometricinterpretationofbalancedsubspaces. SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKYDefinition3.5.Acollectionofpoints,...,visinconvexpositionifforeachachn],viliesontheboundaryoftheconvexhullof,...,Thefollowinglemmamakesprecisetheconnectionbetweenbalancedsubspacesandpointsinconvexposition.Lemma3.6.Supposeisanymatrixwith.Thenisbalancedifandonlyifthecolumnsofareinconvexposition.Proof.WedeferadetailedprooftoAppendixA,givingonlythemainideahere.Wecancheckifacollectionofpointsisinconvexpositionbycheckingthefeasibilityofasystemoflinearinequalities(giveninAppendixA).Anapplicationoflinearprogrammingdualityestablishesthatcerti“catesofinfeasibilityoftheselinearinequalitiestaketheformofelementsofthatarenotbalanced. BycombiningTheorem3.4withLemma3.6,weareinapositiontointerpretTheorem3.4purelyintermsofellipsoid“tting.Corollary3.7.Ifthereisacenteredellipsoidpassingthrough,thentheyareinconvexposition.Ifand,thenthereisacenteredellipsoidpassingthroughthem.Inthisgeometricsettingthenecessaryconditionisclear„wecanonlyhopeto“ndacenteredellipsoidpassingthroughacollectionofpointsiftheyareinconvexposition,i.e.,theylieontheboundaryofsomeconvexbody.Onemaywonderforwhichotherand(ifany)itisthecasethatthereisanellipsoidpassingthroughanysetofpointsinthatareinconvexposition(orequivalentlythatany()-dimensionalbalancedsubspaceofisrealizable).Thisisnotthecaseforgeneraland.Forexample,let+1andchoose,...,vontheunitsphereinsothatthesphereistheuniquecenteredellipsoidpassingthroughthem.Thenchooseintheinteriorofthespherebutnotintheconvexhullof.Theresultingpointsareinconvexpositionbutthereisnocenteredellipsoidpassingthroughthem.Onapositivenote,sincethereisclearlyanellipsoidpassingthroughanysubsetinconvexposition,wehavethefollowingsimpleadditiontoTheorem3.4.Proposition3.8.Ifasubspaceisbalancedand,thenisrealizable.4.Asucientconditionforthethreeproblems.Inthissectionweestab-lishanewsucientconditionforasubspacetoberealizableandconsequentlyasucientconditionfortoberecoverablebyMTFAandtohavetheellipsoid“ttingproperty.OurconditionisbasedonasimplepropertyofasubspaceknownasGivenasubspace,thecoherenceofisameasureofhowclosethesub-spaceistocontaininganyoftheelementaryunitvectors.Thisnotionwasintroduced(withadierentscaling)byCand`esandRechtintheirworkonlow-rankmatrixcom-pletion[5],althoughrelatedquantitieshaveplayedanimportantroleintheanalysisofsparsereconstructionproblemssincetheworkofDonohoandHuo[10].Definition4.1.isasubspaceof,thenthecoherence)=maxxn]PUei22.Thereareanumberofequivalentwaystoexpresscoherence,someofwhichwecollecthereforconvenience.Since(4.1))=max DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONSCoherencealsohasaninterpretationasthesquareoftheworst-caseratiobetweenthein“nitynormandtheEuclideannormonasubspace:(4.2))=maxxn]PUei22=maxxn]maxU\{ =maxU\{ Abasicpropertyofcoherenceisthatitsatis“estheinequality(4.3) foranysubspace[5].Thisholdsbecausehasdim()eigenvaluesequaltooneandtherestequaltozerosothatdim(=tr(maxxn][PU]ii=µ(U).Theinequality(4.3),togetherwiththede“nitionofcoherence,providesusefulintu-itionaboutthepropertiesofsubspaceswithlowcoherence,thatis,coherence.Anysubspacewithlowcoherenceisnecessarilyoflowdimensionandfarfromcontaininganyoftheelementaryunitvectors.Assuch,anysymmetricmatrixwithincoherentrow/columnspacesisnecessarilyoflowrankandquitedierentfrombeingadiagonalmatrix.4.1.Coherence-threshold-typesucientconditions.Inthissectionwefo-cuson“ndingthelargestpossiblesuchthatisrealizable,thatis,“ndingthebestpossiblecoherence-threshold-typesucientconditionforasubspacetoberealizable.Suchconditionsareofparticularinterestbecausethedependencetheyhaveontheambientdimensionandthedimensionofthesubspaceisonlythemilddependenceimpliedby(4.3).Incontrast,existingresults(e.g.,[8,20,19])aboutrealizabilityofsubspacesholdonlyforspeci“ccombinationsoftheambientdimensionandthedimensionofthesubspace.Thefollowingtheorem,ourmainresult,givesasucientconditionforrealizabilitybasedonacoherence-thresholdcondition.Furthermore,itestablishesthatthisisthebestpossiblecoherence-threshold-typesucientcondition.Theorem4.2.isasubspaceofand,thenisrealizable.Ontheotherhand,givenany,thereisasubspacewiththatisnotrealizable.Proof.Wegivethemainideaoftheproof,deferringsomedetailstoAppendixA.InsteadofprovingthatthereissomeU}suchthatforrn],itsucestochooseaconvexconethatisaninnerapproximationtoandestablishthatthereissomesuchthat=1forrn].Onenaturalchoiceistotakediag,whichisclearlycontainedinNotethatthereissomesuchthat=1forallln]ifandonlyifthereis0suchthat(4.4)diag(diagTherestoftheproofofthesucientconditioninvolvesshowingthatifthensuchanonnegativeexists.WeestablishthisinLemmaA.1.Nowletusconstruct,forany2,asubspacewithcoherencethatisnotrealizable.Letbethesubspaceofspannedby ).Then)=maxandyetbyTheorem3.4,isnotrealizablebecausenotbalanced. SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKYComparisonwithresultsonsparseandlow-rankdecompositions.In[6,Corol-lary3]asucientconditionisgivenunderwhichaconvexprogramrelatedtoMTFAcansuccessfullydecomposethesumofanunknownsparsematrixandanun-knownlow-rankmatrix.Theconditionisthatdegmax)inc(12,wheremax)isthemaximumnumberofnonzeroentriesperrow/columnofand,forasymmetricmatrixwithcolumnspace,inc( )isthesquarerootofthecoherenceasde“nedinthepresentpaper.Assuch,Chandrasekaranetal.showthattheconvexprogramtheyanalyzesuccessfullydecomposesthesumofadiagonalmatrixandasymmetricmatrixwithcolumnspaceaslongas1 12orequivalently144.Bycomparison,inthispaperboththeconvexprogramweconsiderandouranalysisofthatconvexprogramexploittheassumptionsthatthesparsematrixisdiagonalandthatthelow-rankmatrixispositivesemide“nite.Thisallowsustoobtainthemuchmorere“nedsucientconditionWenowestablishtwocorollariesofourcoherence-threshold-typesucientcon-ditionforrealizability.Thesecorollariescanbethoughtofasreinterpretationsofthecoherenceinequality2intermsofothernaturalquantities.Anellipsoid-“ttinginterpretation.WiththeaidofProposition3.1wereinterpretourcoherence-threshold-typesucientconditionasasucientconditiononasetofpointsinthatensuresthereisacenteredellipsoidpassingthroughthem.TheconditioninvolvessandwichingŽthepointsbetweentwoellipsoids(thatdependonthepoints).Indeed,given01andpointsthatspan,wede“netheellipsoidDefinition4.3.thepoints,...,vsatisfythe-sandwichcondition,...,vspanand,...,v,...,v,...,vTheintuitionbehindthisde“nition(illustratedinFigure4.1)isthatifthepointssatisfythe-sandwichconditionforclosetoone,thentheyarecon“nedtoathinellipsoidalshellthatisadaptedtotheirposition.OnemightexpectthatitiseasierŽto“tanellipsoidtopointsthatarecon“nedinthisway.Indeedthisisthecase.Corollary4.4.,...,vsatisfythe-sandwichcondition,thenthereisacenteredellipsoidpassingthrough,...,v  3 1 2 Fig.4.1Theellipsoidsshownareand.Thereisanellipsoidpassingthrough,andbecausethepointsaresandwichedbetweenand DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONSProof.Letbethematrixwithcolumnsgivenbythe,andletthe()-dimensionalnullspaceof.ThentheorthogonalprojectionontotherowspaceofandcanbewrittenasOurassumptionthatthepointssatisfythe12-sandwichconditionisequivalenttoassumingthat11PU]ii 1forallln],whichby(4.1)isequivalentto2.FromTheorem4.2weknowthat2impliesthatisrealizable.InvokingProposition3.1wethenconcludethatthereisacenteredellipsoidpassingthrough,...,v Abalanceinterpretation.Insection3.2wesawthatifasubspaceisrealizable,everyisbalanced.ThesucientconditionofTheorem4.2canbeexpressedintermsofabalanceconditionontheelementwisesquareoftheelementsofasubspace.(InwhatfollowsdenotestheelementwisesquareofavectorinLemma4.5.isasubspaceof,thenisstrictlybalancedforallisstrictlybalancedProof.Bythecharacterizationof)in(4.2)maxU\{whichinturnisequivalenttoforalllln]andall,i.e.,strictlybalancedforallSupposeisstrictlybalancedforall.Thenforallln]andallisstrictlybalanced.(Herewehaveusedthefactthatforany Withthisrelationshipestablishedwecanexpressboththeknownnecessarycon-ditionandoursucientconditionforasubspacetoberealizableintermsofthenotionofbalance.ThenecessaryconditionisrestatedfromTheorem3.4;thesu-cientconditionfollowsbycombiningTheorem4.2andLemma4.5.Corollary4.6.Ifasubspaceisrealizable,theneveryisbalanced.Ifisstrictlybalancedforevery,thenthesubspaceisrealizable.Remark.Suppose=spanisaone-dimensionalsubspaceof.Wehavejustestablishedthatifisstrictlybalanced,thenisrealizableandso(byTheorem3.4)mustbebalanced,afactweproveddirectlyinLemma4.5.4.2.Examples.TogainmoreintuitionforwhatTheorem4.2means,wecon-sideritsimplicationsintwoparticularcases.First,wecomparethecharacterizationofwhenitispossibleto“tanellipsoidto+1pointsin(Corollary3.7)withthespecializationofoursucientconditiontothiscase(Corollary4.4).Thiscom-parisonprovidessomegeometricinsightintohowconservativeoursucientconditionis.Second,weinvestigatethecoherencepropertiesofsuitablyrandomsubspaces.Thisprovidesintuitionaboutwhether2isaveryrestrictivecondition.Inparticular,weestablishthatmostŽsubspacesofwithdimensionboundedaboveby(1arerealizable. SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKY 2 1 (a)Theshadedsetis,thosepointsforwhichwecan“tanellipsoidthroughandthestandardbasisvectors. 2 1 (b)Theshadedsetis,thosepointssuchthatw,e,andsatisifytheconditionofCorollary4.4.Fig.4.2Comparingoursucientcondition(Corollary4.4)withthecharacterization(Corol-3.7)inthecaseof“ttinganellipsoidtopointsinFittinganellipsoidtopointsinRecallthattheresultofLedermannandDelormeandPoljak,interpretedintermsofellipsoid“tting,tellsusthatwecan“tanellipsoidto+1points,...,vifandonlyifthosepointsareinconvexposition(seeCorollary3.7).Wenowcomparethischaracterizationwiththe2-sandwichcondition,whichissucientbyCorollary4.4.Withoutlossofgeneralityweassumethatofthepointsare,...,e,thestandardbasisvectors,andcomparetheconditionsbyconsideringthesetoflocationsofthe(+1)stpointforwhichwecan“tanellipsoidthroughall+1points.Corollary3.7givesacharacterizationofthisregionas1forrk],whichisshowninFigure4.2(a)inthecase=2.Thesetofsuchthatw,e,...,esatisfythe12-sandwichconditioncanbewrittenas2forrk]}=wRk:k j=1w2j�1,w2iŠ j=iw2j1forrk],wherethesecondequalityholdsbecause( .TheregionisshowninFigure4.2(b)inthecase=2.ItisclearthatRealizabilityofrandomsubspaces.SupposeisasubspacegeneratedbytakingthecolumnspaceofanmatrixwithindependentandidenticallydistributedstandardGaussianentries.Forwhatvaluesofanddoessuchasubspacehave2withhighprobability,i.e.,satisfyoursucientconditionforbeingreal-izable?Thefollowingresultessentiallyshowsthatforlarge,mostŽsubspacesofdimen-sionatmost(1arerealizable.ThissuggeststhatMTFAisaverygoodheuristic DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONSfordiagonalandlow-rankdecompositionproblemsinthehigh-dimensionalsetting.IndeedmostŽsubspacesofdimensionuptoonehalftheambientdimension„hardlyjustlow-dimensionalsubspaces„arerecoverablebyMTFA.Proposition4.7.LetbeaconstantandsupposeTherearepositiveconstants,(dependingonlyon)suchthatifisarandomdimensionalsubspaceof,thenPr[isrealizable WeprovideaproofofthisresultinAppendixA.Themainideaisthatthecoherenceofarandom-dimensionalsubspaceofisthemaximumofrandomvariablesthatconcentratearoundtheirmeanofr/nforlargeToillustratetheresult,weconsiderthecasewhere4and192.Then(byexaminingtheproofinAppendixA)weseethatwecantake24and=24 8.Henceif192andisarandom4dimensionalsubspaceofwehavethatPr[isrealizable] 5.Tractableblock-diagonalandlow-rankdecompositionsandrelatedproblems.InthissectionwegeneralizeourresultstotheanalogueofMTFAforblock-diagonalandlow-rankdecompositions.Mimickingourearlierdevelopment,werelatetheanalysisofthisvariantofMTFAtothefacialstructureofavariantoftheelliptopeandageneralizationoftheellipsoid“ttingproblem.Thekeypointisthattheseproblemsallpossessadditionalsymmetriesthat,oncetakenintoaccount,essentiallyallowustoreduceouranalysistocasesalreadyconsideredinsections3and4.Throughoutthissection,letbea“xedpartitionof,...,n.Wesayamatrixis-block-diagonalifitiszeroexceptfortheprincipalsubmatricesindexedbytheelementsof.Wedenotebyblkdiagthemapthattakesanmatrixandmapsittotheprincipalsubmatricesindexedby.Itsadjoint,denotedblkdiagtakesatupleofsymmetricmatrices(IPandproducesanmatrixthatis-block-diagonalwithblocksgivenbytheWenowdescribetheanaloguesofMTFA,ellipsoid“tting,andtheproblemofdeterminingthefacialstructureoftheelliptope.Blockminimumtracefactoranalysis.,where-block-diagonaland0islowrank,theobviousanalogueofMTFAisthesemide“niteprogram(5.1)minimizeB,Ltr()subjectto-block-diagonal,whichwecallblockminimumtracefactoranalysis(BMTFA).Astraightforwardmod-i“cationoftheDellaRicciaandShapiroargumentforMTFA[7]showsthatifBMTFAisfeasibleithasauniqueoptimalsolution.Definition5.1.AsubspacerecoverablebyBMTFAifforeverythatis-block-diagonalandeverypositivesemide“nitewithcolumnspaceistheoptimumofBMTFAwithinputFacesofthe-elliptope.JustasMTFAisrelatedtothefacialstructureoftheelliptope,BMTFAisrelatedtothefacialstructureofthespectrahedron0:blkdiag)=(I,I,...,I SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKYWerefertoasthe-elliptope.Weextendthede“nitionofarealizablesubspacetothiscontext.Definition5.2.Asubspace-realizableifthereissomesuchthatGeneralizedellipsoid“tting.Todescribethe-ellipsoid“ttingproblemwe“rstintroducesomeconvenientnotation.Iffn]wewrite(5.2)=0ifI}fortheintersectionoftheunitspherewiththecoordinatesubspaceindexedbySupposeisacollectionofpointsandisthematrixwithcolumnsgivenbythe.Notingthat,andthinkingofalinearmapfrom,weseethattheellipsoid“ttingproblemisto“ndanellipsoidinwithboundarycontainingningn]V(S{i}),i.e.,thecollectionofpoints.The-ellipsoid“ttingproblemisthento“ndanellipsoidinboundarycontainingIP),i.e.,thecollectionofellipsoidsThegeneralizationoftheellipsoid“ttingpropertyofasubspaceisasfollows.Definition5.3.-dimensionalsubspacehasthe-ellipsoid“ttingpropertyifthereisamatrixwithrowspacesuchthatthereisacenteredellipsoidinwithboundarycontainingIP5.1.Relatingthegeneralizedproblems.Thefacialstructureoftheelliptope,BMTFA,andthe-ellipsoid“ttingproblemarerelatedbythefollowingresult,theproofofwhichisomittedasitisalmostidenticaltothatofProposition3.1.Proposition5.4.Letbeasubspaceof.Thenthefollowingareequivalent:isrecoverablebyBMTFA.-realizable.hasthe-ellipsoid“ttingproperty.ThefollowinglemmaistheanalogueofLemma3.2.Itdescribescerti“catesthatasubspaceisnot-realizable.AgaintheproofisalmostidenticaltothatofLemma3.2,soweomitit.Lemma5.5.Asubspacenot-realizableifandonlyifthereisa-block-diagonalmatrixsuchthattr(andforallForthesakeofbrevity,inwhatfollowsweonlydiscusstheproblemofwhether-realizablewithoutexplicitlytranslatingtheresultsintothecontextoftheothertwoproblems.5.2.Symmetriesofthe-elliptope.Wenowconsiderthesymmetriesofthe-elliptope.Ourmotivationfordoingsoisthatitallowsustopartitionsubspacesintoclassesforwhicheitherallelementsare-realizableornoneoftheelementsare-realizable.Itisclearthatthe-elliptopeisinvariantunderconjugationby-block-diagonalorthogonalmatrices.Letdenotethissubgroupofthegroupoforthogonalmatrices.Thereisanaturalactionofonsubspacesofde“nedasfollows.Ifandisasubspaceof,thenistheimageofthesubspacethemap.(Itisstraightforwardtocheckthatthisisawell-de“nedgroupaction.)Ifthereexistssomesuchthat,thenwewriteUUandsaythatandequivalent.Wecareaboutthisequivalencerelationonsubspacesbecausethepropertyofbeing-realizableisreallyapropertyofthecorrespondingequivalenceclasses.Proposition5.6.Supposeandaresubspacesof.IfUU,then-realizableifandonlyif-realizable. DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONSProof.If-realizablethereissuchthat=0forallSupposeforsomeandletPYP.Thenand)=(PYP)=0forall.Bythede“nitionofitisthenthecasethat=0forall.Hence-realizable.Theconverseclearlyalsoholds. 5.3.Exploitingsymmetries:Relatingrealizabilityand-realizability.Forasubspaceof,wenowconsiderhowthenotionsof-realizabilityandrealiz-ability(i.e.,[]-realizability)relatetoeachother.Since,if-realizable,itiscertainlyalsorealizable.Whiletheconversedoesnothold,wecanestablishthefollowingpartialconverse,whichwesubsequentlyusetoextendouranalysisfromsections3and4tothepresentsetting.Theorem5.7.Asubspace-realizableifandonlyifisrealizableforeverysuchthatProof.Wenotethatonedirectionoftheproofisobvioussince-realizabilityimpliesrealizability.Itremainstoshowthatifisnot-realizable,thenthereissomeequivalenttothatisnotrealizable.RecallfromLemma5.5thatifisnot-realizablethereissome-block-diagonalwithpositivetracesuchthat0forall.Since-block-diagonalthereissomesuchthatPXPisdiagonal.Sinceconjugationbyorthogonalmatricespreserveseigenvalues,tr(PXP)=tr(0.Furthermore,PXP0forall.HencePXP0forall.ByLemma3.2,PXPisacerti“catethatisnotrealizable,completingtheproof. ThepowerofTheorem5.7liesinitsabilitytoturnanyconditionforasubspacetoberealizableintoaconditionforthesubspacetobe-realizablebyappropriatelysymmetrizingtheconditionwithrespecttotheactionof.WenowillustratethisapproachbygeneralizingTheorem3.4andourcoherence-basedcondition(Theo-rem4.2)forasubspacetobe-realizable.Ineachcasewe“rstde“neanappropriatelysymmetrizedversionoftheoriginalcondition.Thenaturalsymmetrizedversionofthenotionofbalanceisasfollows.Definition5.8.Avector-balancedifforallIPJP\{I}Wenextde“netheappropriatelysymmetrizedanalogueofcoherence.Justasco-herencemeasureshowfarasubspaceisfromanyone-dimensionalcoordinatesubspace,-coherencemeasureshowfarasubspaceisfromanyofthecoordinatesubspacesin-dexedbyelementsofDefinition5.9.-coherenceofasubspace)=maxIPmaxJustasthecoherenceofcanbecomputedbytakingthemaximumdiagonalelementof,itisstraightforwardtoverifythatthe-coherenceofcanbecom-putedbytakingthemaximumofthespectralnormsoftheprincipalsubmatricescesPU]IindexedbyIPWenowuseTheorem5.7toestablishthenaturalgeneralizationofTheorem3.4.Corollary5.10.Ifasubspace-realizable,theneveryelementof-balanced.If=spanisone-dimensional,then-realizableifandonly-balanced. SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKYProof.Ifthereisthatisnot-balanced,thenthereissuchthatisnotbalanced.(Choosesothatitrotateseachuntilithasonlyonenonzeroentry.)Butthenisnotrealizableandsoisnot-realizable.Fortheconverse,we“rstshowthatifavectoris-balanced,thenitisbalanced.IP,andconsider.Thensince-balanced,JPandsoisbalanced.Nowsuppose=spanisone-dimensionaland-balanced.Sincebalanceditfollowsthat-balanced(andhencebalanced)every.ThenbyTheorem3.4spanisrealizableforevery.HencebyTheorem5.7,-realizable. Similarly,withtheaidofTheorem5.7wecanwriteaconditionthatisasucientconditionforasubspacetobe-realizable.ThefollowingisanaturalgeneralizationofTheorem4.2.Corollary5.11.,then-realizable.Proof.Byexaminingtheconstraintsinthevariationalde“nitionsof)and)weseethat).Consequentlyif2itfollowsfromTheorem4.2thatisrealizable.SinceisinvariantundertheactionofsubspaceswecanapplyTheorem5.7tocompletetheproof. 6.Conclusions.Weestablishedalinkbetweenthreeproblemsofindependentinterest:decidingwhetherthereisacenteredellipsoidpassingthroughacollectionofpoints,understandingthestructureofthefacesoftheelliptope,anddecidingwhichpairsofdiagonalandlow-rankmatricescanberecoveredfromtheirsumusingatractablesemide“nite-programming-basedheuristic,namelyMTFA.Weprovidedasimplesucientcondition,basedonthenotionofthecoherenceofasubspace,whichensuresthesuccessofMTFAandshowedthatthisisthebestpossiblecoherence-threshold-typesucientconditionforthisproblem.Finallywegavenaturalgen-eralizationsofourresultstotheproblemofanalyzingtractableblock-diagonalandlow-rankdecompositions,showinghowthesymmetriesofthisproblemallowustoreducemuchoftheanalysistotheoriginalcaseofdiagonalandlow-rankdecomposi-tions.AppendixA.Additionalproofs.A.1.ProofofLemma3.6.We“rstestablishLemma3.6,whichgivesanin-terpretationofthebalanceconditionintermsofellipsoid“tting.Proof.Theproofisafairlystraightforwardapplicationoflinearprogrammingduality.Throughoutletbethematrixwithcolumnsgivenbythe.Thepointisontheboundaryoftheconvexhullof,...,ifandonlyifthereexistssuchthatx,v=1andx,v| 1forall.Equivalently,thefollowinglinearprogram(whichdependson)isfeasible:(A.1)minimizesubjectto1forallSupposethereissomesuchthatisintheinteriorofconv.Then DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONS(A.1)isnotfeasiblesotheduallinearprogram(whichdependson(A.2)maximizesubjecttoisunbounded.Thisisthecaseifandonlyifthereissomeinthenullspaceofsuchthat.Ifsuchaexists,thenitiscertainlythecasethatandsoisnotbalanced.Conversely,ifisinthenullspaceofandisnotbalanced,theneithersatis“esforsome.Hencethelinearprogram(A.2)associatedwiththeindexisunboundedandsothecorrespondinglinearprogram(A.1)isinfeasible.Itfollowsthatisintheinterioroftheconvexhullof,..., A.2.CompletingtheproofofTheorem4.2.WenowcompletetheproofofTheorem4.2byestablishingthefollowingresultabouttheexistenceofanonnegativesolutiontothelinearsystem(4.4).LemmaA.1.,thendiag(diaghasanonnegativesolutionProof.Wenotethatthelinearsystemdiag(diagcanberewrit-tenas,wheredenotestheentrywiseproductofmatrices.Assuch,weneedtoshowthatisinvertibleand(0.Todoso,weappealtothefollowing(slightrestatement)ofatheoremofWalters[31]regardingpositivesolutionstocertainlinearsystems.TheoremA.2(Walters[31]Supposeisasquarematrixwithnonnegativeentriesandpositivediagonalentries.Letbeadiagonalmatrixwithall.Ifand,thenisinvertibleandForsimplicityofnotationlet.WeapplyTheoremA.2withandandsoneedtocheckthat(,whereisthediagonalmatrixwithforrn].SinceP2=Pitfollowsthatforalllln],nj=1P2ij=[P2]ii=Pii.Ourassumptionthat2impliesthatmininn]Pii�1/2andsoand4forallln].HenceHence(PP)DŠ11]i=n j=1P2ijDŠ1jj=1+=1+4(aswerequire. A.3.ProofofProposition4.7.WenowestablishProposition4.7,givingaboundontheprobabilitythatasuitablyrandomsubspaceisrealizablebyboundingtheprobabilitythatithascoherencestrictlyboundedaboveby1Proof.Itsucestoshowthat)=1forallwithhighprobability.Themainobservationweuseisthatifisarandom-dimensionalsubspaceofandisany“xedvectorwith=1,then2),wherep,q)denotesthebetadistribution[13].Inthecasewhere=(1,usingatailboundforrandomvariables[13]weseethatifis“xedand,thenPr[(1+2 a1 (4Š2/2nŠ1/2eŠak, SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKYwhere3.Takingaunionboundoverevents,aslongasPr[)forsome forappropriatepositiveconstants¯and AppendixB.Usingcomplexscalars.Inthisappendixwebrie”ydiscusstheanaloguesofourmainresultswhenusingcomplexratherthanrealscalars.Ifdenoteby)and)itsrealandimaginarypartsandbyitsmodulus.AnHermitianmatrixispositivesemide“niteif0forall.Notethatispositivesemide“niteifandonlyiftherealsymmetricmatrix(B.1)ispositivesemide“nite.WecallHermitianpositivesemide“nitematriceswithalldiagonalelementsequaltoonecomplexcorrelationmatrices.Wefocushereonestab-lishingsucientconditionsforthecomplexanalogueofrealizability.DefinitionB.1.Asubspace-realizableifthereisancomplexcorrelationmatrixsuchthatNotethatthesetofcomplexcorrelationmatricesisinvariantunderconjugationbydiagonalunitarymatrices(i.e.,diagonalmatriceswithalldiagonalelementshavingmodulusone).B.1.ComplexanalogueofTheorem3.4.Asbefore,wesaybalancedforalllln].Asubspace-balancedifallitselementsare-balancedTheoremB.2.Ifasubspace-realizable,thenitis-balanced.Ifasubspace-balancedanddim()=1,thenitis-realizable.Proof.Letspantheone-dimensionalsubspaceandletbesuchthat.Clearly-balancedifandonlyifisbalancedor,equivalently,spanisrealizable.Nowspanisrealizableifandonlyif-realizable.(Toseethisnotethatifisacomplexcorrelationmatrix=0,then)isacorrelationmatrixwith)=0.)Finallynotethatspanisrealizableifandonlyifspanisrealizable.Thisisbecausethereisadiagonalunitarymatrixsuchthat,soifisacomplexcorrelationmatrixwith=0,thenDMDisacomplexcorrelationmatrix=0.Theaboveargumentalsoestablishesthatif-realizable,thenitisbalanced,aseveryelementofa-realizablesubspacespansaone-dimensionalrealizablesubspaceandsois-balanced. B.2.ComplexanalogueofTheorem4.2.WecouldestablishthecomplexanalogueofTheorem4.2byappropriatelymodifyingtheproofgiveninsection4.Wetakeadierentapproach,insteadrelatingthe-realizabilityofasubspaceoftheblockrealizabilityofarelatedsubspaceofDefinitionB.3.isasubspaceofde“neasubspace=span DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONSNotethatisinvariantundermultiplicationby,correspondingtobeingclosedundermultiplicationbythecomplexunit.Observethatifisacomplexcorrelationmatrix,then)haszerodiagonal,so(de“nedin(B.1))isanelementofthe-elliptopeforthepartition,...,of[2Throughthispartitionof[2]wecanrelaterealizabilitypropertiesofandLemmaB.4.Asubspace-realizableifandonlyif-realizable.Proof.Ifisacomplexcorrelationmatrixsuchthat,then(asde“nedin(B.1))isinthe-elliptopeanditisstraightforwardtocheckthatOntheotherhandsupposethereisanelementofthe-elliptopewithnullspacecontaining.ItisstraightforwardtocheckthattheHermitianmatrix2hasunitdiagonalandsatis“es.ItremainstoshowthatisHermitianpositivesemide“nite.Toseethisnotethatwhichisclearlypositivesemide“nitewhenever De“nethecomplexcoherenceofasubspace)=maxxn]PUei22.Notethatitfollowsdirectlyfromthede“nitionsthat).Finally,thecomplexversionofTheorem4.2isasfollows.TheoremB.5.isasubspaceofand,then-realizable.Proof.Since)wehavethat-realizable=-realizable,wherethelasttwoimplicationsfollowfromCorollary5.11andLemmaB.4. Acknowledgments.TheauthorswouldliketothankProf.SanjoyMitterforhelpfuldiscussionsandtheanonymousreviewersforcarefullyreadingthemanuscriptandprovidingmanyhelpfulsuggestions.[1]A.A.AlbertThematricesoffactoranalysis,Proc.Natl.Acad.Sci.USA,30(1944),pp.90…[2]J.Briet,F.deOliveiraFilho,andF.VallentinGrothendieckInequalitiesforSemide“niteProgramswithRankConstraint,preprint,arXiv:1011.1754,2010.[3]A.E.BrouwerandJ.DraismaEquivariantGr¨obnerbasesandtheGaussiantwo-factormodel,Math.Comp.,80(2011),pp.1123…1133.[4]E.J.Candes,X.Li,Y.Ma,andJ.WrightRobustprincipalcomponentanalysis?,J.ACM,58(2011),pp.11:1…11:37.[5]E.J.CandesandB.RechtExactmatrixcompletionviaconvexoptimization,Found.Comput.Math.,9(2009),pp.717…772.[6]V.Chandrasekaran,S.Sanghavi,P.A.Parrilo,andA.S.WillskyRank-sparsityinco-herenceformatrixdecomposition,SIAMJ.Optim.,21(2011),pp.572…596.[7]G.DellaRicciaandA.ShapiroMinimumrankandminimumtraceofcovariancematricesPsychometrika,47(1982),pp.443…448.[8]C.DelormeandS.PoljakCombinatorialpropertiesandthecomplexityofamax-cutap-proximation,EuropeanJ.Combin.,14(1993),pp.313…333.[9]A.P.Dempster,N.M.Laird,andD.B.RubinMaximumlikelihoodfromincompletedataviatheEMalgorithm,J.R.Stat.Soc.Ser.BStat.Methodol.,39(1977),pp.1…38.[10]D.L.DonohoandX.HuoUncertaintyprinciplesandidealatomicdecomposition,IEEETrans.InformTheory,47(2001),pp.2845…2862. SAUNDERSON,CHANDRASEKARAN,PARRILO,WILLSKY[11]M.Drton,B.Sturmfels,andS.SullivantAlgebraicfactoranalysis:Tetrads,pentadsandbeyond,Probab.TheoryRelatedFields,138(2007),pp.463…493.[12]M.FazelandJ.GoodmanApproximationsforPartiallyCoherentOpticalImagingSystemsTechnicalreport,StanfordUniversity,Stanford,CA,1998.[13]P.FranklandH.MaeharaSomegeometricapplicationsofthebetadistribution,Ann.Inst.Statist.Math.,42(1990),pp.463…474.[14]M.X.GoemansandD.P.WilliamsonImprovedapproximationalgorithmsformaximumcutandsatis“abilityproblemsusingsemide“niteprogramming,J.ACM,42(1995),pp.1115…[15]R.Grone,S.Pierce,andW.WatkinsExtremalcorrelationmatrices,LinearAlgebraAppl.,134(1990),pp.63…70.[16]N.J.HighamComputingthenearestcorrelationmatrix„aproblemfrom“nance,IMAJ.Numer.Anal.,22(2002),pp.329…343.[17]R.E.KalmanIdenti“cationofnoisysystems,RussianMath.Surveys,40(1985),pp.25…42.[18]H.KrimandM.VibergTwodecadesofarraysignalprocessingresearch,IEEESignalProcess.Mag.,13(1996),pp.67…94.[19]M.LaurentandS.PoljakOnapositivesemide“niterelaxationofthecutpolytope,LinearAlgebraAppl.,223(1995),pp.439…461.[20]M.LaurentandS.PoljakOnthefacialstructureofthesetofcorrelationmatrices,SIAMJ.MatrixAnal.Appl.,17(1995),pp.530…547.[21]W.LedermannOnaproblemconcerningmatriceswithvariablediagonalelements,Proc.Roy.Soc.Edinburgh,60(1940),pp.1…17.[22]M.MesbahiandG.P.PapavassilopoulosOntherankminimizationproblemoverapositivesemide“nitelinearmatrixinequality,IEEETrans.Automat.Control,42(1997),pp.239…[23]B.Recht,M.Fazel,andP.A.ParriloGuaranteedminimum-ranksolutionsoflinearmatrixequationsvianuclearnormminimization,SIAMRev.,52(2010),pp.471…501.[24]J.SaundersonSubspaceIdenti“cationviaConvexOptimization,Mastersthesis,Mas-sachusettsInstituteofTechnology,Cambridge,MA,2011.[25]P.ShahandV.ChandrasekaranGroupSymmetryandCovarianceRegularization,preprint,arXiv:1111.7061,2011.[26]A.ShapiroRank-reducibilityofasymmetricmatrixandsamplingtheoryofminimumtracefactoranalysis,Psychometrika,47(1982),pp.187…199.[27]A.ShapiroWeightedminimumtracefactoranalysis,Psychometrika,47(1982),pp.243…264.[28]A.ShapiroIdenti“abilityoffactoranalysis:Someresultsandopenproblems,LinearAlgebraAppl.,70(1985),pp.1…7.[29]C.SpearmanGeneralintelligence,Žobjectivelydeterminedandmeasured,AmericanJ.Psy-chol.,(1904),pp.201…292.[30]L.VandenbergheandS.BoydSemide“niteprogramming,SIAMRev.,38(1996),pp.49…95.[31]J.A.WaltersNonnegativematrixequationshavingpositivesolutions,Math.Comp.,(1969),p.827. SIAMJ.MATRIXNAL.2012SocietyforIndustrialandAppliedMathematicsVol.33,No.4,pp.1395–1416DIAGONALANDLOW-RANKMATRIXDECOMPOSITIONS,CORRELATIONMATRICES,ANDELLIPSOIDFITTINGJ.SAUNDERSON,V.CHANDRASEKARAN,P.A.PARRILOA.S.WILLSKY ReceivedbytheeditorsApril5,2012;acceptedforpublication(inrevisedform)byM.L.OvertonOctober17,2012;publishedelectronicallyDecember19,2012.ThisresearchwasfundedinpartbyShellInternationalExplorationandProduction,Inc.,underP.O.450004440andinpartbytheAirForceOceofScienti“cResearchundergrantFA9550-11-1-0305.http://www.siam.org/journals/simax/33-4/87251.htmlLaboratoryforInformationandDecisionSystems,DepartmentofElectricalEngineeringand