Proceedings of the International Congress of Mathematicians Hyderabad India Nonasymptotic - PDF document

2M.Rudelson,R.VershyninthedimensionsN;ngrowtoinnitywhiletheaspectration=Nconvergestoanon-randomnumbery2(0;1],thespectrumofthenormalizedWishartmatri-ces1 NWN;nisdistributedaccordingtotheMarchenko-Pasturlawwithdensity1 2xyp (b�x)(x�a)supportedon[a;b]wherea=(1�p y)2,b=(1+p y)2.ThemeaningoftheconvergenceissimilartotheoneinWigner'ssemicirclelaw.Itiswidelybelievedthatphenomenatypicallyobservedinasymptoticrandommatrixtheoryareuniversal,thatisindependentoftheparticulardistributionoftheentriesofrandommatrices.Byanalogywithclassicalprobability,whenweworkwithindependentstandardnormalrandomvariablesZi,weknowthattheirnormalizedsumSn=1 p nPni=1Ziisagainastandardnormalrandomvariable.Thissimplebutusefulfactbecomessignicantlymoreusefulwhenwelearnthatitisasymptoticallyuniversal.Indeed,TheCentralLimitTheoremstatesthatifinsteadofnormaldistributionZihavegeneralidenticaldistributionwithzeromeanandunitvariance,thenormalizedsumSnwillstillconverge(indistribution)tothestandardnormalrandomvariableasn!1.Inrandommatrixtheory,universalityhasbeenestablishedformanyresults.Inparticular,Wigner'ssemicirclelawandMarchenko-Pasturlawareknowntobeuniversal{liketheCentralLimitTheorem,theyholdforarbitrarydistributionofentrieswithzeromeanandunitvariance(see[60,6]forsemi-circlelawand[95,5]forMarchenko-Pasturlaw).Asymptoticrandommatrixtheoryoersremarkablyprecisepredictionsasdi-mensiongrowstoinnity.Atthesametime,sharpnessatinnityisoftencoun-terweightedbylackofunderstandingofwhathappensinnitedimensions.Letusbrie yreturntotheanalogywiththeCentralLimitTheorem.OneoftenneedstoestimatethesumofindependentrandomvariablesSnwithxednumberoftermsnratherthaninthelimitn!1.InthissituationonemayturntoBerry-Esseen'stheoremwhichquantiesdeviationsofthedistributionofSnfromthatofthestandardnormalrandomvariableZ.Inparticular,ifEjZ1j3=M1thenjP(Snz)�P(Zz)jC 1+jzj3
M p n;z2R;(1.1)whereCisanabsoluteconstant[11,23].NotwithstandingtheoptimalityofBerry-Esseeninequality(1.1),onecanstillhopeforsomethingbetterthanthepolynomialboundontheprobability,especiallyinviewofthesuper-exponentialtailofthelimitingnormaldistribution:P(jZj�z).exp(�z2=2).Betterestimateswouldindeedemergeintheformofexponentialdeviationinequalities[61,47],butthiswouldonlyhappenwhenwedropexplicitcomparisonstothelimitingdistributionandstudythetailsofSnbythemselves.Inthesimplestcase,whenZiarei.i.d.meanzerorandomvariablesboundedinabsolutevalueby1,onehasP(jSnj�z)2exp(�cz2);z0;(1.2)wherecisapositiveabsoluteconstant.Suchexponentialdeviationinequalities,whichareextremelyusefulinanumberofapplications,arenon-asymptoticresultswhoseasymptoticprototypeistheCentralLimitTheorem.Asimilarnon-asymptoticviewpointcanbeadoptedinrandommatrixtheory.Onewouldthenstudyspectralpropertiesofrandommatricesofxeddimensions. 4M.Rudelson,R.Vershyninthefactorsmax(A).Theextremesingularvaluesareclearlyrelatedtotheoper-atornormsofthelinearoperatorsAandA�1actingbetweenEuclideanspaces:smax(A)=kAkandifAisinvertiblethensmin(A)=1=kA�1k.Understandingthebehaviorofextremesingularvaluesofrandommatricesisneededinmanyapplications.Innumericallinearalgebra,theconditionnumber(A)=smax(A)=smin(A)oftenservesasameasureofstabilityofmatrixalgo-rithms.Geometricfunctionalanalysisemploysprobabilisticconstructionsoflinearoperatorsasrandommatrices,andthesuccessoftheseconstructionsoftendependsongoodboundsonthenormsoftheseoperatorsandtheirinverses.ApplicationsofdierentnatureariseinstatisticsfromtheanalysisofsamplecovariancematricesAA,wheretherowsofAareformedbyNindependentsamplesofsomeunknowndistributioninRn.SomeotherapplicationsarediscussedinSection5.AsymptoticbehaviorofextremesingularvaluesWerstturntotheasymptotictheoryfortheextremesingularvaluesofrandommatriceswithin-dependententries(andwithzeromeanandunitvariancefornormalizationpur-poses).FromMarchenko-PasturlawweknowthatmostsingularvaluesofsuchrandomNnmatrixAlieintheinterval[p N�p n;p N+p n].Undermildadditionalassumptions,itisactuallytruethatallsingularvaluesliethere,sothatasymptoticallywehavesmin(A)p N�p n;smax(A)p N+p n:(2.1)Thisfactisuniversalanditholdsforgeneraldistributions.Thiswasestablishedforsmax(A)byGeman[29]andYin,BaiandKrishnaiah[97].Forsmin(A),Silverstein[71]provedthisforGaussianrandommatrices,andBaiandYin[8]gaveauniedtreatmentofbothextremesingularvaluesforgeneraldistributions:Theorem2.1(Convergenceofextremesingularvalues,see[8]).LetA=AN;nbeanNnrandommatrixwhoseentriesareindependentcopiesofsomerandomvariablewithzeromean,unitvariance,andnitefourthmoment.SupposethatthedimensionsNandngrowtoinnitywhiletheaspectration=Nconvergestosomenumbery2(0;1].Then1 p Nsmin(A)!1�p y;1 p Nsmax(A)!1+p yalmostsurely:Moreover,withoutthefourthmomentassumptionthesequence1 p Nsmax(A)isal-mostsurelyunbounded[7].Thelimitingdistributionoftheextremesingularvaluesisknownanduniversal.ItisgivenbytheTracy-WidomlawwhosecumulativedistributionfunctionisF1(x)=exp�Z1xu(s)+(s�x)u2(s)ds;(2.2)whereu(s)isthesolutiontothePainleveIIequationu00=2u3+suwiththeasymptoticu(s)1 2p s1=4exp(�2 3s3=2)ass!1.TheoccurrenceofTracy-Widomlawinrandommatrixtheoryandseveralotherareaswasthesubjectof 6M.Rudelson,R.VershyninProof(sketch).WewillsketchtheproofforN=n;thegeneralcaseissimilar.Theexpressionsmax(A)=maxx;y2Sn�1hAx;yimotivatesustorstcontroltherandomvariableshAx;yiindividuallyforeachpairofvectorsx;yontheunitEuclideansphereSn�1,andafterwardstaketheunionboundoverallsuchpairs.Forxedx;y2Sn�1theexpressionhAx;yi=Pi;jaijxjyiisasumofindependentrandomvariables,whereaijdenotetheindependententriesofA.Ifaijwerestandardnor-malrandomvariables,therotationinvarianceoftheGaussiandistributionwouldimplythathAx;yiisagainastandardnormalrandomvariable.Thispropertygeneralizestosubgaussianrandomvariables.Indeed,usingmomentgeneratingfunctionsonecanshowthatanormalizedsumofmeanzerosubgaussianrandomvariablesisagainasubgaussianrandomvariable,althoughthesubgaussianmo-mentmayincreasebyanabsoluteconstantfactor.ThusP�hAx;yi�s
2e�cs2;s0:Obviously,wecannotnishtheargumentbytakingtheunionboundoverinnite(evenuncountable)numberofpairsx;yonthesphereSn�1.Inordertoreducethenumberofsuchpairs,wediscretizeSn�1byconsideringits"-netN"intheEuclideannorm,whichisasubsetofthespherethatapproximateseverypointofthesphereuptoerror".Anapproximationargumentyieldssmax(A)=maxx;y2Sn�1hAx;yi(1�")�2maxx;y2N"hAx;yifor"2(0;1):TogainacontroloverthesizeofthenetN",weconstructitasamaximal"-separatedsubsetofSn�1;thentheballswithcentersinN"andradii"=2formapackinginsidethecenteredballofradius1+"=2.Avolumecomparisongivestheusefulboundonthecardinalityofthenet:jN"j(1+2=")n.Choosingforexample"=1=2,wearewellpreparedtotaketheunionbound:P�smax(A)�4s
P�maxx;y2N"hAx;yi�s
jN"jmaxx;y2N"P�hAx;yi�s
5n
2e�cs2:Wecompletetheproofbychoosings=Cp n+twithappropriateconstantC. Byintegration,onecaneasilydeducefromProposition2.4thecorrectexpec-tationboundEsmax(A)C1(p N+p n).Thislatterboundactuallyholdsundermuchweakermomentassumptions.SimilarlytoTheorem2.1,theweakestpossi-blefourthmomentassumptionsuceshere.R.Latala[46]obtainedthefollowinggeneralresultformatriceswithnotidenticallydistributedentries:Theorem2.5(Largestsingularvalue:fourthmoment,non-iidentries[46]).LetAbearandommatrixwhoseentriesaijareindependentmeanzerorandomvariableswithnitefourthmoment.ThenEsmax(A)Chmaxi�XjEa2ij
1=2+maxj�XiEa2ij
1=2+�Xi;jEa4ij
1=4i:ForrandomGaussianmatrices,amuchsharperresultthaninProposition2.4isduetoGordon[31,32,33]: 8M.Rudelson,R.Vershyninforc(n=N)2andlikeexp(�c1N2)forlarger.Forsquarematricesthemeaningofthisphenomenonisespeciallyclear.Largedeviationsofsmax(A)areproducedbyburstsofsingleentries:bothP(smax(A)Esmax(A)+t)andP(ja1;1jEsmax(A)+t)areofthesameorderexp(�ct2)fortEsmax(A).Incontrast,forsmalldeviations(forsmallert)thesituationbecomestrulymultidi-mensional,andTracy-Widomtypeasymptoticsappears.Themethodof[25]alsoaddressesthemoredicultsmallestsingularvalue.ForanNnrandommatrixAwhosedimensionsarenottooclosetoeachotherFeldheimandSodin[25]provedtheTracy{Widomlawforthesmallestsingularvaluetogetherwithanon-asymptoticversionoftheboundsmin(A)p N�p n:Psmin(A)p N�p n�p N
N N�nC 1�p n=Nexp(�c0n3=2):(2.5)3.ThesmallestsingularvalueQualitativeinvertibilityproblemInthissectionwefocusonthebehaviorofthesmallestsingularvalueofrandomNnmatriceswithindependententries.Thesmallestsingularvalue{thehardedgeofthespectrum{isgenerallymoredicultandlessamenabletoanalysisbyclassicalmethodsofrandommatrixtheorythanthelargestsingularvalue,the\softedge".Thedicultyespeciallymanifestsitselfforsquarematrices(N=n)oralmostsquarematrices(N�n=o(n)).Forexample,wewereguidedsofarbytheasymptoticpredictionsmin(A)p N�p n,whichobviouslybecomesuselessforsquarematrices.AremarkableexampleisprovidedbynnrandomBernoullimatricesA,whoseentriesareindependent1valuedsymmetricrandomvariables.Eventhequalitativeinvertibilityproblem,whichaskstoestimatetheprobabilitythatAisinvertible,isnontrivialinthissituation.Komlos[44,45]showedthatAisinvertibleasymptoticallyalmostsurely:pn:=P(smin(A)=0)!0asn!1.LaterKahn,KomlosandSzemeredi[43]provedthatthesingularityprobabilitysatisespncnforsomec2(0;1).Thebasecwasgraduallyimprovedin[78,81],withthelatestrecordofpn=(1=p 2+o(1))nobtainedin[12].ItisconjecturedthatthedominantsourceofsingularityofAisthepresenceoftworowsortwocolumnsthatareequaluptoasign,whichwouldimplythebestpossibleboundpn=(1=2+o(1))n.QuantitativeinvertibilityproblemThepreviousproblemisonlyconcernedwithwhetherthehardedgesmin(A)iszeroornot.Thissaysnothingaboutthequantitativeinvertibilityproblemofthetypicalsizeofsmin(A).Thelatterquestionhasalonghistory.VonNeumannandhisassociatesusedrandommatricesastestinputsinalgorithmsfornumericalsolutionofsystemsoflinearequations.Theaccuracyofthematrixalgorithms,andsometimestheirrunningtimeaswell,dependsontheconditionnumber(A)=smax(A)=smin(A).Basedonheuristicandexperimentalevidence,vonNeumannandGoldstinepredictedthatsmin(A)n�1=2;smax(A)n1=2withhighprobability(3.1) 10M.Rudelson,R.VershyninThisresultaddressesbothqualitativeandquantitativeaspectsoftheinvert-ibilityproblem.Setting"=0weseethatAisinvertiblewithprobabilityatleast1�cn.ThisgeneraizestheresultofKahn,KomlosandSzemeredi[43]fromBernoullitoallsubgaussianmatrices.Ontheotherhand,quantitatively,The-orem3.2statesthatsmin(A)&n�1=2withhighprobabilityforgeneralrandommatrices.Acorrespondingnon-asymptoticupperboundsmin(A).n�1=2alsoholds[66],sowehavesmin(A)n�1=2asinvonNeumann-Goldstine'sprediction.Boththesebounds,upperandlower,holdwithhighprobabilityundertheweakerfourthmomentassumptionontheentries[65,66].Thistheorywasextendedtorectangularrandommatricesofarbitrarydimen-sionsNnin[67].AsweknowfromSection2,oneexpectsthatsmin(A)p N�p n.Butthiswouldbeincorrectforsquarematrices.Toreconcilerectan-gularandsquarematriceswemakethefollowingcorrectionofourprediction:smin(A)p N�p n�1withhighprobability:(3.3)Forsquarematricesonewouldhavethecorrectestimatesmin(A)p n�p n�1n�1=2.ThefollowingresultextendsTheorem3.2torectangularmatrices:Theorem3.3(Smallestsingularvalueofrectangularrandommatrices[65]).LetAbeannnrandommatrixwhoseentriesareindependentandidenticallydistributedsubgaussianrandomvariableswithzeromeanandunitvariance.ThenP�smin(A)"(p N�p n�1)
(C")N�n+1+cN;"0whereC�0andc2(0;1)dependonlyonthesubgaussianmomentoftheentries.Thisresulthasbeenknownforalongtimefortallmatrices,whosetheaspectratio=n=Nisboundedbyasucientlysmallconstant,see[10].Theoptimalboundsmin(A)cp Ncanbeprovedinthiscaseusingan"-netargumentsimilartoProposition2.4.Thiswasextendedin[53]tosmin(A)cp Nforallaspectratios1�c=logn.Thedependenceofcontheaspectratiowasimprovedin[2]forBernoullimatricesandin[62]forgeneralsubgaussianmatrices.Feldheim-Sodin'sTheorem2.3givespreciseTracy-Widom uctuationsofsmin(A)fortallmatrices,butbecomesuselessforalmostsquarematrices(sayforNn+n1=3).Theorem3.3isananoptimalresult(uptoabsoluteconstants)whichcoversmatri-ceswithallaspectratiosfromtalltosquare.Non-asymptoticestimate(3.3)wasextendedtomatriceswhoseentrieshavenite(4+")-thmomentin[93].UniversalityofthesmallestsingularvaluesThelimitingdistributionofsmin(A)turnsouttobeuniversalasdimensionn!1.WealreadysawasimilaruniversalityphenomenoninTheorem2.3forgenuinelyrectangularmatrices.Forsquarematrices,thecorrespondingresultwasprovedbyTaoandVu[87]:Theorem3.4(Smallestsingularvalueofsquarematrices:universality[87]).LetAbeannnrandommatrixwhoseentriesareindependentandidenticallydistributedrandomvariableswithzeromean,unitvariance,andniteK-thmomentwhereK 12M.Rudelson,R.VershyninAnobviousadvantageofspreadvectorsisthatweknowthemagnitudeofalltheircoecients.Thismotivatesthefollowinggeometricinvertibilityargument.IfAperformsextremelypoorsothatsmin(A)=0,thenoneofthecolumnsXkofAliesinthespanHk=span(Xi)i6=koftheothers.Thissimpleobservationcanbetransformedintoaquantitativeargument.Supposex=(x1;:::;xn)2Rnisaspreadvector.Then,foreveryk=1;:::;n,wehavekAxk2dist(Ax;Hk)=distnXi=1xiXi;Hk=dist(xkXk;Hk)=jxkj
dist(Xk;Hk)c1n�1=2dist(Xk;Hk):(3.5)Sincetherighthandsidedoesnotdependonx,wehaveprovedthatminx2SpreadkAxk2c1n�1=2dist(Xn;Hn):(3.6)Thisreducesourtasktothegeometricproblemofindependentinterest{es-timatethedistancebetweenarandomvectorandanindependentrandomhyper-plane.Theexpectationestimate1Edist(Xn;Hn)2=O(1)followseasilybyindependenceandmomentassumptions.Butweneedalowerboundwithhighprobability,whichisfarfromtrivial.ThiswillmakeaseparatestoryconnectedtotheLittlewood-Oordtheoryofsmallballprobabilities,whichwediscussinSection4.InparticularwewillproveinCorollary4.4theoptimalestimateP(dist(Xn;Hn)")C"+cn;"0;(3.7)whichissimplefortheGaussiandistribution(byrotationinvariance)anddiculttoprovee.g.fortheBernoullidistribution.Togetherwith(3.6)thismeansthatweprovedinvertibilityonallspreadvectors:P�minx2SpreadkAxk2"n�1=2
C"+cn;"0:ThisisexactlythetypeofprobabilityboundclaimedinTheorem3.2.Aswesaid,wecannishtheproofbycombiningwiththe(muchbetter)invertibilityonsparsevectorsin(3.4),andbyanapproximationargument.4.Littlewood-OordtheorySmallballprobabilitiesandadditivestructureWeencounteredthefol-lowinggeometricproblemintheprevioussection:estimatethedistancebetweenarandomvectorXwithindependentcoordinatesandanindependentrandomhyper-planeHinRn.Weneedalowerboundonthisdistancewithhighprobability.LetusconditiononthehyperplaneHandleta2Rndenoteitsunitnormalvector.Writingincoordinatesa=(a1;:::;an)andX=(1;:::;n),weseethatdist(X;H)=ha;Xi=nXi=1aii:(4.1) 14M.Rudelson,R.VershyninAdditivestructureofthecoecientvectoraisrelatedtotheshortestarithmeticprogressionintowhichitembeds.Thislengthisconvenientlyexpressedastheleastcommondenominatorlcd(a)denedasthesmallest�0suchthata2Znn0.ExamplessuggestthatLevyconcentrationfunctionshouldbeinverselyproportionaltotheleastcommondenominator:lcd(a0)=n1=21=L(S;0)in(4.2)andlcd(a00)=n3=21=L(S;0)in(4.3).Thisisnotacoincidence.Buttostateageneralresult,wewillneedtoconsideramorestableversionoftheleastcommondenominator.Givenanaccuracylevel�0,wedenetheessentialleastcommondenominatorlcd(a):=inf�0:dist(a;Zn)min(1 10kak2;) :Therequirementdist(a;Zn)1 10kak2ensuresapproximationofabynon-trivialintegerpoints,thoseinanon-trivialconeinthedirectionofa.Theconstant1 10isarbitraryanditcanbereplacedbyanyotherconstantin(0;1).Onetypicallyusesthisconceptforaccuracylevels=cp nwithasmallconstantcsuchasc=1 10.Theinequalitydist(a;Zn)yieldsthatmostofthecoordinatesofaarewithinasmallconstantdistancefromintegers.Forsuch,inexamples(4.2)and(4.3)onehasasbeforelcd(a0)n1=2andlcd(a00)n3=2.HerewestateandsketchaproofofageneralLittlewood-Oordtyperesultfrom[67].Theorem4.2(Levyconcentrationfunctionviaadditivestructure).Let1;:::;nbeindependentidenticallydistributedmeanzerorandomvariables,whicharewellspread:p:=L(k;1)1.Then,foreverycoecientvectora=(a1;:::;an)2Sn�1andeveryaccuracylevel&#x]TJ/;ø 9;&#x.962; Tf;&#x 11.;ڙ ;� Td;&#x [00;0,thesumS=Pni=1aiisatisesL(S;")C"+C=lcd(a)+Ce�c2;"0;(4.4)whereC;c&#x]TJ/;ø 9;&#x.962; Tf;&#x 11.;ڙ ;� Td;&#x [00;0dependonlyonthespreadp.Proof.AclassicalEsseen'sconcentrationinequality[24]boundstheLevyconcen-trationfunctionofanarbitraryrandomvariableZbytheL1normofitscharac-teristicfunctionZ()=Eexp(iZ)asfollows:L(Z;1)CZ1�1jZ()jd:(4.5)OnecanprovethisinequalityusingFourierinversionformula,see[80,Section7.3].WewillshowhowtoproveTheorem4.2forBernoullirandomvariablesi;thegeneralcaserequiresanadditionalargument.Withoutlossofgeneralitywecanassumethatlcd(a)1 ".Applying(4.5)forZ=S=",weobtainbyindependencethatL(S;")CZ1�1jS(=")jd=CZ1�1nYj=1jj(=")jd;wherej(t)=Eexp(iajjt)=cos(ajt).Theinequalityjxjexp(�1 2(1�x2))whichisvalidforallx2Rimpliesthatjj(t)jexp�1 2sin2(ajt)exp�1 2dist(ajt ;Z)2: 16M.Rudelson,R.VershyninProof.Aswasnoticedin(4.1),wecanwritedist(Xn;Hn)asasumofindependentrandomvariables,andthenbounditusing(4.7). Corollary4.4oersusexactlythemissingpiece(3.7)inourproofoftheinvert-ibilityTheorem3.2.Thiscompletesouranalysisofinvertibilityofsquarematrices.Remark.Thesemethodsgeneralizetorectangularmatrices[67,93].Forexample,Corollary4.4canbeextendedtocomputethedistancebetweenrandomvectorsandsubspacesofarbitrarydimension[67]:forHn=span(X1;:::;Xn�d)wehave(Edist(Xn;Hn)2)1=2=p dandP�dist(Xn;Hn)"p d
(C")d+cn;"0:5.ApplicationsTheapplicationsofnon-asymptotictheoryofrandommatricesarenumerous,andwecannotcoveralloftheminthisnote.Insteadweconcentrateonthreedierentresultspertainingtotheclassicalrandommatrixtheory(CircularLaw),signalprocessing(compressedsensing),andgeometricfunctionalanalysisandtheoreticalcomputerscience(shortKhinchin'sinequalityandKashin'ssubspaces).CircularlawAsymptotictheoryofrandommatricesprovidesanimportantsourceofheuristicsfornon-asymptoticresults.Wehaveseenanillustrationofthisintheanalysisoftheextremesingularvalues.Thisinteractionbetweentheasymptoticandnon-asymptotictheoriesgoestheotherwayaswell,asgoodnon-asymptoticboundsaresometimescrucialinprovingthelimitlaws.Oneremarkableexampleofthisisthecircularlawwhichwewilldiscussnow.ConsiderafamilyofnnmatricesAwhoseentriesareindependentcopiesofarandomvariableXwithmeanzeroandunitvariance.LetnbetheempiricalmeasureoftheeigenvaluesofthematrixBn=1 p nAn,i.e.theBorelprobabilitymeasureonCsuchthatn(E)isthefractionoftheeigenvaluesofBncontainedinE.Along-standingconjectureinrandommatrixtheory,whichiscalledthecircularlaw,suggestedthatthemeasuresnconvergetothenormalizedLebesguemeasureontheunitdisc.TheconvergenceherecanbeunderstoodinthesamesenseasintheWigner'ssemicirclelaw.ThecircularlawwasoriginallyprovedbyMehta[56]forrandommatriceswithstandardnormalentries.Theargumentusedtheexplicitformulaforjointdensityoftheeigenvalues,soitcouldnotbeextendedtootherclassesofrandommatrices.WhiletheformulationofWigner'ssemicirclelawandthecircularlawlooksimilar,themethodsusedtoprovetheformerarenotapplicabletothelatter.Thereasonisthatthespectrumofageneralmatrix,unlikethatofaHermitianmatrix,isunstable:asmallchangeoftheentriesmaycauseasignicantchangeofthespectrum(see[6]).Girko[30]introducedanewapproachtothecircularlawbasedonconsideringtherealpartoftheStieltjestransformofmeasuresn.Forz=x+iytherealStieltjestransformisdenedby 18M.Rudelson,R.Vershyninwherekxk0=jsupp(x)j.Thisoptimizationproblemishighlynon-convexandcomputationallyintractable.Sooneconsidersthefollowingconvexrelaxationof(5.1),whichcanbeecientlysolvedbyconvexprogrammingmethods:minimizekxk1subjecttoAx=Ax;(5.2)wherekxk1=Pni=1jxijdenotesthe`1norm.Onewouldthenneedtondconditionswhenproblems(5.1)and(5.2)areequivalent.CandesandTao[16]showedthatthisoccurswhenthemeasurementmatrixAisarestrictedisometry.Foranintegersn,therestrictedisometryconstants(A)isthesmallestnumber0whichsatises(1�)kxk22kAxk22(1+)kxk22forallx2Rn;jsupp(x)js:(5.3)Geometrically,therestrictedisometrypropertyguaranteesthatthegeometryofs-sparsevectorsxiswellpreservedbythemeasurementmatrixA.InturnsoutthatinthissituationonecanreconstructxfromAxbytheconvexprogram(5.2):Theorem5.1(Sparsereconstructionusingconvexprogramming[16]).Assume2sc.Thenthesolutionof(5.2)equalsxwheneverjsupp(x)js.Aproofwithc=p 2�1isgivenin[15];thecurrentrecordisc=0:472[13].RestrictedisometrypropertycanbeinterpretedintermsoftheextremesingularvaluesofsubmatricesofA.Indeed,(5.3)equivalentlystatesthattheinequalityp 1�smin(AI)smax(AI)p 1+holdsforallmssubmatricesAI,thoseformedbythecolumnsofAindexedbysetsIofsizes.InlightofSections2and3,itisnotsurprisingthatthebestknownrestrictedisometrymatricesarerandommatrices.ItisactuallyanopenproblemtoconstructdeterministicrestrictedisometrymatricesasinTheorem5.2below.Thefollowingthreetypesofrandommatricesareextensivelyusedasmeasure-mentmatricesincompressedsensing:Gaussian,Bernoulli,andFourier.Herewesummarizetheirrestrictedisometryproperties,whichhavethecommonremark-ablefeature:therequirednumberofmeasurementsmisroughlyproportionaltothesparsitylevelsratherthanthe(possiblymuchlarger)dimensionn.Theorem5.2(Randommatricesarerestrictedisometries).Letm;n;sbepositiveintegers,";2(0;1),andletAbeanmnmeasurementmatrix.1.SupposetheentriesofAareindependentandidenticallydistributedsub-gaussianrandomvariableswithzeromeanandunitvariance.AssumethatmCslog(2n=s)whereCdependsonlyon",,andthesubgaussianmoment.Thenwithprobabilityatleast1�",thematrixA=1 p mAisarestrictedisometrywiths(A).2.LetAbearandomFouriermatrixobtainedfromthenndiscreteFouriertransformmatrixbychoosingmrowsindependentlyanduniformly.AssumethatmCslog4(2n):(5.4) 20M.Rudelson,R.Vershyninthisargumentfor()runsintotothesameproblemsasforthesmallestsingularvalue.Foranyxed�0thesolutionwasrstobtainedrstbyJohnsonandSchechtman[38]whoshowedthatthereexistsVsatisfying(5.5)with()=c1=.In[54]thiswasestablishedforarandomsetV(orarandommatrixA)withthesameboundon().Furthermore,theresultremainsvalidevenwhendependsonn,aslongasc=logn.Theproofusesthesmallestsingularvalueboundfrom[53]inacrucialway.Theboundon()hasbeenfurtherimprovedin[2],alsousingthesingularvalueapproach.Finally,atheoremin[62]assertsthatforarandomsetVtheinequalities(5.5)holdwithhighprobabilityfor()=c2;()=C:Moreover,theresultholdsforall�0andn,withoutanyrestrictions.Theproofcombinesthemethodsof[63]andageometricargumentbasedonthestructureofasectionofthe`n1ball.Theprobabilityestimateof[62]canbefurtherimprovedifonereplacesthesmallballprobabilityboundof[63]withthatof[65].TheshortKhinchininequalityshowsalsothatthe`1and`2normsareequiv-alentonarandomsubspaceE:=ARnRN.Indeed,ifAisanNnran-dommatrix,thenwithhighprobabilityeveryvectorx2Rnsatises()kxk2N�1kAxk1N�1=2kAxk2Ckxk2.ThesecondinequalityhereisCauchy-Schwartz,andthethirdoneisthelargestsingularvaluebound.ThiereforeC�1()kyk2N�1=2kyk1kyk2forally2E:(5.6)SubspacesEpossessingproperty(5.6)arecalledKashin'ssubspaces.TheclassicalDvoretzkytheoremstatesthatahigh-dimensionalBanachspacehasasubspacewhichisclosetoEuclidean[59].Thedimensionofsuchsubspacedependsonthegeometryoftheambientspace.Milmanprovedthatsuchsubspacesalwaysexistindimensionclogn,wherenisthedimensionoftheambientspace[58](seealso[59]).Forthespace`n1thesituationismuchbetter,andsuchsubspacesexistindimension(1�)nforanyconstant�0.ThiswasrstprovedbyKashin[41]alsousingarandommatrixargument.Obviously,as!0,thedistancebetweenthe`1and`2normsonsuchsubspacegrowsto1.TheoptimalboundforthisdistancehasbeenfoundbyGarnaevandGluskin[28]whousedsubspacesgeneratedbyGaussianrandommatrices.Kashin'ssubspacesturnedouttobeusefulintheoreticalcomputerscience,inparticularinthenearestneighborsearch[36]andincompressedsensing.AtpresentnodeterministicconstructionisknownofsuchsubspacesofdimensionnproportionaltoN.Theresultof[62]showsthatab(1+)ncnrandomBernoullimatrixdenesaKashin'ssubspacewith()=c2.ArandomBernoullimatrixiscomputationallyeasiertoimplementthanarandomGaussianmatrix,whilethedistancebetweenthenormsisnotmuchworsethanintheoptimalcase.Atthesametime,sincethesubspacesgeneratedbyaBernoullimatrixarespannedbyrandomverticesofthediscretecube,theyhaverelativelysimplestructure,whichispossibletoanalyze. 22M.Rudelson,R.Vershynin[21]P.Erdos,OnalemmaofLittlewoodandOord,Bull.Amer.Math.Soc.51(1945),898{902[22]P.Erdos,Extremalproblemsinnumbertheory,1965Proc.Sympos.PureMath.,Vol.VIII,pp.181{189AMS,Providence,R.I.[23]C.G.Esseen,Fourieranalysisofdistributionfunctions.AmathematicalstudyofthelaplaceGaussianlaw,ActaMath.77(1945),1{125.[24]C.G.Esseen,OntheKolmogorov-Rogozininequalityfortheconcentrationfunction,Z.WahrscheinlichkeitstheorieundVerw.Gebiete5(1966),210{216.[25]O.Feldheim,S.Sodin,Auniversalityresultforthesmallesteigenvaluesofcertainsamplecovariancematrices,GeometricandFunctionalAnalysis,toappear[26]P.Frankl,Z.Furedi,SolutionoftheLittlewood-Oordprobleminhighdimensions,Ann.ofMath.(2)128(1988),259{270.[27]O.Friedland,S.Sodin,BoundsontheconcentrationfunctionintermsoftheDio-phantineapproximation,C.R.Math.Acad.Sci.Paris345(2007),513{518.[28]A.Garnaev,E.Gluskin,ThewidthsofaEuclideanball,SovietMath.Dokl.30(1984),200{204.[29]S.Geman,Alimittheoremforthenormofrandommatrices,Ann.Probab.8(1980),252{261.[30]V.L.Girko,Thecircularlaw,TheoryProbab.Appl.29(1984),no.4,694{706.[31]Y.Gordon,OnDvoretzky'stheoremandextensionsofSlepian'slemma,Israelsem-inarongeometricalaspectsoffunctionalanalysis(1983/84),II,TelAvivUniv.,TelAviv,1984.[32]Y.Gordon,SomeinequalitiesforGaussianprocessesandapplications,IsraelJ.Math.50(1985),265{289.[33]Y.Gordon,MajorizationofGaussianprocessesandgeometricapplications,Probab.TheoryRelatedFields91(1992),251{267.[34]F.Gotze,A.Tikhomirov,OntheCircularLaw,arXiv:math/0702386.[35]F.Gotze,A.Tikhomirov,TheCircularLawforRandomMatrices,arXiv:0709.3995,toappearinAnn.Prob.[36]P.Indyk,Dimensionalityreductiontechniquesforproximityproblems,ProceedingsoftheNinthACM-SIAMSymposiumonDiscreteAlgorithms,2000.[37]K.Johansson,Shape uctuationsandrandommatrices,Comm.Math.Phys.209(2000),437{476.[38]W.B.Johnson,G.Schechtman,VerytightembeddingsofsubspacesofLp,1p2,intolnp,Geom.Funct.Anal.13(2003),no.4,845{851.[39]I.M.Johnstone,Onthedistributionofthelargesteigenvalueinprincipalcomponentsanalysis,Ann.Statist.29(2001),295{327.[40]G.Halasz,Estimatesfortheconcentrationfunctionofcombinatorialnumbertheoryandprobability,PeriodicaMathematicaHungarica8(1977),197{211[41]B.Kashin,Sectionofsomenite-dimensionalsetsandclassesofsmoothfunctions(inRussian)Izv.Acad.Nauk.SSSR41(1977)334{351.[42]A.Kolmogorov,SurlesproprietesdesfonctionsdeconcentrationsdeM.P.Levy,Ann.Inst.H.Poincare16(1958),27{34. 24M.Rudelson,R.Vershynin[63]M.Rudelson,Invertibilityofrandommatrices:normoftheinverse,AnnalsofMath-ematics168(2008),575{600.[64]M.Rudelson,R.Vershynin,OnsparsereconstructionfromFourierandGaussianmeasurements,CommunicationsonPureandAppliedMathematics61(2008),1025{1045.[65]M.Rudelson,R.Vershynin,TheLittlewood-Oordproblemandinvertibilityofran-dommatrices,Adv.Math.218(2008),600{633.[66]M.Rudelson,R.Vershynin,TheleastsingularvalueofarandomsquarematrixisO(n�1=2),Comptesrendusdel'Academiedessciences-Mathematique346(2008),893{896.[67]M.Rudelson,R.Vershynin,Smallestsingularvalueofarandomrectangularmatrix,Comm.PureAppl.Math.62(2009),1707{1739.[68]A.Sankar,D.A.Spielman,S.-H.Teng,Smoothedanalysisoftheconditionnumbersandgrowthfactorsofmatrices,SIAMJ.MatrixAnal.Appl.28(2006),446{476.[69]A.Sarkozy,E.Szemeredi,UbereinProblemvonErdosundMoser,ActaArithmetica11(1965),205{208[70]G.Schechtman,RandomembeddingsofEuclideanspacesinsequencespaces,IsraelJournalofMathematics40,No.2,1981,187?192.[71]J.Silverstein,Thesmallesteigenvalueofalarge-dimensionalWishartmatrix,Ann.Probab.13(1985),1364{1368.[72]D.Slepian,Theone-sidedbarrierproblemforGaussiannoise,BellSystemTech.J.41(1962),463{501.[73]S.Smale,Ontheeciencyofalgorithmsofanalysis,Bull.Amer.Math.Soc.(N.S.)13(1985),87{121[74]A.Soshnikov,Anoteonuniversalityofthedistributionofthelargesteigenvaluesincertainsamplecovariancematrices,J.Statist.Phys.108(2002),1033{1056.[75]D.Spielman,S.-H.Teng,Smoothedanalysisofalgorithms.ProceedingsoftheIn-ternationalCongressofMathematicians,Vol.I(Beijing,2002),597{606,HigherEd.Press,Beijing,2002[76]M.Talagrand,Concentrationofmeasureandisoperimetricinequalitiesinproductspaces,Inst.HautesEtudesSci.Publ.Math.No.81(1995),73{205.[77]M.Talagrand,Anewlookatindependence,Ann.Probab.24(1996),1{34.[78]T.Tao,V.Vu,Onrandom1matrices:singularityanddeterminant,RandomStructuresandAlgorithms28(2006),1{23.[79]S.Szarek,Conditionnumbersofrandommatrices,J.Complexity7(1991),131{149.[80]T.Tao,V.Vu,Additivecombinatorics.CambridgeStudiesinAdvancedMathemat-ics,105.CambridgeUniversityPress,Cambridge,2006.[81]T.Tao,V.Vu,OnthesingularityprobabilityofrandomBernoullimatrices,J.Amer.Math.Soc.20(2007),603?628.[82]T.Tao,V.Vu,Randommatrices:thecircularlaw,Commun.Contemp.Math.10(2008),no.2,261{307.[83]T.Tao,V.Vu,InverseLittlewood-Oordtheoremsandtheconditionnumberofrandomdiscretematrices,AnnalsofMath.169(2009),595-632.