TheAnnalsofStatistics2004,Vol.32,No.5,1781 - PDF document

TheAnnalsofStatistics2004,Vol.32,No.5,1781–1804DOI10.1214/009053604000000391©InstituteofMathematicalStatistics,2004PERIODICBOXCARDECONVOLUTIONANDDIOPHANTINEAPPROXIMATIONAINM.JOHNSTONEAIMONDOStanfordUniversityandUniversityofSydney f(tu)du,a�(2)W(t),t
[Šisastandardtwo-sidedWienerprocessandissmalland ReceivedMay2002;revisedJuly2003.SupportedinpartbyNSFGrantsDMS-95-05151,DMS-00-72661andNIHGrantRO1CASupportedinpartbytheAustralianAcademyofScienceandbyNSFGrantDMS96-31278. I.M.JOHNSTONEANDM.RAIMONDObecauseoftheperiodicandconvolutionstructure,theproblemisdiagonalizedintheFourierbasis.Thus,let(t)ikt,forinteger.Thenwheretheeigenvalues1andsinka ka(3)Furthermore,setting(t)dY(t)f,e:=f(t)e(t)dt,and(t)dW(t),wendthatmodel(1)isequivalentto(4)Forrationalp/q,theeigenvaluesvanishforallintegermultiples.IntheFourierexpansionf,e,allinformationaboutthecoefcientsf,eislostafterconvolution.Forirrational,however,theinversionformulaf,e f,e(5)isatleastwelldened,since0forany.TheobjectofthispaperistostudythequalityofestimationofattainableforirrationalinthesmallnoiseMotivationforstudyingthisspecialproblemarisesfromseveralsources:(i)Itmaybeviewedasanidealizationoftheproblemofrecoveryfromlinearmotionblurplusnoiseinaxedeldofview.Ifacameraispassingoverascenef(x,y)alongadirection,r)atunitspeed,theninexposuretime2theimageacquiredatpoint(x,y)maybemodeledasKf(x,y) f(xu,yru)du.(6)Ourmodelisaone-dimensionalversionofhorizontalmotion,0.Whiletheperiodicityassumptiononmayseemarticial,itdoescapturethepropertythatislocallyperiodicwithperiod2(x,y)(asincertaintextures),thenislocallyconstantnear(x,y).ComparethediscussioninSection5.1.Amoredetaileddiscussionoflinearmotionblur,withphotographicexamples,maybefoundinBerteroandBoccacci[(1998),pages54–58].(ii)Itisrelatedtotheproblemofperiodicdensityestimationwithuniformerrors.Supposearei.i.d.randomvariableswithunknownperiodicdensityonthecircle.However,thearenotobserved;insteadweseejitteredversionswherearei.i.d.uniformlydistributedonna,aandcircularadditionisused.(iii)Asaninverseproblem,(5)isnonstandard:theeigenvaluesoscillateinsideanenvelopedecayinglike1frequency,for(a) PERIODICBOXCARDECONVOLUTIONWemayaskthefollowing:isthequalityofestimation—measuredbyminimaxrateofconvergenceas0—determinedbythe1decay,orisitaffectedbytheoscillatorybehavior?(iv)Letdenotethedistancefromtothenearestinteger.For  |rk (7)andsotheoscillationsin(3)aredrivenby:=inf(8)Thestudyofsuch“Diophantineapproximations”usestheclassicaltheoryofcontinuedfractions,forexample,Lang(1966)andKhinchin(1992),andplaysabasicroleinthispaper.Thereisalargeliteratureonstatisticalinverseproblems—forsomerecentreviewsseeTenorio(2001)andEvansandStark(2002).Inparticular,thesequencespaceformulationstudiedherehasreceivedsubstantialattention:asampleofrecentworks,inadditiontothosecitedbelow,includeWahba(1990),JohnstoneandSilverman(1990),Koo(1993),BelitserandLevit(1995),Donoho(1995),MairandRuymgaart(1996),GolubevandKhas’minski(1999,2001)andCavalier,Golubev,PicardandTsybakov(2002).However,muchofthisliteratureisconcernedwitheigenvaluesequenceshaving(uptoconstants)monotonicbehaviorincreases.PapersthatdospecicallyaddresstheboxcardeconvolutionproblemincludeHall,Ruymgaart,vanGaansandvanRooij(2001),GroeneboomandJongbloed(2003)andO’SullivanandRoyChoudhury(2001);seeSection5.1forsomefurtherdiscussion.Effectivedegreeofill-posedness.Problem(1)isanexampleofalinearstatisticalinverseprobleminwhichoneobservesanoisyversionofforsomelinearoperator,andwishestoreconstruct.Suchlinearinverseproblemsaretypicallyill-posedinthesenseofHadamard:theinversiondoesnotdependcontinuouslyontheobserveddata.Onemanifestationofthisisthatratesofconvergenceofestimatorsas0areslowerthaninthedirectcaseinwhichitselfisobservedwithnoise.Weshallformulatesomewell-knownexistingresultsintermsofanotionof“degreeofill-posedness”(DIP)inordermoreeasilytostatetheresultsofthepresentpaper.Underappropriateconditions,willhaveasingularvaluedecomposition,andintermsofcoefcientsinthesingularsystemexpansions,theobservationsmaybewritteninasequenceform(9)or,equivalently,afterdividingthroughby,as(10) I.M.JOHNSTONEANDM.RAIMONDOwhere
/r.Let.Denethe(nonlinear)minimaxriskofestimationwithrespecttoaparameterspacevia(,
)inf(11)wheretheinmumistakenoverall(measurable)functionsofthedata.Wedenethelinearminimaxriskby(,
)infwhereattentionisrestrictedtothesubclassoflinearestimators,forsomesequenceParameterspacesofprimaryinterestinthispaperinclude,for,C�hyperrectangles(C)(12)ellipsoids(C)(13)1.Withinthesescalesofspaces,theparametermeasuressmoothness:largercorrespondstofasterdecayofcoefcients.WhentheareFouriercoefcients,theellipsoidscorrespondexactlytomean-squaresmoothnessofthederivativesof.[See,e.g.,Kress(1999),Chapter8.1.]Thereisnosuchsimplecharacterizationforhyperrectangles—thedenition(12)ischosentoyieldthesameratesofconvergenceas(13)inthehomogeneouscasesdescribednext.Theparametermeasuressize:itcorrespondstotheradiusofballswithinthesespaces.2.In(5)weusedthecomplexexponentialsikt.Themodelhasthesameformifinsteadoneusestherealtrigonometricbasis(t)ktsinktor1 2accordingas,k0or0.Model(9)–(10)appliestoindices.Forconvenienceintherestofthepaper,werestricttheindex.Indeed,sincespacessuchas(12)and(13)aresymmetricwithrespectto,wehave(,
(,
withtheanalogousstatementvalidalsoforthelinearminimaxrisks.Consequently,ratesofconvergencearecertainlyunaffectedbyworkingon3.Thenotationa(
)b(
)meansthatthereexistconstantssuchthatforsufcientlysmallb(
)a(
)b(
).Theconstantsandothergenericconstants(denotedbyandnotnecessarilythesameateachappearance)maydependonparametersofthesmoothnessclasssuchas,buttheydonot PERIODICBOXCARDECONVOLUTIONdependon
,orthesizeparameter.Whilethesizeconstantclearlydoesnotaffecttherateofconvergenceas0,weconsideritusefultoshowtheorderofdependenceofminimaxriskson.Thenotationmeansthat1.Thenotationmeansthat,forallSupposethattheeigenvaluessatisfyahomogeneousdecayconditionandthat(C)(C).Thenitiswellknown[e.g.,KorostelevandTsybakov(1993),Chapter9]that (14)Fordirectdatawehave1in(9)anditisknownthat ThismotivatesthefollowingdenitionofeffectiveDIP:(K,) sŠ1 (15)Forindirectproblems(K,)givesameasureoftheeffect(ontheconvergencerate)duetotheinversionprocess.Forexample,ifisan-fractionalintegrationoperatorand(C),thenandso,inthiscase,(K,)getslargeritbecomesmoreandmoredifculttorecoverReturningtoboxcardeconvolution,wenotethatcorrespondstoaneffectiveDIPof1.Thequestionstudiedinthispaperiswhethertheoscillationsinof(3)increasetheDIP.CompareFigure1.Theanswerturnsouttodependonthefunctionclass.Themainresults,Theorems1and2,canbeexpressedassaying,solongaslogarithmictermsareignored,thatforellipsoidsandalmostallirrational(K forallwhileforhyperrectangles,(Kif0 2,1+Š3/2 2+1, (16)Thus,theDIPofboxcardeconvolutionliesbetween1and ,andisbetter(i.e.,smaller)formoreuniformsmoothness(hyperrectangles)andforsmaller4.WecautionthattheliteraturecontainsotherdenitionsofDIPofaninverseproblem:forexample,inMathéandPereverzev(2001),itrefersto I.M.JOHNSTONEANDM.RAIMONDO .1.AnillustrationofthedegreeoftheDIPfortheboxcardeconvolutionoperatorwith Usingalog-scalealongtheverticalaxisthefunctionisdepictedfor,...,500(oscillatingsolidlineForcomparisonpurposewealsodepictforahomogeneousoperatorwithDIPtakingeigenvalueswhere58(smoothdashedcurvesanumericalindexofdistancefrominvertibility.Whilethesenotionsarecertainlyrelated,thedenitionusedhereissimplyaconvenienceforinterpretingresultsstatedformallyinSections3and4:itreferstothedropinrateofconvergenceduetopresenceofthedecayingeigenvalues5.Thereisanelbowinratesat forhyperrectanglesbutnotellipsoids.Thiscontrastswithresultsobtainedforhomogeneousopera-tors(14).Observethattheratesofconvergenceareworseforellipsoidsthanforcorrespondinghyperrectangles:thisoccursbecausetheuniformhyperrectan-gleconstraint(12)operatesoneachcoordinateandsoprovideslessscopeformaximizingriskbyconcentratingsignalenergyincoordinateswheresmallthandoestheellipsoidcasewhereonlyatotalenergyconstraint(13)ap-plies. PERIODICBOXCARDECONVOLUTION2.Preliminaries.Diophantineapproximations.Werecallsomepertinentpartsoftheclas-sicaltheory,referringtoLang(1966)andKhinchin(1992)forfurtherdetails.Thestudyofapproximationssuchas(8)isconnectedtotheapproximationofirra-tionalsbyrationalsknownasDiophantineapproximations.Foragivenirrational,wedistinguishthesystematicapproximationsof(8)fromtherationalapproximationsp/q:by-approximationwemeanthatkq(17)Giventhesequenceofsolutionsto(17),therateofapproximationisdenedintermsofthedecayofD(a,q (18)Apartfromthetwobasicgroupsofrealnumbers,rationalsandirrationals,thereexistsamuchnerdivisionofirrationalnumbersbaseduponthedegreetowhichtheycanbeapproximatedbyrationalfractions.Thismayrangefromtoarbitrarilymuchfaster,asexplainedbelow.Theseratesdependcruciallyonthebest-possiblerationalapproximation(17).Thesolutionof(17)isgivenbythecontinuedfractionsalgorithmwhich,unlikesystematicfractions(),capturesthearithmeticpropertiesofthenumbertobeapproximated.Continuedfractionsandconvergents.Anyrealnumberthatisnotanintegermaybeuniquelydeterminedbyitscontinuedfractionexpansion a1+1 a2+1 +······a0;a1,a2,...],(19)whereisanintegerandisaninnitesequenceofstrictlypositiveintegers.Inthealgorithm(19)thenumbersarecalledtheelementspartialdenominators.Toeachinnitesequencecorrespondsauniqueirrationalandviceversa.Atstagethealgorithmusesonlytherstelementssa0;a1,a2,...,a.Forsuchaterminatingcontinuedfractiononlyanitenumberofoperationsareinvolvedandtheresultisclearlyarationalnumber: a1+1 ...+1 an0;a1n (20)Therationalnumbers),narecalledtheconvergentsReturningtotheproblemofapproximatinganirrationalnumberbyrationals,wehavethat,forinf=||=(21) I.M.JOHNSTONEANDM.RAIMONDOInwords,theconvergentssatisfythebest-approximationproperty(17).Indeed,anybest-approximationisaconvergentsince,foristhesmallestinteger�qqsuchthat[see,e.g.,Lang(1966),page9].Thequalityofbest-approximationisgivenby 2qn+1qna1 (22)[Lang(1966),page8].Whileforsystematicapproximation,with1kq,Lang[(1966),page10]showsthat (23)Itisinformativetonotethat,for2,thealgorithm(20)canbewrittenas(24)fromwhichfollowsomebasicpropertiesoftheconvergentsofallirrationalnumbers(i)Thedenominatorsgrowatleastgeometrically:,i�(25)(ii)Forall Thequalitativenatureofrationalapproximationscan,therefore,bemeasuredbythesizeoftheelementsinthecontinuedfractionalgorithm,from(22), (a,q (26)Fasterapproximationwilloccurforthoseirrationalswithlargerelementsviceversa.Familiesofirrationalnumberscanbedenedaccordingtothesizeoftheirelements.EFINITION1.Wesaythatanirrationalnumberisbadlyapproximable(BA)if(a)From(26),weseethatarbitrarilyfastratesofapproximationarepossible.Anaturalquestionarises—aretheregenerallawswhichgoverntheapproxi-mationsofclassicalirrationalnumbers?—Again,someanswersfollowfromthecontinuedfractionalgorithm[Khinchin(1992),ChapterII].Oneclassofresultsconcernsalgebraicnumbers—rootsofpolynomialswithintegercoefcients.For PERIODICBOXCARDECONVOLUTIONexample,itcanbeshownthatquadraticirrationals(suchas 5)haveperiodicel-ementsandsoareBA.Andcubicirrationals(e.g.,5)cannotbeapproximatedwitharatefasterthan1Anotherclassofresultsconstitutesthe“measuretheory”ofcontinuedfractions.Forexample,almostallnumbers(i.e.,exceptasetofLebesguemeasurezero)haveunbounded[Khinchin(1992),Theorem30].Ontheotherhand,foralmostallnumbers,itisalsotruethattherateofapproximationcanbenofasterthanlog0.Forus,animportantconsequence(seetheAppendix)isthefollowing.Foreach0,thereisasetoffullmeasuresuchthatloginnitelyoften,(27)andyetlogforalllarge�nn(a).(28)Henceforth,theusage“almostall”means“forallMinimaxrisk.Werecallsomebasicresults,establishedforthedirectdatasetting1(or)inDonoho,LiuandMacGibbon(1990),andeasilyextendedtotheindirectsetting(10)(seetheAppendix).Ifiscompact,orthosymmetricandquadraticallyconvex,then(,
)(,
)(,
),(29)where25istheIbragimov–Khasminskiiconstant;seeDonoho,LiuandMacGibbon(1990).Forsuchsets,wealsohave (,
)(,
)(,
),wherewedene(,
)(30)Inthelightofbounds(7)andRemark2,ourtaskis,then,toevaluate(,
)forselected,small,fortheboxcaroperator,whichhas 
k 2 forall(31)Anequidistributionlemma.Whileprecisebounds(22)areavailableforbest-possiblerationalapproximationstoanirrationalnumber,thequalityofsystematicrationalapproximations,...,changesconsiderablyasvaries.Asaresult,oscillatewidelyaschanges;seeFigure1.However,theaveragebehaviorismuchlesssusceptibletouctuations.Indeed,runsoverablockoflength,thevaluesofhaveadistributionthatisincertainrespectsclosetodiscreteuniformon I.M.JOHNSTONEANDM.RAIMONDOp/qbesuccessiveprincipalconvergentsinthecontinuedfractionexpansionofarealnumberbeapositiveintegerwithqqbeanonincreasingfunctionThenwehaveupperandlowerh(µ/q)h(µ/q)(32)ROOF.Theargumentisamodicationofthatusedby[Lang(1966),page37].Sincep/qisaprincipalconvergent,wemaywriteintheformp/q /q1.Writing,...,q,onegets p/q/q.Sincearerelativelyprime,thesets p/q, ,...,qµ/q,µ,...,qareequalmodulo.Toeachthereisassociatedaunique,andsetting,wehaveµ(k)µ(k)Thepoints,...,qformanequispacedsetwithexactlyonepointineachintervalal(µŠ1)/q,µ/q).R())]denotetheremainderofarealnumber.Considerrsttheofindicesforwhichthecorrespondingpointslieinclearly,3.Sincekq,wehavefromtheremarkfollowing(22)thatR(ka)Hence,thesumofh(R(ka)),for,isboundedby3bethesetofremainingindices,...,N,sothatthecorrespondingpointsliein···.Sinceall,eachoftheleftendpointsof,...,IisalowerboundforexactlyoneR(ka)andtherightendointsofeachareupperboundsforexactlyoneR(ka)Combiningthiswiththeupperboundfor,weobtainh(µ/q)h(R(ka))h(µ/q)(33)Thisinequalityremainsvalidifwereplaceh(R(ka))R(ka))—indeed,theproofissimply“reectedabout ,”andwenotethatforinthe(reected)wehave1R(ka)SinceR(x),R(x),wehavemaxh(R(x)),hR(x)andusingb)/maxa,b,thelemmafollowsfrom(33)appliedtoR(ka)and1R(ka) PERIODICBOXCARDECONVOLUTION6.Theproofshowsthattheupperboundcontinuestoholdifthemiddlesumistakenover,whereandweassumeonly7.Theboundsprovidedbythislemmaareoftensharpuptoconstants.Forexample,ifisBAandh(x)log3.Hyperrectangles.Statementandoutline.Tostatethemainresults,introducetworateconstants 2+5 2,¯r=+3 andnotethatifandonlyifMorepreciseresultsarepossibleintheBAcase,whileforgenericirrationals,theconsequences(27)and(28)ofKhinchin’stheoremleadtoonlyslightlyweakerstatements.ForBAwehave(C),
log(C/
), (34)Foralmostallthepreviousboundsremainvalidfor whilefor foreach(C),
logC/
)forallsmalllogC/
)forinÞnitelymany(35)Thereisthusan“elbow”intheratesofconvergenceat .Comparisonwith(14)showsthatfor ,theDIPis1(asifthesinusoidaltermwerenotpresentin).However,for ,theDIPgivenby(16)increasesgraduallyfrom1toalimitingvalueof forlargeThisresultdoesnotcoverirrationalswithfastratesofapproximation(e.g.,1orhigher,asdiscussedinSection2.2),but,ofcourse,suchnumbersformasetofLebesguemeasurezero.Weoutlinethemainstepsoftheproof,withdetailstofollowinSection3.3.First,asnotationalconvention,weintroduceaparameter ,sothat(C).Withtheseconventions,(30)becomes(,
)(
).(36) I.M.JOHNSTONEANDM.RAIMONDOFirst,weusethecontinuedfractionapproximationto,...,andforfrequenciesnear,splitthesumintoblocksoflength.Thus,(
)blocksblock(
),(37)whereblocksisthesumoverallblocksasvaries,theblocksbeingoflengthbetween.Wethenapplytheequidistributionlemmatothesumwithinblocks.Theblocksumsarethencollectedintooneofthreezones:(,
)(
)V(
)M(
)B(
).(38)Thesezones(variance,mixedandbias)areillustratedinFigure2,anddenedformallyat(45). .2.Anillustrationofthevariance-mixed-biaszonesUsingalog-scalealongtheverticalaxistheplotshowsbothfunctionsoscillatingdottedcurvesmoothdashedcurve whichcorrespondstoSolidverticallinesindicatethebordersofthekeyzonesThethicksolidlineplots(
) PERIODICBOXCARDECONVOLUTIONFrequencypartitionsdeterminedbyanirrational.Anyirrationalnum-denesauniquesequenceofconvergents:1asthelargestintegerstrictlylessthan,thus,Consideranonuniformgrid,....Introduceindices(n,l),l,...,l,....Thebivariateindices(n,l)aretotallyorderedbylexicographicorderingandwerefertotheircomponentsbythefunctionsn( )l( ).Furthermore,eachindexhasanimmediatesuccessor,whichinslightabuseofnotationwedenoteby1.Soourgridisl( )qn( )(39)thisgriddenesapartitionofbyblockswhichbetweenhavelengthhN ,N +1).(40)Clearly,n( )unlessl( )n( ))1,qn( )l( )n( )Tosimplifycertaincalculationsweuseblocksoflengthn( )only,introducinggN ,N +qn( )(41)Byconstruction,foragiveninteger,thereareatmosttwosuchthat.Hence,summingoverallinplaceofwillonlyaffecttheratebyamultiplicativeconstantofatmost2.ProofofTheoremKeyzonesandbounds.First,recallthat(
)isdenedat(36)andusebounds(31);byconstructionn( )sothatforinablockkN ,N +qn( ),hence,(
) 2 C2NŠ2
2N2 (42)Wesuppresstheindexwhennotnecessary.Fromtheequidistributionlemma, qk
C hN()cqµ=1hNµ q+1 (43)Toestimatethesesums,weuseaneasilyveriedbound. I.M.JOHNSTONEANDM.RAIMONDOthen , ,qwheretheconstantsneededfordependonlyonNowapplythisto(x).Writingalso
/C,weobtain(µ/q),Nq,q(44)Wecannowformallydenethezonetowhichablock(or)belongsintermsofthevalueofn( ).Againsuppressingthesubscript,wesayVariancezoneMixedzoneNBiaszoneƆ&#x.600;qq.(45)Thus,thezonedescribeswhichtermappearsintheminimizerin(44).Letbethelastindicesforwhichn( )1and1,respectively,andset()()(46)Frequencieskklieinthevariancezone,thosewithkkinthemixedzone,andthosewithinthebiaszone.Considernowthesecondtermintheupperboundof(43):1,then,ofcourse,soisandsocanbeignoredincomparisonwith(44).Ontheotherhand,if1,thenandthisbounddominates.Insummary,wehavederivedthefollowingkeybounds:(
)(variancezone),q, (mixedzone),q, (biaszone).(47)Thevariancezone.Considerrstvalueskk(
)suchthatthecontributiontotheminimaxriskisduetooscillationsoccasionedbyDiophantineapproximationonly.Heretherstboundof(47)appliesandthehyperrectangleconstrainthasnotyetanysmoothingeffect.Werstderiveanexpressionforintermsof.If1,wehavebydenition,n( )andso PERIODICBOXCARDECONVOLUTIONOntheotherhand,againbydenition,.Writingfor,weobtainForBA,whileforalmostallandalllarge,(28)showsthatlog.Tosummarize,(C/
)forBAc(C/
)logC/
)foralmostallFirst,sumoverblocksusingpartition(40)andapplybound(47)inthevarianceV(
)(
)(
)n( )(48)Usinggrid(39),andsetting,l),l,weobtainV(
)(49)wherewehavesetmaxThedenominatorsgrowatleastexponentially[cf.(25)]andsousing,wend(C/
)IntheBAcase,,whileforalmostallwehaveloglog.Insummary,V(
)forBAlog(C/
)foralmostall(50)Themixedzone.Wearenowinterestedinindicescesk0,k1)wherebothoscillationsandthehyperrectangleconstraintcontributetotheminimaxrisk;itendswheretheoscillationsstop.Bydenition,satises.Sincealways,itfollowsthat(C/
)Usingbound(47)inthemixedzone,togetherwithn( ),andyields(
)C
Nn( ) I.M.JOHNSTONEANDM.RAIMONDOwhichshowsthatforsumsoverblocksoflengthn( )inthemixedzone,wemayreplace(
).Sincetheblocksformacoveroftheintegers1ofredundancyatmosttwo,M(
)(
)Thus,inthemixedzone,M(
)loglog(C/
), (51)Thebiaszone.Notethatfor,sincealways1,wehaveandsothereisnolongeranyeffectofoscillation,(
)in(36).Hence,B(
)(
)(52)Weemphasizethatbounds(51)and(52)applytoallirrationalsSummary.Wereturnto(38).IntheBAcase(andalsothea.a.casewhen 2),itisapparentfrom(50),(51)and(52)that,whichestablishes(34).Itremainstoconsiderthea.a.casewithTheupperboundin(35)isapparentfrom(50).Forthelowerbound,letbeanarbitraryirrationalwithconvergents,....Simplybychoosingtobezeroexceptinthethcoordinate—inwhich—weobtaintheelementarylowerbound(,
)(53)Since
k/,wendusing(22)thatforUsing(27)in(53),wededucethatforalmostallthereexistsasequencesuchthat(,
)log(54)Constructasequencesequencel]),l,...,,l]2q4nllogwhichgivesesl]loggl]Š1)Š1/(2+),(55)andusingsuchananl]-sequencein(54),togetherwith(55),yieldstherequired(,

l])C2qŠ2nl log(C/
PERIODICBOXCARDECONVOLUTION4.Ellipsoids.Foranellipsoid(C)denedasin(13),let/(Thegoalofthissectionistoestablishthefollowing:ForandBAwehave(C),
Foralmostall(35)holdfor(C),
)withreplacedbyforallSince/(,theDIP(K(C)) forallellipsoids,regardlessofthevalueofthesmoothnessindexUpperbound.Aswithhyperrectangles,theaimistousesumsoverblocksoflength.Todoso,wedeneslightlylargerellipsoidsbasedonthepartitionof(40):(C)(56)wheretheindexindicatesthatthegriddependsonnumbertheoreticalproperties.Bydenition(40)ofthepartition,impliesthatsothatand,hence,R(,
)R(,
)Wemaynowsplittheoptimizationacrossandwithinblocks:,
)(57),
)wheretheoptimizationwithinblockissubjecttothequota,
)(58)Theequidistributionlemmacanbeappliedtothislastsum:.Ondroppingthesubscript,weobtain 24N2k
B 1 28N2qµ=1q2 Hence,from(57)and(58),,
)n( )(59) I.M.JOHNSTONEANDM.RAIMONDOObservethatforanypositivesequences),(cnondecreas-ing,(60)foranyvalueforwhich(61)Applyingthisto(59)withn( ),weobtain,
)n( )(62)Here(lq(n,l).Letandnote,sincethatbetherstindexforwhich1:since1,wehaveandso(63)Since(62),togetherwith(63),isexactlythesituationreachedat(48)inthehyperrrectanglecase(withreplacedby)weconcludethatthebounds(50)apply(withreplacedbyLowerbound.Arguingexactlyasat(53),butwithreplacedby(,
)(64)IntheBAcase,letbethelastindexforwhich,sothat/(.From(64)at1,wend(,
)Forthealmostallcase,theargumentisthesameasbeforeat(54)andbelow. PERIODICBOXCARDECONVOLUTION5.Discussion.Periodicvs.nonperiodic.RecentpapersbyHall,Ruymgaart,vanGaansandvanRooij(2001)andGroeneboomandJongbloed(2003)considerinpartadensityestimationversionofthedeconvolutionprobleminwhichthedataconsistofani.i.d.sampleinwhicharei.i.d.withunknowndensityarei.i.d.uniformonona,aandindependentofthe.GroeneboomandJongbloed(2003)derivepointwiselimitingdistributionsofestimatorsofbasedonkernelsmoothsofnonparametricMLEsofthedistributionfunction.TheworkofHall,Ruymgaart,vanGaansandvanRooij(2001)looksatmaximumglobalestimationerrors,andsoisperhapscloserinspirittothepresentinvestigation.Insteadofanyperiodicityassumptions,itisassumedtherethatthedensityhascompactsupporton.Thecompactsupportpermitsanexplicitinversionformula:ifischosenlargeenoughthatIainfsupp,thenf(x)ia).InthiscaseHall,Ruymgaart,vanGaansandvanRooij(2001)showthattheDIP(K1forofbothhyperrectangleandellipsoidtype,incontrasttotheresultsfoundfortheperiodicmodelconsideredhere.Thedifferenceinresultsmayperhapsbeunderstoodbyobservingthatsinusoids,whicharebasictotheperiodicmodel,donothavecompactsupport.Thus,themodelscapturegenuinelydifferentphenomena.EffectofrationalapproximationstoInpractice,computercodeworkswithrationalnumbers—whateffectwillthishaveonourconclusions?Afewremarkscanbemadeevenwithoutgettingintospecicsofparticularmodelsofcomputationorattemptingafullanalysis.Abasicissueiswhethertheboxcarwidthisundertheinvestigator’scontrol.Ifitis—ourrstscenario—thenwemightimaginereplacingsay,sothatmodel(4)becomessink k(65)HeremightbeoneofthesequenceofbestrationalapproximationstoTheapproximationresultsofSection2.2showthatouranalysisofestimationinmodel(65)isunchangedfromthatofirrational,atleastforfrequenciessincewillhavethesameconvergentsfor.Thus,onecouldsimplychooselargeenoughthatthetailbiasaccruingtofrequenciesaboveisnegligible.Tobemorespecic,assumethatisahyperrectangle(C),and I.M.JOHNSTONEANDM.RAIMONDOthatisknown.Let0besmall[wecouldlet(
)0withtopreserveratesofconvergence].Wecanchoose(
)[denedat(46)]sothatthetailbias�kk(C),
andthenchooselargeenoughthat.Aminimaxestimatorfor(C)undermodel(65)willbeessentiallyidenticalinstructurewithonefortheoriginalirrational,sinceineithercase,thezeroestimatorisusedatallfrequencies�kqInthesecondscenario,theboxcarwidthisdeterminedbynatureandtheinvestigatormustworkwiththedatafrommodel(4).Westillassumethatthevalueofisknown,butmustuserationalapproximationstoinourestimatorsbasedon.Fordeniteness,consideragainthecase(C)andset.Considertheriskoflinearrules(y)(y)otherwise.If,thentheriskofsucharuleisr(c,))c2k
2+(1Šckrk)22k]+k/
S2k.Supposethatisirrational:withinniteprecision,wecoulduseanestimatorthatmakesr(c,).Nowconsiderthedifferenceinriskthatresultsfromanapproximation,wheresina)/(kforsomerationalapproximationc,)r(c,) rk2Š1
2k+1Šrk ifwewrite,andassumethatc,)r(c,)(,
)(66)Usingaderivativeboundonsinkaandthen(7), sinka sin aŠ1 Ša| a sin ak kq,thenfrom(26),(23)and(25), aqr qm28 Consequently,theriskdifferenceduetousingarationalapproximationcanbemadeassmallasdesiredbyrstselectingsothatsup(
)andthensothattheboundonand,hence,isassmallasneeded. PERIODICBOXCARDECONVOLUTIONGeneralizations.1.Itseemslikelythatestimatorswhichareadaptivewithrespecttocouldbeconstructed(foraxedirrational)bygroupingfrequencieswithinagivenblockockqn,qn+1)intoanumberofsubblocksaccordingtothevalueofandthenusingsomeformofJames–Steinshrinkagewithineachsubblock.Thismethodologyisnowquitewellestablishedonotherinverseproblemswithmonotoneeigenvalues;see,forexample,CavalierandTsybakov(2002).Alternatively,adaptivity(uptologarithmicterms)isestablishedviaawaveletdeconvolutionapproachinJohnstone,Kerkyacharian,PicardandRaimondo(2004)foraclassofBesovspacesincludingellipsoids(13).2.TheellipsoidresultsmightalsohavebeenderivedusingtheexplicitevaluationofminimaxriskgivenbyPinsker(1980).However,themethodusedhereallowsextensionoftherateresultstoweightedbodiesoftheformfor1usingessentiallythesameargumentasforellipsoids.Forexample,theanalogof(58)statesthatiftheorderedincreasing(k)correspondingtoindiceswithinablocksatisfysomebound(k)(ashappensfortheboxcar),then,
)(j)where(j),andsuchsumscanbeestimatedbythemethodsofthispaper.3.Itisstraightforwardtoextendtheresultsofthispapertoiteratedkernelssa,awitheigenvaluessinka)/(ka)However,kernelsoftheforma,bba,aab,bhaveeigenvaluessinka kasinkb kb whilethelinearmotionkernel(6)hassin(kra) (kra)ConsiderableworkexistsonsimultaneousDiophantineapproximationproblems[Schmidt(1980),Chapter2],butwhetherthisenablesrateofconvergencecalculationsisanopenquestion.APPENDIXROOFOF(27)(28).Werecalltheconvergence/divergencetheoremofKhinchin[(1992),Theorem32].Let(x)beapositivecontinuousfunctionof0,suchthatx(x)isnonincreasing.Thentheinequality(q)has,foralmostall,aniteorinnitenumberofsolutionsinpositiveintegersaccording(x)dxconvergesordiverges. I.M.JOHNSTONEANDM.RAIMONDOFor(27),consider(x)log.Sincetheintegraldiverges,letoneoftheinnitelymanysolutionsto(q)andchoosesothatqq.Itthenfollowsfrom(22)andthepropertystatedafter(21)that  log logfromwhich(27)isimmediate.For(28),consider(x)log.Sincetheintegralconverges,forall�qq(a, ),wehave(q).Inparticular,from(22),forlarge qn+1na logfromwhichweobtain(28).ROOFOF(29).Themethodusedtoestablish(29)fordirectdatamaybeextendedinastraightforwardmannertomodel(9),forexample,bysteppingthroughtheargumentsinJohnstone[(2003),Hyperrectangleschapter].Thekeystepinthisapproach,asinDonoho,LiuandMacGibbon(1990),istoestablishthat(,
)(),
(67)where()isthehyperrectangleei,i].ThiscanbereducedtotheKneser–Kuhnminimaxtheorem[Johnstone(2003),CorollaryA.4]appliedtopayofffunctionf(c,s))
2c2k+(1Šck)2sk],(68)denedfor(c,s)Butresult(67)extendsimmediatelytomodel(9)byreplacingin(68)andchangingthedomainoftotheweightedHilbertspace,(
,andapplyingtheminimaxtheoreminthesameway.Acknowledgments.IainM.JohnstoneisgratefulforhospitalityoftheAustralianNationalUniversity,wherethisprojectbegan.MarcRaimondoisgratefultoPeterHallforhelpfulreferencesonDiophantineapproximations.PartofthispaperwaswrittenwhileMarcRaimondovisitedStanfordUniversity.Bothauthorsaregratefulforthecommentsofareferee,particularlyforraisingtheissuediscussedinSection5.2.ELITSER,E.andLEVIT,B.(1995).Onminimaxlteringoverellipsoids.Math.MethodsStatist. PERIODICBOXCARDECONVOLUTIONERTERO,M.andBOCCACCI,P.(1998).IntroductiontoInverseProblemsinImaging.InstituteofPhysics,Bristol.AVALIER,L.,GOLUBEV,G.K.,P,D.andTSYBAKOV,A.B.(2002).Oracleinequalitiesforinverseproblems.Ann.Statist.AVALIER,L.andTSYBAKOV,A.B.(2002).Sharpadaptationforinverseproblemswithrandomnoise.Probab.TheoryRelatedFields,D.L.(1995).Nonlinearsolutionoflinearinverseproblemsbywavelet-vaguelettedecomposition.Appl.Comput.Harmon.Anal.,D.L.,L,R.C.andM,K.B.(1990).Minimaxriskoverhyperrectangles,andimplications.Ann.Statist.VANS,S.N.andSTARK,P.B.(2002).Inverseproblemsasstatistics.InverseProblemsR55–R97.OLUBEV,G.K.andK,R.Z.(1999).Astatisticalapproachtosomeinverseproblemsforpartialdifferentialequations.ProblemyPeredachiInformatsiiOLUBEV,G.andK,R.(2001).AstatisticalapproachtotheCauchyproblemfortheLaplaceequation.InStateoftheArtinProbabilityandStatisticsFestschriftforWillemR.vanZwet419–433.IMS,Beachwood,OH.ROENEBOOM,P.andJONGBLOED,G.(2003).Densityestimationintheuniformdeconvolutionmodel.Statist.NeerlandicaALL,P.,RUYMGAART,F.,VAN,O.andVAN,A.(2001).Invertingnoisyintegralequationsusingwaveletexpansions:Aclassofirregularconvolutions.InStateoftheArtinProbabilityandStatisticsFestschriftforWillemR.vanZwet533–546.IMS,Beachwood,OH.OHNSTONE,I.M.(2003).FunctionestimationandGaussiansequencemodels.Draftofamonograph.Availableatwww-stat.stanford.edu/˜imj.OHNSTONE,I.M.,KERKYACHARIAN,G.,P,D.andR,M.(2004).Waveletdeconvolutioninaperiodicsetting(withdiscussion).J.R.Stat.Soc.Ser.BStat.Methodol.547–573,627–652.OHNSTONE,I.M.andSILVERMAN,B.(1990).Speedofestimationinpositronemissiontomographyandrelatedinverseproblems.Ann.Statist.,A.Y.(1992).ContinuedFractions.Dover,NewYork.,J.-Y.(1993).Optimalratesofconvergencefornonparametricstatisticalinverseproblems.Ann.Statist.OROSTELEV,A.andTSYBAKOV,A.(1993).MinimaxTheoryofImageReconstruction.LectureNotesinStatist..Springer,NewYork.RESS,R.(1999).LinearIntegralEquations,2nded.Springer,NewYork.,S.(1966).IntroductiontoDiophantineApproximations.Addison–Wesley,Reading,MA.,B.andRUYMGAART,F.H.(1996).StatisticalinverseestimationinHilbertscales.SIAMJ.Appl.Math.ATHÉ,P.andPEREVERZEV,S.V.(2001).OptimaldiscretizationofinverseproblemsinHilbertscales.Regularizationandself-regularizationofprojectionmethods.SIAMJ.Numer.Anal.1999–2021(electronic).ULLIVAN,F.andRHOUDHURY,K.(2001).AnanalysisoftheroleofpositivityandmixturemodelconstraintsinPoissondeconvolutionproblems.J.Comput.Graph.Statist.673–696.INSKER,M.(1980).OptimallteringofsquareintegrablesignalsinGaussianwhitenoise.ProblemsInform.Transmission,W.(1980).DiophantineApproximationLectureNotesinMath..Springer,Berlin.ENORIO,L.(2001).Statisticalregularizationofinverseproblems.SIAMRev.(electronic). I.M.JOHNSTONEANDM.RAIMONDOAHBA,G.(1990).SplineModelsforObservationalData.SIAM,Philadelphia.EPARTMENTOFTATISTICSTANFORDNIVERSITYTANFORDALIFORNIAMAIL:imj@stanford.eduCHOOLOFATHEMATICSTATISTICSARSLAWUILDINGNIVERSITYOFNSW2006USTRALIAMAIL:marcr@maths.usyd.edu.au