IEEETRANSACTIONSONIMAGEPROCESSING,VOL.23,NO.1,JANUARY2014[10]M.Chang,A - PDF document

IEEETRANSACTIONSONIMAGEPROCESSING,VOL.23,NO.1,JANUARY2014[10]M.Chang,A.Tekalp,andA.Erdem,“Bluridenticationusingthebispectrum,”IEEETrans.SignalProcess.,vol.39,no.10,pp.2323–2325,Oct.1991.[11]R.Fergus,B.Singh,A.Hertzmann,S.T.Roweis,andW.Freeman,“Removingcamerashakefromasinglephotograph,”ACMTrans.Graph.SI,vol.25,pp.787–794,Aug.2006.[12]K.Gao,X.-X.Li,Y.Zhang,andY.-H.Liu,“Motion-blurparameterestimationofremotesensingimagebasedonquantumneuralnetwork,”Proc.Int.Conf.Opt.Instrum.Tecnol.,Optoelectron.Imag.Process.Technol.,2011,pp.82001L-1–82001L-11.[13]A.GoldsteinandR.Fattal,“Blur-kernelestimationfromspectralirreg-ularities,”inProc.ECCV.2012,pp.622–635.[14]A.Hyvärinen,J.Hurri,andP.O.Hoyer,NaturalImageStatistics:AProbabilisticApproachtoEarlyComputationalVision.,2nded.NewYork,NY,USA:Springer-Verlag,2009.[15]A.Jain,FundamentalsofDigitalImageProcessing.UpperSaddleRiver,NJ,USA:Prentice-Hall,1989.[16]H.JiandC.Liu,“Motionbluridenticationfromimagegradients,”inProc.IEEEConf.CVPR,Jun.2008,pp.1–8.[17]J.Jia,“Singleimagemotiondeblurringusingtransparency,”inProc.IEEEConf.CVPR,Jun.2007,pp.1–8.[18]N.Joshi,R.Szeliski,andD.J.Kriegman,“PSFestimationusingsharpedgeprediction,”inProc.IEEEConf.CVPR,Jun.2008,pp.1–8.[19]F.Krahmer,Y.Lin,B.McAdoo,K.Ott,J.Wang,D.Widemannk,,“Blindimagedeconvolution:Motionblurestimation,”Inst.Math.Appl.,Univ.Minnesota,Minneapolis,Minnesota,Tech.Rep.2133-5,[20]D.KundurandD.Hatzinakos,“Blindimagedeconvolution,”IEEESignalProcess.Mag.,vol.13,no.3,pp.43–64,May1996.[21]D.KundurandD.Hatzinakos,“Blindimagedeconvolutionrevisited,”IEEESignalProcess.Mag.,vol.13,no.6,pp.61–63,Nov.1996.[22]L.Lelégard,E.Delaygue,M.Brédif,andB.Vallet,“Detectingandcor-rectingmotionblurfromimagesshotwithchannel-dependentexposuretime,”inProc.Annal.ISPRS,vols.1–3.2012,pp.341–346.[23]A.Levin,“Blindmotiondeblurringusingimagestatistics,”inProc.Adv.NIPS,2006,pp.841–848.[24]A.Levin,Y.Weiss,F.Durand,andW.Freeman,“Efcientmarginallikelihoodoptimizationinblinddeconvolution,”inProc.IEEEConf.CVPR,Jun.2011,pp.2657–2664.[25]M.Liu,G.Liu,J.Xiu,H.Kuang,andL.Zhai,“Aerialimageblurringcausedbyimagemotionanditsrestorationusingwavelettransform,”Proc.SPIE,vol.5637,pp.425–433,Feb.2005.[26]X.LiuandA.Gamal,“Simultaneousimageformationandmotionblurrestorationviamultiplecapture,”inProc.IEEEICASSP,vol.3.May2001,pp.1841–1844.[27]J.MiskinandD.Mackay,“Ensemblrelearningforblindimagesepara-tionanddeconvolution,”inProc.Adv.Independ.Compon.Anal.,2000,pp.123–141.[28]M.MoghaddamandM.Jamzad,“Motionbluridenticationinnoisymotionbluridenticationinnoisyimagesusingfuzzysets,”inProc.5thIEEEInt.Symp.SignalProcess.Inf.Technol.,Dec.2005,pp.862–866.[29]M.Moghaddam,“Amathematicalmodeltoestimateoutoffocusblur,”Proc.ISPA5thInt.Symp.,Sep.2007,pp.278–281.[30]F.Natterer,TheMathematicsofComputerizedTomography.NewYork,NY,USA:Wiley,1986.[31]S.K.NayarandM.B.Ezra,“Motion-basedmotiondeblurring,”IEEETrans.PatternAnal.Mach.Intell.,vol.26,no.6,pp.689–698,Jun.2004.[32]J.P.Oliveira,“Advancesintotalvariationimagerestoration:Blurestimation,parameterestimationandefcientoptimization,”Ph.D.dis-sertation,Inst.SuperiorTécnico,Univ.Montana,Missoula,MT,USA,Jul.2010.[33]J.P.Oliveira,M.A.T.Figueiredo,andJ.M.Bioucas-Dias,“Blindestimationofmotionblurparametersforimagedeconvolution,”inProc.3rdIberianConf.,IbPRIA,2007,pp.604–611.[34]A.OppenheimandR.Schafer,Discrete-TimeSignalProcessing,2nded.UpperSaddleRiver,NJ,USA:Prentice-Hall,1999.[35]J.G.ProakisandD.G.Manolakis,DigitalSignalProcessing.UpperSaddleRiver,NJ,USA:Prentice-Hall,2007.[36]R.Raskar,A.Agrawal,andJ.Tumblin,“Codedexposurephotography,”ACMTrans.Graph.SI,vol.3,no.25,pp.95–804,2006.[37]A.Rav-AchaandS.Peleg,“Twomotionblurredimagesarebetterthanone,”PatterRecognit.Lett.,vol.25,pp.311–317,Feb.2005.[38]W.H.Richardson,“Bayesian-basediterativemethodofimagerestora-tion,”J.Opt.Soc.Amer.,vol.62,no.1,pp.55–59,1972.[39]M.Sakano,N.Suetake,andE.Uchino,“Robustidenticationofmotionblurparametersbyusinganglesofgradientvectors,”inProc.ISPACS2006,pp.522–525.[40]A.SavakisandH.Trussell,“OntheaccuracyofPSFrepresentationinimagerestoration,”IEEETrans.ImageProcess.,vol.2,no.2,pp.252–259,Apr.1993.[41]Q.Shan,J.Jia,andA.Agarwala,“High-qualitymotiondeblurringfromasingleimage,”ACMTrans.Graph.SI,vol.27,no.3,pp.1–5,[42]E.SteinandG.Weiss,IntroductiontoFourierAnalysisonEuclideanp.Princeton,NJ,USA:PrincetonUniversityPress,1971.[43]T.-Y.Sun,S.-J.Ciou,C.-C.Liu,andC.-L.Huo,“Out-of-focusblurestimationforblindimagedeconvolution:Usingparticleswarmopti-mization,”inProc.IEEEInt.Conf.SMC,Oct.2009,pp.1627–1632.[44]M.Tanaka,K.Yoneji,andM.Okutomi,“Motionblurparameteridenticationfromalinearlyblurredimage,”inProc.Int.Conf.ICCE2007,pp.1–2.[45]A.TorralbaandA.Oliva,“Statisticsofnaturalimagecategories,”Netw.,Comput.NeuralSyst.,vol.14,no.3,pp.391–412,2003.[46]Y.WangandW.Yin,“Compressedsensingviaiterativesupportdetec-tion,”Dept.Comput.Appl.Math.,RiceUniv.,Houston,TX,USA,Tech.Rep.TR09–30,2009.[47]L.XuandJ.Jia,“Two-phasekernelestimationforrobustmotiondeblurring,”inProc.11thECCV,2010,pp.157–170.[48]L.Xu,S.Zheng,andJ.Jia,“Unnaturalsparserepresentationsfornaturalimagedeblurring,”inProc.IEEEConf.CVPR,Jan.2013,pp.2–4.[49]L.Yuan,J.Sun,L.Quan,andH.Y.Shum,“Imagedebluringwithblurred/noisyimagepairs,”ACMTrans.Graph.SI,vol.26,no.3,pp.1–5,2007. JoãoP.OliveirareceivedtheE.E.andPh.D.degreesinelectricalandcomputerengineeringfromInstitutoSuperiorTecnico,EngineeringSchool,UniversityofLisbon,Portugal,in2002and2010,respectively.HeiscurrentlyaResearcherwiththePatternandImageAnalysisGroup,InstitutodeTelecomunicações,Lis-bon,Portugal.HeisanAssistantProfessorwiththeDepartmentofInformationScienceandTechnol-ogy,InstitutoUniversitáriodeLisboa(ISCTE-IUL).Hispresentresearchinterestsincludesignalandimageprocessing,patternrecognition,andmachine MárioA.T.FigueiredoreceivedtheE.E.,M.Sc.,Ph.D.,andAgre-gadodegreesinelectricalandcomputerelectri-calfromInstitutoSuperiorTécnico(IST),Engi-neeringSchool,UniversityofLisbon(ULisbon),Portugal,in1985,1990,1994,and2004,respec-tively.HehasbeenwiththefacultyoftheDepartmentofElectricalandComputerEngineer-ing,IST,whereheiscurrentlyaProfessor.HeisagroupandareacoordinatoratInstitutodeTelecomunicações,aprivatenon-protresearchinstitution.Hisresearchinterestsincludesignalprocessingandanalysis,machinelearning,andoptimization.HeisafellowoftheInternationalAssoci-ationforPatternRecognition.From2005to2010,hewasamemberoftheImage,Video,andMultidimensionalSignalProcessingTechnicalCommitteeoftheIEEESignalProcessingSociety(SPS).Hereceivedthe2011IEEESPSBestPaperAward,the1995PortugueseIBMSci-enticPrize,the2008UTL/Santander-TottaScienticPrize.HehasbeenanAssociateEditorofseveraljournals,namelytheIEEETRANSACTIONSONMAGEROCESSING,theIEEETRANSACTIONSONNALYSISANDACHINENTELLIGENCE,theIEEETRANSACTIONSONOBILEOMPUTING,theSIAMJournalonImagingSci-ence,PatternRecognitionLetters,SignalProcessing,andStatisticsandComputing.HewasaCo-Chairofthe2001and2003WorkshopsonEnergyMinimizationMethodsinComputerVisionandPatternRecog-nition,aguestco-editorofspecialissuesofseveraljournals,andprogram/technical/organizingcommitteememberofmanyinternationalconferences. IEEETRANSACTIONSONIMAGEPROCESSING,VOL.23,NO.1,JANUARY2014 Fig.4.IllustrationofRadon-dintegrationlimits:thegraysquarerepresentsthemaximuminscribedsquare.thesameintegrationareafordifferentangles;(ii)integratingalongcircles,ratherthanparallelstraightlines.A.Radon-dTransformTheRadon-dmodicationoftheRTperformsintegrationoverthesamearea,independentlyofthedirectionofintegra-tion.Thisisachievedby,insteadofcomputingtheRTofthewholeimage,changingtheintegrationlimitstocontainonlythemaximuminscribedsquare,asillustratedinFig.4,i.e.,,),otherwise 2(wheremin,foranimage).ThismodiedRT(calledRadon-d)oflog(,)hasapproximatelythesameenergy,independentlyofConsiderthenaturalimagerepresentedinFig.5.ThecorrespondingRadon-dtransformofthelogarithmofthemagnitudeofitsFouriertransformisdepictedinFig.6-(a),fordifferentangles.Asshownin[32],thisRadon-dtransformofanaturalimagecanbeapproximatedbyaline,asaconsequenceofthefactthatthespectrumfollowsthepowerlawmentionedinSectionIII-A.However,thespectralirregularitiespointedoutin[13],aswellasthetwolinesthatcanbeobservedat0and90(duetotheuseoftheFFT[34]),maketheintegrationnotexactlyaline.Thus,tobetterapproximatetheRadon-dtransformofanaturalimage,weproposettingathirdorderpolynomial,log,,)InFig.6-(b)weplotalineoftheRadon-dtransformofthelogarithmofthespectrummagnitudeofthenaturalimageinFig.5,andtheapproximationgivenbyEquation(7).B.Radon-cTransformLimitingtheintegrationintervalisnottheonlywaytocapturethequasi-invariantangularbehavioroflog(,)Instead,wemayintegratealongcircleswithradius,i.e.,performintegrationdirectlyinpolarcoordinates,,) ,whichwecall.Noticethatifequals1(inthe2-Dwillbeequalto1,independentlyof,duetothenormalizationfactor1 ) Fig.5.Imageofsize32642448(acquiredwithaCanonIxus850). Fig.6.(a)Radon-dtransformofthelogarithmofthespectralmagnitudeoftheimageinFig.5(inpixelunits).(b)Fittedfunction(7).(c)Radon-ctransformofFig.5.(d)Fittedfunction(9).InFig.6(c),weplottheRadon-ctransformofthelogarithmofthespectrummagnitudeofthenaturalimageinFig.5.Sincetheintegrationisalongcircles,theRadon-ctransformiscloselyrelatedwiththeapproximationgivenbyEquation(5).Afteranexhaustiveexperimentalstudy,theRadon-ctransformofnaturalimagesisverysimilartotheonedepictedinFig.6(c).Tobetterapproximateit,speciallyinthehigherfrequencies,weproposeatwo-regionpowerlawfunction,log,),�where ,sincetheapproximatefunctionmustbecontinousat.Fig.6-(d)showstheRadon-ctransform,togetherwiththeapproximatemodel(9),forthenaturalimageofFig.5.V.PROPOSEDLGORITHMWenowintroducetheproposedalgorithmstoinfertheparametersoflinearuniformmotionblursandout-of-focusblurs.Forthelinearuniformmotioncase,theparametersto OLIVEIRAetal.:PARAMETRICBLURESTIMATIONFORBLINDRESTORATIONOFNATURALIMAGES475 Fig.19.Closeupsofrestoredimagesandestimatedkernels.(a)Proposedmethod.(b)Xuetal[47].(c)Goldsteinetal[13].(d)Fergusetal Fig.20.Closeupsofrestoredimagesandestimatedkernels.(a)Proposedmethod.(b)Xuetal[47].(c)Goldsteinetal[13].(d)Fergusetalalonlythecloseupversions(800by800pixels),ourmethodtookaround33seconds,Xuetal[47]58seconds,andGoldsteinetal[13]92seconds.2)Out-of-Focus:Theimagesusedintheseexperimentsweretakenonatripod,toensurethattheimagesarefreefrommotionblur.Thescenesarefarawayfromthecamera,makingthefocaldistanceatinnityassumptionvalid.Fig.18showstheoriginalblurredimages.TheestimatedkernelsaswellthecloseupsoftherestoredimagesaredepictedinFigs.19and20.Ascanbeseen,therestoredimageswithourmethodarevisuallygood.Theestimatedkernelsare,differentinshape,butconsistentinthesupportsize.Onceagain,ourbetterresultscomesfromthefactthattheproposedkerneliscompactandclosetothetrueone.Intermsofspeed,therestorationtimesweresimilartothelinearmotionblurcase.VII.CONCLUSIONWehaveproposedanewmethodtoestimatetheparametersfortwostandardclassesofblurs:linearuniformmotionblurandout-of-focus.Theseclassesofblursarecharacterizedbyhavingwelldenedpatternsofzerosinthespectraldomain.Themethodproposedinthispaperworksonthespectrumoftheblurredimages,andissupportedontheweakassumptionthattheunderlyingimagessatisfythefollowingnaturalimageproperty:thepower-spectrumisapproximatelyisotropicandhasapower-lawdecaywithrespecttothedistancetotheoriginofthespatialfrequencyplaneToidentifythepatternsoflinearmotionblurandout-of-focusblur,weintroducedtwomodicationstotheRadontransform,termed.Theformerischaracterizedbyperformingintegrationoverthesameareaoftheimagespectrum,whilethelaterperformsintegrationalongcircles.Theidenticationoftheblurparametersismadebyttingappropriatefunctionsthataccountseparatelyforthenaturalimagespectrumandtheblurspectrum.Theaccuracyoftheproposedmethodwasvalidatedbysimulations,anditseffectivenesswasassessedbytestingthealgorithmonrealblurrednaturalimages.Therestoredimageswerealsocomparedwiththoseproducedbystate-of-the-artmethodsforblindimagedeconvolution.EFERENCES[1]M.AlmeidaandL.Almeida,“Blindandsemi-blinddeblurringofnaturalimages,”IEEETrans.ImageProcess.,vol.19,no.1,pp.36–52,Jan.2010.[2]L.Bar,N.Sochen,andN.Kiryati,“Variationalpairingofimagesegmentationandblindrestoration,”inProc.8thECCV,2004,pp.166–177.[3]M.BerteroandP.Boccacci,IntroductiontoInverseProblemsinImag-.England,U.K.:IOPPublishing,1998.[4]B.P.Bogert,M.J.R.Healy,andJ.W.Tukey,“Thequefrencyalanysisoftimeseriesforechoes:Cepstrum,PseudoAutocovariance,Cross-Cepstrumandsaphecracking,”inProc.Symp.TimeSer.Anal.,vol.15,1963,pp.209–243.[5]R.Bracewell,Two-DimensionalImaging.UpperSaddleRiver,NJ,USA:Prentice-Hall,1995.[6]P.CampisiandK.Egiazarian,BlindImageDeconvolution:TheoryandApplications.Cleveland,OH,USA:CRCPress,2007.[7]A.S.Carasso,“Directblinddeconvolution,”SIAMJ.Appl.Math.vol.61,no.6,pp.1980–2007,2001.[8]T.F.ChanandJ.Shen,ImageProcessingandAnalysis-Variational,PDE,Wavelet,StochasticMethods.Philadelphia,PA,USA:SIAM,2005.[9]T.F.ChanandC.K.Wong,“Totalvariationblinddeconvolution,”IEEETrans.ImageProcess.,vol.7,no.3,pp.370–375,Mar.1998. OLIVEIRAetal.:PARAMETRICBLURESTIMATIONFORBLINDRESTORATIONOFNATURALIMAGES469 Fig.2.(a)Naturalcolorimage(size32642448)withlinearmotionblur.(b)Naturalcolorimage(size51843456)without-of-focusblur(bothacquiredwithaCanonIxus850).assumethatthenoiseisweak,supportingtheapproximationlog(,)log(,)(,)log(,)log(,)i.e.,thecoarsebehavioroflog(,)dependsessentiallyonlog(,)log(,).Sincethecoarsebehaviorlog(,)alonglinesinthe(,)planeisapproximatelyindependentof(see(5)),thestructureoflog(,),namelyitszeros,ispreservedinlog(,)However,thepresenceofnoisemaypreventthese“zeros”frombeingexact.Nevertheless,theyremainclosetozero,andmoreimportantly,theyarelocalminima.Sincelinearuniformmotionblurismodeledbyalinesegment,thecorrespondingspectrumisasinc-likefunctioninthedirectionoftheblur.Inthiscase,thespectrumexhibitszerosalonglinesperpendiculartothemotiondirection,sepa-ratedfromeachotherbyadistancethatdependsontheblurlength.Fig.3(a)showsthelogarithmofthepowerspectrumofthenaturalimageshowninFig.2(a),whichsufferedlinearuniformmotionblur.Namelyduetothepresenceofnoiseandothermodelmismatches,thezerosbecomelocalminima;nevertheless,onecaneasilyrecognizethemotionblurpattern.Toidentifythemotionangle,weproposetouseamodiedRadontransform(RT)describedindetailinSectionIV.Theideaistointegratethespectrumoftheblurredimagealongdifferentdirections;theintegrationperformedperpendicularlytotheangleofthemotionblurwillbestexhibitthesinc-likebehavior,namelybecausethelogpowerspectrumoftheunderlyingnaturalimageis(approximately)angle-independent.ThisisillustratedinFig.3(b). Fig.3.ImageofFig.2-(a):(a)logarithmofthepowerspectrum(whitelinesegmentindicatesmotiondirection),(b)Radontransformofspectrumatthemotionblurangle().ImageofFig.2-(b):(c)logarithmofthepowerspectrum(magnied),(d)Radon-ctransformofspectrum.Theout-of-focusblur,ontheotherhand,ismodeledbyanuniformdisk,andhasaBessel-likespectrum[42].Inthiscase,thelocalminimaarealongcircles,theradiiofwhichdependonthePSFradius.Tocapturethesecircularzeropatterns(orlocalminima),weproposeaRadon-typetransform(termedRadon-c)thatintegratesalongcircles,ratherthanstraightlines,asdescribeinSectionIV.Fig.2(b)showsanaturalcolorimagecorruptedbyout-of-focusblur;inFig.3(c)and(d)wecanobservethecircularpattern,bothinthepowerspectrumoftheimageandonthecircularRadontransform.IV.MODIFIEDRANSFORMSTheRadontransform(RT)isanintegraltransformthatconsistsoftheintegralofafunctionalongstraightlines[5].Formally,theRTofareal-valuedfunctiondenedon,atangle,anddistancefromtheorigin,isgivenby( ,,))(dxdywheredenotestheDiracdeltafunction.Equivalently,( ,,) (,TheRT( ,,)istheintegralofalongalineformingananglewiththe-axis,atadistancefromtheorigin[5].TheRadontransformisusedinmanyscienticandtechnicalelds,inparticularincomputedtomography[15],[30].Inthispaper,weintroducetwomodicationstotheRT.Asnotedabove,naturalimageshaveanapproximatecoarsebehavioroflog(,)alonglinesthatpassthroughtheorigin,independentlyoftheangle.Wecapturethisbehaviorintwodifferentways:(i)performingtheRadonTransformwith OLIVEIRAetal.:PARAMETRICBLURESTIMATIONFORBLINDRESTORATIONOFNATURALIMAGES477 JoséM.Bioucas-Dias(S’87–M’95)receivedtheE.E.,M.Sc.,Ph.D.,andAgregadodegreesinelectri-calandcomputerengineeringfromInstitutoSupe-riorTécnico(IST),EngineeringSchool,UniversityofLisbon(ULisbon),Portugal,in1985,1991,1995,and2007,respectively.Since1995,hehasbeenwiththeDepartmentofElectricalandComputerEngineering,IST,wherehewasanAssistantProfessorfrom1995to2007andanAssociateProfessorsince2007.Since1993,hehasbeenaSeniorResearcherwiththePatternandImageAnalysisGroup,InstitutodeTelecomunicações,whichisaprivatenonprotresearchinstitution.Hisresearchinterestsincludeinverseproblems,signalandimageprocessing,patternrecognition,optimization,andremotesensing.Dr.Bioucas-DiaswasanAssociateEditorfortheIEEETRANSACTIONSIRCUITSANDYSTEMSfrom1997to2000andisanAssociateEditorfortheIEEETRANSACTIONSONMAGEROCESSINGandtheIEEERANSACTIONSONEOSCIENCEANDEMOTEENSING.HewasaGuestEditoroftheSpecialIssueonSpectralUnmixingofRemotelySensedDataoftheIEEETRANSACTIONSONEOSCIENCEANDEMOTEENSINGoftheSpecialIssueonHyperspectralImageandSignalProcessingoftheIEEEJOURNALOFOPICSINPPLIEDARTHBSERVATIONSEMOTEENSING,andisaGuestEditoroftheSpecialIssueonSignalandImageProcessinginHyperspectralRemoteSensingoftheIEEEIGNALROCESSINGAGAZINE.HewastheGeneralCo-Chairofthe3rdIEEEGRSSWorkshoponHyperspectralImageandSignalProcessing,EvolutioninRemotesensing(WHISPERS’2011),andhasbeenamemberofprogram/technicalcommitteesofseveralinternationalconferences. IEEETRANSACTIONSONIMAGEPROCESSING,VOL.23,NO.1,JANUARY2014ParametricBlurEstimationforBlindRestorationofNaturalImages:LinearMotionandOut-of-FocusJoãoP.Oliveira,Member,IEEE,MárioA.T.Figueiredo,Fellow,IEEEandJoséM.Bioucas-Dias,Member,IEEE—Thispaperpresentsanewmethodtoestimatetheparametersoftwotypesofblurs,linearuniformmotion(approximatedbyalinecharacterizedbyangleandlength)andout-of-focus(modeledasauniformdiskcharacterizedbyitsradius),forblindrestorationofnaturalimages.Themethodisbasedonthespectrumoftheblurredimagesandissupportedonaweakassumption,whichisvalidforthemostnatural OLIVEIRAetal.:PARAMETRICBLURESTIMATIONFORBLINDRESTORATIONOFNATURALIMAGES473 Fig.11.RMSE(inpixelunits)ofthelengthestimationalgorithm,fortwonoisescenarios.(a)BSNR40dB.(b)BSNR Fig.12.RMSE(inpixelunits)oftheout-of-focusblurestimationalgorithm,fortwonoisescenarios. Fig.13.Estimatedparametersasafunctionofimagesize.(a)and(b)Images1and2arethoseinFig.14,Images3and4arethoseinFig.18.inequivalentblurlengthswithsub-pixelprecision,dependingonthesamplingrate.Fig.12showstheaccuracyoftheproposedalgorithmfortheout-of-focuscase.Theseresultsshowthatthealgorithmisaccurateand,asexpected,theerrorsarerelativelylargerforsmallerblurs.Forsmallblurs,therstzero(localminimum)oftheblurspectrumcorrespondstolargervaluesof,whichisapproximatedbythesecondtermin(9).Nevertheless,thealgorithmcorrectlycopeswiththesecases.Finally,Fig.13showstheestimatedblurangleandlengthobtainedfromthenaturalblurredimages,asafunctionofimagesize.WeconsidersquarecropsoftheimagesdepictedinFig.14andFig.18.Asexpected,theperformanceofthealgorithmdecreaseswiththeimagesize,butitonlydegradesconsiderablyforimagesizesbellow600600pixels,whichistotallyacceptable.B.NaturalBlurredImagesWeconsidernowasetofimagesobtainedwithacommonhand-heldcamera,corruptedwith(approximatelylinear)motionblurandout-of-focusblur.Duetothelargesizeoftheimages,deblurringwasdonewiththeRichardson–Lucyalgorithm[38],separatelyforeachcolorchannel.Sincewe Fig.14.Naturalimagescorruptedwith(approximatelylinear)motionblur,acquiredwithaCanonIxus850. Fig.15.CloseupsoftheblurredimagesfromFig.2-(a)(top)andfromFig.14(bottom).don’thavegroundtruth,onlyaqualitativevisualcomparisoncanbemade.Wecompareourresultswiththreestate-of-the-artBIDmethods,forwhichthereiscodeavailable:(i)themethodproposedbyFergusetalal(ii)themethodofGoldsteinetal[13],whichisrelatedtoourmethod;(iii)themethodofXuetal[47],consideredstate-of-the-artwhencomparedagainstothersmethods(wearethusindirectlyalsocomparingourmethodwithallthemethodsconsideredin[47]).Fullsizeimagesandmoreexamplescanbeseenathttp://preview.tinyurl.com/ce96nsb.1)MotionBlur:Tosimulatemotionblur(not“camerashake”),weperformedanoutofplanerotationofafarawayscene.Thisway,alltheelementsoftheimagemoveapproximatelythesame,makingvalidthespaceinvariantblurapproximation.Notethatthisisanapproximation,andthatsomeinplanerotationmaybepresent.InFigs.2–(a)and14,weshownaturallinearmotionblurredimages.Agraphicalrepresentationoftheblurestimatesobtained,aswellascloseupsshowingthecorruptedimagesandcorrespondingrestorationsaredepictedinFigs.15,16and17.Theimageestimatesproducedbyourapproacharevisuallyquitegood.Comparingwiththeresultsoftheothermethods,wecanobservethatsomedetailsarerecoveredbetter.Notice,inparticular,somedetailsforwhichweknowaprioritheiroriginalshape,suchasthe“P”signinFig.16oracircularlampinFig.17.Wecanseeinalltheexamples(andalsoonthoseavailableathttp://preview.tinyurl.com/ce96nsb)thatthedifferentmethodsproducekernelestimateswithsimilarlengthsanddirections.However,unliketheothers,ourmethodimposesthecontinuityofthekernel. IEEETRANSACTIONSONIMAGEPROCESSING,VOL.23,NO.1,JANUARY2014 Fig.16.Closeupsoftherestoredimagesandestimatedkernels.(a)Proposedmethod.(b)MethodofXuetal[47].(c)MethodofGoldsteinetalal(d)MethodofFergusetal Fig.17.Closeupsofrestoredimagesandestimatedkernels.(a)Proposedmethod.(b)MethodofXuetal[47].(c)MethodofGoldsteinetalal(d)MethodofFergusetal Fig.18.Naturalimagescorruptedwithout-of-focusblur(acquiredwithaCanonD60)andcloseupsthereof.AMATLABimplementationofouralgorithm(runningon2.2GHzCore2Duo)tookaround10minutestorestorethenaturalcolorimagesshowninthissection.Themethodproposedin[11]takesaround1hour.Itwasnotpossibletorestorethefullsizeimageswithmethodsandcodeproposedin[47]and[13],duetoitshugedimensions.Considering OLIVEIRAetal.:PARAMETRICBLURESTIMATIONFORBLINDRESTORATIONOFNATURALIMAGES471estimatearetheangleandthelength.Intheout-of-focuscase,theonlyparameteristheradius.OncewehavecomputedoneofthemodiedRTsmentionedintheprevioussection,theblurparameter(i.e.,themotionlengthorthediskradius)estimationwillbeperformedbyttinganappropriatefunctiontotheresult.Accordingto(5),andthelinearityoftheRT,theproposedfunctionhastwoterms:onefortheimagespectrum,andtheotheronefortheblurfrequencyresponsei.e.,omittingthedependencyon()log,,) ()log,,) ()Thepreviousequationreferstothelinearuniformmotionblurcase;fortheout-of-focusblur,wesimplyreplaceTheimagespectrumterm()isapproximatedby(7)or(9),accordingly.Theblurspectrumtermisapproximated()loglog()where()isdenedinthefollowingsubsections,andparametersareintroducedtotakeintoaccountthenonlinearitiesandthenoise.Sincenoisepreventsthe“zeros”oftheblurspectrafrombeingexact,thiscanonlybeachievedbytheterm1insidethelogarithm.Parametercontrolstherelativeweightoftheblurspectraltermagainsttheimagespectra.ThistermisproportionaltotheintegrationlimitsoftheRT,becausethemagnitudeoftheblurspectrumisconstantintheintegrationdirection,i.e.,alongstraightlinesforlinearuniformmotionblur,andcircularlinesforout-of-focusblur.Theseparametersareneededsince(5)isjustanapproximation.A.MotionBlurThesinc-likestructureofthemotionblurkernel[3]iswellcapturedbythetransformattheblurangle.Thus,motionblurestimationwillbedoneintwophases:(i)angleestimation;(ii)motionlengthestimation.In[28],theangleestimateisthatforwhichthemaximumoftheRToccurs;naturally,thisonlyworksforverylongblurs,sothattheblurredimageisverysmoothinthemotionblurdirection,leadingtoaclearmaximumoftheRT.Ontheotherhand,in[19],theangleestimateistheoneforwhich( ,,),asafunctionof,hasthehighestentropy.ThespectralirregularitiesandtheartifactsintroducedbytheFFTmakeitdifcultforthepreviousapproachestoworkwellforshortblurs.Toincreasetherobustnessandtakeadvantageofthequasi-invarianceofthespectra,[32]computesthedifferenceoftheRTatperpendicularanglesandchoosestheonethathasthemaximumenergy.Inthispaper,wefollowasimplerapproach,wherethemaingoalistoidentifytheblurpatternintheRadon-dtransform.ComputingtheRadon-dtransformofthelinearmotionblurspectrum,weobtainasincstructureintheblurdirection,andaconstantlineintheperpendiculardirection.Thus,byttingthemodelinEquation(7)totheRadon-dtransform,whichintegratesthequasi-invarianceoftheimagespectraplustheblurspectra,thettingerrorwillbemaximumpreciselyatthemotion Fig.7.Illustrationofthemotionblurangleestimationcriterion.(a)(,)asafunctionof,representedbygraylevels.(b)Residual(,)(,)asafunctionof.(c)(,)(,)asafunctionof(inpixelunits),for.(d)(,)(,)(thecorrectangle).angle.Let(,)denotetheintegraloflog(,)alongadirectionperpendiculartoi.e.(,)log(,),,).Consideralsothefunction(,)givenbyttinganapproximationoftheform(7)to(,).Theproposedangleestimateisthatwhichmaximizesthemeansquarederror(MSE)ofthist,argmax(,)(,)InFig.7,severalplotsillustratetheangleestimationcriteriongivenby(13),appliedtotheimageofFig.2(a).Oncewehave,weproceedtoestimatethelengthoftheblurkernel.Giventhatthesinc-likebehaviorispreservedintheRadontransformatangle,webasetheblurlengthestimationon.Weproceedbytting()(see(10))to(,.Inthiscase,()isgivenby(7),and()mustbeproportionaltoasincfunction[3],i.e.()| |sinc()wheresinc istheblurlength.Thejointestimateofalltheparametersi.e.,, ,,maynotyieldtherightsolution,asthecorrespondingleastsquarescriterionishighlynon-convex,thusanyiterativeminimizationalgorithmisdoomedtobetrappedatlocalminima.Instead,werstminimizewithrespectto, ,,withxed,thusobtainingafunctionofalone,whichisthenminimizedbylinesearch.Thepreviouslyestimatedparametersareused,inarenementstage,asinitialvaluestotEquation(7)Theintegrationoftheblurspectrum,perpendicularlytothemotiondirection,yieldsaconstantvalue,wellapproximatedbyEq.(7). IEEETRANSACTIONSONIMAGEPROCESSING,VOL.23,NO.1,JANUARY2014 Fig.8.RTandcorrespondingapproximatefunction.(a)MSEoftted()asafunctionof.(b)(,andadjustedfunction()tothedata,with ,initializedwithpositivevalues(typically1).Parameterischosentobethevalueleadingtotheminimummeansquarederror.InFig.8,weshowtheRadon-dtransformat,therootmeansquarederrorasafunctionof,andtheapproximatedfunction(10)forthemotionblurredimageofFig.2(a).ThenormalizeddiscreteFourierangularfrequencyrelatedtothecontinuousfrequency[34];sincewehavedifferentangularfrequencies(isthenumberofpoints),eachrealfrequencyisgivenby: ,...,Assumingthattheimageissquarewithsize,from(15),wenallyhave B.Out-of-FocusBlurToinfertheradiusoftheout-of-focusblur,weproceedasinthemotionblurcase.However,sincethepatternofzerosinthespectrumisnowcircular,weusethetransformanddonothaveanyangletoestimate.Thettingfunctionisagaintheonein(10),where()isgivenby(9),and()() whereistheBesselfunctionoftherstkind,withparameter[42].Thesetofparameterstoestimateis,, ,.Again,sincethecriteriumishighlynon-convex,weproceedasinthepreviouscase:wexandopti-mizefortherestoftheparameters;weinitializethevaluesthatt(9)alone,andassignasmallpositivenumber ,;wepickthatleadstotheminimummeansquarederror.Likeinthepreviouscase,from(16)wehave InFig.9,weshowtheRadon-dtransformat,therootmeansquarederrorasafunctionof,andtheapproximatedfunction(10)forthemotionblurredimageofFig.2(b).VI.EXPERIMENTALESULTSWeassesstheperformanceoftheproposedmethodintwodifferentways.First,weusesyntheticallyblurredimages,exactlygivenbythemodelsdescribedinSectionIII.Theaccuracyisassessedbytherootmeansquarederror(RMSE)oftheestimatedparameters:RMSE 1 ni(xŠ xi)2, Fig.9.RTandcorrespondingapproximatefunction.(a)MSEoftted()asafunctionof.(b),)andadjustedfunction() Fig.10.RMSE(indegrees)oftheangleestimationalgorithm,fortwonoisescenarios.(a)BSNR40dB.(b)BSNR20dB(whereBSNRdenotes“blurredSNR”,givenby10logvarrblurredimage,andisthenoisevariance,asdenedinSectionI).whereisthenumberofruns,arethetrueparameteranditsestimateinthe-thrun,respectively.Finally,weapplythemethodtorealBIDproblems;inthiscase,thelinearmotionblurandout-of-focusassumptionsareonlyapproximations.IntherealBIDexperiments,wecomparetheproposedmethodwithseveralstate-of-the-artalternatives.A.AccuracyofProposedAlgorithmTheaccuracyoftheproposedmethodisassessedintermsofRMSEover10runs(indegreesfortheangularpara-meter,andpixelunitsforlengthparameters).Tothisend,weconsidererasetof7well-knownimages:cameraman,Lena,Barbara,boats,peppers,goldhill(256256),ngerprint512),andalsothenaturalimageofFig.5.Fig.10showstheaccuracyoftheproposedmethod:theerrorsaresimilarandessentiallyindependentofthetrueangle.Thehighesterrorsareobtainedforthesmallestlengths,whichisanaturalresult;infact,foraveryshortmotionblur,thekernelsobtainedwithtwocloseanglesarealmostidentical.Theaccuracyofthealgorithmalsodependsonthenaturalimageassumption(namelyitsspectralisotropy):ifanimageisnotanaturalimage,thequasi-invarianceoftheimagespectrumdoesnothold,makingtheangleidenticationmoredifcult.Concerninglengthestimation,theerrorsarealsoquitesmall,evenforlargeblurlengths(Fig.11).Thisisamajorimprovementoverourpreviousalgorithm[33],forwhichoneoftheweaknesseswaspreciselyforlongblurs.Byusingthettingfunction()(10),themethodisnolongerdependentonthelocationoftherstlocalminimum,andcanalsoachievesub-pixelprecision.Thisisimportantinthecaseofnaturalmotionblurredimages,wherethelengthoftheblurcanresult