Two Algorithms for the Minimum Enclosing Ball Problem - PDF document

alongwithu0.Asimplemanipulationoftheoptimalityconditionsrevealsthat A=(u);(4)whichimpliesthatu2Rmisanoptimalsolutionof(D)andthatstrongdualityholdsbetween(P2)and(D).NotethatthecentercAoftheminimumenclosingballofAisgivenbyaconvexcombinationoftheelementsofAby( 3b ).Inaddition,itfollowsfrom( 3d )thatonlythecomponentsofucorrespondingtothepointsontheboundaryofMEB(A)canhaveapositivevalue. Lemma2.1 LetA=fa1;:::;amg.TheminimumenclosingballofAexistsandisunique.Letu2Rmdenotetheoptimalsolutionof(D).Then,MEB(A)=BcA;A,wherecA=mXi=1uiai;A=p (u):(5)Proof.NotethatAB0;u,whereu:=maxi=1;:::;mkaik.Byaddingtheredundantconstraint (u)2to(P2),thefeasibleregionbecomesaclosedandboundedsetandtheobjectivefunctioniscontinuous,whichestablishestheexistenceofMEB(A).Ifthereweretwodierentminimumenclosingballs,onecanthenconstructaballofsmallerradiusthatenclosestheintersectionofthetwoballsandhencealsoA,whichisacontradiction.Therelationships( 5 )directlyfollowfromthediscussionsprecedingthelemma. ByLemma 2.1 ,MEB(A)canbecomputedbysolvingthedualproblem(D),whichwillbethebasisofbothofouralgorithmsinthispaper.Weclosethissectionbythefollowingtechnicalresult,whichwillplayanimportantroleinndingagoodinitialfeasiblesolutioninouralgorithms.Thereaderisreferredto[ 16 ]fortheproofofthisresult. Lemma2.2 LetA=fa1;:::;amgandletMEB(A)=BcA;A.Then,anyclosedhalfspacethatcontainscAalsocontainsatleastonepointaj2Asuchthatkaj�cAk=A.3TheFirstAlgorithmGivenA:=fa1;:::;amgRnand�0,wepresentourrstalgorithmthatcomputesa(1+)-approximationtoMEB(A)inthissection. 5 iseasytoverifythatk=argmax2[0;1]�(1�)uk+e
:WeremarkthatAlgorithm 3.1 usesonlytherst-orderapproximation.Assuch,eachiterationisfairlycheapbutthenumberofiterationsisusuallysignicantlyhigherthanotheralgorithmsthatusesecond-orderinformationsuchasinterior-pointmethods.However,suchgeneral-purposealgorithmsbecomecomputationallyinfeasibleforlargerproblemssinceeachiterationisusuallymuchmoreexpensive.Thisobservationprovidesoneofourmotivationstodevelopaspecializedalgorithmforthisproblem.3.1AnalysisoftheFirstAlgorithmThissubsectionisdevotedtotheanalysisofAlgorithm 3.1 . Lemma3.1 u02Rmsatises 0= (u0)(1=3) (u)=(1=3) A,whereu2Rmand Aaretheoptimalsolutionandtheoptimalvalueof(D),respectively.Furthermore,08.Proof.Notethat (u0)=(1=2)(a)Ta+(1=2)(a)Ta�(c0)Tc0;(6a)=(1=4)(a)Ta+(1=4)(a)Ta�(1=2)(a)Ta;(6b)=(1=4) a�a 2;(6c)whereweusedthefactthatc0=(1=2)�a+a
.TheproofisbasedonestablishingthatatleastoneofaandaissucientlyawayfromthecentercAofMEB(A).First,supposethatka1�cAk(1=p 3)A,whereAistheradiusofMEB(A).LetHbethehyperplanepassingthroughcAthatisperpendiculartoa1�cA.LetH+denotetheclosedhalfspacewhoseboundaryisHandwhichdoesnotcontaina1.ByLemma 2.2 ,H+containsapointaj2Asuchthatkaj�cAk=A.Therefore,ka�a1k2ka1�cAk2+(A)2(4=3) A,where A= (u)=(A)2istheoptimalvalueof(D).Itfollowsfrom( 6 )that (u0)=(1=4) a�a 2(1=4) a1�a 2(1=3) (u):Supposenowthatka1�cAk=A,where1=p 3.Inthiscase,ka1�akka1�cAk+kcA�ak,whichimpliesthatkcA�ak a1�a � a1�cA (1+2)1=2A�A=[(1+2)1=2�]A;whereweagaininvokedLemma 2.2 toobtainalowerboundonka1�ak.Therefore,onemoreapplicationofLemma 2.2 yields (u0)=(1=4) a�a 2;(1=4)�ka�cAk2+(A)2
;(1=4)�1+2+2�2(1+2)1=2+1
A;=(1=2)�1+2�(1+2)1=2
A: 7 Proof.Letusrstconsider0.Ateachiterationk0,wehavek&#x-277;1.ByLemma 3.2 , k+1= k1+2k 4(1+k); k(1+1=8):Iteratingthisinequality,weobtain k+1(9=8)k+1 0.ByLemma 3.1 andthefeasibilityofuk+1,wehave A k+1(9=8)k+1 0(9=8)k+1( A=3);whichimpliesthatk+1log(3)=log(9=8)10,orequivalentlythatk9.Therefore,09.Letusnowconsider��1for=1;2;:::.Let:=�1.Ateachiterationkwithk�1=2,wesimilarlyhave k+1= k1+2k 4(1+k); k1+1 22+(2+1):Atiteration,wehave1=2�1.Sincetheballcenteredatcwithradius[(1+) ]1=2enclosesA,itfollowsthat  A(1+) (1+(1=2�1)) .Togetherwiththerepeatedapplicationoftheinequalityabove,wehave A +k 1+1 22+(2+1)k A 1+(1=2�1)1+1 22+(2+1)k;whichimpliesthat��1log�1+1 2�1
log1+1 22+(2+1)1 2�11 22+(2+1)+1 1 22+(2+1)825 24(2+1)17(2);whereweusedtheinequalitieslog(1+x)xforx��1andlog(1+x)x=(x+1)forx��1. Thefollowinglemmaestablishesanupperboundonthenumberofiterationstoobtainaniteratewithk. Lemma3.4 Let2(0;1).Algorithm 3.1 computesaniteratekwithkinatmost9+68=iterations.Proof.Letbeanintegersuchthat1=22=2.Therefore,afteratmostiterations,Algorithm 3.1 computesaniteratekwithk.ByLemma 3.3 ,=0+X=1(��1)9+17X=129+34(2)9+68=: 9 4TheSecondAlgorithmInthissection,wedescribeoursecondalgorithm,whichisamodicationofAlgorithm 3.1 .Algorithm 4.1 startsowiththesameinitialsolutionu0astheonecomputedbyAlgorithm 3.1 .Ateachiteration,thefurthestpointinAfromthecenterckofthetrialballiscomputedasinAlgorithm 3.1 .Incontrast,eachiterationofAlgorithm 4.1 alsoincludesthecomputationoftheclosestpointtockamongallpointsinXkA.Geometrically,thisisequivalenttoshrinkingthecurrenttrialballbythesmallestamountsothattheshrunkenballdoesnotcontainanypointsinXk.Algebraically,thiscorrespondstomovingawayfromthevertexoftheunitsimplexthatminimizesthelinearapproximationto (u)atuk,wheretheminimizationisovertheverticesfej:ukj�0g.Thefeasiblesolutionukisupdatedindierentwaysbasedonthesetwocomputations.Ifk=+k,thenAlgorithm 4.1 usestheexactsameupdateasinAlgorithm 3.1 .Otherwise,thenewcenterck+1isobtainedbymovingthecurrentcenterckawayfromtheclosestpointa2Xk.Therefore,Algorithm 4.1 isobtainedbyincorporating\away"stepsintoAlgorithm 3.1 .Forawaysteps,itiseasytoverifythatk=argmax2[0;uk=(1�uk)] �(1+)uk�e
:(10)Notethattherangeofischosentoensurethatthedualfeasibilityconstraintuk+10issatised.4.1AnalysisoftheSecondAlgorithmTheanalysisofAlgorithm 4.1 isverysimilartothatofAlgorithm 3.1 .Asin[ 34 ],wecalliterationkaplus-iterationifk=+k.Ifk=�kandk=(�k)=[2(1��k)],thenwecallitaminus-iteration.Theworkingcoresetremainsunchangedataminus-iteration.Finally,ifk=�kandk=uk=(1�uk),wethencallitadrop-iterationsincethethcomponentofukdropsto0andaisremovedfromtheworkingcoreset.Ouranalysismimicstheanalysisof[ 34 ]forasimilaralgorithmthatcomputesanapproximationtotheminimum-volumeenclosingellipsoidofanitesetofpoints.Thenextlemmaestablishesalowerboundontheimprovementateachplus-orminus-iteration. Lemma4.1 Ateachplus-orminus-iteration, k+11+2k 4(1+k);k=0;1;::::(11)Proof.Ataplus-iteration,theresultsdirectlyfollowsfromLemma 3.2 .Ataminus-iteration, k+1= �(1+k)uk�ke
;= k1+(�k)2 4(1��k): 11 Algorithm4.1Thesecondalgorithmthatcomputesa(1+)-approximationtoMEB(A). Require: InputsetofpointsA=fa1;:::;amgRn;�0. 1:  argmaxi=1;:::;mkai�a1k2; argmaxi=1;:::;mkai�ak2; 2: u0i 0;i=1;:::;m; 3: u0 1=2;u0 1=2; 4: X0 fa;ag; 5: c0 Pmi=1u0iai; 6: 0 (u0); 7:  argmaxi=1;:::;mkai�c0k2; argmini:ai2X0kai�c0k2; 8: +0 ka�c0k2= 0�1;�0 1� a�c0 2= 0; 9: 0 maxf+0;�0g; 10: k 0; 11: Whilek�(1+)2�1,do 12: loop 13: ifk��kthen 14: k k=[2(1+k)]; 15: k k+1; 16: uk (1�k�1)uk�1+k�1e; 17: ck (1�k�1)ck�1+k�1a; 18: Xk Xk�1[fag; 19: else 20: k minn�k 2(1��k);uk 1�uko; 21: k k+1; 22: ifk=uk=(1�uk)then 23: Xk Xk�1nfag; 24: else 25: Xk Xk�1; 26: endif 27: uk (1+k�1)uk�1�k�1e; 28: ck (1+k�1)ck�1�k�1a; 29: endif 30: k (uk); 31:  argmaxi=1;:::;m ai�ck 2; argmini:ai2Xk ai�ck 2; 32: +k  a�ck 2= k�1;�k 1� a�ck 2= k; 33: k maxf+k;�kg; 34: endloop 35: Outputck;Xk;uk;p (1+k) k. 12 Theresulteasilyfollowsfromtheobservationthat(�k)2 4(1��k)(�k)2 4(1+�k);andthat�k=kataminus-iteration. Lemma 4.1 establishesthatAlgorithm 4.1 makesatleastasmuchimprovementasAlgorithm 3.1 ateachplus-orminus-iteration.Atadrop-iteration,itiseasytoshowthat k+1 k.However,wecannolongerndapositivelowerboundon k+1� k0.Usingasimilarreasoningasin[ 34 ],eachdrop-iterationcanbepairedwithapreviousplus-iterationkatwhichukwasincreasedfrom0,exceptforthethandthcomponentswhichwerepositiveattheinitialsolutionandmaybedecreasedtozeroforthersttime.Therefore,wecandoubletheiterationcount(andaddtwoiterationstoaccountfortheinitialpositivecomponentsofu0)intheanalysisofAlgorithm 3.1 toestablishthatAlgorithm 4.1 cancomputea(1+)-approximationtoMEB(A)inatmosttwiceasmanyiterationsasthatrequiredbyAlgorithm 3.1 .NotethatthisdoesnotaecttheasymptoticiterationboundofAlgorithm 3.1 .Furthermore,eachiterationstillrequiresO(mn)operations,whichimpliesthattheasymptoticoverallcomplexityofAlgorithm 4.1 alsoremainsthesameasthatofAlgorithm 3.1 .Finally,theasymptoticboundonthesizeofthecoresetisalsounaected.However,weremarkthatAlgorithm 4.1 hasthepotentialtocomputeevensmallercoresetsthanthosereturnedbyAlgorithm 3.1 duetothepossibleinclusionofminus-anddrop-iterations.Wesummarizetheseresultsinthefollowingtheorem. Theorem4.1 GivenA:=fa1;:::;amgRnand2(0;1),Algorithm 4.1 computesa(1+)-approximationtoMEB(A)inO(mn=)operations.Furthermore,upontermination,XAisan-coresetandjXj=O(1=),whereistheindexofthenaliteratecomputedbyAlgorithm 4.1 .5ExtensionsInthissection,weestablishthatthealgorithmicframeworksofSections 3 and 4 canbeusedtocomputeanapproximationtotheminimumenclosingballofmoregeneralinputsets.Whilethecostofeachiterationofthecorrespondingalgorithmsmaydependontheinputset,theiterationcomplexityandtheasymptoticsizeofthecoresetremainunchanged.Therefore,theexistenceofan-coresetofsizeO(1=)extendstomoregeneralsetsincludingthosewithuncountablymanyelements.Notethatbothofthealgorithmsinthispapercomputetheinitialfeasiblesolutioninasimilarfashion.Thiscomputationentailsndingthefurthestpointintheinputsetfromaxedpoint.Inaddition,similarfurthestpointcomputationsareperformedateachiterationofbothofthealgorithms.Therefore,theextentofinputsetswhichareamenabletothesealgorithmshighlydependsontheeciencywithwhichsuchcomputationscanbeperformed.Inaddition,eachiterationofAlgorithm 4.1 requiresthecomputationoftheclosestpoint 13 inXkfromaxedpoint.SinceXkisalwaysaniteset,thiscomputationcanalwaysbeecientlyperformed.Forinputsetswithinnitelymanypoints,notethatthedimensionofthevectorsukgeneratedbyeitherofthetwoalgorithmsisnolongerxedandmayincreasebyoneateachiteration.AnotherkeyobservationisthatLemma 3.1 ,whichestablishesthequalityoftheinitialfeasiblesolutioncomputedbyeachofthetwoalgorithms,extendstomoregeneralsets.ThisfollowsfromthefactthatLemma 2.2 remainstrueforarbitraryinputsets.TheproofofLemma 2.2 isbasedontheargumentthatanenclosingballofsmallerradiuscanbeconstructedbymovingthecenterawayfromthehalfspaceinthedirectionofthenormalvectoroftheboundinghyperplaneifthehypothesisofLemma 2.2 isnotsatisedbythathalfspace.Therefore,thequalityoftheinitialsolutionisindependentoftheinputset.Wenowspecifyseveralinputsetsforwhichsimilaralgorithmicframeworkscanbeapplied.5.1SetofBallsLetA=fB1;:::;BmgRnbeasetofmballs.GivenBc;andx2Rn,thefurthestpointinBc;fromxisgivenbyx=c+(c�x)=kc�xk,whichcanbecomputedinO(n)operations.Therefore,eachiterationofAlgorithm 3.1 stillrequiresO(mn)operations,whichimpliesthatAlgorithm 3.1 computesa(1+)-approximationtoMEB(A)inO(mn=)operationsandreturnsan-coresetofsizeO(1=).Inadditiontocomputingthefurthestpointateachiteration,Algorithm 4.1 alsorequiresthecomputationoftheclosestpointinaniteset.ThesizeofthissetisboundedabovebyO(1=),whichimpliesthateachiterationcanbeperformedinO(mn+n=)operations.Therefore,Algorithm 4.1 cancomputea(1+)-approximationtoMEB(A)inO(mn=+n=2)operationsandreturnsan-coresetofsizeO(1=).5.2SetofEllipsoidsLetA=fE1;:::;EmgRnbeasetofmellipsoidsgivenbyEi:=fx2Rn:(x�ci)TQi(x�ci)1g,whereci2RnandQi2Rnnissymmetricandpositivedenitefori=1;:::;m.ThefurthestpointinanellipsoidfromagivenpointcanbecomputedusingatightsemideniteprogrammingrelaxationwithaxednumberofconstraintsinO(nO(1))operationsintherealnumbermodelofcomputation[ 37 ],whereO(1)denotesauniversalconstantgreaterthanthree.Therefore,Algorithm 3.1 computesa(1+)-approximationtoMEB(A)inO(mnO(1)=)operationsandreturnsan-coresetofsizeO(1=).Similarly,Algorithm 4.1 cancomputea(1+)-approximationtoMEB(A)inO(mnO(1)=+nO(1)=2)operationsandreturnsan-coresetofsizeO(1=).5.3SetofHalfEllipsoidsHissaidtobeahalfellipsoidifitisgivenbytheintersectionofanellipsoidwithahalfspace.LetA=fH1;:::;Hmg,whereHi:=fx2Rn:(x�ci)TQi(x�ci)1;(fi)Tx!ig,whereci2Rn;fi2Rn;!i2R,andQi2Rnnissymmetricandpositivedenitefori=1;:::;m. 14 time,Algorithm 4.1 hasasignicantlybetterperformancethanAlgorithm 3.1 andtheBCalgorithm,bothofwhichhavesimilarrunningtimes.Allthreealgorithmscomputedverysmallcoresetsofsimilarsizes.Algorithm 4.1 alwaysreturnedthesmallestcoresetsforeachinputset.ThecoresetscomputedbyAlgorithm 3.1 andtheBCalgorithmhavesimilarsizeswiththeformerbeingslightlybetterthanthelatter.Intermsofthenumberofiterations,Algorithm 4.1 onceagainsignicantlyoutperformstheothertwoalgorithms,bothofwhichexhibitasimilarperformance.AcloseexaminationofthecomputationalresultsrevealsthatAlgorithm 4.1 resultedinreductionsof73%to88%intermsofrunningtimeandof74%to90%intermsofnumberofiterationsincomparisonwiththeothertwoalgorithms.ThisimprovedbehaviorofAlgorithm 4.1 resultingfromtheincorporationofawaystepsintoAlgorithm 3.1 wasanalyzedbyAhipasaoglu,Todd,andSun[ 1 ],whoestablishthelinearconvergenceofthisalgorithmicframeworkunderfairlygeneralassumptions.Furthermore,duetoallowingpointstobedroppedfromtheworkingcoreset,thesizesofthecoresetscomputedbyAlgorithm 4.1 areabout10%to30%smallerthanthosereturnedbytheothertwoalgorithms.Thenextdatasetweconsideredistheverticesoftheunitsimplex.BadoiuandClarkson[ 3 ]establishatightupperboundofd1=eonthesizeofthecoresetforsuchaninputsetundertheassumptionthatnb1=c.Inanattempttoassesstheperformancesofthethreealgorithmsonsuchadataset,weconsideredtheverticesoftheunitsimplexwithn=1000using2f1;:1;:01;:001g.TheresultsofthisexperimentarepresentedinTable 2 ,whichisorganizedsimilarlytoTable 1 . Time CoreSetSize Iterations  A1A2BC A1A2BC A1A2BC 1 .010.01 222 001 .1 .03.03.02 111111 9910 .01 1.851.861.84 101101101 9999100 .001 183.06181.97182.48 100010001000 998998999 Table2:Verticesoftheunitsimplex(n=1000) AsillustratedbyTable 2 ,allthreealgorithmshavesimilarperformancesontheverticesoftheunitsimplexinRnwithn=1000.Notethatboththesizeofthecoresetandthenumberofiterationsgrowproportionallyto1=.Theseresultsareinagreementwiththetightcoresetboundof[ 3 ].ThisexampleillustratesthattheasymptoticboundsonthecoresetsizeandthenumberofiterationsforAlgorithms 3.1 and 4.1 ingeneralcannotbeimproved.However,allthreealgorithmscomputedtheexactminimumenclosingballfor=10�3(andforany10�3).Therefore,theupperboundofd1=eonthesizeofthecoresetisnolongertightfornb1=c. 17 7ConcludingRemarksInthispaper,weproposedandanalyzedtwoalgorithmsthatcomputeanapproximationtotheminimumenclosingballofagivennitesetofvectors.Bothalgorithmsexploittheunderlyinggeometricstructureoftheproblem.Eachofthetwoalgorithmsisespeciallywell-suitedforlarge-scaleinstancesoftheminimumenclosingballproblemforwhichamoderateapproximationsuces.Bothalgorithmscancomputeasmallcoresetwhosesizedependsonlyontheapproximationparameter.Wehavediscussedhowouralgorithmscanbeextendedtomoregeneralinputsetswithoutsacricingtheiterationcomplexityandhencethesizeofthecoreset.Inparticular,severalclassesofinputsetsadmitsmallandnitecoresets.Ourcomputationalexperimentsrevealthatbothofouralgorithmsarecapableofquicklycomputingagoodapproximationtotheminimumenclosingballofanitesetofvectors.Algorithm 4.1 ,whichisobtainedbyincorporatingawaystepsintoAlgorithm 3.1 ,seemstoexhibitasignicantlybetterperformancethanotherrst-orderalgorithms.Thesizesofthecoresetscomputedbyouralgorithmsareusuallyfairlysmall.Theexamplethatconsistsoftheverticesoftheunitsimplexillustratesthatouranalysisingeneralcannotbeimproved.Whilethediscoveryofecientalgorithmssuchasinterior-pointmethodsrevolutionizedconvexoptimization,thecomputationalcostofeachiterationofsuchalgorithmsquicklybecomesprohibitiveasthesizeoftheproblemsincreases.Therefore,itseemsdesirabletodesignspecializedalgorithmsforlarge-scaleproblemsthatexploittheunderlyingspecialstructureoftheproblem.Wehavedevelopedtwosuchalgorithmsfortheminimumenclosingballprobleminthispaper.Weintendtocontinueourworkondevelopingspecializedalgorithmsforotherclassesoflarge-scalestructuredoptimizationproblemsinthenearfuture.References [1] D.Ahipasaoglu,P.Sun,andM.J.Todd.LinearconvergenceofamodiedFrank-Wolfealgorithmforcomputingminimum-volumeenclosingellipsoids.TechnicalReportTR1452,CornellUniversity,SchoolofOperationsResearchandIndustrialEngineering,CornellUniversity,Ithaca,NewYork,2006. [2] M.BadoiuandK.L.Clarkson.Smallercore-setsforballs.InProceedingsofthe14thAnnualSymposiumonDiscreteAlgorithms,pages801{802,2003. [3] M.BadoiuandK.L.Clarkson.Optimalcore-setsforballs.Unpublishedmanuscript,2006. [4] M.Badoiu,S.Har-Peled,andP.Indyk.Approximateclusteringviacore-sets.InProceedingsofthe34thAnnualACMSymposiumonTheoryofComputing,pages250{257,2002. 18 [32] J.J.Sylvester.Aquestioninthegeometryofsituation.QuarterlyJournalofPureandAppliedMathematics,1(79),1857. [33] J.J.Sylvester.OnPoncelet'sapproximatelinearvaluationofSurdforms.PhilosophicalMagazine,xx:203{222,1860.FourthSeries. [34] M.J.ToddandE.A.Yldrm.OnKhachiyan'salgorithmforthecomputationofminimumvolumeenclosingellipsoids.DiscreteAppliedMathematics,2007.Toappear. [35] E.Welzl.Smallestenclosingdisks(ballsandellipsoids).InH.Maurer,editor,NewResultsandNewTrendsinComputerScience,volume555ofLectureNotesinComputerScience,pages359{370.Springer-Verlag,1991. [36] S.Xu,R.M.Freund,andJ.Sun.Solutionmethodologiesforthesmallestenclosingcircleproblem.ComputationalOptimizationandApplications,25(1{3):283{292,2003. [37] E.A.Yldrm.Ontheminimumvolumecoveringellipsoidofellipsoids.SIAMJournalonOptimization,17(3):621{641,2006. [38] G.Zhou,K.C.Toh,andB.Sun.Ecientalgorithmsforthesmallestenclosingballproblem.ComputationalOptimizationandApplications,30(2):147{160,2005. 21