IEEETRANSACTIONSONINFORMATIONTHEORY,VOL.53,NO.1,JANUARY2007LPDecodingC - PDF document

IEEETRANSACTIONSONINFORMATIONTHEORY,VOL.53,NO.1,JANUARY2007LPDecodingCorrectsaConstantFractionofErrorsJonFeldman,TalMalkin,RoccoA.Servedio,CliffStein,andMartinJ.Wainwright,Member,IEEEWeshowthatforlow-densityparity-check(LDPC)codeswhoseTannergraphshavesufÞcientexpansion,thelinearprogramming(LP)decoderofFeldman,Karger,andWainwrightcancorrectaconstantfractionoferrors.ArandomgraphwillhavesufÞcientexpansionwithhighprobability,andrecentworkshowsthatsuchgraphscanbeconstructedefÞciently.Akeyelementofourmethodistheuseofadualwitness:azero-valueddualsolu-tiontothedecodinglinearprogramwhoseexistenceprovesde- repeatÐaccumulatecodes[17],[21],[22],andaproofthatatleast errorscanbecorrectedingeneralLDPCcodesunderbit-ßippingchan-nels.Inthispaper,weshowthatLPdecoderscancorrectup underthebinary-symmetricchannel(BSC).Thisresultconsti-tutestheÞrstproofthatLPdecodinghasaninverse-exponen-tialWERonaconstant-ratecode.Furthermore,nosuchWERboundisknownformessage-passingdecoderssuchasmin-sumandsum-product(beliefpropagation)onÞnite-lengthLDPCOurresultisbasedontheexpansionoftheTannergraph,ratherthanitsgirth.MorespeciÞcally,aTannergraph isa0018-9448/$25.00©2007IEEE etal.:LPDECODINGCORRECTSACONSTANTFRACTIONOFERRORS expanderifforallsets ofvariablenodeswhere ,atleast checknodesareincidentto .Withthisdetion,ourmaintheoremisgivenexplicitlyasfollows.Theorem1: beanLDPCcodewithlength andrateatleast describedbyaTannergraph with variable checknodes,andregularleftdegree .Suppose isa -expander,where and isaninteger.ThentheLPdecodersucceeds,aslongasatmost bitsareippedbythechannel.RandomTannergraphswillmeettheconditionsofthisthe-oremwithhighprobability,andrecentworkbyCapalboetal.[4]givesefcientdeterministicconstructionsofsuchgraphs.TheproofofTheorem1isbasedonshowingthatwheneverthenumberoferrorsinthechannelisbounded,andthegraphex-pandssufciently,wecanndadualwitness:azero-valueddualsolutiontothedecodingLP.ThisdualsolutionimpliesthatthetransmittedcodewordisoptimalfortheprimalLP,andsoLPdecodingsucceeds.ApreliminaryversionofthispaperwaspresentedattheISIT2004symposium[23].Wealsonotethatsometechniquesde-velopedinthispaperhavesincebeenusedtoobtainresultsformoregeneralexpandercodes[24].A.RelatedWork:LDPCCodesThebit-ippingalgorithmforexpandercodesdevelopedbySipserandSpielman[25]hasatheoreticalperformanceguar-anteesimilartoTheorem1.Infact,whentheexpansionparam- equals ,theerror-correctionguaranteegiveninThe-orem1forLPdecodingmatchestheSipserSpielmanboundexactly.BurshteinandMiller[26]alsousegraphexpansiontoanalyzetheperformanceofvariousrelatediterativealgorithms.Withrespecttothefractionoferrorscorrectedonregulargraphs,ourresultsareroughlythesameastheirs;specically,for closeto ,bothresultsshowtheabilitytocorrect(closeto) er-rors,where istheexpansionparameter.Lentmaieretal.al.analyzeiterativecodingforLDPCcodes(aswellasotheren-sembles),andestablishthatundersuitabletechnicalconditions,theerrorprobabilitydecaysas forsomeconstant TheLPdecoderaswellasiterativealgorithmssuchassum-productandmin-sumhavetheadvantagethattheyapplyinmoregeneralsettingssuchastheadditivewhiteGaussiannoise(AWGN)channel,andexploitsoftinformationthechannel.Althoughthisentailsanincreaseinrunningtime,ourpreliminaryexperimentsindicatethatevenforthebinary-symmetriccase,LPdecoding(andotheralgorithmsthatexploitsoftinformation)performsignicantlybetterthantheippingalgorithmofSipserandSpielman[25].B.RelatedWork:OtherCodesBuiltFromExpandersExpandersalsoplayaroleingeneralizedLDPCconstruc-tions.Zmor[28]andBargandZmor[29][31]haveaseriesofpapersanalyzingexpandercodeswherethechecknodesareallowedtorepresentarbitrarylinearsubcodes(asopposedtosingleparitychecksinthecaseofastandardLDPCcode).Theyshowthatsuchcodes,togetherwithefcientmessage-passingalgorithms,canachievethecapacityoftheBSC,aswellascor-rectadversarialerrorsupto(andbeyond)theZyablovbound.GuruswamiandIndyk[32](aswellasRothandSkatchek[33])codesusingexpanders.Inlaterwork[34],theyachievetheGilbertVarsharmov(GV)boundforlowThenaturalLPdecoderforthesecodesisstrongerthantheoneobtainedbyreducingthecodetoanLDPCcodeandap-plyingthetree-basedLP[1],[2]totheassociatedfactorgraph;therefore,theresultsinthispapershouldnotbecomparedtothesemorepowerfulcodes.Furthermore,sincethepreliminaryversionofthiswork[23],wehaveshownthatLPdecodingcanachievechannelcapacity[24],usingafamilyofexpandercodesalongthelinesof[29].Whilethesemoresophisticatedexpander-basedconstructionsyieldstrongertheoreticalboundsonerrorcorrectionthanthoseknownforLDPCcodes,thecodesthemselvesaremostlyimpracticalforuseincommunicationsystems,duetotheirdependenceonlargesubcodes.(Thesizeofthesesubcodesisoftenexponentiallylargein ,where isthegapbetweenthecoderateandthedesiredbound.)There-fore,thestudyofLDPCcodesisofindependentinterest.C.OutlineTheremainderofthispaperisorganizedasfollows.InSec-tionII,weprovidebackgroundonLDPCcodes,andtheassoci-atedLPdecoderfrom[2].InSectionIII,weshowhowtoproveanerrorboundusingadualwitness;itisworthnotingthatthismethodappliestoanyLPdecoder,notjusttheoneforLDPCcodes.SectionIVisdevotedtotheproofofourmainresultusingtheexpansionoftheTannergraph.WeconcludewithsomeremarksandopenquestionsinSectionV.IntheAppendix,weshowthatgraphswithsufcientexpansionexistandcanbecon-structedefII.BACKGROUNDWebeginbyprovidingbackgroundonLDPCcodes,aswellasLPdecodingappliedtothem[2].A.LDPCCodes and beindicesforthecolumns(respectively,rows)ofthe parity-checkmatrix ofabinarylinearcode withrateatleast .TheTannergraphrepresentationofthecode isabipar-titegraph withnodesets and ,andedges variablenode andchecknode forall where .Iftheparity-checkmatrixhasaboundednumber(independentof )ofnonzeroentriesineachcolumn,wesaythatithas;thisconditiontranslatestoeachnodein havingboundeddegree.Inthispaper,wedonotrequirethatthechecknodeshaveboundeddegree.Thecodecanbevisualizeddirectlyfromthegraph .Imagineassigningtoeachvariablenode avaluein ,representingthevalueofaparticularcodebit.Aparity-checknode ifthebitsassignedtothevariablenodesinitsneighbor-hoodhaveevenparity(sumtozero ).The bitsassignedtothevariablenodesformacodewordifandonlyifallchecknodesaresatisWeassumethatthegraph isleft-regular;i.e.,thedegreeofeachvariablenode isexactlysomeconstant .Let IEEETRANSACTIONSONINFORMATIONTHEORY,VOL.53,NO.1,JANUARY2007denotetheneighborsofanodeset .Forasinglenode ,welet .Foreachcheck ,let even representsalocalcodeword;inotherwords,ifweseteachbitin to ,andallotherbitsin to ,thenwesatisfycheck .Let betheofnode ,where isthelog-likelihoodratioforthe thcodebit.WhentransmittingovertheBSCwithcrossoverprobability ,wemayrescalethelog-like-lihoodratiossuchthat ifa isreceivedfromthechannelforbit ,and ifa isreceived.Weassumethatthecodeword issentoverthechannel;thisassump-tionisvalidsincethepolytopeforLDPCcodes[2]is foranybinary-inputoutput-symmetric(BIOS)channel;seeFeldmanetal.[3]forfurtherdetails.Therefore,fortheBSCwithcrossoverprobability ,wehave withprobability ,and otherwise.Foraparticularsettingofthecostvector ,let bethesetofnegative-costvariablenodes.B.TheLPDecoderforLDPCCodesrst-orderLPdecoderforLDPCcodes[2]hasanLPvariable foreachnode ,indicatingthevalueofthe codebit.Inaddition,foreachparitycheck andeachset thereisanLPvariable ,whichservesasanindicatorforusingthelocalcodeword tosatisfy .Notethatthevariable isalsopresentforeachparitycheck,andrepresentssettingallbitsin tozero.WenowgivethedecodingLPalongwithitsdual,whichweuseinthenextsection: minimize suchthat (1a) edges (1b) (1c) maximize suchthat (2a) (2b) free edges free (2c) Notethattheconstraints and areimpliedbytheotherconstraints,assumingthateverybitisconnectedtoatleastonecheck.Let bethesettingofthe variablesappropriateforwhen ;i.e.,forall and ,wehave if ,and Thedecodingalgorithmworksasfollows.First,wesolvethedecodingLPtoobtainanoptimalsolution .If ,then mustrepresentthemaximum-likelihood(ML)codeword[3].Inthiscase,weoutput ;otherwise,ifsome hasafractionalvalue,wedeclareanerror.OurLPdecoderwillsucceedif istheuniqueoptimumsolutionoftheLP.(Weremindthereaderofourpreviousassumptionthatwearesendingtheall-zeroscodeword .)AnimportantfactisthatthedecodingLPissolvableinpolynomialtimeevenifsomeofthechecknodeshavelargedegree;wereferthereadertothepapers[2],[18]fordetails.III.PROVINGRRORSINGAUALInordertoprovethatLPdecodingsucceeds,wemustshow istheuniqueoptimumoftheLP.Tobeconser-vative,weassumefailureintheeventthattheLPhasmultipleoptima,sothattheLPdecodersucceedsifandonlyif theuniqueoptimumsolution.ConsiderthedualofthedecodingLPgivenabove.IfthereisafeasiblepointofthedualLPthathasthesamecost(i.e.,zero)asthepoint hasinthede-codingLP,then isalsoanoptimalpointofthedecodingLP.Therefore,usingstandardresultsonLPduality[35],inordertoprovethattheLPdecodersucceeds,itsufcestoexhibitazero-costpointinthedual.Actually,sincetheexistenceofthezero-costdualpointonlyprovesthat isoneofpossiblymanyprimaloptima,weneedtobeabitmorecareful;inpartic-ular,wegiveadualfeasiblepointthatisstrictlyboundedawayfromitscostconstraints(2b),whichimpliesusingcomplemen-taryslackness[35]that istheuniqueoptimalsolutiontotheLP.Wecallsuchadualpointadualwitness.Thisargu-mentismadepreciseintheupcomingproof.Werefertothevalues edgeweights.Thefollowingnitionunderliesasufcientconditionforauniquezero-costdualsolution:DeÞnition1:Asettingofedgeweights ifi)forallchecks anddistinct ,wehave ,andii)forallnodes ,wehave Proposition2:Ifthereisafeasiblesettingofedgeweights,thenthepoint istheoptimumofthedecodingProof: beafeasiblesettingofedgeweights.Taking forall givesazero-costdualsolution;itiseasilyveriedthatthissolutionsatisesthedualconstraints(2a)and(2b)byapplying,respectively,conditionsi)andii)fromDetion1.(For(2a),notethatwhen ,theconstraintdescribedby(2a)isredundantforall where .)Itfollowsfromtheprecedingdiscussionthat isoptimalforthecost inthedecodingLP.Wenowshowthat istheuniqueoptimum.Thestrictinequalityinpartii)ofDenition1impliesthat forsomepositivenumber ,fromwhichitfollowsthat isanoptimalpointofthedecodingLPunderthecost where forall Nowsuppose isnottheuniqueLPoptimumundertheoriginalcostfunction .Since istheonlyfeasiblesettingofthe variableswhen ,theremustbesomeother etal.:LPDECODINGCORRECTSACONSTANTFRACTIONOFERRORSfeasiblepoint where and .But ,wehave ,whichcontradictsthefact isoptimalunder . TheprecedingresultcaneasilybegeneralizedtoanyLPdecoder,wherethedualwitnesstakesonadifferentformde-pendingonthestructureofthecodeandtheLPrelaxation.Infact,avariantofthisideawasexploredinpreviouswork[36]inthecontextofturbocodes,andhasrecentlybeenexploredformoregeneralexpandercodes[24].Atonelevel,tryingtondadualwitnessissimplyareformu-lationoftheproblemoftryingtoprovethatthetransmittedcode-wordistheoptimalprimalLPsolution.Thevalueoflookingatthedualliesintheanalyticalexibilitythatitaffordsinpartic-ular,anabilitytotradeofferrorboundqualityforeaseofanal-ysis.Take,forexample,theextremecaseinwhichthechannelisnoiseless,sothat forall .Inthiscase,ndingadualwitnessreducestondingafeasiblesettingofedgeweights,andisveryeasy:simplysetall .Ingeneral,asthenoiseincreases(andhencemorebitsgetipped),itbecomesincreas-inglydifculttondadualwitness.IV.UXPANSIONTOINDAUALThissectionisdevotedtotheproofofourmainresult,pre-viouslystatedasTheorem1.Webeginbydeningaprocedureforassigningfeasibleedgeweights ,whichthenallowsustoapplyProposition2.Ourprocedureusesaspecialsubsetofedgescalleda matching,denedinSectionIV-A.The -matchingisafunctionoftheerrorpatternreceivedfromthechannel.InSectionIV-B,weshowthatifa exists,thenwecanndafeasibleassignmentofedgeweights.InSectionIV-C,weprovethata -matchingdoesindeedexistaslongasthenumberofbitsippedbythechannelisatmostaconstantfractionof ,wheretheconstantdependsontheexpansionpropertiesofthegraph.WenishtheproofofThe-orem1inSectionIV-D.A.DenitionandNotationFortheremainderofthissection,let beaTannergraph variablenodeseachofdegree ,andmoreoverlet bean -expander,where and isaninteger.Wealsoxthefollowingparametersandsets,whichareimplicitfunctionsof and/orthecostvector .Let Notethat ,andthat isaninteger.De ,andlet bethesetofpositive-costvariablenodesoutside thathavemorethan neighborsin (i.e., Finally,wede nition2: matching isasubset oftheedgesincidentto suchthati)everycheckin isinci-denttoatmostoneedgeof ,ii)everynodein isincidenttoatleast edgesof ,andiii)everynodein isincidenttoatleast edgesof B.AssigningWeightsUsinga -MatchingWegiveourweightassignmentschemeinthefollowingthe-orem(alsoinFig.1).Theexistenceofsuchanassignmentim-pliesdecodingsuccess,byProposition2. Fig.1.AweightingschemethatsatisestheLPdualconstraints.Givenanerrorset ,weletbethenodesnotinmorethanneighborsin,andlet _U.Thematching(solidedges)containsatmostoneedgeincidenttoeachchecknode,atleastedgesincidenttoeachnodein,andatleastedgesincidenttoeachnode.Forallsuchthat,weset ,and =xforalli .Forallother,wesetall Proposition3:Ifthereisa -matchingof ,thenthereisafeasibleedgeweightassignment.Proof:Callachecknode in activatedif isincidenttoanedge of ,and .Notethatanactivatedcheckisincidenttoexactlyoneedgeof ,bythedenitionof .Weassignedgeweightsasfollows(seealsoFig.1),usingapositive thatwedenelater.Forallactivatedchecks ,wehave forsome ,and forallother .Set ,andset forallother Forallotherchecks,setallincidentedgeweightstozero.Thisweightingclearlysatisesconditioni)ofafeasibleweightassignment.Forconditionii),wedistinguishthreecases.Forthefollowingargument,notethatalledgesin incidenttonodes receiveweight ,allotheredgesin receiveweight andalledgesnotin receiveweighteither or 1)Foravariablenode ,wehave .Also,atleast oftheedgesincidentto arein (andeachhasweight ).Allotherincidentedgeshaveweighteither or Ineithercase,eachhasweightatmost ,andsothetotalweightofincidentedgesisatmost .Thisislessthan aslongas 2)If ,then .Atleast of sincidentedgesarein ,but(trivially)notincidentto ;theseedgeshave .Allotherincidentedgeshaveweighteither or .Ineithercase,theyeachhaveweightatmost ,andsothetotalweightofincidentedgesisatmost whichislessthan aslongas 3)Theremainingcaseiswhen ,andinthiscase .Thedenitionof impliesthat hasatleast neighborsnotin ,andsoatmost incidentto havenonzeroweight.Wearethereforeinthesamesituationasinthepreviouscase:allnonzeroweightsareatmost ,andsothetotalweightofincidentedgesisatmost ,whichislessthan aslongas Summarizingourconditionson ,wehave IEEETRANSACTIONSONINFORMATIONTHEORY,VOL.53,NO.1,JANUARY2007 Fig.2.Aninstanceofmax-owusedtoshowthatgraphexpansionimpliesa;-matching(Proposition4).Alledgeshaveunitcapacity,excepttheedgesleavingthesource,whichhavecapacityThereisafeasible satisfyingtheseconditionsaslongas ,whichistruebythedenitionof . C.ExpansionImpliesa -MatchingToconstructourfeasibleweightassignment,itremainstoshowthatwecanconstructa -matching.Todoso,weusetheexpansionofthegraph.Proposition4: isa -expanderwith ,and ,then hasa Proof:Weconstructthe -matching bysettingupaowinstance(seethebooks[37],[38]forbackgroundonow).Wewillconstructthisowinstanceusingthevari-ablenodes ,thechecknodes ,anddirectedversionsoftheedgesincidentto .Wewillalsointroducetwonewnodes(asourceandasink),aswellasedgesincidenttothosenodes.Weconstructtheowinstanceasfollows(seeFig.2),withallintegercapacities:Foreveryedge in where and ,makeadirectededge withcapacity Createasource ,andmakeanewedgewithcapacity from toeveryvariablenode.Createasink ,andmakeanewedgewithcapacity fromeachchecknode to Weclaimthatifthereexistsaowofvalue inthisinstance,thenthereisa -matching .Let beaowofvalue ;withoutlossofgenerality,wemayassume integral[38].Weset tobethesetoforiginaledges(from to )withunitowin .Since hasvalue ,everyedgeoutofthesource tothenodesof mustbesaturated.Itfollowsthatexactly edgesoutofeach haveaunitofowin .Thus, esconditionii)ofa andsince ,theset ismorethansufcienttosatisfyconditioniii)aswell.Theedgesfromeachcheck thesinkhavecapacity ,andsoatmostoneincomingedgetoeachcheckiscarryingowin .Itfollowsthatatmostoneedge isincidenttoeachcheckin ,andthus, conditioni)ofa Soitremainstoshowthatthereexistsaowofvalue orequivalently[38]thattheminimum - cutisatleast .Let describetheminimum - cutasfollows: and arethevariableandchecknodes,respectively,onthesamesideofthecutasthesource .Similarly, and arethevariableandchecknodes,respectively,onthethesamesideofthecutasthesink .Thisminimum - cutisdepictedinFig.3.Fornodesets and ,let denotethetotalcapacityofedgesgoingfrom to .Thevalueoftheminimum - cutis Fig.3.Aminimum C ;V ;C intheowgraphofFig.2.Oneofeachedgetypeisshown,andtheedgesthatcontributetothesizeofthecutareshownwithsolidlines.Everyedgehasunitcapacityexceptthoseleavingthesource,whichhavecapacityexactly .Notethat ,and Weclaimthatwithoutlossofgenerality,therearenoedgesintheminimum - cutfrom to ;i.e., .Toseethis,consideranedge ,where and .Ifwemove tothesourcesideofthecut,thenweaddatmost tothecutvalue,sincetheonlyedgeleaving istheonetothe .However,wealsosubtractatleast fromthecutvalue,becausetheedge isnolongerinthecut.So,wehavethattheminimum - cuthasvalue (3a) where(3a)followsfrom (sincetherearenoedges to ),and(3b)followsfromtheexpansionof . WenotethatwehaveessentiallytakentheLPdualtwice:onceinreasoningaboutadualwitness,andthenagainbyap-plyingthemax-owmin-cuttheorem.Itmightbeinterestingtoseeamoredirectconstructionofthematching.ProofofOurMainTheoremBeforeproceedingtotheproofofTheorem1,werequirethefollowing.Lemma5: ,where .Then,wehave Proof:Assumetothecontrarythat .Thenthereissomesubset where .Considerthe .Since wehave byourassumptionon .Therefore,thissetexpands,andwehavei): Furthermore,wehave etal.:LPDECODINGCORRECTSACONSTANTFRACTIONOFERRORSConsidertheset .Thesearetheedgesfrom arenotincidentto .Eachnodein hasatmost suchedges,bythedenitionof .Therefore, andwehaveii): Combiningtheinequalitiesi)andii)andusingthede ,weobtain whichisacontradiction. Wearenowreadytoproveourmaintheorem.Theorem1: beaLDPCcodewithlength andrateat describedbyaTannergraph with variable checknodes,andregularleftdegree .Suppose isa -expander,where and isaninteger.ThentheLPdecodersucceeds,aslongasatmost bitsareippedbythechannel.Proof:Byassumption, andso,byLemma5,wehave .Thisimplies .Therefore,byProposition4,thereexistsa -matchingof ,andsobyProposition3thereexistsafeasibleweightassignment.UsingProposition2,weconcludethat istheuniqueoptimumoftheLP,andsothedecodersucceeds. Foranyconstantratebetween and ,arandomgraphwillmeettheconditionsoftheabovetheoremforsome asrequiredandsomeconstant ;alsoexplicitfamiliesofsuchgraphscanbeconstructedefciently(wediscussthismoreintheAp-pendix).AsanexampleofTheorem1,letusset .Using -expander,Theorem1assertsthattheLPdecoderwillsucceediffewerthan bitsareippedbythechannel.Interestingly,thisresultmatchestheparametersofthestatementgivenbySipserandSpielman[25]intheoriginalpaperonex-pandercodes(i.e.,decodingsuccessiffewerthan usingan -expander).V.CWehavegiventherststrongWERboundforLPdecoding;furthermore,thisboundisbetterthananynite-lengthboundknownfortheconventionalmessage-passingdecoders.Thispaperraisesanumberofopenquestions.ItwouldbeinterestingtoseeanimprovementintheresultsforLDPCcodes.Thefractionoferrorprovedhere( ,seeAppendix)isquitefarfromtheperformanceofLDPCcodesobservedinpractice.Also,constraining tobeanintegercouldrequirearatherlargedegree.Bothoftheseproblemsarearesultoftheparticularmethodwefoundforconstructingadualwitness,andarenotnecessarilyadeciencyinLPdecodingitself.Onecouldimproveourresultsbyamorecarefulweightingscheme,perhapsusinggraphstructuresthataremorelocalizedthansetexpansion.Thenextlogicalstepistoadaptourtechniquestodifferentcodesandchannels.Theideaofconstructingadualsolutionwithvaluezerotoprovedecodingsuccessappliestoany [3]LPdecoderandmemorylesssymmetricchannel(suchastheAWGNchannel).Inafollow-uptothiswork,asubsetofthecurrentauthorshasshown[24],usingadualwitness,thatLPdecodingwithexpandercodescanachievethecapacityofanymemorylesssymmetricchannelinwhichthebitwiselog-likelihoodratioisboundedbysomeconstant.Additionally,aboundisprovedfortheadversarialchannelthatisstrongerthantheonegivenhere.Itshouldbenoted,however,thatexpandercodesaremuchlesspracticalthanLDPCcodes.Expandercodesmighthaveveryalbeitconstantvariabledegree,andthusarenotusefulforsmallnitelengths.Turbocodespresentanotherpromisingapplicationofthetechniquesdevelopedinthispaper.Sincethedistanceofturbocodesisingeneralsublinear[39],onecannotproveafractionoferrorresult.However,provingthattheWER(oftheturbocodeLPdecoderin[17],[18])goestozeroastheblocklengthincreases(forreasonablyhighrates)wouldbeasignicantresult.Asfarastheauthorsareaware,nosuchnite-lengtherrorboundisknownforturbocodesofanyconstantrate(otherthantheresultsin[17],[22],[21]forthe cyclecodeacodewithlogarithmicdistance).XISTENCEANDONSTRUCTIONOFXPANDERSInthisappendix,wegivetheoremsshowingthatthereexistfamiliesofexpandergraphs,andweciteresultsprovingthattheseexpanderscanbeconstructedefciently.Wegivethesere-sultssimplyforthesakeofcompletenessofourmainresultthatLDPCcodeswithLPdecodingcancorrectaconstantfractionoferrors.Theresultinggraphswillhavelargedegreerequirementsandsmallerror-correctingcapability,andsoimprovingthesere-sultsisanimportantsteptowardmakingthesecodespractical.A.ExpansionFromRandomGraphsUsingtheprobabilisticmethod,onecanshowthefollowing.Proposition6: and beanyxedconstants,andlet besuchthat isanintegerwhichisat .Thenforany suchthat thereisaTannergraphwith variablenodes, checknodes,andregularleftdegree whichisa -expander,where Proof:Weconsiderrandom -bipartitegraphswhichareformedasfollows.For thvariablenodeuniformlypicksa -elementsubsetof andformsedgestothesecheckAnygraphformedthiswayis -regularontheleft.Welet denote ,theaveragedegreeofthechecknodes.Wenoterstthateachsetconsistingofasinglevariablenodeclearlyexpandsbyafactorofexactly .Nowxavalue ,aset IEEETRANSACTIONSONINFORMATIONTHEORY,VOL.53,NO.1,JANUARY2007 ofleft-verticeswhere ,andaset ofright-verticesofsize (notethat isaninteger).Foreachindividualvertexin ,theprobabilitythatall ofitsneighborsliein is Sinceeachleft-vertexchoosesitsneighborsindependentlyoftheotherleft-vertices,theprobabilitythat isatmost .Sincethereare sets of left-verticesand sets of right-vertices,theprobabilitythatsetof left-verticeshasitsneighborhoodofsizeatmost isatmost (5)Let and ,so(5) .Itiseasilycheckedthatfor thequantity isatmost ,andthuswehave Thuswithprobabilityatleast ,wehavethatarandomgraph formedasdescribedaboveisa -expanderfor .Plugginginfor and andrecallingthat thepropositionisproved. TogetherwithTheorem1,Proposition6impliesthatthereareLDPCcodesofanyconstantrateforwhichLPdecodingcorrectsaconstantfractionoferror.Asaconcreteexample,ifwetake , ,and ,wehavethatthereisafamilyofLDPCcodesofrate forwhichLPdecodingcan fractionoferrors.WenotethatamorecarefulanalysisoftherandombipartitegraphsusedtoproveProposition6givesastrongerboundon butthisbounddoesnothaveaconvenientclosedform.UsingthisstrongerbounditcanbeshownthatforthespecicfamilyofLDPCcodesdescribedabove(with , ,and )LPdecodingcancorrect fractionoferrors.Toseethis,notethattheproofofProposition6impliesthattheprobability(overourchoiceofarandomgraph)thatanysetofsizeupto failstoexpandisatmost ,where isde-nedin(4).Usingadifferentboundonbinomialcoefwecanshowthatforsome tobedescribedbelow,allsetsofsize failtoexpandwithexponentiallylowprob-ability,where isanyxedconstantvalueintheopeninterval .Combiningthesefacts,wehavethatarandom isa -expanderwithprobabilityatleast Inordertoobtainthesharperresult,wenowapplythefol-lowingentropybound[40]onthebinomialcoef whichisatighterboundthanthepreviouslyused bound.Using , and thisgives Thus,if isanyconstantvaluesuchthat wethenhavethat,withprobability ,allsetsofsize satisfytherequiredexpansion.Inequality(6)doesnotseemtoyieldaniceclosed-formexpressionfor .However,onecanverifythat,e.g.,for , ,and ,anyvalue causes(6)tobenegative.ThisgivesthestrongerLPdecodingperformanceboundclaimedearlier.B.ExplicitConstructionsofExpandersRecently,Capalboetal.[4]gavetherstexplicitconstruc-tionofexpanders(namely,with arbitrarilycloseto ),usingthezig-zaggraphproduct[41]throughtheframeworkofrandomnessconductors.Theirworkimpliesthefollowing.Proposition7: and anyxedconstants.Thenforany suchthat thereisanefcientlyconstructibleTannergraphwith vari-ablenodes, checknodes,andregularleft-degree whichis -expander,where ,and Thus,thereareefcientlyconstructibleLDPCcodesofanyconstantrateforwhichLPdecodingcorrectsaconstantfractionoferrors.Notethatwhiletheabovepropositiondoesnotdirectly tobeaninteger,thisisnotaproblemsincegivenany thereissome suchthat and isaninteger(notethatany -expanderisclearlyalso -expanderforany ).Thus,inordertoapplyTheorem1,itissufcienttochoosesome fortheCapalboetal.CKNOWLEDGMENTWewouldliketothankG.DavidForney,Jr.,DavidKarger,RalfKoetter,andPascalVontobelforhelpfuldiscussions.Inad-dition,wethanktheanonymousreviewersfortheirconstructivesuggestionsthathelpedtoimprovethepaper.[1]J.Feldman,M.J.Wainwright,andD.R.Karger,Usinglinearpro-grammingtodecodelinearcodes,Proc.37thAnnu.Conf.Informa-tionSciencesandSystems(CISS,Baltimore,MD,Mar.2003.2003.ÑÑ,ÒUsinglinearprogrammingtodecodebinarylinearcodes,Trans.Inf.Theory,vol.51,no.3,pp.954972,Mar.2005.[3]J.Feldman,D.R.Karger,andM.J.Wainwright,LPdecoding,Proc.41stAnnu.AllertonConf.Communication,Control,andCom-,Monticello,IL,Oct.2003.[4]M.Capalbo,O.Reingold,S.Vadhan,andA.Wigderson,nessconductorsandconstant-degreeexpansionbeyondthedegree/2barrier,Proc.34thACMSymp.TheoryofComputing,Montreal,Canada,May2002,pp.659[5]R.Koetter,2002,personalcommunication. etal.:LPDECODINGCORRECTSACONSTANTFRACTIONOFERRORS[6]R.KoetterandP.O.Vontobel,Graph-coversanditerativedecodingofnitelengthcodes,Proc.3rdInt.Symp.TurboCodes,Brest,France,Sep.2003.[7]C.Berrou,A.Glavieux,andP.Thitimajshima,NearShannonlimiterror-correctingcodinganddecoding:Turbo-codes,Proc.IEEEInt.Conf.Communications,Geneva,Switzerland,May1993,pp.[8]R.Gallager,Low-densityparity-checkcodes,IRETrans.Inf.Theoryvol.IT-8,pp.2128,Jan.1962.[9]T.RichardsonandR.Urbanke,Thecapacityoflow-densityparity-checkcodesundermessage-passingdecoding,IEEETrans.Inf.Theory,vol.47,no.2,pp.599618,Feb.2001.[10]T.Richardson,A.Shokrollahi,andR.Urbanke,Designofcapacity-approachingirregularlow-densityparitycheckcodes,IEEETrans.Inf.Theory,vol.47,no.2,pp.619637,Feb.2001.[11]M.Luby,M.Mitzenmacher,A.Shokrollahi,andD.Spielman,provedlow-densityparity-checkcodesusingirregulargraphsandbe-liefpropagation,Proc.1998IEEEInt.Symp.InformationTheoryCambridge,MA,Oct.1998,p.117.[12]R.M.Tanner,Arecursiveapproachtolowcomplexitycodes,Trans.Inf.Theory,vol.IT-27,no.5,pp.533547,Sep.1981.[13]C.Di,D.Proietti,T.Richardson,E.Telatar,andR.Urbanke,lengthanalysisoflow-densityparitycheckcodes,IEEETrans.Inf.,vol.48,no.6,pp.15701579,Jun.2002.[14]N.Wiberg,CodesandDecodingonGeneralGraphs,Ph.D.disserta-tion,LinkpingUniv.,Linkping,Sweden,1996.[15]G.D.Forney,Jr.,R.Koetter,F.R.Kschischang,andA.Reznik,theeffectiveweightsofpseudocodewordsforcodesdenedongraphswithcycles,Codes,SystemsandGraphicalModels.NewYork:Springer,2001,pp.101[16]B.Frey,R.Koetter,andA.Vardy,Signal-spacecharacterizationofit-erativedecoding,IEEETrans.Inf.Theory,vol.47,no.2,pp.766Feb.2001.[17]J.FeldmanandD.R.Karger,Decodingturbo-likecodesvialinearProc.43rdAnnu.IEEESymp.FoundationsofCom-puterScience(FOCS),Vancouver,Canada,Nov.2002.[18]J.Feldman,DecodingError-CorrectingCodesviaLinearProgram-Ph.D.dissertation,MIT,Cambridge,MA,2003.[19]M.J.Wainwright,T.S.Jaakkola,andA.S.Willsky,MAPestimationviaagreementon(hyper)trees:Message-passingandlinearprogram-mingapproaches,Proc.40thAnnu.AllertonConf.Communication,ControlandComputing,Monticello,IL,Oct.2002.2002.ÑÑ,ÒExactMAPestimatesviaagreementon(hyper)trees:Linearprogrammingandmessage-passing,IEEETrans.Inf.Theory,vol.51,no.11,pp.36973717,Nov.2005.[21]N.HalabiandG.Even,Improvedboundsontheworderrorproba-bilityofRA(2)codeswithlinear-programming-baseddecoding,Trans.Inf.Theory,vol.51,no.1,pp.265280,Jan.2005.[22]G.EvenandN.Halabi,ImprovedboundsontheworderrorprobabilityofRA(2)codeswithlinearprogrammingbaseddecoding,Proc.41stAnnu.AllertonConf.Communication,Control,andComputingMonticello,IL,Oct.2003.[23]J.Feldman,T.Malkin,R.A.Servedio,C.Stein,andM.J.Wainwright,LPdecodingcorrectsaconstantfractionoferrors,Proc.IEEEInt.Symp.InformationTheory,Chicago,IL,Jun./Jul.2004,p.68.[24]J.FeldmanandC.Stein,LPdecodingachievescapacity,Proc.Symp.DiscreteAlgorithms(SODA,Vancouver,Canada,Jan.2005.[25]M.SipserandD.Spielman,Expandercodes,IEEETrans.Inf.,vol.42,no.6,pp.17101722,Nov.1996.[26]D.BurshteinandG.Miller,Expandergraphargumentsformes-sage-passingalgorithms,IEEETrans.Inf.Theory,vol.47,no.2,pp.790,Feb.2001.[27]M.Lentmaier,D.V.Truhachev,K.S.Zigangirov,andD.J.Costello,AnanalysisoftheblockerrorprobabilityperformanceofiterativeIEEETrans.Inf.Theory,vol.51,no.11,pp.3834Nov.2005.[28]G.Zmor,Onexpandercodes,IEEETrans.Inf.Theory,vol.47,no.2,pp.835837,Feb.2001.[29]A.BargandG.ZErrorexponentsofexpandercodes,Trans.Inf.Theory,vol.48,no.6,pp.17251729,Jun.2002.2002.ÑÑ,ÒErrorexponentsofexpandercodesunderlinear-complexityde-SIAMJ.Discr.Math.,vol.17,no.3,pp.426445,2004.2004.ÑÑ,ÒConcatenatedcodes:Serialandparallel,IEEETrans.Inf.,vol.51,no.5,pp.16251634,May2005.[32]V.GuruswamiandP.Indyk,Near-optimallinear-timecodesforuniquedecodingandnewlist-decodablecodesoversmalleralpha-Proc.34thAnnu.Symp.TheoryofComputing(STOC)Montreal,Canada,May2002.[33]R.M.RothandV.Skachek,Onnearly-MDSexpandercodes,Proc.IEEEInt.Symp.InformationTheory,Chicago,IL,Jun.2004,p.[34]V.GuruswamiandP.Indyk,cientlydecodablelow-ratecodesmeetingtheGilbertVarshamovbound,Proc.ACM-SIAMSymp.DiscreteAlgorithms(SODA),NewOrleans,LA,Jan.2004.[35]A.Schrijver,TheoryofLinearandIntegerProgramming.NewYork:Wiley,1987.[36]J.Feldman,D.R.Karger,andM.J.Wainwright,Linearprogram-mingbaseddecodingofturbo-likecodesanditsrelationtoiterativeProc.40thAnnu.AllertonConf.Communication,Con-trol,andComputing,Monticello,IL,Oct.2002.[37]T.Cormen,C.Leiserson,R.Rivest,andC.Stein,IntroductiontoAl-.Cambridge,MA:MITPress,2001.[38]R.K.Ahuja,T.L.Magnanti,andJ.B.Orlin,NetworkFlows.Engle-woodCliffs,NJ:Prentice-Hall,1993.[39]L.Bazzi,M.Mahdian,S.Mitter,andD.Spielman,Theminimumdistanceofturbo-likecodes,unpublishedmanuscript,2001.[40]T.CoverandJ.Thomas,ElementsofInformationTheory.NewYork:Wiley,1991.[41]O.Reingold,S.Vadhan,andA.Wigderson,Entropywaves,thezigzaggraphproduct,andnewconstant-degreeexpandersandextractors,Proc.41stIEEESymp.FoundationsofComputerScience(FOCS),Re-dondo,CA,Nov.2000,pp.3