Contents1Introduction11.1Overview..................................... - PDF document

Contents1Introduction11.1Overview..........................................11.2Notationandterminology.................................41.3Ourresults.........................................61.3.1Randomerasures-theBECchannel.......................61.3.2Weightdistributionandlistdecoding......................71.3.3Randomerrors-theBSCchannel........................71.4Prooftechniques......................................81.5Relatedliterature......................................101.6Organization........................................112Preliminaries112.1Basiccodingdenitions..................................112.2Equivalentrequirementsforprobabilisticerasures....................142.3BasicpropertiesofReed-Mullercodes..........................163WeightdistributionofReed-Mullercodes174RandomsubmatricesofE(m;r)204.1RandomsubmatricesofE(m;r),forsmallr,havefullcolumn-rank..........224.2RandomsubmatricesofE(m;r),forsmallr,havefullrow-rank............255Reed-Mullercodeforerasures275.1Low-rateregime......................................275.2High-rateregime......................................276Reed-Mullercodeforerrors286.1Low-rateregime......................................286.2High-rateregime......................................306.2.1Paritycheckmatrixandparityofpatterns....................316.2.2Thecaser=1...................................326.2.3Thedegree-rcase..................................346.2.4Ageneralreductionfromdecodingfromerrorstodecodingfromerasures..366.2.5Thedegree-2counterexample...........................377Futuredirectionsandopenproblems38AProofsofClaim4.15andClaim5.342BAproofofLemma4.10usinghashing43 1Introduction1.1OverviewWestartbygivingahighleveldescriptionofthebackgroundandmotivationfortheproblemswestudy,andofourresults.Reed-Muller(RM)codeswereintroducedin1954,rstbyMuller[Mul54]andshortlyafterbyReed[Ree54],whoalsoprovidedadecodingalgorithm.Theyareamongtheoldestandsimplestcodestoconstruct;thecodewordsaretheevaluationvectorsofallmultivariatepolynomialsofagivendegreebound.Moreprecisely,inanRM(m;r)codeoveraniteeldF,amessageisinterpretedasthecoecientsofamultivariatepolynomialfofdegreeatmostroverF,anditsencodingissimplythevectorofevaluationsf(a)forallpossibleassignmentsa2Fmtothevariables.Thus,RMcodesarelinearcodes.Theyhavebeenextensivelystudiedincodingtheory,andyetsomeoftheirmostbasiccoding-theoreticparametersremainamysterytodate.Specically,xingtherateofanRMcode,whileitiseasytocomputeitstolerancetoerrorsanderasuresintheworst-case(oradversarial)model,ithasprovedextremelydiculttoestimatethistoleranceforeventhesimplestmodelsofrandomerrorsanderasures.Thequestionsregardingerasurescanbeinterpretedfromalearningtheoryperspective,aboutinterpolatinglowdegreepolynomialsfromlossyornoisyevaluations.ThequestionsregardingerrorsrelatesparserecoveryfromrandomBooleanerrors.Thispapermakessomeprogressonthesebasicquestions.Reed-Mullercodes(overbothlargeandsmallniteelds)havebeenextremelyin uentialinthetheoryofcomputation,playingacentralroleinsomeimportantdevelopmentsinseveralareas.Incryptography,theyhavebeenusede.g.insecretsharingschemes[Sha79],instancehidingconstructions[BF90]andprivateinformationretrieval(seethesurvey[Gas04]).Inthetheoryofrandomness,theyhavebeenusedintheconstructionsofmanypseudo-randomgeneratorsandrandomnessextractors,e.g.[BV10].Theseinturnwereusedforhardnessamplication,programtestingandeventuallyinvariousinteractiveandprobabilisticproofsystems,e.g.thecelebratedresultsNEXP=MIP[BFL90],IP=PSPACE[Sha92],NP=PCP[ALM+98].Incircuitlowerboundsforsomelowcomplexityclassesonearguesthateverycircuitintheclassisclosetoacodeword,soanyfunctionfarfromthecodecannotbecomputedbysuchcircuits(e.g.[Raz87].Indistributedcomputingtheywereusedtodesignfault-tolerantinformationdispersalalgorithmsfornetworks[Rab89].Thehardnessofapproximationofmanyoptimizationproblemsisgreatlyimprovedbythe\shortcode"[BGH+12],whichusestheoptimaltestingresultof[BKS+10].Andthelistgoeson.Needlesstosay,thepropertiesusedintheseworksarepropertiesoflow-degreepolynomials(suchinterpolation,linearity,partialderivatives,self-reducibility,heredityundervariousrestrictionstovariables,etc.),andinsomeofthesecases,speciccoding-theoreticperspectivesuchasdistance,unique-decoding,list-decoding,localtestinganddecodingetc.playimportantroles.Finally,polynomialsarebasicobjectstounderstandcomputationallyfrommanyperspectives(e.g.testingidentities,factoring,learning,etc.),andthisstudyinteractswellwiththestudyofcodingtheoreticquestionsregardingRMcodes.Todiscussthecoding-theoreticquestionswefocuson,andgiveappropriateperspective,weneedsomemorenotation.First,wewillrestrictattentiontobinarycodes,themostbasiccasewhereF=F2,theeldoftwoelements1.Toreliablytransmitk-bitmessagesweencodeeachbyann-bitcodewordviaamappingC:Fk2!Fn2.WeabusenotationanddenotebyCboththemappingand 1Thisseemsalsothemostdicultcaseforthesequestions,andweexpectourtechniquestogeneralizetolargerniteelds.1 constant,themostpopularregimeincodingtheory.TheconjecturethatRMcodesachievecapacityhasbeenexperimentally\conrmed"insimu-lations[Ari08,MHU14].Moreover,despitebeingextremelyold,newinterestinitresurgedafewyearsagowiththeadventofpolarcodes[Ari09].Toexplaintheconnectionbetweenthetwo,aswellassomeofthetechnicalproblemsarisinginprovingtheresultsabove,considerthefollowing2m2mmatrixEm(for\evaluation").Indextherowsandcolumnsbyallpossiblem-bitvectorsinFm2inlexicographicorder.InterpretthecolumnssimplyaspointsinFm2,andtherowsasmonomials(whereanm-bitstringcorrespondtothemonomialwhichistheproductofvariablesinthepositionscontaininga1).Finally,Em(x;y)isthevalueofthemonomialxonthepointy(namelyitis1ifthesetof1'sinxiscontainedinthesetof1'siny).Thus,everyrowofEmisthetruthtableofonemonomial.ItisthuseasytoseethatthecodeR(m;r)issimplythespanofthetop(or\highweight")krowsofEm,withk=�mr
;thesearethetruthtablesofalldegreerpolynomials.Incontrast,polarcodesofthesameratearespannedbyadierentsetofkrows,sotheyformadierentsubspaceofpolynomials.Whilethemonomialsindexingthepolarcoderowshavenoexplicitdescription(sofar),theycanbecomputedecientlyforanykinpoly(n)=2O(m)time.Itissomehowintuitively\better"topreferhigherweightrowstolowerweightonesasthebasisofthecode(asthe\chancesofcatchinganerror"seemhigher).Giventheamazingresultthatpolarcodesachievecapacity,thisintuitionseemstosuggestthatRMcodesdosoaswell.Infact,experimentalresultsin[MHU14]suggestthatRMcodesmayoutperformpolarcodesfortheBECandBSCwithmaximum-likelihood7decoding.DenotingbyE(m;r)thetopsubmatrixofEmwithk=�mr
rows,onecanexpresssomenaturalproblemsconcerningitwhichareessentialforourresults.Toobtainsomeofourresultsonachievingcapacityfortheerasurechannel,wemustunderstandthefollowingtwonaturalquestionsregardingE(m;r).First,whatisthelargestnumberssothatsrandomcolumnsofE(m;r)arelinearlyindependentwithhighprobability.Second,whatisthesmallestnumbertsuchthattrandomcolumnshavefullrow-rank.Capacityachievingforerasuresmeansthats=(1�o(1))kandt=(1+o(1))k,respectively.Weprovethatthisisthecaseforsmallvaluesofr.Thesecondpropertygivesdirectlytheresultforlow-ratecodesRM(m;r),andtherstimpliestheresultforhigh-ratecodesusingadualitypropertyofRMcodes.Bothresultsmaybeviewedfromalearningtheoryperspective,showingthatintheserangesofparametersanydegreerpolynomialinmvariablescanbeuniquelyinterpolatedwithhighprobabilityfromitsvaluesontheminimumpossiblenumberofrandominputs.Forerrors,furtheranalysisisneededbeyondtherankpropertiesdiscussedabove.Fromtheparity-checkmatrixviewpoint,decodingerrorsisequivalenttosolving(withhighprobability)anunderdeterminedsystemofequations.Recallthatalinearcodecanbeexpressedasthenullspaceofan(n�k)nparity-checkmatrixH.IfZisarandomerrorvectorwithabout(oratmost)sone'scorruptingacodeword,applyingtheparity-checkmatrixtothecodewordyieldsY=HZ,wherethe\syndrom"Yisoflowerdimensionn�k.DecodingrandomerrorsmeansreconstructingZfromYwithhighprobability,usingthefactthatZissparse(hencetheconnectionwithsparserecovery).Notehoweverthatthisdiersfromcompressedsensing,asZisrandomandHZisoverGF(2).ItrelatestorandomnessextractioninthatacapacityachievingcodeshouldproduceanoutputYofdimensionmnh(s=n)containing8alltheentropyofZ.Comparedtotheusualnotionofrandomnessextraction,thechallengehereistoextractwithaverysimplemapH(seedlessandlinear),whilethesourceZismuchmorestructured,i.e.ithasi.i.d.components,comparedto 7MLdecodinglooksforthemostlikelycodeword.FortheBEC,thisrequiresinvertingamatrixoverGF(2),whereasfortheBSC,MLcanbeapproximatedbyasuccessivelist-decodingalgorithm.8See[Abb11]forfurtherdiscussiononthis.3 matrixAandI[a],wedenotewithAI;
thematrixobtainedbykeepingonlythoserowsindexedbyI,anddenotesimilarlyA
;JforJ[b].Channels,capacityandcapacity-achievingcodesWenextdescribethechannelsthatwewillbeworkingwith,andprovideformaldenitionsinSection2.Throughoutpwilldenotethecorruptionprobabilitypercoordinate.TheBinaryErasureChannel(BEC)withparameterpactsonvectorsv2f0;1gn,bychangingeverycoordinateto\?"withprobabilityp.Thatis,afteramessagevistransmittedintheBECthereceivedmessage^vsatisesthatforeverycoordinateieither^vi=vior^vi=\?"andPr[^vi=\?"]=p.TheBinarySymmetricChannel(BSC)withparameterpis ipsthevalueofeachcoordinatewithprobabilityp.Thatis,afteramessagevistransmittedintheBSCthereceivedmessage^vsatisesPr[^vi6=vi]=p.Infact,wewilluseasmallvariationonthesechannels;forcorruptionprobabilitypwewillxthenumberoferasures/errorstos=pn.WenotethatbytheCherno-Hoedingbound(seee.g.,[AS92]),theprobabilitythatmorethanpn+!(p pn)erasures/errorsoccurforindependentBernoullichoicesiso(1),andsowecanrestrictourattentiontotalkingaboutaxednumberoferasures/errors.Thus,whenwediscussscorruptions,wewilltakethecorruptionprobabilitytobep=s=n.WerefertoSection2forthedetails.Wenowdenethenotionsof\capacity-achieving"forthechannelsabove.WeconsiderRM(m;r)wherer=r(m)typicallydependsonm.WesaythatRM(m;r)cancorrectrandomerasures/errors,ifitcancorrecttherandomerasures/errorswithhighprobabilitywhenntendstoinnity.Thegoalistorecoverfromthelargestamountoferasures/errorsthatisinformation-theoreticallyachievable.Wenotethatwhilerecoveringfromerasures,wheneverpossible,isalwayspossibleeciently(bylinearalgebra),thisneednotbethecaseforrecoveryfromerrors.Aswefocusontheinformationtheoreticlimits,weallowmaximum-likelihood(ML)decodingrule.Obtaininganecientalgorithmisamajoropenproblem.NotethatMLminimizestheerrorprobabilityforequiprobablemessages,henceifMLfailstodecodethecodewordswithhighprobability,nootheralgorithmscansucceed.Recallthatthecapacityofachannelisthelargestpossiblecoderateatwhichwecanrecover(whp)fromcorruptionprobabilityp.Thiscapacityisgivenby1�pforBECerasures,andby1�h(p)forBSCerrors.Namely,ShannonprovedthatforanycodeofrateRthatallowstocorrectcorruptionsofprobabilityp,thenR1�pfortheBECandR1�h(p)fortheBSC.AchievingcapacitymeansthatRisclosetotheupperbound,saywithin(1+")factoroftheoptimalboundsabove.ForxedcorruptionprobabilitiespandratesRin(0;1)thisiseasytodene(previousparagraph).Howeveraswedealwithveryloworveryhighratesabove,deningthisneedsabitmorecare,andisdescribedinthetablebelow,andformallyinSection2.AcodeofrateRis"-closetoachievecapacityifitcancorrectfromacorruptionprobabilitypthatsatisestheboundsbelow9.Itiscapacity-achievingifitis"-closetoachievecapacityforall"&#x]TJ/;ø 1;�.90;‘ T; 22;&#x.913;&#x 0 T; [0;0. BEC BSC Lowcode-rate(R!0) p1�R(1+") h(p)1�R(1+") Highcode-rate(R!1) p(1�R)(1�") h(p)(1�R)(1�") 9NotethatforR!0,intheBECwehavep!1,whilefortheBSCwehavep!1 2.Also,wehavestatedtheboundsthinkingofRxedandputtingarequirementonp.OnecanequivalentlyxpandrequirethecodetocorrectacorruptionprobabilitypforarateRthatsatisestheboundsinthetable.5 1.3.2WeightdistributionandlistdecodingBeforemovingtoourresultsonrandomerrors,wetakeadetourtodiscussourresultsontheweightdistributionofReed-Mullercodesaswellastheirlistdecodingproperties.Thesearenaturallyimportantbythemselves,and,furthermore,tightweightdistributionboundsturnsouttobecrucialforachievingcapacityfortheBECinTheorem1.1above,aswellasforachievingcapacityfortheBSCinTheorem1.7below.OurboundextendsanimportantrecentresultofKaufman,LovettandPoratontheweight-distributionofReed-Mullercodes[KLP12],usingasimplevariantoftheirtechnique.Kaufmanetal.gaveaboundthatwastightforr=O(1),butdegradesasrgrows.Ourimprovementextendsthisresulttodegreesr=O(m).DenotewithWm;r()thenumberofcodewordsofRM(m;r)thathaveatmostfractionofnonzerocoordinates.Theorem1.5(SeeTheorem3.3).Let1`r�1m=4and0"1=2.Then,Wm;r((1�")2�`)(1=")O`4(m�`r�`):Asinthepaperof[KLP12],almosttheexactsameproofasourproofofTheorem1.5yieldsaboundforlist-decodingofReed-Mullercodes,forwhichwegetsimilarimprovements.Following[KLP12]wedenote:Lm;r()=maxg:Fm2!F2jff2RM(m;r)jwt(f�g)gj:Thatis,Lm;r()denotesthemaximalnumberofcodewordsofRM(m;r)inahammingballofradius2m.Theboundconcernsoftheform(1�")2�`for1`r�1,andourmaincontributionismakingtherstfactorintheexponentdependon`(ratherthanonrin[KLP12]).Theorem1.6.Let1`r�1and0"1=2.Then,ifrm=4thenLm;r((1�")2�`)(1=")O`4(m�`r�`):1.3.3Randomerrors-theBSCchannelWenowreturntodiscussdecodingfromrandomerrors.OurnextresultshowsthatReed-Mullercodesachievecapacityalsoforthecaseofrandomerrorsatthelowrateregime.TheproofofthisresultreliesonTheorem1.5.Theorem1.7(SeeTheorem6.1).Forr=o(m),RM(m;r)achievescapacityfortheBSC.Moreprecisely,forevery&#x-278;0and=O(1=log(1=))thefollowingholds:Foreveryrm,RM(m;r)is-closetocapacityfortheBSC.Toobtainresultsaboutthebehaviorofhigh-rateReed-MullercodeswithrespecttorandomerrorsweuseanovelconnectionbetweenrobustnesstoerrorsandrobustnesstoerasuresinrelatedReed-Mullercodes.Theorem1.8(SeeTheorem6.13).IfasetofcolumnsUarelinearlyindependentinE(m;r)(namely,RM(m;m�r�1)cancorrecttheerasurepattern1U),thentheerrorpattern1Ucanbecorrected(i.e.,itisuniquelydecodable)inRM(m;m�(2r+2)).UsingTheorem1.3thisgivesanewresultoncorrectingrandomerrorsinReed-Mullercodes.Theorem1.9(SeeTheorem6.2).Forr=O(p m=logm),RM(m;m�(2r+2))cancorrectarandomerrorpatternofweight(1�o(1))
�mr
withprobabilitylargerthan1�o(1).7 WhilethisresultfallsshortofshowingthatReed-MullercodesachievecapacityfortheBSCinthisparameterrange,itdoesshowthattheycancopewithmanymoreerrorsthansuggestedbytheirminimumdistance.RecallthattheminimumdistanceofR(m;m�(2r+2))is22r+2.Achievingcapacityforthiscodemeansthatitshouldbeabletocorrectroughly�m2r
randomerrors.Insteadweshowthatitcanhandleroughly�mr
randomerrors,whichisapproximatelythesquarerootofthenumberoferrorsatcapacity.TheproofofTheorem1.8revealsamoregeneralphenomenon,thatofreducingerrorcorrectiontoerasurecorrection.WeprovethatforanylinearcodeC,ofveryhighrate,thereisanotherlinearcodeC0ofrelatedhighrate,sothatifCcancorrecttheerasurepattern1UthenC0cancorrecttheerrorpattern1U.FurthermoreC0isverysimplydenedfromC.ThedeclineinqualityofC0relativetoCisbestexplainedintermsoftheco-dimension(namelythenumberoflinearconstraintsonthecode,orequivalentlythenumberofrowsofitsparity-checkmatrix).Weprovethattheco-dimensionofC0isroughlythecubeoftheco-dimensionofC.Wenowstatethisgeneraltheorem.ForamatrixHwedenotebyHrthecorrespondingmatrixthatcontainstheevaluationsofallcolumnsofHbyalldegreermonomials(inananalogouswaytothedenitionofUrfromU).Theorem1.10(SeeTheorem6.17).IfasetofcolumnsUislinearlyindependentinaparitycheckmatrixH,thenthecodethathasH3asaparitycheckmatrixcancorrecttheerrorpattern1U.NotethatapplyingthisresultasistoE(m;r)wouldgiveaweakerstatementthanTheorem1.8,inwhichE(m;2r+1)wouldbereplacedbyE(m;3r).Weconcludebyshowingthatthisresultistight,namelyreplacing3by2inthetheoremabovefails,evenforRMcodes.Theorem1.11(SeeSection6.2.5).TherearesubsetsofcolumnsUthatarelinearlyindependentinE(m;1),butsuchthatthepatterns1UarenotuniquelydecodableinE(m;2).1.4ProoftechniquesAlthoughthestatementsofTheorems1.3and1.1soundverysimilar,theirproofsareverydierent.WerstexplaintheidesbehindtheproofsofthesetwotheoremsandthengivedetailsfortheproofsofTheorems1.5,1.7,1.8and1.10.ProofofTheorem1.3TheproofofTheorem1.3reliesonestimatingthesizeofvarieties(setsofcommonzeros)oflinearsubspacesofdegreerpolynomials.Hereisahighlevelsketch.RecallthatwehavetoshowthatifwepickarandomsetofpointsUFm2,ofsize(1�o(1))
�mr
,andwitheachpointassociateitsdegree-revaluationvector,thenwithhighprobabilitythesevectorsarelinearlyindependent.Whileprovingthisissimplewhenconsideredoverlargeelds,itisquitenon-trivialoververysmallelds.Weareabletoprovethatthispropertyholdsfordegreesrupto(roughly)p m=logm.Itisaveryinterestingquestiontoextendthistolargerdegreesaswell.ToprovethatarandomsetUofappropriatesizegivesrisetolinearlyindependentevaluationvectorsweconsiderthequestionofwhatittakesforanewpointtogenerateanevaluationvectorthatislinearlyindependentofallpreviouslygeneratedvectors.Asweprove,thisboilsdowntounderstandingwhatistheprobabilitythatarandompointisacommonzeroofalldegreerpolynomials,inacertainlinearspaceofpolynomialsdenedbythepreviouslypickedpoints.Ifthissetofcommonzerosissmall,thenthesuccessprobability(i.e.,theprobabilitythatanewpointwillyieldanindependentevaluationvector)ishigh,andwecaniteratethisargument.Toboundthenumberofcommonzerosweyetagainmovetoadualquestion.NoticethatifasetofKlinearlyindependentpolynomialsofdegreervanishesonasetofpointsV,thenthereareat8 enablesustomakethisdelicatecalculationforeach(relevant)dyadicintervalofweights.Heretooourimprovementof[KLP12]isessential.ProofsofTheorems1.8,1.9and1.10Consideranerasurepattern1Usuchthatthecor-respondingsetofdegree-revaluationvectors,Ur,islinearlyindependent.NamelythecolumnsindexedbyUinE(m;r)arelinearlyindependent.Wewouldliketoprovethat1UisuniquelydecodablefromitssyndromeunderH(m;m�2r�2)=E(m;2r+1).Weactuallyprovethatif1Visanothererasurepattern,whichhasthesamesyndromeunderH(m;m�2r�2),thenU=V.Theproofmaybeviewedasareconstruction(albeitinecient)ofthesetUfromitssyndrome.Hereisahighleveldescriptionofourargumentthatdierent(linearlyindependent)setsoferasurepatternsgiverisetodierentsyndromes.Werstprovethispropertyforthecaser=1(detailsbelow).ThisimmediatelyimpliesTheorem1.10aseveryparitycheckmatrixofanylinearcodeisasubmatrixofE(m;1)forsomem.Thisisageneralreductionfromtheproblemofrecoveringfromerrorstothatofrecoveringfromerasures(inarelatedcode).Asaspecialcase,italsoimpliesthatforanyr,H(m;m�3r�1)uniquelydecodesanyerrorpattern1UsuchthatthecolumnsindexedbyelementsofUinE(m;r)=H(m;m�r�1)arelinearlyindependent.WethenslightlyrenetheargumentforlargerdegreertoreplaceH(m;m�3r�1)abovebyH(m;m�2r�2),whichgivesTheorem1.8.Forthecaser=1,theproofdividestotwologicalsteps.IntherstpartweprovethatthecolumnsofVmustspanthesamespaceasthecolumnsofU.ThisrequiresonlythesubmatrixE(m;2),namelyatpairsofcoordinatesineachpoint(degree2monomials).InthesecondpartweusethispropertytoactuallyidentifyeachvectorofUinsideV.ThisalreadyrequireslookingatthefullmatrixE(m;3),namelyattriplesofcoordinates.Itisinterestingthatgoingtotriplesofcoordinateisessentialforr=1(andsothisresultaretight).WeprovethatevenifthecolumnsofUarelinearlyindependent,thentherecanbeadierentsetVthathasthesamesyndromeinE(m;2).ThisresultisgiveninSection6.2.5.Wedonotknowwhatistherightboundforgeneralr.1.5RelatedliteratureRecoveryfromrandomcorruptionsBesidestheconjecturesmentionedintheintroductionthatRMcodesachievecapacity,resultsfallshortofthatforallbutveryspacialcases.Wearenotfamiliarofworkscorrectingrandomerasures.SeveralpapershaveconsideredthequalityofRMcodesforcorrectingrandomerrorswhenusingspecicalgorithms,focusingmainlyonecientalgorithms.In[Kri70],themajoritylogicalgorithm[Ree54]isshowntosucceedinrecoveringallbutavanishingfractionoferrorpatternsofweightuptodlog(d)=4,whered=2m�risthecodedistance,requiringhoweverapositiverateR�0.Thiswaslaterimprovedtoweightsuptodlog(d)=2in[DS06].Wenotethatforaxedrate0R1,thisisroughlyp n,whereastoachievecapacityoneshouldcorrect (n)erasures/errors.AlineofworkbyDumer[Dum04,DS06,Dum06]basedonrecursivealgorithms(thatexploitstherecursivestructureofRMcodes),obtainsresultsmainlyforlow-rateregimes.In[Dum04],itisshownthatforaxedorderr,i.e.,fork(m;r)=(mr),analgorithmofcomplexityO(nlog(n))cancorrectmosterrorpatternsofweightupton(1=2�")giventhat"exceedsn�1=2r.In[DS06],thisisimprovedtoerrorsofweightupton=2(1�(4m=d)1=2r),requiringthatr=log(m)!0.Further,[Dum06]showsthatmosterrorpatternsofweightupton=2(1�(4m=d)1=2r)canberecoveredintheregimewherelog(m)=(m�r)!0.Notethatwhilethepreviousresultsrelyonecientdecodingalgorithms,theyarefarfrombeingcapacity-achieving.Concerningmaximum-likelihooddecoding,knownalgorithmsareof10 Denition2.3.(i)Alinearcodeofblocklengthnallowstocorrects=snrandomerasures(resp.errors)ifitcancorrectthemfromtheuniformerasure(resp.error)distributionUs,i.e.,theuniformprobabilitydistributionon@B(n;s)=fz2Fn2:w(z)=dseg.(ii)Alinearcodeofblocklengthnallowstocorrecterasures(resp.errors)fortheBEC(p)channel(resp.BSC(p)channel),wherep=pn,ifitcancorrectthedistributionBp,whereBpisthei.i.d.distribution10onFn2withBernoulli(p)marginal.Notethatforn=Us,i.e.,theuniformdistributionover@B(n;s),theabovedenitionreduces11tojfz2@B(n;s):9z0s.t.z6=z0;Hz=Hz0gj �ns
!0;asn!1;i.e.,thefractionofbaderrorpatterns,whichhavenon-uniquesyndrome,isvanishing.ThefollowingLemma12followsfromstandardprobabilisticarguments.Lemma2.4.(i)Ifalinearcodecancorrects=snrandomerasures(resp.errors),thenitcancorrecterasures(resp.errors)fromtheBEC((s�!(p s))=n)channel(resp.BSC((s�!(p s))=n)channel).(ii)Ifalinearcodecancorrecterasures(resp.errors)fromtheBEC(p)channel(resp.BSC(p)channel),thenitcancorrectnp�!(p np)randomerasures(resp.errors).Wenowdenethenotionsofcapacity-achieving.Sinceintherestofthepapertypicallyconsiderscodesatagivenrate,andinvestigatehowmanycorruptionstheycancorrect,thedenitionsarestatedaccordingly.NotethatwhatfollowsissimplyarestatementofShannon'stheoremsforerasuresanderrors,namelythatacodeCofrateR=(log2(jCj))=ncorrectingacorruptionprobabilitypmustsatisfyR1�pforerasuresandR1�H(p)forerrors.However,sinceweconsidercoderatesthattendto0and1,therequirementsarebrokendowninvariouscasestopreventmeaninglessstatements.Denition2.5.Acodeiscapacity-achieving(orachievescapacity)ifitis"-closetocapacityforall"&#x]TJ/;ø 1;�.90;‘ T; 1.; ;� 0 ;
28;.18; 36;.56; Tm;&#x [00;0.Wenowdenethenotionof"-closetocapacityinthefourcongurations:AlinearcodeCofrateR=o(1)is"-closetocapacity-achievingforerasuresorfortheBECifitcancorrectnprandomerasuresforapsatisfyingp1�R(1+"):AlinearcodeCofrateR=o(1)is"-closetoachievingcapacityforerrorsorfortheBSCifitcancorrectnprandomerrorsforapthatsatisesh(p)1�R(1+");whereh(p)=�plog2(p)�(1�p)log2(1�p)istheentropyfunction. 10Thismeanstheproductdistributionwithidenticalmarginals.11Wedene�ns
as�ndse
foranon-integers.12Thestatementsarerelevantforsnornpnthatare!(1).13 Wenowswitchtotheparity-checkmatrixinterpretation.ThefollowinglemmashowsthatabaderasurepatternforacodeC=ker(H)canbeidentiedasasubsetoflineardependentcolumnsintheparity-checkmatrix.Lemma2.8.ForamatrixHwithncolumns,forS[n],andH[S]thesubsetofcolumnsofHindexedbyS,thenthesetofbaderasurepatternsisgivenbyS2[n]s:9x;y2ker(H);x6=y;x[Sc]=y[Sc]S2[n]s:rk(H[S])s;whererk(H[S])ssimplymeansthecolumnsofH[S]arelinearlydependent.ProofofLemma2.8.LetBadSet:=D2[n]s:rk(H[D])s;BadEra:=S2[n]s:9x;y2ker(H);x6=y;x[Sc]=y[Sc];denoterespectivelythesetofbadsetsforwhichthecolumnsofHdonothavefullrankandthesetofbaderasurepatternsthatcanconfusecodewordsinkerH.Sincethecodeislinear,BadEra=S2[n]s:9v2ker(H);v6=0;v[Sc]=0:HenceforanyS2BadEra,thereexistsv2ker(H),suchthatv6=0andsupp(v)S,andthecolumnsofHindexedbySarenotfullrank.Conversely,ifD2BadSet,DcontainsasubsetVsuchthatV6=;andPi2VH[j]=0,henceD2BadEra(takingvastheindicatorvectorofV). Corollary2.9.ForamatrixHwithncolumnsands2[n],denotebyH[s]therandomsub-matrixofHobtainedbyselectingscolumnsuniformlyatrandom.Then,thecodeker(H)cancorrectsrandomerasuresifandonlyifPrfrk(H[s])=sg!1;asn!1:Inotherwords,correctingsrandomerasuresisequivalenttoaskingthatarandomsubsetofscolumnsintheparity-checkmatrixHisfullrankwhp.Whiletherequirementtocorrectprobabilisticerasures(Corollaries2.9and2.7)issimilartotherequirementfortheworst-casemodelbut\withhighprobability,"thesituationismoresubtleinthecaseoferrors.NotethatforacodeCwithparity-checkmatrixH,thesetofbaderrorpatternsaretheoneswhichleadtoanon-uniquesyndrome,i.e.fz2@B(n;s):9z02@B(n;s)s.t.z6=z0;Hz=Hz0gfz2@B(n;s):9z02@B(n;s)s.t.z6=z0;z+z02Cg:Inotherwords,thesetofbaderrorpatternsareobtainedbytakingthesetofcodewordsandsplittingthecodewordsintoelementsofweights.Itisofcourseenoughtoconsiderthecodewordsofweightatmost2s.However,eveniftheprobabilityofdrawingacodewordofweightatmost2sisvanishing,itdoesnotfollowthattheprobabilityofhavingabadvectorofweightsisalsovanishing.Therearemultiplewaystosplitacodewordinvectorsofweights,andtheseleadtooverlappingsetsofvectors.Hence,theprobabilityofabaderrorpatterndependsonthestructureofHbeyondtheprobabilityofhavingdependentcolumns.15 NotethatG(m;r)issimplyapermutationoftherowsofE(m;r),henceitisalsoageneratormatrixforRM(m;r).Moreover,itcanbeconstructedrecursivelyasfollows:G(m;r)=G(m�1;r)G(m�1;r)0G(m�1;r�1):Thepolynomialinterpretationofthisrecursionissimplythefactthatam-variatepolynomialfofdegreeatmostrcanbeexpressedasf(x1;:::;xm)=f1(x1;:::;xm�1)+xmf2(x1;:::;xm�1);wheref1andf2arem-variatepolynomialsofdegreesatmostrandr�1respectively.Withthisrecession,thedualityproperty(Lemma2.11)(aswellasthefactthatthedistanceofRM(m;r)is2m�r)aredirecltyprovedbyinduction.Wereferto[MS77]forcompleteproofs.3WeightdistributionofReed-MullercodesInthissectionwestudytheweightdistributionofReed-Mullercodes.OuranalysisisbasedonthetechniqueofKaufman,LovettandPorat[KLP12].Westartwithsomehighlevelintuition.Naturally,oneexpectsthatmostcodewordsofRM(m;r)(oranylinearcode,forthatmatter)tohaveweightaroundn=2=(2m)=2.Atrivialupperboundonthenumberofcodewordshavingsuchweight(orlarger)isthetotalnumberofcodewords,i.e.,2(mr).Thequestionisthushowdoesthisnumberchangeswhenweconsidersmallerweights.Specically,whatisthenumberofcodewordsthathaveweightatmost2m�`forsomeparameter`.Ifwedenotethisnumberwith2c(m;r;`)
(mr),thenweareaskingforthevalueofthetermc(m;r;`).Atriviallowerboundonthenumberofsuchcodewordsis2m`+(m�`r�`),whichisobtainedbycountingallpolynomialsofdegree-rthataredivisibleby`linearfunctions.Ifthiswastight,thenc(m;r;`)1,whichsuggeststhatthenumberofsuchpolynomialsgrowsroughlylikethenumberofdegreer�`polynomialsonm�`variables.Kaufmanetal.provedthatindeedthisnumberisessentiallytherightanswerforconstantr.Moreprecisely,theyprovedthatc(m;r;`)=O(r2).Ourcontributionisreplacingthisestimatewithroughlyc(m;r;`)=O(`4).Thischangeismostsignicantwhen`isverysmallcomparedtor,e.g.,whenconsideringthenumberofwordsofweighte.g.roughlyn=4(so`is2)andwhenrislarge,e.g.r= (m).Thisimprovementturnsouttobecriticalfortwoofourresultsonachievingcapacity{forerasureinlowratesanderrorsinhighrate.Itremainsopenifonecanprovethatc(m;r;`)=O(1),namelyisaconstantindependentofallparameters.Westartbygivingthehighlevelviewoftheproofof[KLP12]andthenexplainhowtoimprovetheiranalysis.Werstintroducesomenotation.Forafunctionf:Fm2!F2(equivalently,awordf2Fn2)wedenotebywt(f)therelative(Hamming)weightoff,i.e.,wt(f)=1 2mjfv2Fm2jf(v)6=0gj:ThecumulativeweightdistributionofRM(m;r)atarelativeweight01,denotedWm;r(),isthenumberofcodewordsofRM(m;r)whoserelativeweightisatmost,Wm;r(),jff2RM(m;r)jwt(f)gj:17 WenowshowhowKaufmanetal.deducedTheorem3.1fromLemma3.2.Therstideaistoset=2�r�1.Thepointisthatthereisatmostonedegree-rpolynomialfatdistancefromthefunctionA(x;Y1;:::;Yt;
Y1f(
);:::;
Ytf(
)).Indeed,bythetriangleinequality,thedistancebetweenanytwopolynomialsthatare-closetoA(x;Y1;:::;Yt;
Y1f(
);:::;
Ytf(
))isatmost22�r,whichissmallerthantheminimumdistanceofRM(m;r).Hence,toboundthenumberofpolynomialsf2RM(m;r)ofrelativeweightatmostwt(f)(1�")2�`,itisenoughtoboundthepossiblenumberoffunctionsoftheformA(x;Y1;:::;Yt;
Y1f(
);:::;
Ytf(
))fortheappropriatet.ThesecondstepintheproofofKaufmannetal.istogiveanupperboundonthenumberofexpressionsoftheformA(x;Y1;:::;Yt;
Y1f(
);:::;
Ytf(
)).SinceAisxed,theyonlyhavetoboundthenumberofsetsYiandthenumberofpolynomialsoftheform
Yifandraiseittothepowert.Theynowusethefactthat
Yifisapolynomialofdegreeatmostr�`sothenumberofsuchpolynomialsis2(mr�`).Combiningeverything,andletting=2�r�2sothatt=O�rlog(1=")+r2
andWm;r((1�")2�`)2m`
2(mr�`)t=2m`+(mr�`)O(rlog(1=")+r2):(2)Onedownsideoftheresultof[KLP12]isthatduetothedependenceonroftheconstantinthebigO,theirestimateistightonlyforconstantr,andbecomestrivialatr=~O(p m).Indeed,theboundintheexponentgoesdownroughlylike(r=m)`.Hence,themaximumisobtainedforsmallvaluesof`,i.e.,`=1or`=2.Forthesevalues,thetermlog(1=")r+r2intheexponentbasicallyeliminatesanysavingthatcomesfrom(r=m)`.Thus,toimprovetheboundontheweightdistributionitiscrucialtoimprovetheboundforsmallvaluesof`.Ourresultdoesexactlythis,weareabletoreplacethepowerofrintheexponentwithapowerof`,whichgivestherequiredsavingforsmallvaluesof`.Wenowexplainhowwemodifytheargumentsof[KLP12]inordertotightentheestimategiveninTheorem3.1,thatholdforabroaderrangeofparameters.Ourrstobservationisthatonecanrelaxthesettingof.Weset=(1�")2�`�2,insteadof=2�r�2.Theeectisthatnowtherecanbemanypolynomialsgthatare-closetoA(x;Y1;:::;Yt;
Y1f(
);:::;
Ytf(
)).Indeed,allweknowisthatthedistancebetweenanytwosuchpolynomialsisatmost2=(1�")2�`�1.ThepointisthatthenumberofsuchpolynomialscanbeboundedfromabovebyWm;r(2�`�1)whichisrelativelysmallcomparedtoWm;r(2�`)andsowecan(almost)thinkofitas1.Theeectontheexpression(2)isthatintheexpressionfort,wecan(almost)replacerby`.Asexplainedbefore,thisgivesasignicantsavingovertheboundof[KLP12].Oursecondimprovementcomesfromthesimpleobservationthat
YifcanbedenedbyitsvalueonthequotientspaceFm2=hYii.Asthisisaspaceofdimensionm�`,foraxedYi,wecanupperboundthenumberofpolynomialsoftheform
Yifby2(m�`r�`),insteadof2(mr�`),whichagainyieldsatighterestimate.WenowstateourboundontheweightdistributionofReed-Mullercodes.Theorem3.3.Let1`r�1and0"1=2.Then,ifrm=4,Wm;r((1�")2�`)(1=")8c`4(m�`r�`);wherecisanabsoluteconstant(sameasinLemma3.2).19 Question4.2.WhatisthesmallestsforwhichthesubmatrixUrhasfullrow-rankwithhighprobability?Inthissectionweprovideananswertoeachofthesequestions.16Notethatforanydegree-r,thenumberofrowsofE(m;r),namely�mr
,isanupperboundonthevalueofsfortherstquestionandalowerboundforthesecond.Forsmallrweprovethatwecanapproachthisoptimalboundasymptoticallyinboth.Notethat,interestingly,thedualitypropertyofRMcodesallowstorelatequestion4.1and4.2toeachotherbutfordierentrangesoftheparameters.Namely,thefollowingholds.Lemma4.3.ForasetS[n],denotebyE(m;d)[S]thesub-matrixofE(m;d)obtainedbyselectingthecolumnsindexedbyS.Foranysn,S2[n]s:rk(E(m;d)[S])=s=S2[n]s:rk(E(m;m�d�1)[Sc])=n�md:NotethatE(m;d)[S]=smeansthatE(m;d)[S]hasfullcolumn-rankandE(m;m�d�1)[Sc]=n��md
meansthatE(m;m�d�1)[Sc]hasfullrow-rank.Corollary4.4.Foranintegers2[n],denotebyE(m;d)[s]therandommatrixobtainedbysamplingscolumnsuniformlyatrandominE(m;d).Then,Prfrk(E(m;d)[s])=sg=Prrk(E(m;m�d�1)[n�s])=n�md;wherebothtermsaretheprobabilityofdrawingauniformerasurepatternofsizeswhichcanbecorrectedwiththecodekerE(m;d).ThiscorrespondencefollowsfromLemmas2.8,2.6and2.11.Weprovidetheproofbelowforconvenience.ProofofLemma4.3.NotethatfS2[n]s:rk(E(m;d)[S])sgfS2[n]s:9z2ker(E(m;d));s.t.supp(z)S;z6=0gfS2[n]s:9z2ker(E(m;d))s.t.z[Sc]=0;z6=0g;andusingLemma2.11,previoussetisequaltofS2[n]s:9z2Im(E(m;m�d�1))s.t.z[Sc]=0;z6=0gfS2[n]s:E(m;m�d�1)[Sc]isnotfullrow-rankgfS2[n]s:rk(E(m;m�d�1)[n�s])n�mdg: Thisequivalencepropertyimpliesthatitsucienttoanswereachquestioninoneofthetwoextremalregimes,whichwenextcover. 16Usingtensoringtoproducelinearlyindependentvectorshasalsobeenstudiedrecentlyinthecontextofrealvectors[BCMV14].21 4.1RandomsubmatricesofE(m;r),forsmallr,havefullcolumn-rankThefollowingtheoremaddressesQuestion4.1inthecaseoflowdegree-r.Theorem4.5.Let"�0andk;m;rintegerssuchthats�m�log((mr))�log(1=")r
.Then,withprobabilitylargerthan1�"ifwepicku1;:::;us2Fm2uniformlyatrandomwegetthattheevaluationvectors,ur1;:::;ursarelinearlyindependent.Observethatforr=o(p m=logm)theboundonsis(1�o(1))�mr
,whichwillgiveusacapacity-achievingresult.AsdiscussedinSection1.4(Theorem1.3),toprovethetheoremwehavetounderstandthesetofcommonzeroesofdegree-rpolynomials.Moreaccurately,weneedtogiveanupperboundonthenumberofcommonzeroesofpolynomialsinsomelinearspace.WestartbyintroducingsomenotationandthendiscussthereductionfromTheorem4.5totheproblemofdeterminingthenumberofcommonzeroesofaspaceofpolynomials.Givenasetofpointsu1;:::;us2Fm2wedeneI(u1;:::;us)=ff2P(m;r)j8if(ui)=0g:WhenUisanmsmatrixwedeneI(U)=I(u1;:::;us),whereuiistheithcolumnofU.ItisclearthatI(U)isavectorspace.Similarly,forasetofpolynomialsFP(m;r)wedenoteV(F)=fu2Fm2j8f2Ff(u)=0g:Inotherwords,V(F)isthesetofcommonzeroesofF.FromthedenitionitisclearthatifF1F2thenV(F2)V(F1)andsimilarly,ifU1U2thenI(U2)I(U1).ThenextlemmasexploretheconnectionbetweenthedualspaceofUr,I(U)andV(I(U)).Hereafterweinterpretavectorfoflength�mr
asapolynomialinP(m;r),byviewingitscoordinatesascoecientsoftherelevantmonomials.Weabusenotationandcallthispolynomialfaswell.Lemma4.6.LetUbeanmsmatrix.Then,avectorfoflength�mr
satisesf
Ur=0ifandonlyifthecorrespondingpolynomialf(x1;:::;xm)isinI(U),namely,f2I(U).Proof.TheproofisimmediatefromthecorrespondencebetweenvectorstopolynomialsandfromthedenitionofUr.Indeed,foracolumnuiwehavethatthecoordinatesofuricorrespondtoallevaluationsofmonomialsofdegreeronui.Similarly,thecoordinatesofthevectorfcorrespondtocoecientsofthepolynomialf(x1;:::;xm).Thus,f
uiisequaltof(ui).Hence,f
U=0ifandonlyiff(u1)=:::;f(us)=0,i.e.ifandonlyiff2I(U). Lemma4.7.LetUbeanmsbinarymatrix.Then,foranyu2Fm2wehavethaturisinthelinearspanofthecolumnsofUrifandonlyifI(U)=I(U[fug);namely,everydegreerpolynomialthatvanishesonthecolumnsofUalsovanishesonu.Proof.ItisclearthaturlinearlydependsonthecolumnsofUrifandonlyifforeveryvectorfsuchthatf
Ur=0,itholdsthatf
ur=0,namely,thatf(u)=0.ByLemma4.6thisisequivalenttosayingthatI(U)=I(U[fug). Similarly,wegetanequivalencewhenconsiderthecommonzerosofthepolynomialsthatvanishonthecolumnsofU.22 proofthatwegiveinthemainbodyofthepaper.Forcompleteness,andasthehashingargumentismoreselfcontainedwegiveitinAppendixB.WestartbydiscussingthenotionofgeneralizedHammingweight.LetCFn2bealinearcodeandDCalinearsubcode.Wedenotesupp(D)=fi:9y2D;suchthatyi6=0g:Inotherwords,thesupportofDistheunionofthesupportsofallcodewordsinD.Denition4.11(GeneralizedHammingweight).ForacodeCoflengthnandanintegerawedeneda(C)=minfsupp(D)jDCisalinearsubcodewithdim(D)=ag:Thus,da(C)istheminimalsizeofasetofcoordinatesS,suchthatthereexistsasubcodeD,ofdimensiondim(D)=a,thatissupportedonS.ThereasonforthisdenitionisthatforanycodeCifweleta=dim(C)thenda(C)=n�d,wheredistheminimaldistanceofC.ByconsideringthecomplementsetScthenextlemmagivesanequivalentdenitionofda(C).Lemma4.12.ForacodeCoflengthnandanintegerawehavethatda(C)=maxfbj8jSjbwehavethatdim(C[Sc])&#x-278;dim(C)�ag:Proof.Theprooffollowsimmediatelyfromasimplelinearalgebraargument.IfDisasubcodeofCthatissupportedonasetofcoordinatesSthendim(C)=dim(D)+dim(C[Sc]). ThealternativedenitiongiveninLemma4.12isveryclosetowhatweneed.WewishtoshowthatforanylargeV,therearemanylinearlyindependentdegreerpolynomialsthataredenedonV.Inotherwords,wewishtoprovethatdo(1)(mr)(RM(m;r))2m�"
2m=mr:Indeed,thiswillimplythatforanyjVj"
2m=�mr
thereareatleast(1�o(1))�mr
linearlyindependentdegreermonomialsdenedonV(VplaystheroleofScinLemma4.12).ThenexttheoremofWei[Wei91]computesexactlythegeneralizedHammingweightofReed-Mullercodes.Forstatingthetheoremweneedthefollowingtechnicalclaim.Lemma4.13(Lemma2of[Wei91]).Forevery0a�mr
thereisauniquewayofexpressingaasa=P`i=1�miri
,wheremi�ri=m�r�i+1.Theorem4.14([Wei91]).Let0a�mr
beaninteger.Then,da(RM(m;r))=P`i=12mi,wherea=P`i=1�miri
istheuniquerepresentationofaaccordingtoLemma4.13.WearenowreadytoproveLemma4.10.ProofofLemma4.10.Fora=Pti=1�m�ir�1
,Theorem4.14impliesthatda(RM(m;r))=Pti=12m�i=2m�2m�t.Thus,ifjVj�2m�da(RM(m;r))=2m�tthentherearemorethan�mr
�a=�mr
�Pti=1�m�ir�1
manylinearlyindependentdegree-rpolynomialsdenedonV.Tomakesenseofparametersweshallneedthefollowingsimplecalculation.WegivethestraightforwardproofinSectionA.24 notvanish.Thus,bythediscussioninSection4.1,thismeansthatthedualspacecontainsonlythezeropolynomial,whichisexactlywhatwewishtoprove.Toapplythisstrategywewillneedtoknowthenumberofpolynomialsthathaveacertainnumberofnonzeroes.SuchanestimatewasgiveninTheorem3.3,following[KLP12].Proof.Set"==4.Foraninteger1`rwedenotewithP`thesetofdegreerpolynomialswhosefractionofnonzerosisbetween(1�")2�`�1and(1�")2�`.ByTheorem3.3,jP`jWm;r((1�")2�`)(1=")8c`4(m�`r�`):LetfbesomepolynomialinP`.Whenpickingspointsatrandom,theprobabilitythatfvanishesonallofthemisatmost(1�(1�")2�`�1)s.Thus,theprobabilitythatanyf2P`vanishesonallspointsisatmostjP`j
(1�(1�")2�`�1)s.Wewouldliketoshowthatthisprobabilityissmall,whenrangingoverall`.Werststudythecasethat`�0,i.e.,ofthecontributionfromthepolynomialsthathaveatmost(1�")=2nonzeros.Althoughinthiscasetheprobabilityofhittinganonzerogetssmallerandsmalleras`grows,thesizeofP`goesdownatafasterrate(byTheorem1.5),andthereforewegetanexponentiallysmallprobabilityofmissingsomepolynomial.When`=0,althoughthenumberofpolynomialsofsuchhighweightishuge,theprobabilityofmissinganyofthemistiny,andsoweareokinthiscaseaswell.Wenowdotheformalcalculation.Bytheabove,theprobabilitythatsomepolynomialwhosefractionofnonzerosisatmost(1�")=2vanishesonallspointsisatmostr�1X`=1jP`j
(1�(1�")2�`�1)s:(3)Indeed,anydegree-rpolynomialisnonzerowithprobabilityatleast2�randhencetheabovesummationgoesoverallsuchpolynomials.Letusestimateatypicalsummand,jP`j
1�(1�")2�`�1s(1=")8c`4(m�`r�`)
(1�(1�")2�`�1)s(1=")8c`4(m�`r�`)
exp(�(1�")2�`�1s)(1=")8c`4(m�`r�`)
exp�(1+)(1�")mr2�`�1:Let=O(1=log(1="))=O(1=log(1=)).Fromthefactthatm�`r�`mr
r m`;wegetbysimplemanipulationsthatifrmthenjP`j
1�(1�")2�`�1sexpmr
3�`�(1+)(1�")2�`�1exp� mr
2�`:GoingbacktoEquation(3)weseethatr�1X`=1jP`j
(1�(1�")2�`�1)sr�1X`=1exp� mr
2�`exp� 2�rmr:26 UsingnowTheorem4.5andthefactthatE(m;r)isaparity-checkmatrixforRM(m;m�r�1)(seeLemma2.11),weobtainourresultfortheperformanceofhigh-rateRMcodesontheBEC.Corollary5.2.Let"�0,rmbetwopositiveintegersands=b�m�log((mr))�log(1=")r
c.Then,RM(m;m�r)cancorrectsrandomerasureswithprobabilitylargerthan1�".Inparticular,ifm�r=o(p m=logm),thenRM(m;r)iscapacity-achievingontheBEC.Thefollowingcalculationgivesabettersenseoftheparameters(theproofisinSectionA).Claim5.3.Forrq m 4log(m)and"�m�r=2wehavethat�[m�log((mr))�log(1=")r
�(1�)�mr
.6Reed-MullercodeforerrorsWepresentnextresultsforerrorsatbothlowandhighrate.Theresultsatlow-raterelyontheweightdistributionresultsofSection3,whereasthehigh-rateresultsrelyonanovelrelationbetweendecodingfromerrorsanddecodingfromerasures.6.1Low-rateregimeTheorem6.1.Let�0.Thereexists17=O(1=log(1=))suchthatthefollowingholds.Foranytwointegersrandmsatisfyingr=m,andanypsatisfying1�h(p)=(1+)R;whereR=�mr
n;RM(m;r)cancorrectpnrandomerrorswithprobabilityatleastexp� min(;2�r)
�mr
Inparticular,forr=o(m),RM(m;r)iscapacity-achievingontheBSC.Beforegivingtheproofwemakeasmallcalculationtogetabettersenseofwhatparametersweshouldexpect.SinceRissmallwecanexpecttocorrectafractionoferrorsapproaching1=2.Letusdenotep=(1�)=2.Wenowwishtogurehowsmallshouldbe.Atcorruptionratecloseto1=2wehavethath(p)=h(1=2�=2)=1�2=(2ln(2))+(4):(4)Thus,ifwewishtohave(1+)R=1�h(p)thenshouldsatisfy(1+)R=1�h(p)=2=(2ln(2))�(4):(5)Hence,2=(R).WenowgivetheprooffollowingtheoutlinedescribedinSection1.4.Proof.Letsbethenumberoferrors,i.e.,s=pnandp=1=2�=2.Abaderrorpatternz2Fn2isoneforwhichthereexistsanothererrorpatternz02Fn2,ofweights,suchthatz+z0isacodewordinRM(m;r).Weconcentrateonthecasethatw(z0)=sasthisisthemostinterestingcase.Notethatsincebothzandz0aredierentandhavethesameweight,theweightofz+z0mustbeevenandinfd;:::;2sg.Asbothz+z0andtheall1vectorarecodewords,wealsohavethattheweightofz+z0isatmostn�d,hencew(z+z0)2fd;:::;n�dg.Therefore,countingthenumber 17Theexactvalueofisgivenin(11).28 ofbaderrorpatternsisequivalenttocountingthenumberofweightsvectorsthatcanbeobtainedby\splitting"codewordsofevenweightinfd;:::;n�dg.Notethatforacodewordyofweightw(y)=w,therearew=2choicesforthesupportofzinsidesupp(y)ands�w=2choicesoutsidethecodeword'ssupport.Indeed,zandz0mustcanceleachotheroutsidethesupportofyandhencetheyhavethesameweightinsidesupp(y).18Itfollows,thatforaxedythereareww=2n�ws�w=2possibilitiestopickabaderrorpatternwithintersectionw=2withsupp(y).DenotingbyBthesetofbaderrorpatternsandNm;r(w)thenumberofcodewordsofweightwinRM(m;r),aunionboundgivesPrfBgXw2fd;:::;n�dgNm;r(w)�ww=2
�n�ws�w=2
�ns
:WearenowgoingtoprovethatPr[B]isexponentiallysmall,foroursettingofparameters,whichwillimplythetheorem.Sincefor192(0;1),2nh()�O(log(n))nn2nh();andrecallingtheentropyapproximationof(4),wehave,bydening=w=n,PrfBgX2fd=n;:::;1�d=ngNm;r(n)2n2n1��2 (1�)2ln(2)+O4 (1�)2 2n1�2 2ln(2)�O(logn)=X2fd=n;:::;1�d=ngNm;r(n)2�n2 2ln(2) (1�)+nO4 (1�)2+O(logn):Let"==3.Wenextupperboundtheabovesummationbygroupingcodewordsofweightsbetween(1�")2�`�1and(1�")2�`,with`2f1;2;:::;r�1g.Forcodewordsofweightcloseto1=2,weusethefactthatthereare2(mr)codewordsinRM(m;r).Sincethefunction!=(1�)isincreasing,weobtainthefollowingboundwherejP`jis,asbefore,thenumberofcodewordshavingweightsbetweenbetween(1�")2�`�1and(1�")2�`:PrfBgX`2f1;2;:::;r�1gjP`j2�n2 2ln(2)(1�")2�`�1 (1�(1�")2�`�1)+nO4 (1�(1�")2�`�1)2+O(logn)(6)+2(mr)2�n2 2ln(2)(1�")=2 (1�(1�")=2)+nO(4=(1+")2)+O(logn):(7)UsingtheinequalityofTheorem3.3:jP`j(1=")8c`4(m�`r�`); 18Whenzandz0donothavethesameweighttheystillhavetocanceleachotheroutsidey.Thus,supp(z)musthaveintersectionw=2+(w(z)�w(z0))=2withsupp(y).19Thebinomialcoecientshouldbedenedfortheroundingofnwitheitherceilingor oorfunctions.29 Corollary6.3.Let";�0,rmtwopositiveintegerssuchthatrq m 4log(m)and"�m�r=2.ThenRM(m;m�(2r+2))cancorrectarandomerrorpatternofweight(1�)�mr
withprobabilitylargerthan1�".Therestofthissectionisorganizedasfollows.WerstgiveacombinatorialviewofthesyndromeofanerrorpatternunderE(m;r)(Section6.2.1).WethenstudythecaseofE(m;3),whichcorrespondstothecaser=1inTheorem6.2(asE(m;3)=H(m;m�4)),inSection6.2.2.Thecaseofgeneraldegree-rishandledinSection6.2.3.Then,inSection6.2.4weextendthecaser=1toholdforarbitrarylinearcodesofhighdegreeandinSection6.2.5weprovethatourresultsforthecaser=1aretight,insomesense.6.2.1ParitycheckmatrixandparityofpatternsInthissectionwegiveacombinatorialinterpretationofthesyndromeofanerrorpattern.ConsiderthecodeRM(m;m�r�1).ItsparitycheckmatrixisH(m;m�r�1)=E(m;r).LetUFm2beasetofsizes.WeassociatewithUtheerrorpattern1U2Fn2.Clearlyw(1U)=jUj=s.Wedenotewithujthej'thelementofU.WeshallalsothinkofUasanmsmatrixwhosej'thcolumnisuj.AsbeforewedenotewithUrthesubmatrixofE(m;r)whosecolumnsareindexedbyU.Alternatively,thisisthesetofallevaluationvectorsofU'scolumns.WeshallusethesameconventionforanothersubsetVFm2.ThefollowingdenitioncapturesacombinatorialpropertythatwewilllatershowitsrelationtosyndromesunderE(m;r).Denition6.4.FortwomatricesA;Bofsamedimensionn1n2,wedenoteArBifanypatternofsizeatmostrinthecolumnsofAappearswiththesameparityinthecolumnsofB.I.e.,foreverysubsetI[n1]ofsizerandeveryz2Fr2thenumberofcolumnsinAI;
thatequalzisequal,modulo2,tothenumberofcolumnsinBI;
thatequalz.Forexample,thematricesA=0BBBBBB@1000000100000010000001000000100000011CCCCCCAandB=0BBBBBB@1110001101001100101100011011110111111CCCCCCA(12)satisfyA2BbutA63B.ToseethatA2Bonecanobservethat:thenumberof1'sinrowiinAisequal(modulo2)tothatnumberinB;theinnerproduct(modulo2)betweenrowsiandjinAisthesameasinB.Indeed,thatinnerproductbetweenrowsiandjcountsthenumberofcolumnsthathave1inbothrows.Togetherwiththeinformationaboutthenumberof1'sinrowiandinrowjweareguaranteedthatanypatternonrowsiandjhasthesameparityinbothmatrices.Ontheotherhand,thepattern(1;1;;;1;),whichstandsfor1intherst,secondandfthrows(intheterminologyofthedenition,I=f1;2;5gandz=(1;1;1)),appearsonceinBbutitdoesnotappearinA.Thenextlemmashowsthattwoerrorpatterns1Vand1UhavethesamesyndromeunderE(m;r)ifandonlyifthetwomatricesUandVsatisfyUrV.WedenotewithM(m;r)thesetofallm-variatemonomialsofdegreeatmostr.31 oferrorpatternsunderH(m;m�(2r+2))=H(m;m�4)=E(m;3),namelyevaluationsbyalldegree-3monomials.Wewillproverstadeterministicresult:ifUisanysetoflinearlyindependentcolumns,thenforanyV6=U,wehavethatU63V.Thus,anysetoferrorsthatissupportedonlinearlyindependentcoordinates(whenviewedasvectorsinFm2)canbeuniquelycorrected.Thisimmediatelygivesanaverage-caseresult.Ifwehavem�log(m=")randomerrors,thenwithprobabilityatleast1�"theirlocationscorrespondtolinearlyindependentm-bitvectorsandthereforewecancorrectsuchamountoferrorswithhighprobability.20Noticethatthisisalreadyhighlynontrivial,asR(m;m�4)has(absolute)distance16,sointheworstcaseonecannotcorrectmorethan8worst-caseerrors!Lemma6.7.LetUFm2beasetoflinearlyindependentvectors,suchthatjUj=s.Then,foranyV6=U,suchthatjVjs,wehavethatV63U.Inparticularthismeansthatwecancorrecttheerrorpattern1UinRM(m;m�4).Proof.BymultiplyingUwithaninvertibleA(changingthebasisFm2)wecanassume,w.l.o.g.,thatthecolumnsofUaretheelementarybasisvectors,e1;:::;es.21Indeed,sinceAisinvertibleitfollowsfromLemma6.6thatitisenoughtoprovetheclaimforAU.LetVFm2besuchthatjVj=sandV3U.OurtaskistoshowthatV=U.Thiswillbeshownintwosteps.First,we'llshowthatspan(V)=span(U),whichinparticularimpliesthatVislinearlyindependentaswell.ProvinglinearindependencerequiresonlythatV2U,namelyevaluationsbydegree-2monomials.UsingV3U,we'llprovethattheyactuallyhavethesamespan,andfromthatderivethatV=U.LetusrstarguelinearindependenceofV.We'llthinkofUandVnotonlyassetsofvectors,butalsoasmsmatrices,anddenotebyU0thetransposeofU.Notethat,asthecolumnsofUareunitvectors,wehaveU0U=Is.Nowsincediagonalelementsofthisproductcapturethevalueofdegree-1monomialsofthesyndrome,ando-diagonalelementsoftheoftheproductcorrespondtoinnerproductsofrows,namely(asintheexampleoftheprevioussection),todegree-2monomialsofthesyndrome.AsV2UwealsohavethatV0V=IsandsothedimensionofVissaswell.Wewilllatershow6.2.5thatthislinearindependenceistheonlythingwecaninferfromV2U.WenowactuallyprovethestrongerstatementthatinfactUandVspanthesamesubspace.ThiswillrequireV3U.ItwillbesucienttoprovethatVspansthevectore1,asforothervectorsinUtheproofisidentical.Considerthepattern(1;0)inthersttworowsofU.Thatis,considerallcolumnsofUthathave1intheirrstcoordinateand0inthesecond.Itisclearthatthispatternonlyappearsine1andhenceitsparityinUis1.Thus,theremustbeanoddnumberofcolumnsinVwhosersttworowsequal(1;0).Themainobservationisthatifweaddupthecolumnsthenweobtainthevectore1.Claim6.8.Undertheconditionsofthelemma,thesumofallcolumnsinVwhosersttwocoordinatesequal(1;0)ise1.Moregenerally,fori2[s],ifweconsiderthepatternthathas1inthei'thcoordinateand0insomej6=icoordinate,thenthesumofallcolumnsinVthathavethispatternisequaltoei.Proof.Assumethatthisisnotthecase,namely,thesumisavectorw6=e1.Werstnotethatthersttwocoordinatesofwequal(1;0).Indeed,thisholdsaswesummedanoddnumberofvectorsthathasthesevalues.Hence,theremustexistacoordinatei�2suchthatwi=1.Thus,the 20Toeliminatepossibleconfusionwerepeat:anerrorpatternisann-bitvector,whosecoordinatesareindexedbym-bitvectors.21Thisisnotreallynecessary,butitmakestheargumentsimplertoexplain.33 Proof.Denotewithuithei'thcolumnofU.Recallthatthei'thcolumnofUrcorrespondstoallevaluationsofmonomialsofdegreeatmostratthepointui,i.e.,itisequaltouri.Thisinterpretationwillbehelpfulthroughouttheproof.AssumethatVFm2)issuchthatjVj=s0sandV2r+1U.Similarly,wedenotethecolumnsofVwithv1;:::;vs02Fm2andnotethatthei'thcolumnofVrisvri.AsthecolumnsofUrarelinearlyindependent,thereexistvectorsfisothatfi
Ur=ei2Fs2.Asthecoordinatesofeachfiareindexedbymonomialsofdegreer,wecaninterpretfiasadegree-rpolynomialfi(x1;:::;xm)andrewritefi
Urasfi
Ur=(fi(u1);:::;fi(us)).Inotherwords,fi(uj)=i;j.22OurnextgoalisprovingthatifU2r+1Vthenu12V.Thiswillclearlyimplythelemmaaswecanprovethesameforanyotherui.Ourmainhandlewillbethepolynomialf1thatseparatesu1fromtheotherui's.Letusassumewlogthat(u1)1=1,i.e.,thattherstcoordinateofu1equals1.23Considerthepolynomialx1
f1.Thisisapolynomialofdegreer+1andbythedenitionoff1andtheassumptionon(u1)1wehavethatPsi=1(x1f1)(ui)=1.AsU2r+1V,itmustholdthatPs0i=1(x1f1)(vi)=1mod2.FollowingthefootstepsoftheproofofLemma6.7,wedenoteJf1;1=fij(x1f1)(vi)=1g.ThenextclaimisanalogoustoClaim6.8.Claim6.12.Xi2Jf1;1vri=ur1:Proof.LetMbesomemonomialofdegreer.ToeasethenotationassumethatM(u1)=1andconsiderthepolynomialM
x1
f1.24Itisclearthat(M
x1
f1)(u1)=1.SinceV2r+1Uanddeg(M
x1
f1)2r+1,itfollowsthatPs0i=1(Mx1f1)(vi)=1.FromdenitionofJf1;1wehavethats0Xi=1(Mx1f1)(vi)=Xi2Jf1;1(Mx1f1)(vi)=1:Indeed,foreveryi62Jf1;1wehavethat(x1f1)(vi)=0.Wethusconcludethatthereisanoddnumberofvectorsvi,i2Jf1;1,suchthat(Mx1f1)(vi)=1.Inparticular,theM'thcoordinateinthesumPi2Jf1;1vriequals1,i.e.itisequaltoM(u1).AsMwasarbitraryweconcludethatPi2Jf1;1vri=ur1,asrequired. Aswecanproveananalogouslemmaforeveryui,weconcludethethecolumnsofUrbelongtothespanofthecolumnsofVr.Inparticular,thecolumnsofVrarelinearlyindependentandjVj=s(earlierwecalledthisobservationClaim??).Ournextstepisprovingthat,uptopermutationofcolumns,Ur=Vr.Thiswillimplythatui=viaswewanted.Toshowthis,forevery`2[m],wedenoteJf1;`=fij(vi)`=(u1)`andf1(vi)=1g: 22Intuitively,thepolynomialsficorrespondtotherowsofthematrixAthatwereusedintheproofofLemma6.7tomakethecolumnsofUequaltheelementaryunitvectorsthere.23Ifitequals0thenweconsiderthepolynomial(1+x1)f1inwhatfollows.24IfM(U1)=0thenweconsiderthepolynomial(1+M)
x1
f1instead.35 WenotethatthereisanalternativewaytodeneJf1;`byconsideringeitherthepolynomialx`
f1orthepolynomial(1+x`)
f1.Wethushavethatforevery`2[m],Xi2Jf1;`vri=ur1:However,asthecolumnsofVrarelinearlyindependentandXi2Jf1;`vri=ur1=Xi2Jf1;1vriwegetthatJf1;1=Jf1;2=:::=Jf1;m.Hence,Jf1;1=\m`=1Jf1;`=fij8`2[m](vi)`=(u1)`andf1(vi)=1g:Thus,foreveryi2Jf1;1wehavethatvi=u1.Inparticular,sinceJf1;16=;,itfollowsthatthereissomei2[s]suchthatvi=u1.Aswecanprovethisforeveryuj,weconcludethatU=Vasclaimed.ThisconcludestheproofofLemma6.11. Wethusprovedthatifanerrorpattern1Uissuchthatitscoordinatesuisatisfythaturiarelinearlyindependent,thenwecancorrectthaterrorpatterninRM(m;m�(2r+2)).Wesummarisethisinthefollowingtheorem.Theorem6.13.IfasetofcolumnsUarelinearlyindependentinE(m;r)(namely,RM(m;m�r�1)cancorrecttheerasurepattern1U),thentheerrorpattern1UcanbecorrectedinRM(m;m�(2r+2)).Proof.Lemma6.11tellsusthatifthecolumnsindexedbyUarelinearlyindependentinE(m;r),thenthereisnootherVFm2ofsizessuchthatV2r+1U.Lemma6.5nowimpliesthatE(m;2r+1)
1U6=E(m;2r+1)
1V;foranyU6=VFm2ofsizes.AsE(m;2r+1)=H(m;m�(2r+2)),itfollowsthatthesyndromeof1Uisuniqueandhence1UisuniquelydecodableinRM(m;m�(2r+2). TheproofofTheorem6.2immediatelyfollowsfromTheorems4.5and6.13.ProofofTheorem6.2.Theorem4.5guaranteesthatasetUFm2ofs=b�m�log((mr))+log(")r
c�1randomlychosenvectors,satisfythatthecolumnsofUrarelinearlyindependent.ByLemma6.11welearnthatthereisnototherVFm2ofsizessuchthatV2r+1U.Lemma6.5impliesthatforanysuchV,E(m;2r+1)
1U6=E(m;2r+1)
1V:AsE(m;2r+1)=H(m;m�(2r+2)),itfollowsthatthesyndromeof1Uisuniqueandhence1UisuniquelydecodableinRM(m;m�(2r+2). 6.2.4AgeneralreductionfromdecodingfromerrorstodecodingfromerasuresInthissectionweshowthattheresultsprovedinSection6.2.2areinfactmoregeneralandapplytoanydegreethreetensoringofalinearcodewithitself.Werstsetuptherequireddenitions.Denition6.14.TheHadamardproductoftwovectorsy;z2Fn2isthevectorw=yzobtainedfromthecoordinatewiseproductwi=yi
zi.36 7FuturedirectionsandopenproblemsWebelievethatourworkrenewshopeforprogressonsomeclassicalquestions,andsuggestssomenewconcretedirectionsandopenproblems.ThemostobviousofallisthequestionofwhetherReed-Mullercodesachievecapacityforallrangesofparameters,eitherforrandomerasuresorforrandomerrors.Weonlyhandleheretheextremecasesofveryhighorverylowrates,whereasmostinterestistraditionallyfocusedonconstantratecodes.Webelievethatthetechniquesforeachofourfourboundscanbeimprovedtoalargersetofparameters(seebelow),butfeelthattheyfallshortofreachingconstantrate,andpossiblynewtechniquesareneeded.OnewaytoimproveourboundsinbothTheorem1.1(low-rateBEC)andTheorem1.7(lowrateBSC)isthroughtighterboundsontheweightenumerationofReed-Mullercodes,aswellastighterboundsontheprobabilityoferrorforTheorem1.7.WebelievethatinTheorem1.5onecaneliminatethefactor`4intheexponent,resultinginaboundthatisaxedpolynomial(independentofm;r;`)ofthelowerboundin[KLP12].Whilesuchatightresultwouldnotgetus(ineitherTheorem1.1and1.7)totheconstantrateregime,thisquestionofweightenumerationisofcoursebasicinitsownright.Moreover,bothin[KLP12]andourpaper,italsoimpliessimilarboundsforlist-decoding,whichisanotherbasicquestion.Theorem1.4(highrateBEC)isquantitativelymuchweakerthanTheorems1.1and1.7,inthatthelattertwocanhandlepolynomialsofdegree-rwhichislinearinm,whereastheformeronlyreachesdegreesrwhichareaboutp m.Thebottleneckintheargument,whichprobablypreventsitfromreachingalineardegree,istheuseoftheunionbound.Weupperboundtheprobabilitythat,whenaddingasubsequentrandomvectorutooursetU,itsevaluationurwillbelinearlyindependentoftheevaluationsofallpreviouslychosenpoints.Thiscurrentproofdoesnotuseatallthatpreviouspointswerechosenrandomly,aswedon'tknowhowtotakeadvantageofthis.Forhigh-rateBSC(Theorem1.9),whileweareabletocorrectmanymoreerrorsthanpreviouslyknown,wearenotevenabletoachievecapacity.Herewefeelthatoneimportantbottleneckisourinabilitytoarguedirectlyaboutcorruptionpatters(setsU)whicharelinearlydependent.Ouruniquedecodingproof,evenforr=1(onwhichwefocusnow),showingthatasetU2Fm2isuniquelydeterminedbyitssyndromeunderevaluationsbydegree-3monomialsi.e.,byE(m;3)
1U,isespeciallytailoredtolinearlyindependentsetsU.Thegapbetweenourlowerbound(namelythatE(m;2)
1Udoesnotsuce)andtheaboveupperbound(thatE(m;3)
1Usuces)isintriguing,andwebelievewecanndasubsetofquadraticallymanymonomialsofdegreeatmost3whichguaranteeuniquedecoding-sucharesultisinformationtheoreticallyoptimal;numberoferrorpatternsUwhicharelinearlyindependentisaboutexp(m2),andthusO(m2)bitsareneededinanyuniqueencoding.Anotherburningquestionregardingthisresultisitsineciency.Whileuniquedecodingisguaranteed,thebestwayweknowtoidentifythesetUisbruteforce,requiringexp(m2)stepsforindependentsetsUofsizem.Wefeelthatagoodstartingplaceis(perhapsusingouruniquenessproof)whichrecoversUinexp(m)=poly(n)stepsfromitsevaluationonalldegree-3monomials(orevendegree-10monomials).Ofcourse,itisquitepossiblethatapoly(m)algorithmexists.Inparticular,recursivealgorithms(thatexploittherecursivenatureofRMcodes)couldbeusedtothateect26. 26Practically,onecandecodeRMcodesontheBSCbyusingarecursivedecoder(e.g.,likeforpolarcodes)foreachmissingcomponentinthesyndrome,andbygrowingalistofpossiblecodewordseachtimethedecoderhasdoubts,orpruningdownthetreeeachtimethedecodercancheckthevalidityofapath(fromtheavailablecomponentsofthesyndrome).38 [BF90]D.BeaverandJ.Feigenbaum,Hidinginstancesinmultioraclequeries,STACS90,Springer,1990,pp.37{48.1[BFL90]L.Babai,L.Fortnow,andC.Lund,Nondeterministicexponentialtimehastwo-proverinteractiveprotocols,FoundationsofComputerScience,1990.Proceedings.,31stAnnualSymposiumon,IEEE,1990,pp.16{25.1[BGH+12]B.Barak,P.Gopalan,J.Hastad,R.Meka,P.Raghavendra,andD.Steurer,Makingthelongcodeshorter,53rdAnnualIEEESymposiumonFoundationsofComputerScience,FOCS2012,NewBrunswick,NJ,USA,October20-23,2012,2012,pp.370{379.1[BKS+10]A.Bhattacharyya,S.Kopparty,G.Schoenebeck,M.Sudan,andD.Zuckerman,Optimaltestingofreed-mullercodes,Proceedingsofthe2010IEEE51stAnnualSymposiumonFoundationsofComputerScience,FOCS'10,2010,pp.488{497.1[BV10]A.BogdanovandE.Viola,Pseudorandombitsforpolynomials,SIAMJ.Comput.39(2010),no.6,2464{2486.1[DG07]D.J.Costello,Jr.andG.D.Forney,Jr.,Channelcoding:Theroadtochannelcapacity,ProceedingsoftheIEEE95(2007),no.6,1150{1177.2[DS06]I.DumerandK.Shabunov,RecursiveerrorcorrectionforgeneralReed-Mullercodes,DiscreteAppliedMathematics154(2006),no.2,253{269,CodingandCryptography.10[Dum04]I.Dumer,Recursivedecodinganditsperformanceforlow-rateReed-Mullercodes,InformationTheory,IEEETransactionson50(2004),no.5,811{823.10[Dum06] ,Soft-decisiondecodingofReed-Mullercodes:asimpliedalgorithm,InformationTheory,IEEETransactionson52(2006),no.3,954{963.10[Eli55]P.Elias,Codingfornoisychannels,IREConventionRecord4(1955),37{46.2[For67]G.D.Forney,Concatenatedcodes,PhDThesis,MassachusettsInstituteofTechnology,1967.2,39[Gas04]W.Gasarch,Asurveyonprivateinformationretrieval,BulletinoftheEATCS,Citeseer,2004.1[GKZ08]P.Gopalan,A.R.Klivans,andD.Zuckerman,List-decodingReed-Mullercodesoversmallelds,Proceedingsofthe40thAnnualACMSymposiumonTheoryofComputing,Victoria,BritishColumbia,Canada,May17-20,2008,2008,pp.265{274.4[GX13]V.GuruswamiandP.Xia,Polarcodes:Speedofpolarizationandpolynomialgaptocapacity,Proceedingsofthe2013IEEE54thAnnualSymposiumonFoundationsofComputerScience,2013,pp.310{319.39[Ham50]R.W.Hamming,ErrorDetectingandErrorCorrectingCodes,BellSystemTechnicalJournal26(1950),no.2,147{160.2[Has13]S.H.Hassani,PolarizationandSpatialCoupling:TwoTechniquestoBoostPerformance,PhDDissertation,IC,EPFL,2013.3940 [HKL05]T.Helleseth,T.Klove,andV.I.Levenshtein,Error-correctioncapabilityofbinarylinearcodes,InformationTheory,IEEETransactionson51(2005),no.4,1408{1423.11[ILL89]R.Impagliazzo,L.A.Levin,andM.Luby,Pseudo-randomgenerationfromone-wayfunctions(extendedabstracts),Proceedingsofthe21stAnnualSTOC,1989,pp.12{24.43,44[KLP12]T.Kaufman,S.Lovett,andE.Porat,WeightDistributionandList-DecodingSizeofReed-MullerCodes,InformationTheory,IEEETransactionson58(2012),no.5,2689{2696.1,4,7,9,10,11,17,18,19,26,38[Kri70]R.E.Krichevskiy,OnthenumberofReed-Mullercodecorrectableerrors,Dokl.Sov.Acad.Sci.191(1970),541{547.10[KRU11]S.Kudekar,T.J.Richardson,andR.L.Urbanke,Thresholdsaturationviaspatialcoupling:WhyconvolutionalLDPCensemblesperformsowellovertheBEC,InformationTheory,IEEETransactionson57(2011),no.2,803{834.2[KT70]T.KasamiandN.Tokura,OntheweightstructureofReed-Mullercodes,InformationTheory,IEEETransactionson16(1970),no.6,752{759.11[KTA76]T.Kasami,N.Tokura,andS.Azumi,Ontheweightenumerationofweightslessthan2.5dofReed-Mullercodes,InformationandControl30(1976),no.4,380{395.11[LMS+97]M.Luby,M.Mitzenmacher,A.Shokrollahi,D.A.Spielman,andV.Stemann,Practicalloss-resilientcodes,29thannualACMSymposiumonTheoryofComputing,1997,pp.150{159.2[MHU14]M.Mondelli,S.H.Hassani,andR.L.Urbanke,FromPolartoReed-MullerCodes:ATechniquetoImprovetheFinite-LengthPerformance,Communications,IEEETransac-tionson62(2014),no.9,3084{3091.2,3[MS77]F.J.MacWilliamsandN.J.A.Sloane,Thetheoryoferrorcorrectingcodes,North-Hollandmathematicallibrary,no.v.2,North-HollandPublishingCompany,1977.11,17[Mul54]D.E.Muller,Applicationofbooleanalgebratoswitchingcircuitdesignandtoerrordetection,ElectronicComputers,TransactionsoftheI.R.E.ProfessionalGrouponEC-3(1954),no.3,6{12.1[Rab89]M.O.Rabin,Ecientdispersalofinformationforsecurity,loadbalancing,andfaulttolerance,JournaloftheACM(JACM)36(1989),no.2,335{348.1[Raz87]A.A.Razborov,Lowerboundsonthesizeofboundeddepthcircuitsoveracompletebasiswithlogicaladdition,Math.Notes41(1987),no.4,333{338.1[Ree54]I.Reed,Aclassofmultiple-error-correctingcodesandthedecodingscheme,InformationTheory,TransactionsoftheIREProfessionalGroupon4(1954),no.4,38{49.1,10[SB70]N.J.A.SloaneandE.Berlekamp,Weightenumeratorforsecond-orderReed-Mullercodes,InformationTheory,IEEETransactionson16(1970),no.6,745{751.11[Sha48]C.E.Shannon,Amathematicaltheoryofcommunication,TheBellSystemTechnicalJournal27(1948),379{423,623{.241 ProofofClaim5.3.WeshallneedthefollowingtwosimpleinequalitiesthatholdforeveryC�A=4�B:�AB
�CB
�A�B CBandAB2AB:Thus,forany0r0r,�m�3log((mr))+log(")r0
�mr0
� m�3log(�mr
)+log(")�r0 m!r0= 1�3log(�mr
)�log(")+r0 m!r01�3rlog(m)+3�log(")+r0 mr01�4rlog(m) mr01�4r0
rlog(m) m(1�):Hence,m�3log(�mr
)+log(")r=rXr0=0m�3log(�mr
)+log(")r0�rXr0=0(1�)mr0=(1�)mr;asclaimed. BAproofofLemma4.10usinghashingInthissectionweprovethefollowingslightlyweakerversionofLemma4.10.LemmaB.1.LetVFm2suchthatjVj�2m�t.Thentherearemorethan�m�t�2dlog((m�tr))er
linearlyindependentpolynomialsofdegreerthataredenedonV.NoticethattheonlydierencebetweenLemma4.10andLemmaB.1istheconstant2inthelowerbound.ThemainideaintheproofisshowingthatthereexistsalineartransformationTsuchthattheprojectionofthesetT(V)onto(roughly)therstlog(jVj)coordinatescontainsaballofradiusraroundsomepoint.Sincerestrictingmonomials,ofdegreer,toaballofradiusryieldslinearlyindependentfunctions,theclaimfollows.ToprovethatarandomtransformationhasalargeprojectionontotherstcoordinatesweusetheleftoverhashlemmaofImpagliazzoetal.[ILL89].ThisiswherewelosecomparedtoLemma4.10.Thelemmaof[ILL89]givesmoreinformationthanjustalargeprojection(i.e.,thatthedistributionontheprojectionisclosetouniform)andsoitdoesnotgetthesameparametersthatwecangetusingtheresultofWei(Theorem4.14).Proof.Westartbyprovingthat,afterasuitablelineartransformation,theprojectionofVontotherstcoordinatescontainsalargeball.43