Suppose we are interested in ary codes not necessarily linear of block length and minimum distance that have many codewords What is the largest size such a code can have This is a fundamental quantity for which we de64257ne a notation below De64257n ID: 39545
Download Pdf The PPT/PDF document "Introduction to Coding Theory CMU Spring..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
TherealsoexistlinearcodesofsizegivenbytheGilbert-Varshamovbound:Exercise1Byasuitableadaptationofthegreedyprocedure,provethattherealsoexistsalinearcodeoverFqofdimensionatleastn�blogqVolq(n;d�1)c.TheGilbert-Varshamovboundwasactuallyprovedintwoindependentworks(Gilbert,1952)and(Varshamov,1957).Thelatteractuallyprovedtheexistenceoflinearcodesandinfactgotaslightlysharperboundstatedbelow.(YoucanverifythattheHammingcodeinfactattainsthisboundford=3.)Exercise2Foreveryprimepowerq,andintegersn;k;d,provethatthereexistsan[n;k;d]qlinearcodewithkn�blogqd�2Xj=0n�1j(q�1)jc�1:Infact,onecanprovethatarandomlinearcodealmostmatchestheGilbert-Varshamovboundwithhighprobability,sosuchlinearcodesexistinabundance.Butbeforestatingthis,wewillswitchtotheasymptoticviewpoint,expressingthelowerboundintermsoftheratevs.relativedistancetrade-o .1.1EntropyfunctionandvolumeofHammingballsWenowgiveanasymptoticestimateofthevolumeVolq(n;d)whend=pnforp2[0;1�1=q]heldxedandngrowing.Thisvolumeturnsouttobeverywellapproximatedbytheexponentialqhq(p)nwherehq()isthe\entropyfunction"denedbelow.Denition4(Entropyfunction)Forapositiveintegerq2,denetheq-aryentropyfunctionhq:[0;1]!Rasfollows:hq(x)=xlogq(q�1)�xlogqx�(1�x)logq(1�x):Ofspecialinterestisthebinaryentropyfunctionh(x)=xlog1 x+(1�x)log1 1�xwhereweusethenotationalconventionthatlog=log2.IfXisthef0;1g-valuedrandomvariablesuchthatP[X=1]=pandP[X=0]=1�p,thentheShannonentropyofX,H(X),equalsh(p).Inotherwords,h(p)istheuncertaintyintheoutcomeofap-biasedcointoss(whichlandsheadswithprobabilitypandtailswithprobability1�p).Thefunctionhqiscontinuousandincreasingintheinterval[0;1�1=q]withhq(0)=0andhq(1�1=q)=1.Thebinaryentropyfunctionissymmetricaroundthex=1=2line:h(1�x)=h(x).Wecandenetheinverseoftheentropyfunctionasfollows.Fory2[0;1],theinverseh�1q(y)isequaltotheuniquex2[0;1�1=q]satisfyinghq(x)=y.2 1.4Somecommentsonattaining/beatingtheGVboundWehaveseenthatthereexistbinarylinearcodesthatmeettheGilbert-Varshamovbound,andthushaverateapproaching1�h()foratargetrelativedistanceof,01=2.Theproofofthiswasnon-constructive,basedonanexponentialtimealgorithmtoconstructsuchacode(byagreedyalgorithm),orbypickingageneratormatrix(oraparitycheckmatrix)atrandom.ThelatterleadstoapolynomialtimerandomizedMonteCarloconstruction.Iftherewereawaytoascertainifarandomlychosenlinearcodehastheclaimedrelativedistance,thenthiswouldbeapracticalmethodtoconstructcodesofgooddistance;wewillhaveaLasVegasconstructionthatpicksarandomlinearcodeandthenchecksthatithasgoodminimumdistance.Unfortunately,givenalinearcode,computing(orevenapproximating)thevalueofitsminimumdistanceisNP-hard.Anaturalchallengethereforeistogiveanexplicit(i.e.,deterministicpolynomialtime)constructionofacodethatmeetstheGilbert-Varshamovbound(i.e.,hasrateRandrelativedistanceclosetoh�1q(1�R)).Givingsuchaconstructionofbinarycodes(evennon-linearones)remainsanoutstandingopenquestion.Forprimepowersq=p2kforq49,explicitconstructionsofq-arylinearcodesthatnotonlyattainbutsurpasstheGVboundareknown!Thesearebasedonalgebraicgeometryandabeautifulconstructionofalgebraiccurveswithmanyrationalpointsandsmallgenus.Thisisalsooneoftherareexamplesincombinatoricswhereweknowanexplicitconstructionthatbeatstheparametersobtainedbytheprobabilisticmethod.(AnothernotableexampleistheLubotzky-Phillips-SarnakconstructionofRamanujangraphswhosegirthsurpassestheprobabilisticbound.)Whataboutcodesoversmalleralphabets,andinparticularbinarycodes?TheHammingupperboundonsizeofcodes(Lemma13inNotes1)leadstotheasymptoticupperboundR1�h(=2)ontherate.Thisiso byafactorof2inthecoecientofcomparedtotheachievable1�h()rate.WewilllaterseeimprovementstotheHammingbound,butthebestboundwillstillbefarfromtheGilbert-Varshamovbound.Determiningthelargestratepossibleforbinarycodesofrelativedistance2(0;1=2)isanotherfundamentalopenprobleminthesubject.ThepopularconjectureseemstobethattheGilbert-Varshamovboundonrateisasymptoticallytight(i.e.,abinarycodeofrelativedistancemusthaverate1�h()+o(1)),butarguablythereisnostrongevidencethatthismustbethecase.WhilewedonotknowexplicitconstructionsofbinarycodesapproachingtheGVbound,itisstillinterestingtoconstructcodeswhichachievegoodtrade-o s.Thisleadstothefollowingquestions,whicharethecentralquestionsincodingtheoryforanynoisemodel(oncesomeexistentialboundsareestablishedonthetrade-o s,thequestionsbelowpertainingtotheworst-caseoradversarialnoisemodelwhereweimposenorestrictiononthechannelotherthanalimitonthetotalnumberoferrorscaused):1.Canoneexplicitlyconstructanasymptoticallygoodfamilyofbinarycodeswitha\good"ratevs.relativedistancetrade-o ?2.Canoneconstructsuchcodestogetherwithanecientalgorithmtocorrectafractionoferrorsapproachinghalf-the-relativedistance(orevenbeyond)?Wewillanswerboththequestionsinthearmativeinthiscourse.5