Chapter Cartesian Genetic Programming Julian F - PDF document

Chapter2 CartesianGeneticProgramming JulianF.Miller 2.1OriginsofCGP Cartesiangeneticprogramminggrewfromamethodofevolvingdigitalcircuitsde- velopedbyMilleretal.in1997[ 8 ].Howevertheterm‘Cartesiangeneticprogram- ming’rstappearedin1999[ 5 ]andwasproposedasageneralformofgenetic programmingin2000[ 7 ].Itiscalled‘Cartesian’becauseitrepresentsaprogram usingatwo-dimensionalgridofnodes(seeSect. 2.2 ). 2.2GeneralFormofCGP InCGP,programsarerepresentedintheformofdirectedacyclicgraphs.These graphsarerepresentedasatwo-dimensionalgridofcomputationalnodes.Thegenes thatmakeupthegenotypeinCGPareintegersthatrepresentwhereanodegetsits data,whatoperationsthenodeperformsonthedataandwheretheoutputdatare- quiredbytheuseristobeobtained.Whenthegenotypeisdecoded,somenodesmay beignored.Thishappenswhennodeoutputsarenotusedinthecalculationofout- putdata.Whenthishappens,werefertothenodesandtheirgenesas‘non-coding’. Wecalltheprogramthatresultsfromthedecodingofagenotypeaphenotype.The genotypeinCGPhasaxedlength.However,thesizeofthephenotype(intermsof thenumberofcomputationalnodes)canbeanythingfromzeronodestothenumber ofnodesdenedinthegenotype.Aphenotypewouldhavezeronodesifallthepro- gramoutputsweredirectlyconnectedtoprograminputs.Aphenotypewouldhave thesamenumberofnodesasdenedinthegenotypewheneverynodeinthegraph wasrequired.Thegenotype–phenotypemappingusedinCGPisoneofitsdening characteristics. ThetypesofcomputationalnodefunctionsusedinCGParedecidedbytheuser andarelistedinafunctionlook-uptable.InCGP,eachnodeinthedirectedgraph J.F.Miller(ed.), CartesianGeneticProgramming ,NaturalComputingSeries, DOI 10.1007/978-3-642-17310-3 2 ,©Springer-VerlagBerlinHeidelberg2011 17 JulianF.Millerrepresentsaparticularfunctionandisencodedbyanumberofgenes.Onegeneistheaddressofthecomputationalnodefunctioninthefunctionlook-uptable.Wecallthisafunctiongene.Theremainingnodegenessaywherethenodegetsitsdatafrom.Thesegenesrepresentaddressesinadatastructure(typicallyanarray).Wecalltheseconnectiongenes.Nodestaketheirinputsinafeed-forwardmannerfromeithertheoutputofnodesinapreviouscolumnorfromaprograminput(whichissometimescalledaterminal).Thenumberofconnectiongenesanodehasischosentobethemaximumnumberofinputs(oftencalledthearity)thatanyfunctioninthefunctionlook-uptablehas.Theprogramdatainputsaregiventheabsolutedataaddresses0tominus1whereisthenumberofprograminputs.Thedataout-putsofnodesinthegenotypearegivenaddressessequentially,columnbycolumn,startingfrom1,whereistheuser-determinedupperboundofthenumberofnodes.ThegeneralformofaCartesiangeneticprogramisshowninFig.Iftheproblemrequiresprogramoutputs,thenintegersareaddedtotheendofthegenotype.Ingeneral,theremaybeanumberofoutputgenes()whichspecifywheretheprogramoutputsaretakenfrom.Eachoftheseisanaddressofanodewheretheprogramoutputdataistakenfrom.Nodesincolumnscannotbeconnectedtoeachother.Thegraphisdirectedandfeed-forward;thismeansthatanodemayonlyhaveitsinputsconnectedtoeitherinputdataortheoutputofanodeinapreviouscolumn.ThestructureofthegenotypeisseenbelowtheschematicinFig..Allnodefunctiongenesareintegeraddressesinalook-uptableoffunctions.Allconnectiongenesaredataaddressesandareintegerstakingvaluesbetween0andtheaddressofthenodeatthebottomofthepreviouscolumnofnodes.CGPhasthreeparametersthatarechosenbytheuser.Thesearetheofcolumns,thenumberofrowslevels-back.Thesearedenotedby,respectively.Theproductofthersttwoparametersdeterminethemaximumnumberofcomputationalnodesallowed:.Theparametercontrolstheconnectivityofthegraphencoded.Levels-backconstrainswhichcolumnsanodecangetitsinputsfrom.If1,anodecangetitsinputsonlyfromanodeinthecolumnonitsimmediateleftorfromaprimaryinput.If2,anodecanhaveitsinputsconnectedtotheoutputsofanynodesintheimmediatelefttwocolumnsofnodesoraprimaryinput.Ifonewishestoallownodestoconnecttoanynodesontheirleft,then.Varyingtheseparameterscanresultinvariouskindsofgraphtopologies.Choosingthenumberofcolumnstobesmallandthenumberofrowstobelargeresultsingraphsthataretallandthin.Choosingthenumberofcolumnstobelargeandthenumberofrowstobesmallresultsinshort,widegraphs.Choosinglevels-backtobeoneproduceshighlylayeredgraphsinwhichcalculationsarecarriedoutcolumnbycolumn.Animportantspecialcaseofthesethreeparametersoccurswhenthenumberofrowsischosentobeoneandlevels-backissettobethenumberofcolumns.InthiscasethegenotypecanrepresentanyboundeddirectedgraphwheretheupperboundisdeterminedbythenumberofAswesawbrieyinChap.,oneofthebenetsofagraph-basedrepresentationofaprogramisthatgraphs,bydenition,allowtheimplicitreuseofnodes,asa 2CartesianGeneticProgramming19 Fig.2.1GeneralformofCGP.Itisagridofnodeswhosefunctionsarechosenfromasetofprimitivefunctions.Thegridhascolumnsandrows.Thenumberofprograminputsisthenumberofprogramoutputsis.Eachnodeisassumedtotakeasmanyinputsasthemaximumfunctionarity.Everydatainputandnodeoutputislabeledconsecutively(startingat0),whichgivesitauniquedataaddresswhichspecieswheretheinputdataornodeoutputvaluecanbeaccessed(showninthegureontheoutputsofinputsandnodes).nodecanbeconnectedtotheoutputofanypreviousnodeinthegraph.Inaddition,CGPcanrepresentprogramshavinganarbitrarynumberofoutputs.InSect.wewilldiscusstheadvantagesofnon-codinggenes.ThisgivesCGPanumberofadvantagesovertree-basedGPrepresentations.2.3AllelicConstraintsIntheprevioussection,wediscussedtheinteger-basedCGPgenotyperepresenta-tion.Thevaluesthatgenescantake(i.e.alleles)arehighlyconstrainedinCGP.Whengenotypesareinitializedormutated,theseconstraintsshouldbeobeyed.Firstofall,theallelesoffunctiongenesmusttakevalidaddressvaluesinthelook-uptableofprimitivefunctions.Letrepresentthenumberofallowedprimitivenodefunctions.ThenmustobeyConsideranodeincolumn.Thevaluestakenbytheconnectiongenesofallnodesincolumnareasfollows.If JulianF.MillerOutputgenescanconnecttoanynodeorinput:2.4ExamplesCGPcanrepresentmanydifferentkindsofcomputationalstructures.Inthissection,wediscussthreeexamplesofthis.TherstexampleiswhereaCGPgenotypeen-codesadigitalcircuit.Inthesecondexample,aCGPgenotyperepresentsasetofmathematicalequations.Inthethirdexample,aCGPgenotyperepresentsapicture.Intherstexample,thetypeofdatainputisasinglebit;inthesecond,itisrealnumbers.Intheartexample,itisunsignedeight-bitnumbers.2.4.1ADigitalCircuitTheevolvedgenotypeshowninFig.aroseusingCGPgenotypeparameters1and10.Itrepresentsadigitalcombinationalcircuitcalledatwo-bitparallelmultiplier.Itmultipliestwotwo-bitnumberstogether,soitrequiresfourinputsandfouroutputs.Therearefourprimitivefunctionsinthefunctionset(logicgates).Lettherstandsecondinputstothesegatesbe.Thenthefourfunc-tions(withthefunctiongeneinparentheses)areAND()(0),AND(,NOT()(2)andOR()(3).OnecanseeinFig.thattwonodes(withlabels6and10)arenotused,sincenocircuitoutputrequiresthem.Thesearenon-codingnodes.Theyareshowningrey.showsaCGPgenotypeandthecorrespondingphenotype.2.4.2MathematicalEquationsSupposethatthefunctionsofnodescanbechosenfromthearithmeticoperationsplus,minus,multiplyanddivide.Wehaveallocatedthefunctiongenesasfollows.Plusisrepresentedbythefunctiongenebeingequaltozero,minusisrepresented 2CartesianGeneticProgramming21 Fig.2.2ACGPgenotypeandcorrespondingphenotypeforatwo-bitmultipliercircuit.Theunder-linedgenesinthegenotypeencodethefunctionofeachnode.Thefunctionlook-uptableisAND(0),ANDwithoneinputinverted(1),XOR(2)andOR(3).Theaddressesareshownunderneatheachprograminputandnodeinthegenotypeandphenotype.Theinactiveareasofthegenotypeandphenotypeareshowningreydashes(nodes6and10).byone,multiplybytwoanddividebythree.Letussupposethatourprogramhastworeal-valuedinputs,whichsymbolicallywedenoteby.Letussupposethatweneedfourprogramoutputs,whichwedenote.Wehavechosenthenumberofcolumnstobethreeandthenumberofrowstobetwo.Inthisexample,assumethatlevels-back,istwo.AnexamplegenotypeandaschematicofthephenotypeareshowninFig..Thephenotypeisthefollowingsetofequations: x02Šx1OD=x02. JulianF.Miller Fig.2.3ACGPgenotypeandcorrespondingschematicphenotypeforasetoffourmathematicalequations.Theunderlinedgenesinthegenotypeencodethefunctionofeachnode.Thefunctionlook-uptableisadd(0),subtract(1),multiply(2)anddivide(3).Theaddressesareshownun-derneatheachprograminputandnodeinthegenotypeandphenotype.Theinactiveareasofthegenotypeandphenotypeareshowningreydashes(node6).2.4.3ArtAsimplewaytogeneratepicturesusingCGPistoallowtheintegerpixelCartesiancoordinatestobetheinputstoaCGPgenotype.Threeoutputscanthenbeallowedwhichwillbeusedtodeterminethered,greenandbluecomponentsofthepixel’scolour.TheCGPoutputshavetobemappedinsomewaysothattheyonlytakevaluesbetween0and255,sothatvalidpixelcoloursaredened.ACGPprogramisexecutedforallthepixelcoordinatesdeningatwo-dimensionalregion.Inthisway,apicturewillbeproduced.InFig.,agenotypeisshownwithacorrespond-ingschematicofthephenotype.ThesetoffunctiongenesandcorrespondingnodefunctionsareshowninTable.Inalaterchapter,waysofdevelopingartusingCGPwillbeconsideredindetail.ThefunctionsinTablehavebeencarefullychosensothattheywillreturnanintegervaluebetween0and255whentheinputs(Cartesiancoordinates)arebothbetween0and255.TheevolvedgenotypeshowninFig.usesonlyfourfunctiongenes:5,9,6and13.Wedenotetheoutputsofnodesby,wheretheoutputaddressofthenode.Thered,greenandbluechannelsofthepixelvalues)aregivenasbelow: 2CartesianGeneticProgramming23Table2.1Primitivefunctionsetusedinartexample FunctiongeneFunctiondenition 0x1y2 x+y3 4255 2 5255 2 6255 2 7expmod256mod2569coshmod2561025511255 12255 13255 8x+y x2+y2 mod256mod256mod256 JulianF.Miller Fig.2.4ACGPgenotypeandcorrespondingschematicphenotypeforaprogramthatdenesapicture.Theunderlinedgenesinthegenotypeencodethefunctionofeachnode.Thefunctionlook-uptableisgiveninTable.Theaddressesareshownunderneatheachprograminputandnodeinthegenotypeandphenotype.Theinactiveareasofthegenotypeandphenotypeareshowningreydashes(node6). +2 ,g3=2 2+2 mod256 3+2 Whenthesemathematicalequationsareexecutedforall256pixellocations,theyproducethepicture(theactualpictureisincolour)showninFig.2.5DecodingaCGPGenotypeSofar,wehaveillustratedthegenotype-decodingprocessinadiagrammaticway.However,thealgorithmicprocessisrecursiveinnatureandworksfromtheoutputgenesrst.Theprocessbeginsbylookingatwhichnodesare‘activated’bybeingdirectlyaddressedbyoutputgenes.Thenthesenodesareexaminedtondoutwhichnodestheyinturnrequire.AdetailedexampleofthisdecodingprocessisshowninItisimportanttoobservethatinthedecodingprocess,non-codingnodesarenotprocessed,sothathavingnon-codinggenespresentslittlecomputationaloverhead. 2CartesianGeneticProgramming25 Fig.2.5PictureproducedwhentheprogramencodedintheevolvedgenotypeshowninFig.executed.Itaroseinthesixthgeneration.Theuserdecideswhichgenotypewillbethenextparent.Therearevariouswaysthatthisdecodingprocesscanbeimplemented.Onewaywouldbetodoitrecursively;anotherwouldbetodeterminewhichnodesareactive(inarecursiveway)andrecordthemforfutureuse,andonlyprocessthose.Proce-inthenextsectiondetailthelatter.Thepossibilityofimprovingefciencybystrippingoutnon-codinginstructionspriortophenotypeevaluationhasalsobeensuggestedforLinearGP[2.5.1AlgorithmsforDecodingaCGPGenotypeInthissection,wewillpresentalgorithmicproceduresfordecodingaCGPgeno-type.Thealgorithmhastwomainparts:therstisaprocedurethatdetermineshowmanynodesareusedandtheaddressesofthosenodes.ThisisshowninProcedure.Thesecondpresentstheinputdatatothenodesandcalculatestheoutputsforasingledatainput.WedenotethemaximumnumberofaddressesintheCGPgraph,thetotalnumberofgenesinthegenotypeby,thenumberofgenesinanodeby,andthenumberofactiveorusednodesbyIntheprocedure,anumberofarraysarementioned.Firstly,ittakestheCGPgenotypestoredinanarrayarrayLg]asanargument.Itpassesbackasanargumentanarrayholdingtheaddressesofthenodesinthegenotypethatareused.Wedenotethisby.Italsoreturnshowmanynodesareused.Internally,itusesaBooleanarrayholdingwhetheranynodeisused,calledcalledM].ThisisinitializedtoFALSEAnarrayisusedtostorethenodegenesforanyparticularnode.ItalsoassumesthatafunctionArityreturnsthearityofanyfunctioninthefunctionset.Oncewehavetheinformationaboutwhichnodesareused,wecanefcientlydecodethegenotypewithsomeinputdatatondoutwhatdatatheencodedpro- JulianF.Miller Fig.2.6ThedecodingprocedureforaCGPgenotypeforthetwo-bitmultiplierproblem.(a)Output)connectstotheoutputofnode4;movetonode4.(b)Node4connectstotheprograminputs0and2;thereforetheoutputAisdecoded.MovetooutputB.(c)OutputB()connectstotheoutputofnode9;movetonode9.(d)Node9connectstotheoutputofnodes5and7;movetonodes5and7.(e)Nodes5and7connecttotheprograminputs0,3,1and2;thereforeoutputBisdecoded.MovetooutputC.TheprocedurecontinuesuntiloutputC()andoutputD(aredecoded(steps(f)to(h)andsteps(i)to(j)respectively).Whenalloutputsaredecoded,thegenotypeisfullydecoded.gramgivesasanoutput.ThisprocedureisshowninProcedure.ItassumesthatinputdataisstoredinanarrayDIN,andtheparticularitemofthatdatathatisbeingusedasinputtotheCGPgenotypeisitem.Theprocedurereturnsthecalculatedoutputdatainanarray.Internally,itusestwoarrays:,whichstoresthecalcu-latedoutputsofusednodesand,whichstorestheinputdatabeingpresentedtoanindividualnode.Thesymbolistheaddressinthegenotypeoftherstgeneinanode.Thesymbolistheaddressofanodeinthearray.Theprocedureassumesthatafunctionimplementsthefunctionsinthefunctionlook-uptable.ThetnessfunctionrequiredforanevolutionalgorithmisgiveninProcedure.ItisassumedthatthereisaprocedureEvaluateCGPthat,giventheCGPcal- 2CartesianGeneticProgramming27 Procedure2.1Determiningwhichnodesneedtobeprocessed NodesToProcess//returnthenumberofnodestoprocessforallsuchthat00i]=FALSEendforforallsuchthatthatG[i]]TRUEendforforallsuchthat//Findactivenodesnodesi]TRUEindexforallsuchthat0//storenodegenesinNGGj]G[indexendforforallsuchthat0ArityynnŠ1])do15:NU[NG[j]]TRUEendforendifendforforallsuchthat//storeactivenodeaddressesinNPPj]=TRUEEnu]j23:nunu+124:endifendforreturn Procedure2.2DecodingCGPtogettheoutput DecodeCGPDINitemforallsuchthat00i]DINNitemendforforallsuchthat00j]Šni7:gnnn8:forallsuchthat0//storedataneededbyanodenodei]o[G[g+i]]10:endfororg+nnŠ1]//getfunctiongeneofnodenoden+ni]=NF(in,f)//calculateoutputofnodeendforforallsuchthat00j]o[G[LgŠno+j]]16:endfor JulianF.MillerculatedoutputsandthedesiredprogramoutputsDOUT,calculatesthetnessofthegenotypeforasingleinputdataitem.Theprocedureassumesthattherearetnesscasesthatneedtobeconsidered.Indigital-circuitevolutiontheusualnumberoftnesscasesis.Note,however,thatProcedureonlyneedstobeexecutedonceforagenotype,irrespectiveofthenumberoftnesscases. Procedure2.3CalculatingthetnessofaCGPgenotype FitnessCGPNodesToProcessfitforallsuchthat0DecodeCGPDINitemEvaluateCGPDOUTfitfitendfor 2.6EvolutionofCGPGenotypes2.6.1MutationThemutationoperatorusedinCGPisapointmutationoperator.Inapointmutation,analleleatarandomlychosengenelocationischangedtoanothervalidrandomvalue(seeSect.).Ifafunctiongeneischosenformutation,thenavalidvalueistheaddressofanyfunctioninthefunctionset,whereasifaninputgeneischosenformutation,thenavalidvalueistheaddressoftheoutputofanypreviousnodeinthegenotypeorofanyprograminput.Also,avalidvalueforaprogramoutputgeneistheaddressoftheoutputofanynodeinthegenotypeortheaddressofaprograminput.Thenumberofgenesinthegenotypethatcanbemutatedinasingleapplicationofthemutationoperatorisdenedbytheuser,andisnormallyapercentageofthetotalnumberofgenesinthegenotype.Werefertothelatterasmutationrate,andwillusethesymboltorepresentit.Oftenonewantstorefertotheactualnumberofgenesitesthatcouldbemutatedinagenotypeofagivenlength.Wegivethisquantitythesymbol,sothat.WewilltalkaboutsuitablechoicesfortheparametersinSect.AnexampleofthepointmutationoperatorisshowninFig.,whichhighlightshowasmallchangeinthegenotypecansometimesproducealargechangeinthe 2CartesianGeneticProgramming29 Fig.2.7AnexampleofthepointmutationoperatorbeforeandafteritisappliedtoaCGPgenotype,andthecorrespondingphenotypes.Asinglepointmutationoccursintheprogramoutputgene(changingthevaluefrom6to7.Thiscausesnodes3and7tobecomeactive,whilstmakingnodes2,5and6inactive.Theinactiveareasareshowningreydashes.2.6.2RecombinationCrossoveroperatorshavereceivedrelativelylittleattentioninCGP.Originally,aone-pointcrossoveroperatorwasusedinCGP(similartothe-pointcrossoveringeneticalgorithms)butwasfoundtobedisruptivetothesubgraphswithinthechro-mosome,andhadadetrimentalaffectontheperformanceofCGP[].SomeworkbyCleggetal.[]hasinvestigatedcrossoverinCGP(andGPingeneral).Theirapproachusesaoating-pointcrossoveroperator,similartothatfoundinevolu-tionaryprogramming,andalsoaddsanextralayerofencodingtothegenotype,inwhichallgenesareencodedasaoating-pointnumberintherangeange0,1].Alargerpopulationandtournamentselectionwerealsousedinsteadoftheevolu-tionarystrategynormallyusedinCGP,totryandimprovethepopulationdiversity. JulianF.MillerTheresultsofthenewapproachappearpromisingwhenappliedtotwosymbolicregressionproblems,butfurtherworkisrequiredonarangeofproblemsinordertoassessitsadvantages[].Crossoverhasalsobeenfoundtobeusefulinanimage-processingapplicationasdiscussedinSect..Crossoveroperators(cone-basedcrossover)havebeendevisedfordigital-circuitevolution(seeSect.3.6.2).Insitua-tionswhereaCGPgenotypeisdividedintoacollectionofchromosomes,crossovercanbeveryeffective.Sect.discusseshownewgenotypescreatedbyselectingthebestchromosomesfromparents’genotypescanproducesuper-individuals.Thisallowsdifcultmultiple-outputproblemstobesolved.2.6.3EvolutionaryAlgorithmAvariantonasimpleevolutionaryalgorithmknownasa1evolutionaryalgo-rithm[]iswidelyusedforCGP.Usuallyischosentobe4.ThishastheformshowninProcedure Procedure2.4evolutionarystrategy forallsuchthat02:Randomlygenerateindividualendfor4:Selectthettestindividual,whichispromotedastheparentasolutionisnotfoundthegenerationlimitisnotreachedforallsuchthat07:Mutatetheparenttogenerateoffspringendfor9:Generatethettestindividualusingthefollowingrules:anoffspringgenotypehasabetterortnessthantheparent11:Offspringgenotypeischosenasttest13:Theparentchromosomeremainsthettestendifendwhile Online10oftheprocedurethereisanextraconditionthatwhenoffspringgeno-typesinthepopulationhavethesametnessastheparentandthereisnooffspringthatisbetterthantheparent,inthatcaseanoffspringischosenasthenewparent.Thisisaveryimportantfeatureofthealgorithm,whichmakesgooduseofredun-dancyinCGPgenotypes.ThisisdiscussedinSect. 2CartesianGeneticProgramming312.7GeneticRedundancyinCGPGenotypesWehavealreadyseenthatinaCGPgenotypetheremaybegenesthatareentirelyinactive,havingnoinuenceonthephenotypeandhenceonthetness.Suchinac-tivegenesthereforehaveaneutraleffectongenotypetness.Thisphenomenonisoftenreferredtoasneutrality.CGPgenotypesaredominatedbyredundantgenes.Forinstance,MillerandSmithshowedthatingenotypeshaving4000nodes,thepercentageofinactivenodesisapproximately95%![TheinuenceofneutralityinCGPhasbeeninvestigatedindetail[]andhasbeenshowntobeextremelybenecialtotheefciencyoftheevolution-aryprocessonarangeoftestproblems.JusthowimportantneutraldriftcanbetotheeffectivenessofCGPisillustratedinFig.Thisshowsthenormalizedbesttnessvalueachievedintwosetsof100inde-pendentevolutionaryrunsof10milliongenerations[].Thetargetofevolutionwastoevolveacorrectthree-bitdigitalparallelmultiplier.Intherstsetofruns,anoffspringcouldreplaceaparentwhenithadthesametnessastheparentandtherewasnootherpopulationmemberwithabettertness(asinline10ofProce-).Inthegure,thenaltnessvaluesareindicatedbydiamondsymbols.Inthesecondset,aparentwasreplacedonlywhenanoffspringhadastrictlybettertnessvalue.Theseresultsareindicatedbyplussymbols.Inthecasewhereneutraldriftwasallowed,27correctmultiplierswereevolved.Also,manyoftheothercir-cuitswereverynearlycorrect.Inthecasewherenoneutraldriftwasallowed,therewerenorunsthatproducedacorrectmultiplier,andtheaveragetnessvaluesareconsiderablylower.Itispossiblethatbyanalyzingwithinanevolutionaryalgorithmwhethermu-tationaloffspringarephenotypicallydifferentfromparents,onemaybeabletore-ducethenumberoftnessevaluations.Sincelargeamountsofnon-codinggenesarehelpfultoevolution,itismorelikelythatmutationswilloccuronlyinnon-codingsectionsofthegenotype;suchgenotypeswillhavethesametnessastheirparentsanddonotneedtobeevaluated.ToaccomplishthiswouldrequireaslightchangetotheevolutionaryalgorithminProcedure.Onewouldkeeparecordofthenodesthatneedtobeprocessedinthegenotypethatispromoted(i.e.arrayinSect.).Then,ifanoffspringhadexactlythesamenodesthatwereactiveasintheparent,itwouldbeassignedtheparent’stness.Whetherinpracticethishappenssufcientlyoftentowarranttheextraprocessingrequiredhasnotbeeninvestigated.2.8ParameterSettingsforCGPToarriveatgoodparametersforCGPnormallyrequiressomeexperimentationontheproblembeingconsidered.However,somegeneraladvicecanbegiven.Asuit-ablemutationratedependsonthelengthofthegenotype(innodes).Asaruleofthumb,oneshoulduseabout1%mutationifamaximumof100nodesareused(i.e.100).Letusassumethatallprimitivefunctionshavetwoconnectiongenes JulianF.Miller Fig.2.8Thenormalizedbesttnessvalueachievedintwosetsof100independentevolutionaryrunsof10milliongenerations.Thetargetofevolutionwastoevolveacorrectthree-bitdigitalparallelmultiplier.Inonesetofruns,neutraldriftwasallowed,andintheother,neutraldriftwasnotallowed.Theevolutionaryalgorithmwasunabletoproduceacorrectcircuitinthesecondcase.andtheproblemhasasingleoutput.Thenagenotypewithamaximumof100nodeswillrequire301genes.So1%mutationwouldmeanthatuptothreegeneswouldbemutatedintheparenttocreateanoffspring.Experienceshowsthattoachievereasonablyfastevolutiononeshouldarrangethemutationratetobesuchthatthenumberofgenelocationschosenformutationisaslowlygrowingfunctionofthegenotypelength.Forinstance,in[]itwasfoundthat90provedtobeagoodvaluewhen004(4000two-inputnodesandfouroutputs).Thiscorresponds75%.When154(50two-inputnodesandfouroutputs),agoodvaluewas6,whichcorrespondsto4%.Evensmallergenotypesusuallyrequirehighermutationratesstillforfastevolution.Generallyspeaking,whenoptimalmutationratesareused,largergenotypesre-quirefewertnessevaluationstoachieveevolutionarysuccessthandosmallergeno-types.Forinstance,MillerandSmithfoundthatthenumberoftnessevaluations(i.e.genotypeswhosetnessiscalculated)requiredtosuccessfullyevolveatwo-bitmultipliercircuitwaslowerforgenotypeshaving4000nodesthanforgenotypesofsmallerlength[]!ThewaytounderstandthisistothinkabouttheusefulnessofneutraldriftintheevolutionofCGPgenotypes.LargerCGPgenotypeshaveamuchlargerpercentageofnon-codinggenesthandosmallergenotypes,sothepotential 2CartesianGeneticProgrammingforneutraldriftismuchlarger.ThisisanotherillustrationofthegreatimportanceofneutraldriftinevolutionaryalgorithmsforCGP.So,wehaveseenthatlargegenotypesleadtomoreefcientevolution;however,givenacertaingenotypelength,whatistheoptimalnumberofcolumnsnumberofrows?Theadvicehereisasfollows.Iftherearenoproblemswithimplementingarbitrarydirectedgraphs,thentherecommendedchoiceofthesepa-rametersis1with.However,ifforinstanceoneisevolvingcircuitsforimplementationofevolvedCGPgenotypesondigitaldevices(withlimitedroutingresources),itisoftenusefultochoose.Itshouldbestressedthattheserec-ommendationsare‘rulesofthumb’,asnodetailedquantitativeworkonthisaspecthasbeenpublished.CGPusesverysmallpopulationsizes(5inthecasedescribedinSect.).Sooneshouldexpectlargenumbersofgenerationstobeused.Despitethis,innumer-ousstudies,ithasbeenfoundthattheaveragenumberoftnessevaluationsrequiredtosolvemanyproblemscanbefavourablycomparedwithotherformsofGP(seeforinstance[2.9CyclicCGPCGPhaslargelybeenusedinanacyclicform,wheregraphshavenofeedback.However,thereisnofundamentalreasonforthis.TherepresentaionofgraphsusedinCGPiseasilyadaptedtoencodecyclicgraphs.Onemerelyneedstoremovetherestrictionthatallelesforaparticularnodehavetotakevalueslessthantheposition(address)ofthenode.However,despitethis,therehasbeenlittlepublishedworkwherethisrestrictionhasbeenremoved.OneexceptionistherecentworkofKhanetal.,whohaveencodedarticialneuralnetworksinCGP[].TheyallowedfeedbackandusedCGPtoevolverecurrentneuralnetworks.OtherexceptionsaretheworkofWalkeretal.,whoappliedCGPtotheevolutionofmachinecodeofsequentialandparallelprogramsonaMOVEprocessor[]andLiuetal.,whoproposedadual-layerCGPgenotyperepresentationinwhichonelayerencodedprocessorinstructionsandtheotherloopcontrolparameters[].AlsoSect.describeshowcyclicanaloguecircuitscanbeencodedinCGP.Clearly,suchaninvestigationswouldextendtheexpressivityofprograms(sincefeedbackimplieseitherrecursionoriteration).Itwouldalsoallowbothsynchronousandasynchronouscircuitstobeevolved.Afullinvestigationofthistopicremainsforthefuture.References1.Brameier,M.,Banzhaf,W.:AComparisonofLinearGeneticProgrammingandNeuralNet-worksinMedicalDataMining.IEEETransactionsonEvolutionaryComputation(1),17–26 JulianF.Miller2.Clegg,J.,Walker,J.A.,Miller,J.F.:ANewCrossoverTechniqueforCartesianGeneticPro-gramming.In:Proc.GeneticandEvolutionaryComputationConference,pp.1580–1587.ACMPress(2007)3.Khan,M.M.,Khan,G.M.,Miller,J.F.:EfcientrepresentationofRecurrentNeuralNetworksforMarkovian/Non-MarkovianNon-linearControlProblems.In:A.E.Hassanien,A.Abra-ham,F.Marcelloni,H.Hagras,M.Antonelli,T.P.Hong(eds.)Proc.InternationalConferenceonIntelligentSystemsDesignandApplications,pp.615–620.IEEE(2010)4.Liu,Y.,Tempesti,G.,Walker,J.A.,Timmis,J.,Tyrrell,A.M.,Bremner,P.:ASelf-ScalingInstructionGeneratorUsingCartesianGeneticProgramming.In:Proc.EuropeanConferenceonGeneticProgramming,,vol.6621,pp.299–310.Springer(2011)5.Miller,J.F.:AnEmpiricalStudyoftheEfciencyofLearningBooleanFunctionsusingaCartesianGeneticProgrammingApproach.In:Proc.GeneticandEvolutionaryComputationConference,pp.1135–1142.MorganKaufmann(1999)6.Miller,J.F.,Smith,S.L.:RedundancyandComputationalEfciencyinCartesianGeneticPro-gramming.IEEETransactionsonEvolutionaryComputation(2),167–174(2006)7.Miller,J.F.,Thomson,P.:CartesianGeneticProgramming.In:Proc.EuropeanConferenceonGeneticProgramming,,vol.1802,pp.121–132.Springer(2000)8.Miller,J.F.,Thomson,P.,Fogarty,T.C.:DesigningElectronicCircuitsUsingEvolutionaryAlgorithms:ArithmeticCircuits:ACaseStudy.In:D.Quagliarella,J.Periaux,C.Poloni,G.Winter(eds.)GeneticAlgorithmsandEvolutionStrategiesinEngineeringandComputerScience:RecentAdvancementsandIndustrialApplications,pp.105–131.Wiley(1998)9.Rechenberg,I.:Evolutionsstrategie–OptimierungtechnischerSystemenachPrinzipienderbiologischenEvolution.Ph.D.thesis,TechnicalUniversityofBerlin,Germany(1971)10.Vassilev,V.K.,Miller,J.F.:TheAdvantagesofLandscapeNeutralityinDigitalCircuitEvolu-tion.In:Proc.InternationalConferenceonEvolvableSystems,,vol.1801,pp.252–263.Springer(2000)11.Walker,J.A.,Liu,Y.,Tempesti,G.,Tyrrell,A.M.:AutomaticCodeGenerationonaMOVEProcessorUsingCartesianGeneticProgramming.In:Proc.InternationalConferenceonEvolvableSystems:FromBiologytoHardware,,vol.6274,pp.238–249.Springer12.Walker,J.A.,Miller,J.F.:AutomaticAcquisition,EvolutionandRe-useofModulesinCarte-sianGeneticProgramming.IEEETransactionsonEvolutionaryComputation,397–41713.Yu,T.,Miller,J.F.:NeutralityandtheevolvabilityofBooleanfunctionlandscape.In:Proc.Eu-ropeanConferenceonGeneticProgramming,,vol.2038,pp.204–217.Springer(2001)14.Yu,T.,Miller,J.F.:FindingNeedlesinHaystacksisnotHardwithNeutrality.In:Proc.Euro-peanConferenceonGeneticProgramming,,vol.2278,pp.13–25.Springer(2002)