Nowowiejska 1519 00665 Warsaw Poland grzegorzjaskiewicz Abstract In this paper it is shown an application of an agentoriented programming paradigm with a reasoning layer based on a logic program ming The presented solution was used to build bots in ID: 15780
Download Pdf The PPT/PDF document "PrologBased Reasoning Layer for CounterS..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
2GrzegorzJaskiewicz-terrorists(TT)andcounter-terrorist(CT).Bothteamsghtagainsteachotherusingmodernrearms.Eachgamelasts30minutesbydefaultandconsistsofseveral5minuterounds.Thereareseveralmaptypeswhicha ectgoalsforbothteams.ASassasination.OneofCT'sbecomesVIP,whichmustreachthesafezone.TT'strytoeliminatetheVIP.CShostagerescue.CT'strytomovehostagestorescuezone.DEbombdefusion.TT'strytoplantabomb,CT'smaytrytodefuseitafter-wards.ESescape.TT'stryto eetotheescapezone,CT'shuntthem.Ifteamissuccessfulinachievingitsgoal,thentheteamwinstheround.Eliminationoftheentireopposingteamusuallyalsomeansavictory.ThegameplayofCoutner-Strikeisveryrealistic,eliminatedplayersdonotrespawnimmediatelyasinmostFPSmultiplayergamesandeachplayercanbeeliminatedwithonlyfewwell-aimedbullets.Therefore,Counter-Strikeisaveryteam-orientedgame,wherecooperationismoreimportantthanskillsofindividualsplayers.Playersdooftenformgroupscalledclans,whoplaytogetheragainstotherclans.ThereareevenorganizedtournamentsinCounter-Strike.Thegamehasrankofocialsportinsomecountries[?].Unfortunately,thegameinitsearlyversionswascompletelyunplayableforasingleplayer,becauseitwasmeanttobeplayedentirelyinmultiplayermode.Communityofthegamefansdevelopedgameextensions,whichcontainedcomputer-controlledplayers-bots[?].CSBotscreatorshaddicultproblemtosolve,becausedevelopedbotshadtoactasteamandco-operate.Inthisarticleithasbeenshowextensionofexistingbot,usingfoundationsofmultiagentsystems.Notethat,conceptofapplicationagent-orientedprogram-minginFPSvideogamesisnotanewonee.g[?],[?],[?].However,therearedi erentimplementationsofthisconceptleadingtodi erentbotarchitectures.Oneoftheapproachestobotprogrammingbasedonthisconceptisprovid-ingscriptinglanguagee.g[?],[?],sothatitiseasytoexperimentwithvariousbehaviors,botinteractionsandtactics.Thisapproachreliesonexpertknowl-edge,whichismergedintoscriptorprogramsourcecode.Completelydi erentapproachistoinvolvemachinelearningandletbotsgainknowledgethroughin-teractionwithotherplayers.Thereisalreadysomeresearchinthisarea[?],[?],[?].DuetocomplexnatureoftheproblemMLtechniquesdonotsolveitentirely,butserveassupportfortechniquesinvolvingdomainknowledgeaboutgame.Inthisworkexpertknowledgebasedapproachisapplied.Oneoftheknowledgerep-resentationscouldberstorderlogic.Thisrepresentationisdeclarative,thusreadabletohuman.ThiswasmainmotivationforchoosingPrologasscriptinglanguage.PresentedAIbotscomponentisdividedintomain2layers:{reasoninglayer-high-leveldecisionmaking,correspondstohumandeclara-tiveknowledge,{executionlayer-low-leveldecisionmaking,correspondstohumanproceduralknowledge. 4GrzegorzJaskiewiczquicklytoenvironmentchanges.Agentcanexecuteoneactionatthetimet,butexecutedactioncouldchange.Thischangessatisfyfollowingconditions:at+1 8������&#x]TJ ;� -1;.93; Td;&#x [00;&#x]TJ ;� -1;.93; Td;&#x [00;&#x]TJ ;� -1;.93; Td;&#x [00;&#x]TJ ;� -1;.93; Td;&#x [00;&#x]TJ ;� -1;.93; Td;&#x [00;&#x]TJ ;� -1;.93; Td;&#x [00;:rr(K(bi;g(t)))ifrr(K(bi;g(t)))6=?atifat(K(bi;g(t)))=(d;continue)c1ifat(K(bi;g(t)))=(d;success)^cont(at)=(c1;:::;cn)rh(K(bi;g(t)))ifat(K(bi;g(t)))=(d;fail)rh(K(bi;g(t)))if:motiv(at)2g(t)Theseconditionsaren'tmutuallyexclusive,ifcon ictoccursrstsatisedcon-ditionischosen.2.2PrologPrologisageneral-purposedeclarativeprogramminglanguage,whichbasesstronglyonarstorderlogic.Prologusesnotionsoffactsandrules,whereasexecutionofPrologprogramisdefactoaformalreasoningprocess.Therefore,itisveryconvenienttouseinarticialintelligenceprogramming[?].OftenmanytypesoflogiccouldbeusedasunderlyingProloglogic,e.g.fuzzylogic[?],modallogic[?],multivaluedlogic[?].Thosedi erenttypesoflogicareusedtoexpressreasoningprocessinwayunderstandabletohumanandallofthoselanguagesareTuringcomplete.Prologdisplaysfollowingproperties,whichmakeitusefulinmultiagentpro-gramming.{declarativereasoningrules,{botsknowledgebaseinasymbolicform-easytounderstandandanalyze,{noseparationbetweencodeanddata-rulescouldbeusedasdata,e.g.botleadercommunicatingorderstoanotherbot.2.3Counter-StrikeBotsCounter-Strikegameplayisasimulatedenvironmentandbotscanbeseenasagentswithinthisenvironment.Theyshowtypicaltraitsofagents:{autonomy,{makingintelligentdecisions,{beingmobile,{communicationandcollaboration.CSemphasisestheteamworkstrongly,whereascollaborativeproblemsolvingisamaindomainofmultiagentsystems.Sousingthisparadigmisnaturalinthistask.It'sinterestingthatthismethodologywasevenearlierusedinnon-collaborativegameslikeQuakewithgoodresults[?].ActionsAlareimplementedashardcoded,butyetparametrizable,behaviorswhichcanbeactedbybots.Suchbehaviorsmayincludegoingtoanavigational 6GrzegorzJaskiewicz Fig.2.ConceptualarchitecturediagramenhancementswereintroducedtoE[POD]bots.Itwillbedescribedhowformalmodel,presentedinsection2.1,wasimplemented.MotivationsMotivationcanbeassociatedwitheachaction.Amotivationisalogicalexpression.Theactionisexecutedaslongasmotivationisevaluatedtotruthvalue.ThemotivationcanbeconstructedbySLD-resolutionmechanism.SuchsolutionimprovedbotcontrolbyPrologscripts,becausescriptcouldalsointerruptnativeactionexecution.IntroductionofthistechniqueimprovedtheseparationbetweenC++codeandPrologscripts,becausescriptcreatordoesnothavetoknowdetailedspecicationofanyparticularaction.Exampleofamotivation:action kill(BotID,EnemyID,and(bot alive(Enemy),danger low(Bot))).ThisactionwillmakebotwithidentierBotIDtrykillingthebotwithidentierEnemyIDaslongasenemyisaliveandbotisnotindanger.ContinuationsContinuationsareformofsimpleplanninganddeningcom-plexbehaviorsconsistingofsequenceofseverallow-levelactions.Therecouldbeassignedacontinuationtoanyaction.Acontinuationisalogicaltermwhichis Prolog-BasedReasoningLayerforCounter-StrikeAgents7executedafteractionissuccessfullycompleted.Theresultofexecutingcontinu-ationcouldbenewactiontobeexecutedbyabot.Thenewactioncouldhaveitsownmotivationandcontinuation.Forexample:action goto(BotID,Wp,allcost,andThen(action hostages(B,andThen(X)))),after free hostages(X).ThisactionwillmakeabotwithidentierBotIDgotoawaypointwithidentierWp.Themotivationforthisactionisthetermallcost,whichisalwaysevaluatedtotruth.Afterreachingthewaypointbotwilltrytofreethehostages.Theactionforfreeingthehostageshasalsocontinuation,whichisobtainedbySLD-resolutionandunicationofvariableX.Re exreasoningThereasoningmechanismisadeliberativeprocessanditiscomputationallyexpensive.Itrunsoccasionally.Botagentsalsoexpressreactivebehaviorsrrtorespondforrapidchangesingameenvironment,likebotbeingattackedbyopposingteam.Prologscripthasseveralpredicatesforhandlingsignalssenddirectlybythegameenvironment.ThosepredicateshaveshortSLD-resolutiontreeinordernottoimpedebotperformance.Thepredicatesmaycausesamee ectasregularreasoningprocess.CommunicationCommunicationmechanismofE[POD]botsrelayedonradiomessageswhichwasfeatureofCSitself.Suchmessagesaresimpleimperativestatements,e.g:Takingre,needassistance!,Followme,Teamfallback!.Receivingsuchmessageishandledbythere exreasoning.ImprovedteamworkOneadditionaltaskwasaddedtosetofexistingbottaskswritteninC++.Thepurposeofthismodicationwastoenhanceteamtactics.Allbotsinoneteamweredividedintosmallergroups.Eachgrouphasitsownleader,whocanmaketacticaldecisions.Otherbotsexecutingadditionalactionwasblindlyobeyinghisdecisions,withsomeexceptionsforre exreason-ing.Thisbehaviorwasintroduced,becauseteamworkoforiginalE[POD]botswasmorerandomchancethanreasonabledecision-botswereactingontheirowngroupingonlyoccasionally.2.5ExamplesExamplereasoningprocesshasbeenshowningure3.Inpresentedexampledecisionismade,ifagentshouldbuynewweapon.Botshouldbuyit,ifnewroundhasstartedanditisrespawning,becausehehas Prolog-BasedReasoningLayerforCounter-StrikeAgents9tothehostagewaypoint.Thisactioncanhaveit'scontinuation.Thereisanoptionthatbotwillwaitnearthiswaypointforterroriststocomeandambushthem.Thisactionwouldhaveit'smotivation-itwouldbeexecutedaslongasbotsensesanydangere.g.hearinggunreorenemyfootsteps.Inpresentedexamplebotdoesn'tdecidetoexecutethisaction,becauseitischosenwithsomeprobability.Hedecidestoheadbacktobasewiththeliberatedhostages,instead.3ResultsandConclusionThegoalofthisresearchwastovalidateifitisfeasibletorunbotsusingdescribedarchitectureandprovidesomebasicformaldescription,whichcouldbeimproveduponincourseoffurtherresearch.SomepartsoforiginalFSMdecisionlayerwasconvertedtoPrologreasoningscript.Thenthegamewasrunwith32bots-16foreachteam.Gamewasworkingwithsameframerateasoriginalone,asmostofthetimegameenginespendsinidlestateinordertosynchronizeinputs,networking,displayandeventsinthegameenvironment.Experimentleadtofollowingconclusions:1.C++botcodecanberewrittentoProlog,2.Prologbotscanbeusedeciently.Moredetailedperformanceinformationcouldbefoundonbarchart5.TheguredepictspercentageoftotalgameCPUexecutiontimetakenby3di erentsystemcomponents:Counter-Strikeengine,botlibraryandPrologreasoninglayer(comparewithgure2). Fig.5.CPUtimeconsumedbydi erentsystemcomponents.MostofabotlogicwasdirecttranslationofhardcodedC++decision-makingcodeintoPrologscript,soitwasinitiallyexpectedthatbotswouldn`tplayany 10GrzegorzJaskiewiczdi erentthantheoriginalones.However,newactionwasintroducedtoenhancecooperativeteamtactics.Forthatreason,additionalexperimentswereruntoverifyifnewbotsplaydi erentthantheoldones.Experimentswascarriedoutbysettingseveralgameswithdi erentparameterssetuponde dustdefusionmapwithadditionalactionenabledforoneoftheteams.Oneparameterincludedsizesofgroupswithinteamwhichhasteamactionenabled.Thoseexperimentsleadtoconclusionthatnewactionboostsperformanceofteam.Theoptimumgroupconsistsof3botsincludingtheleader.Largergroupmakesmaneuveringtaskmoredicultandbotstendstostuckinnarrowspaces.ItmakesthemalsopronetoHEgrenades.Groupof2botsdoesnotimprovegameplaythatmuch,asitcanbeoverpoweredeasier,thangroupof3bots.Unintended,butverypositiveoutcomewasabilitytotestreasoninglayerindependentlyofthegameenginewhichwasverycomplexsystem.Duringde-velopmentofPrologscriptsstandardPrologtestsuiteswerecreatedinordertocarryoutregressiontests.Testwereeasytocreatebecauseitwaseasytocreateamockofgameengine,whichwasn'tstartedduringtestsatall.ThegameengineinjectedtoProlognativepredicatesinorderPrologscriptwasabletointeractwithit.IntestscenariosinsteadpredicatescreatedbythegameengineregularPrologpredicatesweredeclaredandlinkedwithscriptexecutionenvironment.Usageofthistestsuiteallowedtoreduceamountoferrorsleadingtodivergen-ciesinexpectedscriptbehavioranddeclaredone.Improvingsuchtestingsuiteframeworkmayallowtoincorporatetestdrivendevelopmentmethodology(seegenerally[?])intobotsAIdevelopment.4FutureWorksInfuturesystemwillbeexpandedtomorecompletesolutioninordertocovermorecomplexinteractionsbetweenbotsandproviderulesforhandlingallCSmaptypes.Alsoitisplannedtochangereasoningmechanismtoonethatsup-portsbasicpropertiesofadefaultlogic[?].The4QLlanguage[?]isconsideredasareasonableoption.Thedefaultlogicwouldallowtodenehierarchicalrea-soningsysteminconsistentway.E.g1.generaldescribingCSgamerulesandgeneralfacts,e.g{ifyouareattackedshootback2.morespecicrulesdescribingrulesspecictogametype,e.g{onASmapsT'saren'tallowedtobuyAWP5ri e3.exceptionstotherulesandspecialrulesforconcretemap{oncs assaultpreferSG552thanAWP4.rulesspecictobot{ifyoucourageislowalwaysbuybulletproofvestbeforebuyingweapon 5forrearmsreferenceseehttp://world.guns.ru/ 12GrzegorzJaskiewicz9.Jacobs,S.,Ferrein,A.,andLakemeyer,G.Unrealgologbots.InIJCAI-05WSonReasoning,Representation,andLearninginComputerGames(2005),D.W.Aha,H.Munoz,andM.vanLent,Eds.,NavyCenterforAppliedResearchinArticalIntelligence,Washington,DC,NavyCenterforAppliedResearchinArticalIntelligence,Washington,DC,p.31{36.10.Jennings,N.R.,andWooldridge,M.Applicationsofintelligentagents,1998.11.Kitano,H.,Asada,M.,Kuniyoshi,Y.,Noda,I.,andOsawa,E.RoboCup:TheRobotWorldCupInitiative.InAGENTS'97:Proceedingsoftherstin-ternationalconferenceonAutonomousagents(NewYork,NY,USA,1997),ACMPress,pp.340{347.12.Malinowski,G.Many-ValuedLogics.OxfordLogicGuides.OxfordUniversityPress,USA,1994.13.Maluszynski,J.,andSzalas,A.Logicalfoundationsandcomplexityof4ql,aquerylanguagewithunrestrictednegation.CoRRabs/1011.5105(2010).14.McPartland,M.,andGallagher,M.Learningtobeabot:Reinforcementlearninginshootergames.InProceedingsoftheSeventhAAAIConferenceonArticialIntelligenceandInteractiveDigitalEntertainment,AIIDE2008(2008).15.McPartland,M.,andGallagher,M.ReinforcementLearninginFirstPer-sonShooterGames.IEEETransactionsonComputationalIntelligenceandAIinGames3,1(mar2011),43{56.16.Miikkulainen,R.Creatingintelligentagentsingames.InProceedingsoftheNationalAcademyofEngineering(2006),TheUniversityofTexasatAustin.17.Morris,S.Wads,botsandmods:Multiplayerfpsgamesasco-creativemedia.InLevelUpConferenceProceedings:Proceedingsofthe2003DigitalGamesResearchAssociationConference(Utrecht,November2003),C.MarinkaandR.Joost,Eds.,UniversityofUtrecht,p.CDRom.18.Mu~nozHernandez,S.,andSariWiguna,W.Fuzzycognitivelayerinrobocup-soccer.InProceedingsofthe12thinternationalFuzzySystemsAssociationworldcongressonFoundationsofFuzzyLogicandSoftComputing(Berlin,Heidelberg,2007),IFSA'07,Springer-Verlag,pp.635{645.19.Nguyen,L.A.Themodallogicprogrammingsystemmprolog.InProceedingsofJELIA2004,LNCS3229(2004),Springer,pp.266{278.20.Patel,P.,andHexmoor,H.Designingbotswithbdiagents.InProceedingsofthe2009InternationalSymposiumonCollaborativeTechnologiesandSystems(Washington,DC,USA,2009),CTS'09,IEEEComputerSociety,pp.180{186.21.Reiter,R.Readingsinnonmonotonicreasoning.MorganKaufmannPublishersInc.,SanFrancisco,CA,USA,1987,ch.Alogicfordefaultreasoning,pp.68{93.22.Shoham,Y.Agent-orientedprogramming.Artif.Intell.60,1(Mar.1993),51{92.23.Stepnik,A.E-sportzperspektywyteoriisportu.HomoLudens1,1(2009),213{222.24.Tastan,B.,andSukthankar,G.R.Learningpoliciesforrstpersonshootergamesusinginversereinforcementlearning.InProceedingsoftheSeventhAAAIConferenceonArticialIntelligenceandInteractiveDigitalEntertainment,AIIDE2011(2011),pp.085{090.25.Vaucheret,C.,Guadarrama,S.,andMu~nozHernandez,S.Fuzzyprolog:Asimplegeneralimplementationusingclp(r).InProceedingsofthe9thInterna-tionalConferenceonLogicforProgramming,ArticialIntelligence,andReasoning(London,UK,UK,2002),LPAR'02,Springer-Verlag,pp.450{464.26.Wielemaker,J.Anoverviewoftheswi-prologprogrammingenvironment,2003.