3 Absorbing Markov Processes Denition A state of a Markov process is called absorbing if once in that state there is no chance of leaving that state Example Does the Markov process represented by this transi tion matrix have an absorbing state If yes ID: 49049
Download Pdf The PPT/PDF document "Math copyright Joe Kahlig A Page Sectio..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Math166-copyrightJoeKahlig,10APage1SectionM.3:AbsorbingMarkovProcesses De nition:AstateofaMarkovprocessiscalledabsorbingifonceinthatstatethereisnochanceofleavingthatstate.Example:DoestheMarkovprocessrepresentedbythistransitionmatrixhaveanabsorbingstate?Ifyes,givewhichstatesareabsorbing.T=ABC26664ABC0.10.300.350.410.550.3037775Example:DoestheMarkovprocessesrepresentedbythesetransitiondiagramshaveanabsorbingstate?Ifyes,givewhichstatesareabsorbing. BACD0.310.150.250.510.20.150.30.15 BADC10.20.70.50.40.60.50.1 Example:Findtheabsorbingstate(s),ifany,ofthetransitionmatrix.Findthelimitingmatrix.T=ABC26664ABC10.05000.450.500.50.537775De nition:Anabsorbingstochasticmatrix,absorbingtransitionmatrix,isastochasticmatrixinwhich1)thereisatleastoneabsorbingstate2)fromanystateitispossibletogettoatleastoneabsorbingstate,eitherdirectlyorthroughoneormoreintermediatestates.AMarkovprocesswithanabsorbingstochasticmatrixissaidtobeanabsorbingMarkovprocess. Math166-copyrightJoeKahlig,10APage2Example:DoesthistransitiondiagramrepresentanabsorbingMarkovprocess? BADC10.20.70.50.40.60.50.1 Example:DoesthetransitionmatrixrepresentanabsorbingMarkovprocess?T=ABCD2666664ABCD100000.800.6001000.200.43777775 De nition:Anabsorbingtransitionmatrixissaidtobeinstandardformwhentheabsorbingstatesarelistedbeforethenonabsorbingstates.Example:Converttheabsorbingtransitionmatrixtostandardform.ABCD2666664ABCD10.100.200.400.2500.310.1500.200.43777775Example:Givethetransistionmatrixinstandardformforthistransitiondiagram. BACD0.310.150.250.510.20.150.30.15 Math166-copyrightJoeKahlig,10APage3Theorem:(Part1)IfTthetransitionmatrixofanabsorbingMarkovprocessisinstandardformthenthelimitingmatrixcanbefoundbythefollowingcalculation.Note:Thislimitingmatrixissometimescalledthestablematrix.T="I A 0 B#L="I A(I B) 1 0 0#Example:Findthelimitingmatrixforthistransitionmatrix.T=ACBD2666664ACBD100.10.2010.30.15000.40.25000.20.43777775Example:Intermsoflongtermbehavior,whatpercentofthetimewillyouendupinstateAifyoustarto inStateB?StateC?StateE?T=ABCDE266666664ABCDE100.10.30.2010.30.20.1000.20.10.3000.20.20.1000.20.20.3377777775 Math166-copyrightJoeKahlig,10APage4Theorem:(Part2)ThecomputationF=(I B) 1iscalledthefundamentalmatrix.a)thesumoftheentriesinacolumnistheexpectednumberoftimeitwilltaketoenteranabsorbingstateifyoustartinthatstate(columnlabel).b)theindividualentriesinacolumnaretheexpectednumberoftimesofbeinginanonabsorbingstate(rowlabel)ifyoustartinparticularstate(columnlabel).Example:Findthefundamentalmatrix.T=ACBD2666664ACBD100.10.1010.20000.60.2000.10.73777775Example:Findthefundamentalmatrix.T=ABCDE266666664ABCDE1000.10.20100.20.30010.30.10000.30.20000.10.2377777775 Math166-copyrightJoeKahlig,10APage5Example:HeatherandBlakeplayacardgameinwhichtheytaketurnsdrawingacardfromastan-darddeckofcards.HeathercanwinthegameifshedrawsaheartandBlakecanwinthegameifhedrawsablackcard.Whenaplayerdoesn'twinontheirturn,theircardisreturnedtothedeck,thedeckisreshued,anditbecomestheotherplayersturn.Thegamehasfourstates:Heatherwins(HW),Blakewins(BW),Heather'sturn(HT),andBlake'sturn(BT).A)DrawthetransitiondiagramforthisMarkovprocess.B)Findthelimitingmatrix.C)WhatistheprobabilitythatHeatherwinsifshegoes rst?D)WhatistheexpectednumberofturnsifHeathergoes rst?E)Ifeachplayertakes2minutestoselecttheircard,whatistheexpectedlengthoftimethatthegamewilltakeifHeathergoes rst?F)IfHeathertakes3minutestoselecthercardandBlaketakes1minutetoselecthiscard,whatistheexpectedlengthoftimethatthegamewilltakeifBlakegoes rst?