Principles Of Knowledge Engineering amp Reconstruction Spring 2010 University of Virginia David Evans A Course on Poker There are few things that are so unpardonably neglected in our country as poker The upper class knows very little about it Now and then you find ambassadors who ha ID: 618749
Download Presentation The PPT/PDF document "cs6501: Imperfect Information Games" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
cs6501: Imperfect Information Games
Principles Of Knowledge Engineering & Reconstruction
Spring 2010
University of Virginia
David EvansSlide2
A Course on Poker?!?
There are few things that are so unpardonably neglected in our country as poker. The upper class knows very little about it. Now and then you find ambassadors who have sort of a general knowledge of the game, but the ignorance of the people is fearful. Why, I have known clergymen, good men, kind+hearted, liberal, sincere, and all that, who did not know the meaning of a “flush”. It is enough to make one ashamed of one’s species.+ Mark Twain (as quoted in A Bibliography of Mark Twain, Merle Johnson)Slide3
John von Neumann
(1903+1957)Pure MathQuantum PhysicsAtomic Bombs Designer of Plutonium Bomb Fission/Fusion Hydrogen Bomb Computer Science First Draft Report on EDVAC von Neumann Architecture Merge Sort Random Number generation
Game Theory
Theory of Games and Economic
Behavior
(with Morgenstern)
Mutual Assured Destruction
Slide4
A+K+Q Game (not von Neumann Poker)
Flickr:cc MalkavSlide5
A+K+Q Game Rules
3 card deck: Ace > King > Queen2 Players, each player gets one card face+upHigher card winsWithout secrecy, stakes, betting, its not poker!Slide6
A+K+Q Game Rules
3 card deck: Ace > King > Queen2 Players, each player gets one card face+downHigher card winsBetting: (half+street game)Ante: 1 chipPlayer 1: bet 1, or checkPlayer 2: call or foldStakes: scheduling signup order by chip count
Loosely based on Bill Chen and Jerrod
Ankenman
,
The Mathematics of Poker.Slide7
A+K+Q Analysis
Better to be player 1 or player 2?Easy Decisions:Hard Decisions:Slide8
Game Payoffs
Player 1:AceKingQueen
Bet
Check
Bet
Check
Bet
Check
Player
2
Ace
Call
Fold
King
Call
Fold
Queen
Call
FoldSlide9
Game Payoffs (Player 1, Player 2)
Player 1:AceKingQueen
Bet
Check
Bet
Check
Bet
Check
Player
2
Ace
Call
(-2, +2)
(-1,+1)
(-2,+2)
(-1,+1)
Fold
(+1,-1)
(+1, -1)
(+1,-1)
(+1,-1)
King
Call
(+2,
-2)
(+1,
-1)
(-2,+2)
(-1,+1)
Fold
(+1,
-1)
(+1,
-1)
(+1,-1)
(+1,-1)
Queen
Call
(+2,
-2)
(+1,
-1)
(+2,
-2)
(+1,-1)
Fold
(+1,
-1)
(+1,
-1)
(+1,-1)
(+1,
-1)Slide10
Zero-Sum Game Slide11
Player 1:
AceKingQueen
Bet
Check
Bet
Check
Bet
Check
Player
2
Ace
Call
-2
-1
-2
-1
Fold
+1
+1
+1
+1
King
Call
+2
+1
-2
-1
Fold+1+1+1+1QueenCall+2+1+2+1Fold+1+1+1+1
Payoffs for Player 1Slide12
Strategic Domination
Strategy A dominates Strategy B if Strategy A always produces a better outcome than Strategy B regardless of the other player’s action.Slide13
Player 1:
AceKingQueen
Bet
Check
Bet
Check
Bet
Check
Player
2
Ace
Call
-2
-1
-2
-1
Fold
+1
+1
+1
+1
King
Call
+2
+1
-2
-1
Fold+1+1+1+1QueenCall+2+1+2+1Fold+1+1+1+1
Eliminating Dominated StrategiesSlide14
Player 1:
AceKingQueen
Bet
Check
Bet
Check
Player
2
Ace
Call
-1
-2
-1
King
Call
+2
-2
-1
Fold
+1
+1
+1
Queen
Fold
+1
+1
Simplified Payoff MatrixSlide15
Player 1:
AceQueenBet
Bet
Check
Player
2
Ace
Call
-2
-1
King
Call
+2
-2
-1
Fold
+1
+1
The Tough Decisions
What if Player 1 never bluffs?Slide16
Expected ValueSlide17
Never Bluff Strategy
Player 1:AKQ
Bet
Check
Check
Player
2
A
Call
-1
-1
K
Fold/Call
+1
-1
Q
Fold
+1
+1Slide18
Player 1:
AceQueenBet
Bet
Check
Player
2
Ace
Call
-2
-1
King
Call
+2
-2
-1
Fold
+1
+1
The Tough Decisions
What if Player 1
always
bluffs?Slide19
Always Bluff Strategy
Player 1:AKQ
Bet
Check
Bet
Player
2
A
Call
-1
-2
K
Call
+2
-2
Fold
+1
+1
+1
Q
Fold
+1
+1Slide20
Recap
If player 1 never bluffs: If player 1 always bluffs:Is this a break-even game for Player 1?Slide21
Course Overview
Topics Game TheoryMachine LearningAnything else relevant to building a poker botFormat: most classes will be student-ledPresent a topic and/or research paperSlide22
Class Leader Expectations
At least two weeks* before your scheduled class: Let me know what you are planning on doing (talk to me after class or email)At least one week before your scheduled class: Post on the course blog a description of the class topic and links to any reading/preparation materialsAt the class: lead an interesting class, bring any needed materialsLater that day: post class materials on the course blogFollow-up: respond to any comments on the course blog* If you signed up for Feb 1, you’re already late!Slide23
Course Project
Build a poker bot capable of competing in the Sixth Annual Computer Poker Competition http://www.computerpokercompetition.org/Note: overlaps with USENIX Security, August 9-12 (also in San Francisco)Work in small (2-4) person teamsA few preliminary projects earlier
Combine ideas/code/results from best teamsSlide24
My (Lack of) Qualifications
I do research in computer securityI have very limited knowledge and experience in game theory, machine learning, etc.I am (probably) a fairly lousy poker playerThis course will be a shared learning experience, and will only work well if everyone contributes to make it interesting and worthwhile.Slide25
Things to Do
Submit course surveyPrint and sign course contract: bring to Tuesday’s classReading for Tuesday: Chapters 1 and 2 of Darse Billings’ dissertationEverything will be posted on the course site (by tomorrow!):http://www.cs.virginia.edu/evans/pokerSlide26
Recap Recap
If player 1 never bluffs: If player 1 always bluffs:Looks like a break-even game for Player 1: is there a better strategy?Slide27
Mixed Strategy
Player 1:AKQ
Bet
Check
Bet
Player
2
A
Call
-1
-2
K
Call
+2
-2
Q
Fold
+1
+1
Always Bluff
Player 1:
A
K
Q
Bet
Check
Check
Player 2ACall-1-1KFold/Call+1-1Q
Fold
+1
+1
Never Bluff
Pure strategy:
always do the same action for a given input state.
Mixed strategy:
probabilistically select from a set of pure strategies.Slide28
Strategies
Player 1 Bluff with Queen Check with QueenPlayer 2 Call with King Fold with King
Finding the best strategy for Player 1: assume Player 2 plays optimally.Slide29
Nash Equilibrium
John Nash (born 1928)Equilibrium Points in N-Person Games, 1950Slide30
Nash Equilibrium
Player 1 is making the best decision she can, taking into account Player 2’s decisions.Player 2 is making the best decision he can, taking into about Player 1’s decisions.Neither player can improve its expected value by deviating from the strategy.Hence, to find the best strategy for Player 1, we need to find a strategy that makes Player 2 indifferent between his options.Slide31
Winning the AKQ Game
Bluff
Check
Call
-1
+1
Fold
+1
0
Player 1 wants to make Player 2 indifferent between
T
Call
and
T
FoldSlide32
Winning the AKQ Game
BluffCheckCall-1+1Fold+1
0
Player 1 wants to make Player 2 indifferent between
T
Call
and
T
FoldSlide33
Charge
Submit course surveyPrint and sign course contract: bring to Tuesday’s classReading for Tuesday: Chapters 1 and 2 of Darse Billings’ dissertationReadings posted now. Everything else will be posted on the course site (by tomorrow!):http://www.cs.virginia.edu/evans/pokerIf you are signed up for February 1, by tomorrow: contact me about plans for class.