Effective Short Term Opponent Exploitation In Simplified Poker (B Hoehn, F. Southey & R.C. Holte

Effective Short Term Opponent Exploitation In Simplified Poker (B Hoehn, F. Southey & R.C. Holte

Info icon This preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
Click this link to download and install Full Tilt Poker. Receive the maximum deposit bonus of 100% up to $600. Bonus Code: 2XBONUS600 Click this link to download and install PokerStars. Receive the maximum deposit bonus of 100% up to $600. Marketing Code: PSP6964
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Click this link to download and install PartyPoker Receive the maximum deposit bonus of 100% up to $500. Bonus Code: 2XMATCH500
Image of page 2
Effective Short-Term Opponent Exploitation in Simplified Poker Bret Hoehn Finnegan Southey Robert C. Holte University of Alberta, Dept. of Computing Science Valeriy Bulitko Centre for Science, Athabasca University Abstract Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is un- known. One approach is to find a pessimistic game theo- retic solution (i.e. a Nash equilibrium), but human play- ers have idiosyncratic weaknesses that can be exploited if a model of their strategy can be learned by observing their play. However, games against humans last for at most a few hundred hands so learning must be fast to be effective. We explore two approaches to opponent modelling in the context of Kuhn poker, a small game for which game theoretic solutions are known. Param- eter estimation and expert algorithms are both studied. Experiments demonstrate that, even in this small game, convergence to maximally exploitive solutions in a small number of hands is impractical, but that good (i.e. better than Nash or breakeven) performance can be achieved in a short period of time. Finally, we show that amongst a set of strategies with equal game theoretic value, in par- ticular the set of Nash equilibrium strategies, some are preferable because they speed learning of the opponent’s strategy by exploring it more effectively. Introduction Poker is a game of imperfect information against an ad- versary with an unknown, stochastic strategy. It rep- resents a tough challenge to artificial intelligence re- search. Game theoretic approaches seek to approximate the Nash equilibrium (i.e. minimax) strategies of the game (Koller & Pfeffer 1997; Billings et al. 2003), but this represents a pessimistic worldview where we assume optimality in our opponent. Human players have weaknesses that can be exploited to obtain win- nings higher than the game-theoretic value of the game. Learning by observing their play allows us to exploit their idiosyncratic weaknesses. This can be done ei- ther directly, by learning a model of their strategy, or indirectly, by identifying an effective counter-strategy. Several factors render this difficult in practice. First, real-world poker games like Texas Hold’em have huge Copyright c 2005, American Association for Artificial Intel- ligence (www.aaai.org). All rights reserved. game trees and the strategies involve many parameters (e.g. two-player, limit Texas Hold’em requires O( 10 18 ) parameters (Billings et al. 2003)). The game also has high variance, stemming from the deck and stochas- tic opponents, and folding gives rise to partial obser- vations. Strategically complex, the aim is not simply
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern