Online Poker Bots

[Furiously]

MSNBC has an article on poker bots.

http://www.msnbc.msn.com/id/6002298/

Slashdot appears to have followed it as well. I'm not surprised someone from MIT pipes in they have one that makes about $20 an hour.

Figured some of you online poker fans might enjoy the read. Me - I'm not touching that stuff.

[Shannow]

*insert robotic poker face joke here*

[Ardent]

Nerf pocket pairs!

[Soukyan]

I was thinking of getting into online poker and then I read that article today and now I'm not so sure I want to bother. I guess I'll go read the mammoth thread of online poker info and try to make an educated decision.

[El Gallo]

My serious poker friend and I were just talking about this.  I think it's only a matter of time until the next generation of these programs destroy online play (not that the current golden age would last longer than the TV poker fad would anyway).

If we can program a computer to play chess competitively with the best players in the world, we can program a computer to obliterate your average moron at PartyPoker (me).  I would guess that it would be easier because poker is a much simpler game.

[Furiously]

I suppose asking each player at the table a question would be a fairly easy way to weed out the bots.

[Paelos]

I think the key point to that article was that the bot in question was only good at poker in heads up situations. Well la-de-freaking da, heads up poker is perhaps the simplest form of poker there is. It's basically a game of steals and starting hands, and it rarely goes beyond the flop. Obviously it would be simple to program a bot to play like that. It's when you are up against 9 opponents with different tactics and styles that the bot loses its ability to cope. I wouldn't want to go heads up on against a bot, but I think it would be very predictable in a tournament style set up. Could a bot bluff? Could it catch a bluff? That's more than just odds and outs.

[Furiously]

Could a bot bluff? Could it catch a bluff? That's more than just odds and outs.

It doesnt need to. It just needs to play the odds.

[Shannow]

wonder how effective a poker bot straight up playing the odds is.

[Paelos]

Could a bot bluff? Could it catch a bluff? That's more than just odds and outs.

It doesnt need to. It just needs to play the odds.

In a tournament that's a losing strategy. If you play the odds the whole time the blinds eat your lunch. Plus, how you you program it what to bet to maximize profit in a game? You may have KK, but can it figure out how much you are willing to call with the type of hand you may have. That's something you get from watching a player's timing, his steals, and his general betting strategy. To combine that into an effective formula when you are lacking the basic input of what is in his hand is nearly impossible.

How do you teach the effective bets via position? What about maniac callers? What about the suckouts? When does it go all in? What about a cold bluff raise, a check-raise, or the semi-bluff? You're basically saying that if you play poker by the odds that you will win, but that's not true. Watch the good players at work. They aren't always playing the odds, they are playing their stack, the table, and their positions, in addition to many other factors. If you lived off EV all the time, you'd be a winner overall, but you'd never be a great player.

[Nebu]

It would be easy to get the program algorhythms and it should be trivial to determine how to beat the system.  The program must consider risk/reward in order to play the hands.  Thus, if you tax the system by betting large you'd only lose on those hands where the computer feels that it has a significant edge.   I think that you could bleed the bot dry with bluffs only losing on the rare occasions where the bot draws a solid hand.  Once you have a chip advantage, it's a matter of bleeding it dry.  

Also, after a number of hands it should be clear that the bot plays very tight.  If the bot is in you'd have the option of breaking the risk/reward threshold (betting big) or folding.

I really don't think that htese bots are any threat to good poker players... well, with the exception that they may run some "fish" out of the game.

[Shavnir]

Well if we're looking at Texas Hold 'Em I'd almost say you'd have to consider two seperate games that are tied.

The game of probability and the game of betting.

Programming an AI to react well to odds and know what hands are worth playing opposed to what ones are insta folds would be relatively easy.

Programming an AI to bet efficively, react to other player's bets, know when its outclassed, know when its opponent is bluffing, know how to bluff and know when other players have called its bluff....that's appalingly difficult.

I would say that, through a carefully trained neural net or genetic algorythm however you could eventually have a 'bot that could analyze a player's betting patterns and respond appropriately.  But being able to instantly judge a player blind is an ability very few people contian and is most likely enterily outside the realm of AI.

[El Gallo]

The program has the same information available about other people's play that you do (i.e. their past moves, how long they took, and the circumstances they were made in).  Remember, we are talking about online play here.  In fact, it would doubtless do a better job at collecting information, since humans tend to extrapolate from a few memorable examples which can often be misleading.

The computer would also be better at things like varying its play at random intervals (e.g. bluffing 30% of the time with hand X), a place where even quite good players fall into recognizable patterns, I bet.  It doesn't have that magic intuition a human does, but that is pure BS anyway in an online game.

I don't think that there's anythinig in principle different about tournaments, other than the need to factor in the tournament winnings in your calculations.  But you'd probably start with limit ring games anyway, since every move is 1 of 3 choices (check, raise, fold).  This would also mostly eliminate Nebu's strategy, assuming it couldn't be dealt with otherwise (why couldn't the program just detect your tendency to play like a drunken sailor and alter its valuation of its own starting hands to compensate, just like a real player would?).  If you want to play too loose, go ahead.  Sklansky and his disciples make plenty of money by trying to play like robots against people who do that.  

I imagine (i.e. this is straight out of my ass) that AI is several orders of magnitude above the assumptions some are making in this thread.  If Gary Kasparov can't find recognizable, repeat patterns that are easily exploited, I bet you won't be able to :)  There are plenty of feints, traps, bluffs, etc in chess too.

[Paelos]

I think you are neglecting that in chess the computer can survey the entire board and access calculations for the best move. That's simple, it has every piece of information to do that. In poker, you don't have that. How does the computer know you are bluffing? How does it know when you are sucking it in? Sometimes players just get the feeling you are walking into a trap, but computers don't do that. When it can never receive all the information, it can never adjust because it doesn't know if it made the right play. Three times out of four I might bet big on a semi-bluff, but that could be a setup for my fourth play when i've got the goods. If a computer plays my pattern he calls on the odds that i don't have it everytime, but when? When the hand is favorable? When the EV is right? Good luck with that in a tournament format with increasing blinds. There aren't enough hands to play that way.

[El Gallo]

That's why I'd start my program on limit ring games :)  But I think that tournaments are different only because luck is a much, much larger factor in tournaments.  You are right that there is less time to gain information about your opponents.  Then again, online you can play a lot of tournaments; to the point where each individual tournament is like one hand in a meta-tournament.  And both the human and the computer can collect a lot of information on your opponents over time.

I think where we really disagree is that I don't think that acting "on a feeling" works.  It's just more memorable when it does for most of us.  It's different in a live game, where I could buy the theory that the "feeling" is really a bunch of subconscious tell recognitions.  But online, I just don't believe in the power of intuition.

To quote the guy who shot Greedo in self-defense, "I call it luck."

edit: cut out a lot of redundant crap

[MrHat]

“We take issue with the poker sites because they do not do enough to communicate the fact that it is impossible to physically secure their no-bots, no-teams policies in an online environment,” he wrote in reply to an e-mail question. “... If the poker sites cannot physically secure their no-bots, no-teams policies then they need to cease their persecution of players who are resorting to such measures in order to adequately defend themselves against opponents that have already seen the light. Players should be free to fight fire with fire without being bullied by the poker rooms.”

Smells like these guys need to play some MMO's.

[WayAbvPar]

If the poker sites cannot physically secure their no-bots, no-teams policies then they need to cease their persecution of players who are resorting to such measures in order to adequately defend themselves against opponents that have already seen the light. Players should be free to fight fire with fire without being bullied by the poker rooms.”

Translation- because you can't prove that there are no bots playing, it is only prudent that all players use bots to combat bots...just in case.

What a bunch of fucking crap. Rao (who is a fantastic player, btw) mentioned that he wasn't sure the sites would do everything they could to prevent bots, since they are still getting their rake. I disagree- the sites have little to gain and everything to lose with that attitude. If online poker becomes known for cheating and computerized assistance, it is OVER. The fish will be too afraid to try it out, and the sharks will quit when the fish stop coming. It will degenerate into a shadow of itself, with bots battling bots. Sure, the poker site will still get its rake, but it will be raking a LOT less pots without new human players.

[Krakrok]

Um, it doesn't take a rocket scientist to mitigate bots and many websites already employ the method to do so. It's called the Turing Number (the little box of gibberish letters that you have to type in from an image). The theory behind it being you can't OCR the text and therefore a human has to type in the numbers/letters from the image.

Requiring a Turing number input every hand would force a human to sit there and at least watch the bot play between typing in the Turing Number each hand.

[Abagadro]

It would also grind the games to a standstill. They are slow enough as it is. I don't fear bots in poker. Playing strictly the percentages may give it a tiny EV, but its not going to kill the game and I could likely spot it pretty fast.  The real danger is getting 4 or 5 of them at one table so they know more dead cards.

[Margalis]

I think it would be pretty easy to create a bot that could win at low fixed limits, since most players there are horrible. I can make money on Stud 7 playing on complete autopilot, with a strategy that can be summarized in about 3 sentences.

IMO a big advantage for a bot is they don't tilt, they don't overthink or get cute, and they don't get tired.

I find playing online tedious because I don't have the money to play high stakes, and it doesn't feel real...my typical pattern is do well for a while then get bored and blow it all quickly.

High stakes games tend to run in pretty tight circles as well, a bot would be found out pretty quickly in those sorts of games.

[Arcadian Del Sol]

[Murgos]

Um, I didn't read the orignial article but I have read about using AI systems to play poker over the last couple of years.

1.  It's orders of magnitude more difficult than programming a chess algorithm.

2.  Some teams of very bright people have been working on this for 20+ years with very little success, there will probably be several Nobels awarded to the people involved in solving this.  Joe Bob VBscripter isn't going to design a winning algorithm in his spare time after building his nth online store at work.

3.  From what I've read by researchers there are one or two bots that are being run from university AI programs that are marginal at best against even very weak players.

EDIT:  I just read the article and the acronym FUD comes to mind.  Poker is booming, there are 3 TV shows and hundreds of thousands of new people playing every month, this is just another example of Enquirer style  tabloid journalism designed to sell ad-space.  That article said nothing but managed to say it in the worst possible way it could.  Fuck, I hate mass-media news.

[UD_Delt]

"But Ray Bornert II, whose company makes WinHoldEm, billed as “a programmable pokerbot,” said his company developed a two-computer strategy to foil attempts to block its users. "

For some dirt on this guy check out RGP.

http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&c2coff=1&threadm=J0nIc.60648%24IQ4.41267%40attbi_s02&rnum=1&prev=/groups%3Fq%3DThe%2Btruth%2Babout%2Bwinholdem%26hl%3Den%26lr%3D%26ie%3DUTF-8%26group%3Drec.gambling.poker%26c2coff%3D1%26selm%3DJ0nIc.60648%2524IQ4.41267%2540attbi_s02%26rnum%3D1

If that doesn't work do a search in google groups for "The Truth About Winholdem".

Rather amusing stuff. From another thread:

Here are the facts on WinHoldem Software:.

1)  The use of WinHoldem software is not permitted on Paradise Poker, Poker Stars, or Party Poker or any of their affliates.
2)  Poker sites are capable of detecting WinHoldem Software.
3)  Players using WinHoldem software have had their accounts closed.
4)  If a player has their account closed for using WinHoldem Software their
bankroll could be confiscated.
5)  After an independant third party test WinHoldem Software won an average of $2 an hour playing $3/$6 Limit Poker.
6)  After 50 hours of play WinHoldem will pay for itself unless during that
time the user's account is closed due to detection.

Here are the facts on Hixoxih Software makers of WinHoldem Software:
1)  WinHoldemSupport has stated several times on this forum that they have several support people and technicians.
2)  WinHoldem Software is developed and published by Hixoxih Software.
3)  A D&B report on Hixoxih Software confirms that Hixoxih Software has a total of 1 employee. (www.dnb.com)
4)  That employee and owner of Hixoxih Software is Ray Bornert.
5)  Ray Bornert poses as WinHoldemSupport on this newsgroup,
RiskDeluxeSupport on rec.games.board.
6)  WinHoldemSupport believes collusion in online poker is ok.
7)  Currently Ray Bornert, owner of Hixoxih Software has a Federal Tax Lien against his person to the tune of $351,635 details can be found at
http://www.gsccca.org/search/Lien/lienindex.asp

A few questions for WinHoldem's potential customers:
1)  Are you willing to risk your entire bankroll to make $2 an hour when it
will take 50 hours just to pay for the software?
2)  Do you wish to support a company that blantalty misrepresents
themselves?
3)  Do you want to give your credit card information to a person who
believes cheating is ok and owes the government $351,000?

[Disco Stu]

If anyone is actually interested in the topic beyond the stupidity of the MSNBC article you might want to head here.
http://www.cs.ualberta.ca/%7Egames/poker/

[Toonces the Driving Cat]

The poker playing algorithm isn't the hardest part. It's not trivial, but trust me, there are plenty of programmers out there who can hammer out a very solid holdem AI for an any-handed game in a matter of weeks.

The hard part to code is the automated interface to the poker site. You have to do one of two things, either decrypt, analyze, and emulate the packet stream, or, read the video buffer to perform visual analysis of the pictures and generate automated mouse clicks. If you go the packets route you have to hope they don't switch up their protocol, and if you go the visual route, then you have to hope the site doesn't change its graphics, interface, or game flow. You also run the risk of getting your account banned and your winnings frozen if you are caught -- and what are you going to do, take them to court in Bermuda?

I can tell you this, the argument that AIs are not good enough at the game to be competitive is foolishness. The reason bots aren't widespread is because of the real hard part, which is the automated interface as i said. But it'll happen, just takes a genius with too much time on his hands, and we all know they are out there. Question is how good will the site administrators be about detecting it.

[Disco Stu]

I can tell you this, the argument that AIs are not good enough at the game to be competitive is foolishness. The reason bots aren't widespread is because of the real hard part, which is the automated interface as i said. But it'll happen, just takes a genius with too much time on his hands, and we all know they are out there. Question is how good will the site administrators be about detecting it.

Unless you are talking about heads up you are compleatly and totally wrong. Look around the web site I linked to for an explination of the problems with designing programs to play multi player hold'em. But just for the hell of it could you please try and provide some evidance to support the cliam that a programmer can make a hold'em AI in a couple of weeks.

[Murgos]

The hard part to code is the automated interface to the poker site. You have to do one of two things, either decrypt, analyze, and emulate the packet stream, or, read the video buffer to perform visual analysis of the pictures and generate automated mouse clicks.

Your and mine ideas of difficulty vary greatly apparently.  A screen scrape and algorithm to determine the value of a cards face or to read the font for the amount of money bet is, while not trivial still a VERY well understood bit of programming.  Heck, one of my roomates did something similar years ago for a college (undergrad) project.

An algorithm that can figure out when your lying about how strong your hand is?  Now that sounds hard to me.  If winning poker was just about betting the odds everyone and thier grandmother with a text editor and a compiler would have done it 30 years ago.  Guess why they haven't?  One hint, its not the nearly trivial interface problem.

[Margalis]

Not to get too nerdy, but scaping the screen, while somewhat tedious, is not hard.

Most poker sites display the exact same images for cards in the exact same position each time, it wouldn't have to be particularly smart. Just look at those locations onscreen and compare to your canned list of images.

It might take some time to get right, but it's basically manual labor. Just slog through it and you're done.

I'm a "lead software engineer/ development manager" in real life, so I knowest of whateth I speaketh.

Edit: There are automated packages like WinRunner that generalize scripting and identification of UI components, and we are talking about a very limited functionality compared to that.

There is the danger that the graphics and flow will change, but in that case you tweak your bot a bit.

I am 100% sure I could program a bot that could win at PokerStars playing very low limit hi/lo stud. (Because I can play that with 10 brain cells) I'm not sure I could progam anything that could win at a high enough level to make it worth the time and risk.

[Shockeye]

Wired has a new article about online poker bots.

[WayAbvPar]

That guy is a fucking scumbag. Him and people like him are going to ruin online poker. If/when word gets out that there are a lot of bots playing, say goodbye to all the newbies (which is where a lot of the profit comes from for decent players).

Nothing like having someone trying to actively destroy one of your favorite hobbies. ARRRRRGGGH.

[Strazos]

Agreed.

And his justification is that everyone should use a bot?

What would be the fucking point of playing then?

[Margalis]

And his justification is that everyone should use a bot?

What would be the fucking point of playing then?

Do donate everyone's money evenly to the gambling site?

I love justifications like "they want you to believe it's safe and secure, but it isn't! (because of the bot I created)"

[Azaroth]

Welcome to the wonderful world of people creating bots to actively destroy shit for their own personal gain.

I'm surprised it took so long to become mainstream. Been going on in MMORPGs for years. ESPECIALLY UO.

[Margalis]

It's not mainstream. I think bots are still very rare overall, and WinHoldEm doesn't get glowing reviews. It's mostly known for constantly spamming the poker newsgroups with ads.

[Trippy]

Why don't they add a CAPTCHA test to their programs to flush out the bots?