Influencers with Andy Serwer: Annie Duke

In this episode of Influencers, Andy is joined by 'How to Decide’ author, Annie Duke, as they discuss her time as a professional poker player and how she’s using her experience to fix some of the most common mistakes in the business world.

Video transcript

[MUSIC PLAYING]

ANDY SERWER: They called her the Duchess of Poker. Annie Duke was once the top female money winner in the history of the World Series of Poker, winning a coveted gold bracelet and earning millions of dollars over the course of her career. An expert in human behavior, Duke had a leg up on the competition, putting her natural instinct and Ivy League education to use at the card table. But with her poker playing days behind her, Annie Duke has pointed her focus towards business, using cognitive science to help companies to navigate uncertainty and make better decisions under pressure.

In this episode of Influencers, I speak with Annie Duke about what she learned in her time as a professional poker player and how she's using her experience to fix some of the most common mistakes in the business world.

[MUSIC PLAYING]

Hello, everyone, and welcome to Influencers. I'm Andy Serwer. And welcome to our guest, Annie Duke, former professional poker champion and author in behavioral decision science and decision education. Annie, great to see you.

ANNIE DUKE: Thanks for having me. I'm excited to be here.

ANDY SERWER: Annie, I guess I'd like to start off by asking you what are the similarities between poker and business and business decision making?

ANNIE DUKE: Oh my gosh. Well everything would be my answer. So poker is a very interesting game, you can think about the difference between, say, a game like poker and chess. So in chess you can obviously see all the pieces of your opponent. So you could figure out exactly what you're supposed to-- like if anybody who's seen Queen's Gambit, where you can see that Beth is sort of playing out all those pieces, for any move that I'm thinking about, I can see what all the possible moves that my opponent could make are and so on and so forth. I could in fact, what makes a chess player really great, one of the things is that they can go many moves deep into the game. So that's number one is that you can see where all the pieces are.

Number two, is that there isn't a really strong element of luck, certainly not in the sense of like someone rolls dice and you get a seven and you get an extra bishop or something. And if it's snake eyes it's like you're in check. Those pieces are only going to move by an act of skill. And you can contrast that with a game like poker where certainly there's a very strong element of luck in terms of the turn of the cards, I don't have any control over those. And there's also a tremendous amount of hidden information. I know what my cards are but I don't know what my opponent is holding and so that makes my judgments under those circumstances subjective.

Now that should sound a lot like business decisions for people because when we make business decisions, we don't really know what the future holds in terms of luck. I mean, just look at the pandemic, like if you were opening up a retail shop in October of last year you couldn't have seen that this was just a matter of luck, it's like a bad turn of a card. There are also good turns of the card.

And then there's just a lot of stuff that you don't know when you enter into negotiation, you don't know what cards your opponents are holding. You have some partial information but not a lot and that creates a whole different type of decision problem and particularly with poker where there's a lot of money at stake, which is true of all of our decisions in business as well, you can see where that maps on really nicely. So that if we can understand poker decision making we could know something better about decision making away from the poker table.

ANDY SERWER: Let's talk about your life a little bit, Annie, because you grew up studying science, were going to get a PhD and then you were doing poker and you went back and forth between the two. Tell us about your biography a little bit.

ANNIE DUKE: Oh my gosh, my biography is so weird and long and winding.

ANDY SERWER: Well, give us an abbreviated version then.

ANNIE DUKE: I will give you the abbreviated version. Five years at the University of Pennsylvania studying cognitive science, specifically thinking about learning actually, which shouldn't surprise you. Right at the end I got sick and I needed to take a year off. I was actually on my way to my first tenure track job interview and I've been struggling with something chronic that became acute. So I decided to take a year off before going out and becoming a professor and it was in that year that I started playing poker really actually out of necessity, I needed money because I was taking a year off and I didn't have my fellowship anymore. So I ended up really falling in love with the game and didn't actually go back to those tenure track opportunities that I had available to me. I played poker exclusively for about eight years. I did really well. At one point I became the leading money winner among women, I'm not anymore, there are women who have won more since then. But I really, really love the game.

But then in 2002 I actually got asked by a hedge fund to come speak to their options traders about how poker might inform decision making around risk. And started thinking really explicitly about the way that my academic training kind of informed these kinds of decisions about risk and how poker was this very interesting framework through which to think about the things that I had learned as an academic. And I started thinking about kind of the space between those two things and that interesting conversation, started diving deep back into that and for 10 years I actually overlapped the two and then cycled out of poker in 2012, ended up writing this book Thinking in Bets, with was followed up by How to Decide, which just came out. And started doing-- spent my time doing consulting, performance coaching, executive coaching, and then also deeply embedding in organizations in order to really work on essentially systems around decision making and how to improve decision making on teams. So I work with both individuals and teams and that's what I've been doing ever since. And now I'm back at Penn in Phil Tetlock's lab and I think I'm going to button up that part of my academic career. And I just finished a class at Wharton and I'm kind of back to where I started so.

ANDY SERWER: Wow, full circle a little bit.

ANNIE DUKE: Full circle.

ANDY SERWER: So let me ask you about the poker days because for most of us, well some of us play and then we all watch the stuff on TV with the guys with the glasses and the characters. What was that life like, Annie?

ANNIE DUKE: Well, so the first thing I think that people, it's hard for people to remember this and this will certainly date me but when I started playing poker it was not on television and it wasn't on the internet, either. There wasn't an internet that would have been fast enough. Like this was the days when if you wanted to download a web page you came back in 10 minutes, we had dial up. So this was really a very small handful of people.

At the time that I started playing people didn't understand that it was something you could do as a profession. I think that generally people kind of categorized it in the sort of vice category I would say, they kind of thought about it as if you said I was a professional poker player that about made as much sense as I'm a professional craps player or something. I think that people kind of thought about it very much as gambling as opposed to high frequency trading, which is really what it is. I only happened to know about it because my brother had done it before me. My brother was a chess player and went to New York to study with a grandmaster and ended up finding out about poker and he had fallen in love with the game prior to me and became very good at it. So that was the only reason why I even knew it was something that you could do.

But just to paint the picture when I started playing, I was actually living in Montana at the time and I went to this place called the Crystal Lounge, which is about everything that would think. It was in downtown Billings, Montana and I think it's more gentrified now but at the time downtown Billings, Montana was quite something. That is where all the vice actually happened. And there was the Crystal Lounge and you went into the top floor and there were kind of banks of $5 poker machines and a bar and then you went down the back steps, you sort of descended into this basement. And it was kind of what you would expect in the middle of Billings, Montana. It was ranchers and people who mostly were retired, older gentlemen, couldn't see through the smoke and I was in my 20s, a girl sort of waltzing into that environment. It was a little bit strange. I was pretty much always the only woman at the table. I can't really remember other times when there were other women at the table.

And that's where I got my start and then I moved to Las Vegas and again, it was still a very insular world. The people who were sort of at the top of the game, probably 30 or 40 people that were playing sort of in the largest stakes and what you would wait is for somebody who was who loves poker as an avocation but had a lot of money to come and play and it was a really good value proposition for them, because they were having a fun time and we were making our living and it sort of almost in some ways considered yourself part entertainer or social director or something like that. In the same way that they would go on a trip or a bottle of wine or something like that, it wasn't really a losing proposition for them. Of course, in 2002 that all changed when they came up with the lipstick cameras and then it was on ESPN and then that brought in a whole influx of players but that certainly wasn't the case when I first started playing.

ANDY SERWER: Were you surprised with what it turned out to be with the television aspect of it and the celebrities coming out of the game and all that?

ANNIE DUKE: Shocked. Shocked. I think that it's a failure of imagination because if you ever tried to watch a poker game not on television where you can't see the hole cards, it's incredibly boring. So you can kind of think when you're actually playing the game you can see your cards but obviously you can't see your opponent's cards and that's kind of the fun of the game. It's like I know what I have and I understand what my hand is but how does it relate to the other players and how are those other players going to react to what I think their cards are. That's actually an incredibly interesting problem to try to tackle.

But let's imagine prior to these little lipstick cameras, you would watch the game and all the cards are face down. So you'd see none of them. That is incredibly boring. It's like watching paint dry, it's horrible. So I couldn't imagine that this was something that would ever make sense on television because it's quite a boring game to watch under those circumstances. Of course, the cameras changed all of that because now all of a sudden the viewer ends up in a superior position to the player. The player can only see their cards but the viewer can now see everybody's cards, so they can see like, no don't bluff there. Why are you folding or you should have raised. So it actually makes it a very exciting game to watch when you can see everybody's hole cards. I just, I didn't imagine that that would happen. I didn't see how you could make it interesting. But a few producers, Steve Lipscomb was one of them with the WPT, sort of figured out that that could be something that people really loved and had the vision to put it on television and obviously it exploded. I was so surprised though.

ANDY SERWER: Right. Right. So you're doing some business consulting as you said, Annie, and do you think that executives are surprised when they hear that you're a poker player and you know about business decision making, do sometimes people go what?

ANNIE DUKE: I think it's interesting. I think, so when I first started doing it, so this was back in 2002, it was sort of through this accident of being introduced to this one hedge fund through a friend of mine who really wanted me to come in and talk to their options traders and then most of my work was from referrals. So I think that people sort of understood that I had this academic background and there was a reason why I could understand why those would speak to each other.

But even so, I would say most of my work ended up being in finance obviously because I think that people in finance can see the connection between the two so much more clearly. I think that people in finance kind of really understand naturally that poker is really a trading problem and a capital allocation problem actually as well. So I think they kind of get it intuitively. Now of course people find me through my books and so it's not at all surprising to them because my books clearly lay out that I understand business decision making and cognitive science.

ANDY SERWER: Right, and of course, a lot of big hedge fund managers play celebrity poker tournaments now.

ANNIE DUKE: They do.

ANDY SERWER: Big thing. So what are some mistakes that people make either investing or in business decision making that sort of relate to some of the things that you're interested in?

ANNIE DUKE: Oh my gosh. OK, so let me think because there are so many, for anybody who's read Thinking, Fast and Slow from Daniel Kahneman they know that mostly we make a lot of mistakes. So let me just, let me think about what sort of the most devastating are. And I think I'm going to think about two. The first one, which we can talk about is called resulting and this is actually a really huge problem people should recognize this one pretty quickly. It's taking basically looking at the result of something, say a trade, seeing whether it won or lost and then using that to determine whether the trade was actually good or bad.

So I'll give you an example outside of finance that I think should help people to understand why this is problematic. So just in the simplest sense, obviously whether you won or lost on one iteration if you think about it doesn't actually tell you very much about the quality of the decision. This is actually a little bit of poker versus chess problem because as soon as you introduce luck into a system and also like information that would be unavailable to you that might have been important for you to know, we can get it, we essentially dis-correlate decision quality from outcome quality in the short run, something that doesn't happen in chess. So in chess because you don't have that element of luck if you lose, it's because you made bad moves in comparison to me but that's not necessarily true in poker. In poker, you can lose because you get bad cards. You can go through a green light and get in an accident even so. So just the fact you got in an accident shouldn't actually tell you much. But the human brain doesn't really work that way and it ties those connections.

So the example that I opened Thinking in Bets with is actually the 2015 Super Bowl with Pete Carroll and the Seattle Seahawks and they're on the 1 yard line of the New England Patriots, people really know this play, they're down by four, there's 26 seconds left in the game and Pete Carroll has one timeout and so obviously they're down by four they have to score a touchdown here and it's pretty clear if they score a touchdown that the Patriots aren't going to have enough time to come back down the field. So this will be the game winning play if they can do it. Everybody has an expectation that Pete Carroll is going to hand the ball off to Marshawn Lynch, who's going to run it through the goal line. I mean, I guess running through the pile of Patriots is an obvious thing that someone would be able to do. I would argue against that. But anyway, he doesn't do that, he chooses to have Russell Wilson pass the ball and we know that ball was very famously intercepted. And when you look at the headlines there, they're just incredibly brutal. I mean, they're mostly-- most people are declaring that this is the worst play in Super Bowl history. USA Today actually said it was the worst play in all of NFL history. That was pretty strong.

ANDY SERWER: Wow!

ANNIE DUKE: Yeah. But I'll do the thought experiment with you really quickly, Andy. So let's imagine that Pete Carroll passes the ball, well he doesn't personally, Russell Wilson does, but he calls for the pass play and the ball is caught for the game winning touchdown. What do the headlines look like the next day?

ANDY SERWER: Brilliant play calling by Pete Carroll.

ANNIE DUKE: Yeah. This is why he's going to the Hall of Fame. He out-Belichicked Belichick. So this is where we can see resulting at work is that when that is caught for a touchdown, it's a brilliant play. It's one of the most-- it will be the most brilliant play in NFL history according to USA Today and when it's intercepted, it's the worst play in NFL history.

And when you look at these analyses, none of these talk about process. None of them talk about what's the probability of the ball getting intercepted or incomplete or it being a touchdown pass, how does that compare to the probability that Marshawn Lynch is actually going to score. And of course, those are all the things that you need to know, in order to know whether it was a good decision not just the result of the play.

For that analysis people can go look at Thinking and Bets and see that. The short story is it was actually quite a brilliant play, the chances of an interception there are going to happen less than 2% of the time. And it actually buys you an option for a third play. It would happen to be second down. So if you hand it off to Marshawn Lynch twice you only get two plays if you pass somewhere in there you get three.

So again, there's more detail but regardless, it makes no sense to look at whether it was intercepted or incomplete or a touchdown pass on that one time. But we do this all the time and you see this in finance and we know this for example just in terms of like capital allocation, that people will re-up with managers who have won in the last year and they will not allocate to managers who have lost and we know that that's actually a losing strategy. That you do worse that way because of regression to the mean and you're not taking luck into account and you're just thinking about short term results in terms of as opposed to process. Certainly true and you don't want to do this in options trading as well, whether you're up or down in a particular moment should matter very little, but it's a real mistake that we make and it's a weakness of human decision making.

ANDY SERWER: And you also talk about this and how it relates to elections as well?

ANNIE DUKE: Yeah, so sort of taking this election aside, which I think is still very hot in people's minds. We could think about Hillary Clinton as well. So again, we don't really think about process. We don't really think about what people knew at the time as we're sort of evaluating people's decision making. We really just think about the result. So we know that Hillary Clinton did not spend a ton of time campaigning in Pennsylvania, Wisconsin, and Michigan. And we also know that she lost those three states by a combined total of somewhere around 80,000-ish votes across the three states. It was a very narrow margin but you could see the takes start coming out, oh this was such a terrible decision, such horrible strategy. Why wasn't she in Pennsylvania, Wisconsin, and Michigan? This was her fault that she lost because she made such bad decisions.

So here's the thing about that and this is how we can sort of see that this is a resulting problem as people have these takes, is that we need to think about what is the information that she had at the time of the decision. So could she have known that she should have been in those states? Well, the input into that decision is going to be the polls and what we found out after the fact that there was quite a large polling error that that polling error happened to go in Donald Trump's favor in 2016. And so when you think about the input into her decision making, which is where I am I ahead, where am I close? She was very close in Arizona, she was very close in Georgia, very close in North Carolina, very close in New Hampshire, very close in Florida. And that's where she was spending a lot of her time and it looked like she was very far ahead in Pennsylvania, Wisconsin, Michigan.

We find out after the fact by definition that there's a polling error and so can we really expect her to have known that beforehand? Well there's an easy way for us to figure out whether that was a mistake and she should have known about it beforehand, you can look on a search engine like Yahoo and you can say what would happen if we search to see whether people were writing about this what feels like an obvious polling error after the fact and the answer was nobody was. In fact, almost all of the writing, and there was very little about it, were questioning why Donald Trump was campaigning in Pennsylvania at all and saying why would he be there, he should be in places where the polls are much closer and in fact they were suggesting he should be spending more time in Ohio.

So this is a really good case of resulting, she lost therefore the decision making must have been bad. As opposed to she lost because there was a problem with the polls that nobody could have known about except after the fact because that's when you see the barrage of articles appearing. So that's one of the ways that you can figure out that this resulting problem is happening.

So in the Pete Carroll case, we can go back and do the math. In the Hillary Clinton case, we can go back and say was it reasonable for her to know. And what you can see pretty quickly is no, it certainly wasn't at all reasonable for her to know because nobody knew. And every Silicon Valley data analyst was going to be all over that and political observer and whatnot. But this is really where we see that we take very bad lessons from history.

ANDY SERWER: Pollsters still have some work to do apparently if you look at this election, Annie. I don't know if you have any thoughts on that or if you wanted to talk about another problem?

ANNIE DUKE: Yeah. Sure, I think that this is a good example of what do you know and what don't you know. So we know, for example, that Biden was polling very far ahead in Pennsylvania, Wisconsin, and Michigan and yet when you look at his ad spending, that's where most of the spending was, including the PACs because I think that at that point they were taking into account that there could possibly be a polling error again.

And I think that that shows the difference between what can you know and what can't you. So having seen that polling error in 2016, I think that what they said was, OK there's a non-zero probability that this polling error is occurring again and you can see that their resource allocation actually took that into account. So that's a very good example actually of resulting versus not resulting. Which is you see that you learned something new and this is the only problem that you can get into a decision making, you learn something new and then you don't take it into account for the next decision. So learn something new in 2016, something that Clinton couldn't have known about it but as we go into this, Biden can know about it and you see that the resource allocation was taking into account a possible polling error.

ANDY SERWER: Fascinating. Did you say you wanted to talk about another kind of mistake that people made in business, make in business?

ANNIE DUKE: Yeah, so I think the other really huge mistake that people make in business is that we often think that we're asking for people's opinion in a genuine way, trying to figure out what people actually think in order to get feedback because obviously feedback is one of the most important tools that we have in order to make better decision making. Like the more perspectives that we can get on a problem the higher the quality of the decision we're generally going to be making. And so we go perspective seeking from our team and our colleagues. But this is actually one of the biggest mistakes that people make in business is that the way that they ask for that feedback actually ensures that they're generally going to get something that looks more like their own opinion fed back to them, which is something that we don't want to do.

And it's a very simple hack to solve for this problem, mainly when we ask for people's opinion on something we offer our opinion first. So we can think about like we see two job candidates or we're thinking two business strategies and I'll say to you, hey, Andy, I'm thinking about these two candidates they both came in, this is what their CVs look like, this is what I like about both of them, these are my concerns, A seems kind of abrasive, but B I'm not sure if they have the experience that I need but given the fact that I care about the culture of the team so much, I'm worried about the abrasiveness first, so I'm leaning for candidate B but what do you think?

Well, I mean, I've just told you everything that I think. And so the chances that I'm going to get kind of an unadulterated like a pure opinion from you, what you actually think, has just really gone down. So the simplest thing that somebody can do is to stop offering their opinion first. One on one that's really easy to do because I can just say to you here are these two job candidates, here's their CVs, I'll show you a tape of their interview or I'll describe the interviews to you and then I'll say what do you think? Who do you think, what would your opinion be on who I should hire? Without offering my opinion first. Obviously when you're in a team discussion that's harder to do. Because I could ask you that but then once you've offered your opinion you've kind of infected the rest of the group.

And so what we want to do is elicit opinions independently and asynchronously. So I would go out to the group, I'd ask everybody to give their opinions independently, then I'd put those together and then we'd have that for the group discussion. And that's going to allow you to see what the range of opinions actually are on your team in order, instead of getting sort of an echo chamber effect back at you.

ANDY SERWER: Yeah, got to sublimate your ego a little bit I guess, right?

ANNIE DUKE: A little bit, yeah. I mean, I think it's kind of the natural way that we talk, not necessarily because of ego but mainly because we think that our opinion is important data. Like it's important for you to know that I thought that candidate A was too abrasive and that candidate B maybe doesn't have the range of experience that I need for the job. I feel like this is an important fact to give you, but of course that's the exact opinion that I'm trying to get from you. And I think that we need to remind ourselves of that as much as we possibly can, that the more that we can find out what people actually think, the better off we are because it gives us a more complete view of the way that we can look at our situation.

ANDY SERWER: Right. And you've been active in philanthropy a little bit, Annie, changing subjects here with the actor Don Cheadle. Tell us about that.

ANNIE DUKE: Oh, gosh that was a while ago. Yeah, so I'll tell you what I did with him and then I'll tell you what I've done since. So if people recall Sudan was a mess for a long time. I mean, I think it's still somewhat messy but it's better, where there was a real attack of the government on the people and people were fleeing, it was a huge humanitarian crisis, big refugee crisis, people were fleeing to eastern Chad and ending up in refugee camps. And it was horrible what was going on there.

And so we founded a charity called Ante Up for Africa that basically used poker to raise money to allocate to NGOs that were doing really great work in Sudan, in terms of trying to really help solve for that problem. So we did that, we raised I think a little over $4 million for that cause. That was I think I would say about 2010 to 2014 maybe, right around there that we did that. No, I guess it was more 2006, probably 2006 to 2010 I'm guessing. That's around when we did that. And then after we had allocated all the capital we finished up and shut down.

Now I'm actually co-founder of an organization called the Alliance for Decision Education and what we're trying to do is take-- business people know about the work of people like Daniel Kahneman and obviously Michael Mauboussin and Phil Tetlock and people who are really working in the space bringing sort of behavioral finance and behavioral economics and cognitive psychology and judgment decision-making into the business space and really helping people to become better decision makers. But that particular type of education hasn't gotten into our K through 12 educational system and we feel like it's incredibly important to teach people to be great decision makers. How do you figure out what's true, how do you figure out what your values are, what do you do with it? How do you think through what the best decision is?

And particularly when I think about the information ecosystem that we live in now, in terms of just navigating the information that's available on the internet and trying to figure out how to make good decisions about that and sort out what's true and what's not true with that. We feel like this is even more important now. And so that's what we do at the Alliance for Decision Education is bring decision education into K through 12.

ANDY SERWER: And finally, Annie, you've had such a varied life. What advice do you have for young people who are starting out and sort of who are interested in various things, what's the best path for them?

ANNIE DUKE: Oh, gosh, this is such good serendipity that you asked me about this because this is what I'm writing about now. Is really start to think about the fact that most of the things you do are not permanent lifetime decisions. So particularly when you're young, be a really good sampler and when you find out that you don't like something be willing to quit, it's OK. It's fine to quit.

If you think about the importance of grit and stick-to-itiveness, you have to figure out what is the thing that I really want to stick to? And you don't want to stick to the first thing you try. You want to try out a whole bunch of different things that you be willing to say, well, I'm just going to try this job and see if I like it, and if I don't like it I'll go on to something else. And try to start to learn what the system, the ecosystem is, what the opportunities are that are available to you, what you like and don't like. And do it while you're young and you have lots and lots of time, because your flexibility to be able to move from thing to thing when you're young until you find the really great match for yourself is-- that's the time to do it, because you have so much in front of you.

And I think that in this particular culture in our country, we sort of teach people, like, you're supposed to go into college knowing exactly what you want to do. You're supposed to come out of college knowing exactly what your career is going to be for the next 50 years. And I hope that what I can show is I bounced around from thing to thing, and then I ended up-- it all has a thread that I was pulling, which was decision making under uncertainty and learning under uncertainty. But I was sort of realizing that passion that I had in so many different ways.

And I'm so happy that that's what I did. And I think that we need to get away from this idea of come out of college and know what your next 50 years is going to look like and understand that during college and after college it should be your sampling time. Try out a whole bunch of different things, see what you like see what you don't like. I'd really recommend the book Range by David Epstein for anybody who's in college right now or just coming out of college, that really talks about what the advantages of this kind of broad sampling are, it's a wonderful, wonderful book.

ANDY SERWER: Annie Duke, expert in behavioral decision science and former professional poker champion. Thanks so much for spending time with us.

ANNIE DUKE: Thank you for having me.

ANDY SERWER: You've been watching Influencers. I'm Andy Serwer. We'll see you next time.