How Experts Make Complex Decisions
Skip to content
Economics Marketing Strategy Nov 1, 2022

How Experts Make Complex Decisions

By studying 200 million chess moves, researchers shed light on what gives players an advantage—and what trips them up.

two people playing chess

Yevgenia Nayberg

Making a simple decision is akin to ordering off a restaurant menu: you evaluate the available options one by one and choose whichever alternative promises to make you happiest or deliver the greatest payoff. But when it comes to more complicated choices—say, shopping for a house, devising a business plan, or evaluating insurance policies—identifying the objective “best” option is impractical, and often impossible.

Choosing a health-insurance plan, for example, requires estimating the likelihood that you’ll need a biopsy or an appendectomy—a multilayered guessing game sure to be fraught with error. Selecting a marketing strategy can be similarly knotty, as every potential move opens the door to myriad reactions from customers and competitors, leading to millions of possible scenarios, any of which the decision-maker can only imperfectly foresee.

“There’s a different dimension to decision-making when the available alternatives are so complex that you can’t even figure out what a given option is worth to you,” says Jörg L. Spenkuch, an associate professor of managerial economics and decision sciences at the Kellogg School.

So what does it take to make a good choice when facing this kind of complexity? Does slowing down or having more experience help—or do these convoluted decisions simply leave everyone grasping at straws, regardless of their expertise or how long they spend pondering their options?

These questions have received limited attention from social scientists, says Yuval Salant, a professor of managerial economics and decision sciences. So he and Spenkuch recently teamed up to shed new light on the dynamics of decision-making in complicated scenarios. In a new study, they derive first-of-their-kind predictions about how people behave when making complex choices by using an unusual laboratory: the chess board. “Complexity and chess go hand-in-hand,” says Salant.

Using an immense dataset of more than 200 million moves from an online chess platform, the researchers draw novel conclusions about how chess players find their way through the fog of complexity. They find that slowing down helps everyone, but that masters of the game benefit considerably more from extra decision time than less-expert players. And counterintuitively, they show that adding a mediocre option into the mix can actually be worse than adding a bad one.

How Chess Players Make Decisions

Chess has several features that make it perfect for studying byzantine decisions.

First, the quality of each move can be objectively ranked. Unlike picking an insurance plan or a marketing strategy, where accurately measuring and ranking alternatives is possible only with a well-functioning crystal ball, certain chess moves can be clearly identified as part of a winning strategy: these moves will (if followed up by subsequent optimal play) guarantee a win, no matter what one’s opponent does. Other moves can similarly guarantee a draw or would result in a loss.

(It may seem bizarre that the famously cerebral game can be boiled down to predefined winning and losing moves. But this was proved more than a century ago by the German mathematician Ernst Zermelo. “He basically said, ‘Chess isn’t an interesting game,’” Salant explains, “‘because either white has a winning strategy no matter what black is doing, or black has a winning strategy no matter what white is doing, or both of them can force a draw.’”)

Yet despite this ability to objectively evaluate every move, the nature of the game often makes it hard for even expert players to discern a good move from a bad one. “Even though there are famous theorems that say that at any point in the game either white or black has a winning strategy, you often cannot find those strategies because the game is so complex,” Salant says.

Consequently, the standard model of decision-making, in which a person evaluates every option one by one and then selects the best alternative, does not typically apply. The sheer number of available moves frequently makes it impossible to evaluate every option one by one (especially in speed chess, where players are limited in how long they can spend strategizing).

Thus, players typically consider only a small subset of all possible options and pick the first one that they consider good enough—that is, the first move that they believe will produce a win—a strategy that economists call “satisficing.”

Given that many real-world decisions also require a satisficing approach, Salant and Spenkuch thought that studying how players move on the chess board could help tease out the dynamics of complex decision-making.

A “Monumental” Database of Chess Moves

Their data came from, one of the world’s largest online chess servers. The database encompasses nearly eight years’ worth of games, capturing hundreds of millions of moves by hundreds of thousands of players, who range from hobbyists to grandmasters. Importantly, the data detail not only which piece moved where, but the full configuration of the board at each point in the game.

The researchers focused on what chess players call “endgames.” These are scenarios with a limited number of pieces left on the board in which both players are trying to achieve checkmate. In particular, endgames with fewer than six pieces were useful because they have been “solved” using computers—that is, for any configuration of six pieces on the board, computer scientists have cataloged all of the possible moves and how they fit into a strategy that can guarantee a win, a loss, or a draw.

“Complexity and chess go hand-in-hand.”

— Yuval Salant

But getting the data on actual moves was only the beginning. For each endgame, Salant and Spenkuch had to determine all of the possible moves that the player could have made, and whether each of those was a winning, losing, or drawing move.

“That was a monumental effort,” Spenkuch says, requiring 600,000 hours on Northwestern’s supercomputer cluster. The resulting dataset, containing 4.6 billion hypothetical moves, spans terabytes.

Salant and Spenkuch also obtained data on the number of moves required to obtain the guaranteed win, loss, or draw, a figure known in the chess world as “depth-to-mate.” They considered this metric a handy proxy for the complexity of a move (the logic being, it’s more difficult to correctly discern the value of a move that requires nine additional steps to obtain a win or loss than a move that only requires two steps.)

With the data wrangled, the researchers could begin testing some predictions.

Who Makes the Right Decision in Complex Situations?

They started by looking at how the degree of complexity affects which options get chosen.

The standard economic model of choice predicts that more-complex winning moves (that is, those that require more intermediary steps to obtain a checkmate) should be picked more frequently than less-complex winning moves. Salant and Spenkuch found that this was not the case in the chess data: The higher the depth-to-mate of a winning move, the less frequently it was chosen, presumably because the path to victory was fuzzier. “At least in this setting, the standard model is off in an important way,” says Spenkuch.

The researchers also found that a player’s skill level affected decision-making in nuanced ways. They were able to distinguish between “titled” players—those recognized as among the world’s best—and untitled players. Perhaps unsurprisingly, titled players were less likely than untitled players to make a mistake (i.e., picking a losing or drawing move when there was a winning option). This gap became most pronounced when the available winning moves were highly complex, suggesting that the best players’ expertise benefitted them most in especially tricky scenarios.

Players’ skill levels also influenced how they performed under time pressure. Predictably, titled players made fewer mistakes than untitled players regardless of how much time they were given to choose moves. But the difference was smallest in the most frantic varieties of speed chess that concluded within minutes. In slower matches, when decision time was much less limited, untitled players were considerably more likely than titled players to choose a bad move.

“It’s the titled players that benefit more from extra time to think,” Salant says.

Finally, the study looked at how changing the mix of available options affected decision-making, by comparing how players performed when they had slightly different sets of alternatives available. For instance, some players had to choose between 3 winning moves, 2 drawing moves, and 2 losing moves; other players had to choose between 3 winning moves, 3 drawing moves, and 1 losing move. Even though both groups have the same number of winning moves and therefore the same chance of accidentally stumbling into a bad strategy, the researchers found that the second group does significantly worse: substituting one losing move with a drawing move made players roughly 14 percent more likely to make a mistake, on average. The reason: “Drawing moves are more easily mistaken for winning moves,” Spenkuch explains.

What’s more, players did not seem to struggle with “choice overload” when facing a large number of possible moves: they actually made fewer mistakes when more moves were available, likely because a large portion of the alternatives in these scenarios were winning moves.

Putting these conclusions together, the takeaway is clear: “You can add as many winning moves as you want,” Salant says. “That will not result in more mistakes. If you want to make the problem as hard as possible, you need to add options that are not optimal but are pretty close to being optimal.”

Navigating Complexity in the Real World

Salant and Spenkuch see their paper as uncovering some fundamental features of complex decision-making. “For all sorts of business problems, it’s important to understand how people make decisions on a very basic level,” Spenkuch says.

He does urge caution before relying on these insights during your next strategy meeting though. “There’s a little bit of a leap of faith required to apply this to other settings,” Spenkuch warns. After all, most complex decisions are higher stakes than an online board game, and further studies are needed to test whether similar dynamics play out in other environments.

Still, there are lessons to be gleaned. Business leaders should not assume that a top-performing employee will make the right call in a split-second crisis, for instance, as the chess data suggest that the benefits of expertise fade when there’s a time crunch.

Businesses can also benefit from applying these findings to consumers, who may be asked to make decisions about complex products or services. When should you keep your marketing plan simple and when should you ramp up the complexity, perhaps by describing more facets of the product, or being ambiguous about potential use cases?

Part of the answer may depend on your market segment: flashy tactics are not likely to woo savvy consumers, whose experience will empower them to tell a great product from a so-so one.

And when facing multiple good options, Salant suspects that consumers, like chess players, may tend toward the alternatives whose benefits are immediately obvious to them— something that marketers would do well to keep in mind. “If you have a really good product, try to keep it simple,” he suggests.

Featured Faculty

Professor of Managerial Economics & Decision Sciences

Associate Professor of Managerial Economics & Decision Sciences

About the Writer

Jake J. Smith is a writer based in Illinois.

Most Popular This Week
  1. Sitting Near a High-Performer Can Make You Better at Your Job
    “Spillover” from certain coworkers can boost our productivity—or jeopardize our employment.
    The spillover effect in offices impacts workers in close physical proximity.
  2. Will AI Kill Human Creativity?
    What Fake Drake tells us about what’s ahead.
    Rockstars await a job interview.
  3. Podcast: How to Discuss Poor Performance with Your Employee
    Giving negative feedback is not easy, but such critiques can be meaningful for both parties if you use the right roadmap. Get advice on this episode of The Insightful Leader.
  4. 2 Factors Will Determine How Much AI Transforms Our Economy
    They’ll also dictate how workers stand to fare.
    robot waiter serves couple in restaurant
  5. How Are Black–White Biracial People Perceived in Terms of Race?
    Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
    How are biracial people perceived in terms of race
  6. The Psychological Factor That Helps Shape Our Moral Decision-Making
    We all have a preferred motivation style. When that aligns with how we’re approaching a specific goal, it can impact how ethical we are in sticky situations.
    a person puts donuts into a bag next to a sign that reads "limit one"
  7. Will AI Eventually Replace Doctors?
    Maybe not entirely. But the doctor–patient relationship is likely to change dramatically.
    doctors offices in small nodules
  8. What’s at Stake in the Debt-Ceiling Standoff?
    Defaulting would be an unmitigated disaster, quickly felt by ordinary Americans.
    two groups of politicians negotiate while dangling upside down from the ceiling of a room
  9. 5 Tips for Growing as a Leader without Burning Yourself Out
    A leadership coach and former CEO on how to take a holistic approach to your career.
    father picking up kids from school
  10. How to Manage a Disengaged Employee—and Get Them Excited about Work Again
    Don’t give up on checked-out team members. Try these strategies instead.
    CEO cheering on team with pom-poms
  11. One Key to a Happy Marriage? A Joint Bank Account.
    Merging finances helps newlyweds align their financial goals and avoid scorekeeping.
    married couple standing at bank teller's window
  12. Why Do Some People Succeed after Failing, While Others Continue to Flounder?
    A new study dispels some of the mystery behind success after failure.
    Scientists build a staircase from paper
  13. Which Form of Government Is Best?
    Democracies may not outlast dictatorships, but they adapt better.
    Is democracy the best form of government?
  14. Daughters’ Math Scores Suffer When They Grow Up in a Family That’s Biased Towards Sons
    Parents, your children are taking their cues about gender roles from you.
    Parents' belief in traditional gender roles can affect daughters' math performance.
  15. Take 5: Research-Backed Tips for Scheduling Your Day
    Kellogg faculty offer ideas for working smarter and not harder.
    A to-do list with easy and hard tasks
  16. What Went Wrong at AIG?
    Unpacking the insurance giant's collapse during the 2008 financial crisis.
    What went wrong during the AIG financial crisis?
  17. The Second-Mover Advantage
    A primer on how late-entering companies can compete with pioneers.
  18. Leave My Brand Alone
    What happens when the brands we favor come under attack?
  19. Take 5: Yikes! When Unintended Consequences Strike
    Good intentions don’t always mean good results. Here’s why humility, and a lot of monitoring, are so important when making big changes.
    People pass an e-cigarette billboard
More in Economics