How Experts Make Complex Decisions
Skip to content
The Insightful Leader Live: What to Know about Today’s AI—and Tomorrow’s | Register Now
Economics Marketing Strategy Nov 1, 2022

How Experts Make Complex Decisions

By studying 200 million chess moves, researchers shed light on what gives players an advantage—and what trips them up.

two people playing chess

Yevgenia Nayberg

Making a simple decision is akin to ordering off a restaurant menu: you evaluate the available options one by one and choose whichever alternative promises to make you happiest or deliver the greatest payoff. But when it comes to more complicated choices—say, shopping for a house, devising a business plan, or evaluating insurance policies—identifying the objective “best” option is impractical, and often impossible.

Choosing a health-insurance plan, for example, requires estimating the likelihood that you’ll need a biopsy or an appendectomy—a multilayered guessing game sure to be fraught with error. Selecting a marketing strategy can be similarly knotty, as every potential move opens the door to myriad reactions from customers and competitors, leading to millions of possible scenarios, any of which the decision-maker can only imperfectly foresee.

“There’s a different dimension to decision-making when the available alternatives are so complex that you can’t even figure out what a given option is worth to you,” says Jörg L. Spenkuch, an associate professor of managerial economics and decision sciences at the Kellogg School.

So what does it take to make a good choice when facing this kind of complexity? Does slowing down or having more experience help—or do these convoluted decisions simply leave everyone grasping at straws, regardless of their expertise or how long they spend pondering their options?

These questions have received limited attention from social scientists, says Yuval Salant, a professor of managerial economics and decision sciences. So he and Spenkuch recently teamed up to shed new light on the dynamics of decision-making in complicated scenarios. In a new study, they derive first-of-their-kind predictions about how people behave when making complex choices by using an unusual laboratory: the chess board. “Complexity and chess go hand-in-hand,” says Salant.

Using an immense dataset of more than 200 million moves from an online chess platform, the researchers draw novel conclusions about how chess players find their way through the fog of complexity. They find that slowing down helps everyone, but that masters of the game benefit considerably more from extra decision time than less-expert players. And counterintuitively, they show that adding a mediocre option into the mix can actually be worse than adding a bad one.

How Chess Players Make Decisions

Chess has several features that make it perfect for studying byzantine decisions.

First, the quality of each move can be objectively ranked. Unlike picking an insurance plan or a marketing strategy, where accurately measuring and ranking alternatives is possible only with a well-functioning crystal ball, certain chess moves can be clearly identified as part of a winning strategy: these moves will (if followed up by subsequent optimal play) guarantee a win, no matter what one’s opponent does. Other moves can similarly guarantee a draw or would result in a loss.

(It may seem bizarre that the famously cerebral game can be boiled down to predefined winning and losing moves. But this was proved more than a century ago by the German mathematician Ernst Zermelo. “He basically said, ‘Chess isn’t an interesting game,’” Salant explains, “‘because either white has a winning strategy no matter what black is doing, or black has a winning strategy no matter what white is doing, or both of them can force a draw.’”)

Yet despite this ability to objectively evaluate every move, the nature of the game often makes it hard for even expert players to discern a good move from a bad one. “Even though there are famous theorems that say that at any point in the game either white or black has a winning strategy, you often cannot find those strategies because the game is so complex,” Salant says.

Consequently, the standard model of decision-making, in which a person evaluates every option one by one and then selects the best alternative, does not typically apply. The sheer number of available moves frequently makes it impossible to evaluate every option one by one (especially in speed chess, where players are limited in how long they can spend strategizing).

Thus, players typically consider only a small subset of all possible options and pick the first one that they consider good enough—that is, the first move that they believe will produce a win—a strategy that economists call “satisficing.”

Given that many real-world decisions also require a satisficing approach, Salant and Spenkuch thought that studying how players move on the chess board could help tease out the dynamics of complex decision-making.

A “Monumental” Database of Chess Moves

Their data came from, one of the world’s largest online chess servers. The database encompasses nearly eight years’ worth of games, capturing hundreds of millions of moves by hundreds of thousands of players, who range from hobbyists to grandmasters. Importantly, the data detail not only which piece moved where, but the full configuration of the board at each point in the game.

The researchers focused on what chess players call “endgames.” These are scenarios with a limited number of pieces left on the board in which both players are trying to achieve checkmate. In particular, endgames with fewer than six pieces were useful because they have been “solved” using computers—that is, for any configuration of six pieces on the board, computer scientists have cataloged all of the possible moves and how they fit into a strategy that can guarantee a win, a loss, or a draw.

“Complexity and chess go hand-in-hand.”

— Yuval Salant

But getting the data on actual moves was only the beginning. For each endgame, Salant and Spenkuch had to determine all of the possible moves that the player could have made, and whether each of those was a winning, losing, or drawing move.

“That was a monumental effort,” Spenkuch says, requiring 600,000 hours on Northwestern’s supercomputer cluster. The resulting dataset, containing 4.6 billion hypothetical moves, spans terabytes.

Salant and Spenkuch also obtained data on the number of moves required to obtain the guaranteed win, loss, or draw, a figure known in the chess world as “depth-to-mate.” They considered this metric a handy proxy for the complexity of a move (the logic being, it’s more difficult to correctly discern the value of a move that requires nine additional steps to obtain a win or loss than a move that only requires two steps.)

With the data wrangled, the researchers could begin testing some predictions.

Who Makes the Right Decision in Complex Situations?

They started by looking at how the degree of complexity affects which options get chosen.

The standard economic model of choice predicts that more-complex winning moves (that is, those that require more intermediary steps to obtain a checkmate) should be picked more frequently than less-complex winning moves. Salant and Spenkuch found that this was not the case in the chess data: The higher the depth-to-mate of a winning move, the less frequently it was chosen, presumably because the path to victory was fuzzier. “At least in this setting, the standard model is off in an important way,” says Spenkuch.

The researchers also found that a player’s skill level affected decision-making in nuanced ways. They were able to distinguish between “titled” players—those recognized as among the world’s best—and untitled players. Perhaps unsurprisingly, titled players were less likely than untitled players to make a mistake (i.e., picking a losing or drawing move when there was a winning option). This gap became most pronounced when the available winning moves were highly complex, suggesting that the best players’ expertise benefitted them most in especially tricky scenarios.

Players’ skill levels also influenced how they performed under time pressure. Predictably, titled players made fewer mistakes than untitled players regardless of how much time they were given to choose moves. But the difference was smallest in the most frantic varieties of speed chess that concluded within minutes. In slower matches, when decision time was much less limited, untitled players were considerably more likely than titled players to choose a bad move.

“It’s the titled players that benefit more from extra time to think,” Salant says.

Finally, the study looked at how changing the mix of available options affected decision-making, by comparing how players performed when they had slightly different sets of alternatives available. For instance, some players had to choose between 3 winning moves, 2 drawing moves, and 2 losing moves; other players had to choose between 3 winning moves, 3 drawing moves, and 1 losing move. Even though both groups have the same number of winning moves and therefore the same chance of accidentally stumbling into a bad strategy, the researchers found that the second group does significantly worse: substituting one losing move with a drawing move made players roughly 14 percent more likely to make a mistake, on average. The reason: “Drawing moves are more easily mistaken for winning moves,” Spenkuch explains.

What’s more, players did not seem to struggle with “choice overload” when facing a large number of possible moves: they actually made fewer mistakes when more moves were available, likely because a large portion of the alternatives in these scenarios were winning moves.

Putting these conclusions together, the takeaway is clear: “You can add as many winning moves as you want,” Salant says. “That will not result in more mistakes. If you want to make the problem as hard as possible, you need to add options that are not optimal but are pretty close to being optimal.”

Navigating Complexity in the Real World

Salant and Spenkuch see their paper as uncovering some fundamental features of complex decision-making. “For all sorts of business problems, it’s important to understand how people make decisions on a very basic level,” Spenkuch says.

He does urge caution before relying on these insights during your next strategy meeting though. “There’s a little bit of a leap of faith required to apply this to other settings,” Spenkuch warns. After all, most complex decisions are higher stakes than an online board game, and further studies are needed to test whether similar dynamics play out in other environments.

Still, there are lessons to be gleaned. Business leaders should not assume that a top-performing employee will make the right call in a split-second crisis, for instance, as the chess data suggest that the benefits of expertise fade when there’s a time crunch.

Businesses can also benefit from applying these findings to consumers, who may be asked to make decisions about complex products or services. When should you keep your marketing plan simple and when should you ramp up the complexity, perhaps by describing more facets of the product, or being ambiguous about potential use cases?

Part of the answer may depend on your market segment: flashy tactics are not likely to woo savvy consumers, whose experience will empower them to tell a great product from a so-so one.

And when facing multiple good options, Salant suspects that consumers, like chess players, may tend toward the alternatives whose benefits are immediately obvious to them— something that marketers would do well to keep in mind. “If you have a really good product, try to keep it simple,” he suggests.

Featured Faculty

Professor of Managerial Economics & Decision Sciences

Associate Professor of Managerial Economics & Decision Sciences

About the Writer

Jake J. Smith is a writer based in Illinois.

Most Popular This Week
  1. What Went Wrong at Silicon Valley Bank?
    And how can it be avoided next time? A new analysis sheds light on vulnerabilities within the U.S. banking industry.
    People visit a bank
  2. How Are Black–White Biracial People Perceived in Terms of Race?
    Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
    How are biracial people perceived in terms of race
  3. What Went Wrong at AIG?
    Unpacking the insurance giant's collapse during the 2008 financial crisis.
    What went wrong during the AIG financial crisis?
  4. Will AI Eventually Replace Doctors?
    Maybe not entirely. But the doctor–patient relationship is likely to change dramatically.
    doctors offices in small nodules
  5. Which Form of Government Is Best?
    Democracies may not outlast dictatorships, but they adapt better.
    Is democracy the best form of government?
  6. Podcast: "It's Hard to Regulate U.S. Banks!"
    Silicon Valley Bank spectacularly collapsed—and a new analysis suggests that its precarious situation is not as much of an outlier as we’d hope. On this episode of The Insightful Leader, we learn what went wrong and what should happen next.
  7. What Happens to Worker Productivity after a Minimum Wage Increase?
    A pay raise boosts productivity for some—but the impact on the bottom line is more complicated.
    employees unload pallets from a truck using hand carts
  8. Why Do Some People Succeed after Failing, While Others Continue to Flounder?
    A new study dispels some of the mystery behind success after failure.
    Scientists build a staircase from paper
  9. Marketers, Don’t Be Too Hasty to Act on Data
    Don’t like the trends you’re seeing? It’s tempting to take immediate action. Instead, consider a hypothesis-driven approach to solving your problems.
    CEO stands before large data wall
  10. Why Well-Meaning NGOs Sometimes Do More Harm than Good
    Studies of aid groups in Ghana and Uganda show why it’s so important to coordinate with local governments and institutions.
    To succeed, foreign aid and health programs need buy-in and coordination with local partners.
  11. Understanding the Pandemic’s Lasting Impact on Real Estate
    Work-from-home has stuck around. What does this mean for residential and commercial real-estate markets?
    realtor showing converted office building to family
  12. How Has Marketing Changed over the Past Half-Century?
    Phil Kotler’s groundbreaking textbook came out 55 years ago. Sixteen editions later, he and coauthor Alexander Chernev discuss how big data, social media, and purpose-driven branding are moving the field forward.
    people in 1967 and 2022 react to advertising
  13. How Much Do Campaign Ads Matter?
    Tone is key, according to new research, which found that a change in TV ad strategy could have altered the results of the 2000 presidential election.
    Political advertisements on television next to polling place
  14. How Peer Pressure Can Lead Teens to Underachieve—Even in Schools Where It’s “Cool to Be Smart”
    New research offers lessons for administrators hoping to improve student performance.
    Eager student raises hand while other student hesitates.
  15. Immigrants to the U.S. Create More Jobs than They Take
    A new study finds that immigrants are far more likely to found companies—both large and small—than native-born Americans.
    Immigrant CEO welcomes new hires
  16. Leaders, Don’t Be Afraid to Admit Your Flaws
    We prefer to work for people who can make themselves vulnerable, a new study finds. But there are limits.
    person removes mask to show less happy face
  17. For Students with Disabilities, Discrimination Starts Before They Even Enter School
    Public-school principals are less welcoming to prospective families with disabled children—particularly when they’re Black.
    child in wheelchair facing padlocked school doors
  18. Executive Presence Isn’t One-Size-Fits-All. Here’s How to Develop Yours.
    A professor and executive coach unpacks this seemingly elusive trait.
    woman standing confidently
  19. How Self-Reflection Can Make You a Better Leader
    Setting aside 15 minutes a day can help you prioritize, prepare, and build a stronger team
    Self-reflection improves leadership over time
More in Economics