How Experts Make Complex Decisions
Skip to content
Podcast | Insight Unpacked Season 1: Extraordinary Brands and How to Build Them
Economics Marketing Strategy Nov 1, 2022

How Experts Make Complex Decisions

By studying 200 million chess moves, researchers shed light on what gives players an advantage—and what trips them up.

two people playing chess

Yevgenia Nayberg

Based on the research of

Yuval Salant

Jörg L. Spenkuch

Making a simple decision is akin to ordering off a restaurant menu: you evaluate the available options one by one and choose whichever alternative promises to make you happiest or deliver the greatest payoff. But when it comes to more complicated choices—say, shopping for a house, devising a business plan, or evaluating insurance policies—identifying the objective “best” option is impractical, and often impossible.

Add Insight
to your inbox.

Choosing a health-insurance plan, for example, requires estimating the likelihood that you’ll need a biopsy or an appendectomy—a multilayered guessing game sure to be fraught with error. Selecting a marketing strategy can be similarly knotty, as every potential move opens the door to myriad reactions from customers and competitors, leading to millions of possible scenarios, any of which the decision-maker can only imperfectly foresee.

“There’s a different dimension to decision-making when the available alternatives are so complex that you can’t even figure out what a given option is worth to you,” says Jörg L. Spenkuch, an associate professor of managerial economics and decision sciences at the Kellogg School.

So what does it take to make a good choice when facing this kind of complexity? Does slowing down or having more experience help—or do these convoluted decisions simply leave everyone grasping at straws, regardless of their expertise or how long they spend pondering their options?

These questions have received limited attention from social scientists, says Yuval Salant, a professor of managerial economics and decision sciences. So he and Spenkuch recently teamed up to shed new light on the dynamics of decision-making in complicated scenarios. In a new study, they derive first-of-their-kind predictions about how people behave when making complex choices by using an unusual laboratory: the chess board. “Complexity and chess go hand-in-hand,” says Salant.

Using an immense dataset of more than 200 million moves from an online chess platform, the researchers draw novel conclusions about how chess players find their way through the fog of complexity. They find that slowing down helps everyone, but that masters of the game benefit considerably more from extra decision time than less-expert players. And counterintuitively, they show that adding a mediocre option into the mix can actually be worse than adding a bad one.

How Chess Players Make Decisions

Chess has several features that make it perfect for studying byzantine decisions.

First, the quality of each move can be objectively ranked. Unlike picking an insurance plan or a marketing strategy, where accurately measuring and ranking alternatives is possible only with a well-functioning crystal ball, certain chess moves can be clearly identified as part of a winning strategy: these moves will (if followed up by subsequent optimal play) guarantee a win, no matter what one’s opponent does. Other moves can similarly guarantee a draw or would result in a loss.

(It may seem bizarre that the famously cerebral game can be boiled down to predefined winning and losing moves. But this was proved more than a century ago by the German mathematician Ernst Zermelo. “He basically said, ‘Chess isn’t an interesting game,’” Salant explains, “‘because either white has a winning strategy no matter what black is doing, or black has a winning strategy no matter what white is doing, or both of them can force a draw.’”)

Yet despite this ability to objectively evaluate every move, the nature of the game often makes it hard for even expert players to discern a good move from a bad one. “Even though there are famous theorems that say that at any point in the game either white or black has a winning strategy, you often cannot find those strategies because the game is so complex,” Salant says.

Consequently, the standard model of decision-making, in which a person evaluates every option one by one and then selects the best alternative, does not typically apply. The sheer number of available moves frequently make it impossible to evaluate every option one by one (especially in speed chess, where players are limited in how long they can spend strategizing).

Thus, players typically consider only a small subset of all possible options, and pick the first one that they consider good enough—that is, the first move that they believe will produce a win—a strategy that economists call “satisficing.”

Given that many real-world decisions also require a satisficing approach, Salant and Spenkuch thought that studying how players move on the chess board could help tease out the dynamics of complex decision-making.

A “Monumental” Database of Chess Moves

Their data came from Lichess.org, one of the world’s largest online chess servers. The database encompasses nearly eight years’ worth of games, capturing hundreds of millions of moves by hundreds of thousands of players, who range from hobbyists to grandmasters. Importantly, the data detail not only which piece moved where, but the full configuration of the board at each point in the game.

The researchers focused on what chess players call “endgames.” These are scenarios with a limited number of pieces left on the board in which both players are trying to achieve checkmate. In particular, endgames with fewer than six pieces were useful because they have been “solved” using computers—that is, for any configuration of six pieces on the board, computer scientists have cataloged all of the possible moves and how they fit into a strategy that can guarantee a win, a loss, or a draw.

“Complexity and chess go hand-in-hand.”

— Yuval Salant

But getting the data on actual moves was only the beginning. For each endgame, Salant and Spenkuch had to determine all of the possible moves that the player could have made, and whether each of those was a winning, losing, or drawing move.

“That was a monumental effort,” Spenkuch says, requiring 600,000 hours on Northwestern’s supercomputer cluster. The resulting dataset, containing 4.6 billion hypothetical moves, spans terabytes.

Salant and Spenkuch also obtained data on the number of moves required to obtain the guaranteed win, loss, or draw, a figure known in the chess world as “depth-to-mate.” They considered this metric a handy proxy for the complexity of a move (the logic being, it’s more difficult to correctly discern the value of a move that requires nine additional steps to obtain a win or loss than a move that only requires two steps.)

With the data wrangled, the researchers could begin testing some predictions.

Who Makes the Right Decision in Complex Situations?

They started by looking at how the degree of complexity affects which options get chosen.

The standard economic model of choice predicts that more-complex winning moves (that is, those that require more intermediary steps to obtain a checkmate) should be picked more frequently than less-complex winning moves. Salant and Spenkuch found that this was not the case in the chess data: The higher the depth-to-mate of a winning move, the less frequently it was chosen, presumably because the path to victory was fuzzier. “At least in this setting, the standard model is off in an important way,” says Spenkuch.

The researchers also found that a player’s skill level affected decision-making in nuanced ways. They were able to distinguish between “titled” players—those recognized as among the world’s best—and untitled players. Perhaps unsurprisingly, titled players were less likely than untitled players to make a mistake (i.e., picking a losing or drawing move when there was a winning option). This gap became most pronounced when the available winning moves were highly complex, suggesting that the best players’ expertise benefitted them most in especially tricky scenarios.

Players’ skill levels also influenced how they performed under time pressure. Predictably, titled players made fewer mistakes than untitled players regardless of how much time they were given to choose moves. But the difference was smallest in the most frantic varieties of speed chess that concluded within minutes. In slower matches, when decision time was much less limited, untitled players were considerably more likely than titled players to choose a bad move.

“It’s the titled players that benefit more from extra time to think,” Salant says.

Finally, the study looked at how changing the mix of available options affected decision-making, by comparing how players performed when they had slightly different sets of alternatives available. For instance, some players had to choose between 3 winning moves, 2 drawing moves, and 2 losing moves; other players had to choose between 3 winning moves, 3 drawing moves, and 1 losing move. Even though both groups have the same number of winning moves and therefore the same chance of accidentally stumbling into a bad strategy, the researchers found that the second group does significantly worse: substituting one losing move with a drawing move made players roughly 14 percent more likely to make a mistake, on average. The reason: “Drawing moves are more easily mistaken for winning moves,” Spenkuch explains.

What’s more, players did not seem to struggle with “choice overload” when facing a large number of possible moves: they actually made fewer mistakes when more moves were available, likely because a large portion of the alternatives in these scenarios were winning moves.

Putting these conclusions together, the takeaway is clear: “You can add as many winning moves as you want,” Salant says. “That will not result in more mistakes. If you want to make the problem as hard as possible, you need to add options that are not optimal but are pretty close to being optimal.”

Navigating Complexity in the Real World

Salant and Spenkuch see their paper as uncovering some fundamental features of complex decision-making. “For all sorts of business problems, it’s important to understand how people make decisions on a very basic level,” Spenkuch says.

He does urge caution before relying on these insights during your next strategy meeting though. “There’s a little bit of a leap of faith required to apply this to other settings,” Spenkuch warns. After all, most complex decisions are higher stakes than an online board game, and further studies are needed to test whether similar dynamics play out in other environments.

Still, there are lessons to be gleaned. Business leaders should not assume that a top-performing employee will make the right call in a split-second crisis, for instance, as the chess data suggest that the benefits of expertise fade when there’s a time crunch.

Businesses can also benefit from applying these findings to consumers, who may be asked to make decisions about complex products or services. When should you keep your marketing plan simple and when should you ramp up the complexity, perhaps by describing more facets of the product, or being ambiguous about potential use cases?

Part of the answer may depend on your market segment: flashy tactics are not likely to woo savvy consumers, whose experience will empower them to tell a great product from a so-so one.

And when facing multiple good options, Salant suspects that consumers, like chess players, may tend toward the alternatives whose benefits are immediately obvious to them— something that marketers would do well to keep in mind. “If you have a really good product, try to keep it simple,” he suggests.

Featured Faculty

Professor of Managerial Economics & Decision Sciences

Associate Professor of Managerial Economics & Decision Sciences

About the Writer

Jake J. Smith is a writer based in Illinois.

Most Popular This Week
  1. Your Team Doesn’t Need You to Be the Hero
    Too many leaders instinctively try to fix a crisis themselves. A U.S. Army colonel explains how to curb this tendency in yourself and allow your teams to flourish.
    person with red cape trying to put out fire while firefighters stand by.
  2. What Triggers a Career Hot Streak?
    New research reveals a recipe for success.
    Collage of sculptor's work culminating in Artist of the Year recognition
  3. What’s the Secret to Successful Innovation?
    Hint: it’s not the product itself.
    standing woman speaking with man seated on stool
  4. Which Form of Government Is Best?
    Democracies may not outlast dictatorships, but they adapt better.
    Is democracy the best form of government?
  5. How Much Do Campaign Ads Matter?
    Tone is key, according to new research, which found that a change in TV ad strategy could have altered the results of the 2000 presidential election.
    Political advertisements on television next to polling place
  6. What Went Wrong with FTX—and What’s Next for Crypto?
    One key issue will be introducing regulation without strangling innovation, a fintech expert explains.
    stock trader surrounded by computer monitors
  7. How Are Black–White Biracial People Perceived in Terms of Race?
    Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
    How are biracial people perceived in terms of race
  8. Immigrants to the U.S. Create More Jobs than They Take
    A new study finds that immigrants are far more likely to found companies—both large and small—than native-born Americans.
    Immigrant CEO welcomes new hires
  9. How Experts Make Complex Decisions
    By studying 200 million chess moves, researchers shed light on what gives players an advantage—and what trips them up.
    two people playing chess
  10. Yes, Consumers Care if Your Product Is Ethical
    New research shows that morality matters—but it’s in the eye of the beholder.
    woman chooses organic lettuce in grocery
  11. Why Well-Meaning NGOs Sometimes Do More Harm than Good
    Studies of aid groups in Ghana and Uganda show why it’s so important to coordinate with local governments and institutions.
    To succeed, foreign aid and health programs need buy-in and coordination with local partners.
  12. Product Q&A Forums Hold a Lot of Promise. Here’s How to Make Them Work.
    The key to these online communities, where users can ask and answer questions, is how many questions get useful answers.
    man sits at computer reading Q&A forum
  13. What Went Wrong at AIG?
    Unpacking the insurance giant's collapse during the 2008 financial crisis.
    What went wrong during the AIG financial crisis?
  14. When Do Open Borders Make Economic Sense?
    A new study provides a window into the logic behind various immigration policies.
    How immigration affects the economy depends on taxation and worker skills.
  15. What the New Climate Bill Means for the U.S.—and the World
    The Inflation Reduction Act won’t reverse inflation or halt climate change, but it's still a big deal.
    energy bill with solar panels wind turbines and pipelines
  16. Post-War Reconstruction Is a Good Investment
    Ukraine’s European neighbors will need to make a major financial commitment to help rebuild its economy after the war. Fortunately, as the legacy of the post–World War II Marshall Plan shows, investing in Ukraine's future will also serve Europe's own long-term interests.
    two people look out over a city
  17. How Has Marketing Changed over the Past Half-Century?
    Phil Kotler’s groundbreaking textbook came out 55 years ago. Sixteen editions later, he and coauthor Alexander Chernev discuss how big data, social media, and purpose-driven branding are moving the field forward.
    people in 1967 and 2022 react to advertising
  18. The Political Divide in America Goes Beyond Polarization and Tribalism
    These days, political identity functions a lot like religious identity.
    people engage in conflict with swords
More in Economics