Podcast: Two Surprising Ways to Predict How Supreme Court Justices Will Vote
Skip to content
Policy Dec 10, 2018

Podcast: Two Surprising Ways to Predict How Supreme Court Justices Will Vote

Here’s what a half century of data can tell us.

a supreme court justice tips the scales with two lawyers.

Riley Mann

Based on the research and insights of

Jörg L. Spenkuch

Tonja Jacobi

Listening: Two Surprising Ways to Predict How Supreme Court Justices Will Vote
0:00 Skip back button Play Skip forward button 12:53

Are Supreme Court justices impartial umpires, or politicians in robes?

In this episode of the Kellogg Insight podcast, we take a fresh look at how the Supreme Court really makes decisions. First, Tonja Jacobi, a professor at Northwestern Pritzker School of Law, explains what oral arguments can tell us about how the Court is planning to vote. Then we hear from Jörg Spenkuch, an associate professor of managerial economics and decision-making at Kellogg, whose research questions whether justices are really as impartial as they think.

Podcast Transcript

Jessica LOVE: When Supreme Court Justice John Roberts spoke at his confirmation hearing, back in 2005, he used a memorable analogy. He compared a good judge to a baseball umpire.

John ROBERTS: Umpires don’t make the rules—they apply them. But it is a limited role. Nobody ever went to a ball game to see the umpire.

LOVE: And in the years since, it’s become clear: Roberts isn’t the only justice who portrays his role this way. Justices Kagan, Sotomayor, and Kavanaugh all used this same baseball comparison in their own confirmation hearings. They promised to stay neutral, to weigh the facts of each case, and make an unbiased decision on the merits. But, Kellogg Professor Jörg Spenkuch doesn’t think much of these promises.

Jörg SPENKUCH: They’re a joke. It’s a public spectacle. No justice is going to admit during his confirmation hearing that the reason they want to be on the Supreme Court is to move the law in one direction or the other. Every justice is going to say, “No, no, no, no, no. I decide cases only based on the merits.” Sure, some cases are decided purely on their merits, but our research shows that others are not.

: Welcome to the Kellogg Insight podcast. I’m your host, Jessica Love. This month we take a fresh look at the Supreme Court—and what really affects how justices make decisions.

First, producer Jake Smith talks with a law school professor who investigates what oral arguments—those long back-and-forths between lawyers and the justices—can tell us about the inner workings of the Court. Then, Smith talks more with Spenkuch, whose research puts the “judges as umpires” idea to the test. Stay with us.

[music interlude]

: Whenever the Supreme Court hears a case, they hold what are called “oral arguments.” During oral arguments, all nine of the justices sit at their bench, while lawyers for both sides (sometimes called “advocates”) plead their cases. Tonja Jacobi is a professor at Northwestern’s Pritzker School of Law who studies the Court, and oral arguments in particular. But as she explains: Oral arguments aren’t what they used to be.

Tonja JACOBI: If you listen to oral arguments from, say, the 1960s, even though it was a period of great turmoil in the country, the oral arguments really weren’t that interesting. They were much more sedate.

Robert GINNANE: Now Section 5(2)(b) of the Interstate Commerce Act provides that such acquisitions by a railroad or its affiliate…

: There weren’t as many questions. There wasn’t as much interaction between the justices and the advocates. It was much more like a speech. Now, it’s incredibly fast paced, it’s feisty, people are arguing.

Edwin KNEEDLER: With respect to severability, we don’t—
Antonin SCALIA
: Mr. Kneedler what happened to the Eighth Amendment? You really want us to go through these 2,700 pages? [laughter]

Sonia SOTOMAYOR: Wow, you’ve been talking a long time. You have two more sentences. [laughter]

: Jacobi wondered what all of this new talking, and arguing, might tell us about the Court. So she’s been using software to analyze oral-argument transcripts—thousands of pages worth. She’s found that today, justices on average spend nearly 25 percent more time talking than they did in the 1990s. But the reason why is surprising.

: During that time there hasn’t been an increase in the number of questions that are asked. It’s all taken up by comments by the justices. But if you look at the Supreme Court website, it says, “The point of an oral argument is for the justices to ask questions and the advocates to inform them through their answers.” But that’s not what we’re seeing. More and more we’re seeing the justices making comments, presenting conclusions, making statements.

One of my favorite examples of this is in a case where Justice Sotomayor was really trying to help out an advocate and she had essentially been making his argument for him, asking leading questions.

SOTOMAYOR: And one day is a day of punishment, isn’t it?
: No question as to that, Your Honor. Yes.
: Isn’t that your point?
: Basically it is that.

: And he was saying, “Oh yes, your honor. Yes, that’s absolutely right, your honor. I agree your honor, yes that’s correct”—at every point she made. And then later on in the argument the same advocate mistook Justice Kagan for Justice Sotomayor. He said the wrong name. And Justice Kagan said, “She’s Justice Sotomayor; she was the one trying to help you.”

: I was the one who wasn’t. [laughter]

: And so, there was an admission by a justice. And this is a paper I’ve got forthcoming where we show empirically that the judges are engaging in more advocacy for a particular side. And that’s really interesting to see because you’ve got some pretty great judicial minds who are supposed to be objectively intaking information and making decisions, but instead you see them arguing for a side. And they’re very good at it, a lot of the time, but at the same time they’re sort of pretending that that’s not what they’re doing.

: Given this new editorializing by the justices, Jacobi and a colleague wondered whether oral arguments can be used to predict how a case will be decided. They’ve created a model that pulls together data about how long each justice talks, how much they interrupt the lawyers and each other, things like that. So far, it’s been pretty successful.

: So the clearest example is Justice Gorsuch. If you want to predict his decisions, you can predict him with 86% accuracy by just looking at which side he interrupts more.

: She and her colleague share their predictions each week on their blog, SCOTUSOA.com (the “OA” standing for oral arguments). But Jacobi acknowledges that all of this analysis raises a big question: If we, WITHOUT knowing much about the case, can predict how a justice is going to vote, does that mean that oral arguments don’t matter?

: Now, I don’t believe that, because I think it matters in lots of different ways—the fact that the public get to see the Court operating. And part of the rule of law, and part of due process, is seeing that people have an opportunity to be heard. And I think that’s very important for the legitimacy of the court, and also for a sense that law matters in society, which I think people are having doubts about at the moment.

[music interlude]

: There is a long history of trying to predict how Supreme Court justices will vote. For example, nearly forty years ago, researchers began scouring newspaper editorials written about nominees to the court. By analyzing the language in these editorials, the researchers measured the ideology of each judge, and then made predictions based on that. It turned out, this strategy worked pretty well. A judge who’s seen as a mild conservative, or a hard­line liberal, will, if confirmed, often go on to vote like one.

: What’s unclear though is why does that correlation exist?

: That’s Jörg Spenkuch again. He’s an associate professor at Kellogg.

: Is it because Justices Scalia and Ruth Bader Ginsburg interpret the law differently? An alternative to that point of view is that Supreme Court justices are essentially politicians in robes. They have an idea of where they want the law to go, in which direction—say, more conservative, or more liberal—and they use their position at the heart of American democracy to shape the law in accordance with their personal views. Our view was, you need data to disentangle what is actually going on.

: Here was Spenkuch’s theory: If it’s the innocent explanation—that liberal and conservative justices really just interpret the law differently—then justices should be equally ideological all the time, no matter the case. But if it’s the more nefarious explanation, and justices really are actively trying to shape the law, then there will be some situations where they become extra liberal, or extra conservative. And he had a hunch when that would be.

The Supreme Court works like this: After oral arguments, the nine justices lock themselves in a room to deliberate. Then, in order of seniority, they take turns discussing what they think the ruling should be, and why. After each judge makes their case, they say how they’re planning to vote—essentially, taking a “straw poll.”

SPENKUCH: Now it’s important to note that the straw poll is not binding. A justice can always change his or her vote until the Court’s official decision has been announced.

: Because they do this informal straw poll, every justice knows how the other eight are likely to vote—which means they know when the majority hinges on their vote. In other words, they know when their vote matters most—when they’re casting the pivotal vote—and they can use that information when deciding which side to vote for. For example: Imagine the Court is deciding a divisive case. In the straw poll, five of the nine justices say they’re going to vote for Side A, and the other four say they’re going to vote for Side B.

: All five justices voting with the majority are pivotal, because if they switched the case would be decided differently. But none of the four justices in the minority are pivotal because if they switched, the outcome of the case would remain the same.

: So the question is: Do those five justices vote differently than usual, since they’re casting the pivotal vote? To find out, Spenkuch and his coauthors used a huge dataset capturing every justice’s vote on every Supreme Court case going back to 1946.

: What we find is that a justices’ final vote correlates much more heavily with his or her ideology or perceived ideology when his vote is pivotal than when it is not.

: This wasn’t just a small change, either. The study found that the average justice becomes roughly twice as ideological when they cast the decisive vote.

: Take a conservative like Scalia. And, suppose it’s a case that involves certain individual liberties. In cases when Scalia is pivotal, he will almost certainly vote conservative. In cases in which he is not pivotal, he will sometimes vote liberally, writing a dissenting opinion. So when it comes to moving the law in the conservative direction, conservatives can count on someone like Scalia, Gorsuch, John Roberts to a certain extent. And liberals can count on people like RBG or Elena Kagan, et cetera.

: To Spenkuch, this means that the idea of justices as impartial umpires just doesn’t hold water. But beyond that, it has serious implications for the justice system. After all, a person having their day in court may receive a certain ruling not because of how well their lawyer pleaded their case, but simply because the justices are voting strategically to further their own agendas. And it means that the U.S. president, who gets to appoint new justices, ends up able to shift Constitutional law in ways the country’s Founders never intended.

: In this day and age, the prospects for a Constitutional amendment are essentially nil. Congress is so polarized that it’s just not going to happen. So who makes Constitutional law in America in the 21st century? It’s the Supreme Court justices. And what our work suggests is that the direction that Constitutional law takes is essentially determined by the ideology of these justices. So this gives the U.S. president an enormous amount of power in moving law further to the left or to the right, and it’s absolutely not what’s intended by our system of checks and balances.

[music interlude]

: This program was produced by Kevin Bailey, Jessica Love, Fred Schmalz, Jake Smith, Emily Stone, and Michael Spikes. It was written by Jake Smith.

Special thanks to our guests, Tonja Jacobi and Jörg Spenkuch.

As a reminder, you can find us on iTunes, Google Play, or our website. If you like this show, please leave us a review or rating—that helps new listeners find our show. And visit us at insight.kellogg.northwestern.edu, where you can read more about public policy and the impact of elections.

We’ll be back next month with another episode of the Kellogg Insight podcast.
Featured Faculty

Associate Professor of Managerial Economics & Decision Sciences

About the Writer
Jake J. Smith is a research editor for Kellogg Insight.
Most Popular This Week
  1. How Are Black–White Biracial People Perceived in Terms of Race?
    Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
    How are biracial people perceived in terms of race
  2. Don’t Wait to Be Asked: Lead
    A roadmap for increasing your influence at work.
    An employee leads by jumping from the bleachers and joining the action.
  3. How (Not) to Change Someone’s Mind
    Psychologists have found two persuasion tactics that work. But put them together and the magic is lost.
    A woman on a street talks through a large megaphone.
  4. Which Form of Government Is Best?
    Democracies may not outlast dictatorships, but they adapt better.
    Is democracy the best form of government?
  5. Knowing Your Boss’s Salary Can Make You Work Harder—or Slack Off
    Your level of motivation depends on whether you have a fair shot at getting promoted yourself.
    person climbin ladder with missing rungs toward rich boss surrounded by money bags on platform
  6. How Autocracies Unravel
    Over time, leaders grow more repressive and cling to yes-men—a cycle that’s playing out today in Putin’s Russia.
    autocrat leaning over battle map surrounded by yes-men.
  7. Will AI Eventually Replace Doctors?
    Maybe not entirely. But the doctor–patient relationship is likely to change dramatically.
    doctors offices in small nodules
  8. Sitting Near a High-Performer Can Make You Better at Your Job
    “Spillover” from certain coworkers can boost our productivity—or jeopardize our employment.
    The spillover effect in offices impacts workers in close physical proximity.
  9. Why Do Some People Succeed after Failing, While Others Continue to Flounder?
    A new study dispels some of the mystery behind success after failure.
    Scientists build a staircase from paper
  10. It’s Performance Review Time. Which Ranking System Is Best for Your Team?
    A look at the benefits and downsides of two different approaches.
    An employee builds a staircase for his boss.
  11. Entrepreneurship Through Acquisition Is Still Entrepreneurship
    ETA is one of the fastest-growing paths to entrepreneurship. Here's how to think about it.
    An entrepreneur strides toward a business for sale.
  12. Four Strategies for Cultivating Strong Leaders Internally
    A retired brigadier general explains how companies can prioritize talent development.
    Companies should adopt intentional leadership strategies since developing leaders internally is critical to success.
  13. Too Much Cross Talk. Too Little Creativity. How to Fix the Worst Parts of a Virtual Meeting.
    Six tools from an unlikely place—improv comedy—to use on your next Zoom call.
    meeting participants improv
  14. Take 5: How to Be Prepared for Important Career Moments
    Expert advice on getting ready to network, negotiate, or make your case to the CEO.
    How to be prepared
  15. Why Do Long Wars Happen?
    War is a highly inefficient way of dividing contested resources—yet conflicts endure when there are powerful incentives to feign strength.
    long line of soldiers marching single file through a field
  16. Podcast: The Case for Admitting (Some) Flaws at Work
    On this episode of The Insightful Leader: Why showing vulnerability can actually be a boon for leaders.
  17. What Went Wrong at AIG?
    Unpacking the insurance giant's collapse during the 2008 financial crisis.
    What went wrong during the AIG financial crisis?
  18. 2 Factors Will Determine How Much AI Transforms Our Economy
    They’ll also dictate how workers stand to fare.
    robot waiter serves couple in restaurant
  19. Take 5: Not So Fast!
    A little patience can lead to better ideas, stronger organizations, and more-ethical conduct at work.
More in Leadership & Careers Careers