Podcast: Two Surprising Ways to Predict How Supreme Court Justices Will Vote

Jessica LOVE: When Supreme Court Justice John Roberts spoke at his confirmation hearing, back in 2005, he used a memorable analogy. He compared a good judge to a baseball umpire.

John ROBERTS: Umpires don’t make the rules—they apply them. But it is a limited role. Nobody ever went to a ball game to see the umpire.

LOVE: And in the years since, it’s become clear: Roberts isn’t the only justice who portrays his role this way. Justices Kagan, Sotomayor, and Kavanaugh all used this same baseball comparison in their own confirmation hearings. They promised to stay neutral, to weigh the facts of each case, and make an unbiased decision on the merits. But, Kellogg Professor Jörg Spenkuch doesn’t think much of these promises.

Jörg SPENKUCH: They’re a joke. It’s a public spectacle. No justice is going to admit during his confirmation hearing that the reason they want to be on the Supreme Court is to move the law in one direction or the other. Every justice is going to say, “No, no, no, no, no. I decide cases only based on the merits.” Sure, some cases are decided purely on their merits, but our research shows that others are not.

LOVE: Welcome to the Kellogg Insight podcast. I’m your host, Jessica Love. This month we take a fresh look at the Supreme Court—and what really affects how justices make decisions.

First, producer Jake Smith talks with a law school professor who investigates what oral arguments—those long back-and-forths between lawyers and the justices—can tell us about the inner workings of the Court. Then, Smith talks more with Spenkuch, whose research puts the “judges as umpires” idea to the test. Stay with us.

[music interlude]

Jake SMITH: Whenever the Supreme Court hears a case, they hold what are called “oral arguments.” During oral arguments, all nine of the justices sit at their bench, while lawyers for both sides (sometimes called “advocates”) plead their cases. Tonja Jacobi is a professor at Northwestern’s Pritzker School of Law who studies the Court, and oral arguments in particular. But as she explains: Oral arguments aren’t what they used to be.

Tonja JACOBI: If you listen to oral arguments from, say, the 1960s, even though it was a period of great turmoil in the country, the oral arguments really weren’t that interesting. They were much more sedate.

Robert GINNANE: Now Section 5(2)(b) of the Interstate Commerce Act provides that such acquisitions by a railroad or its affiliate…

JACOBI: There weren’t as many questions. There wasn’t as much interaction between the justices and the advocates. It was much more like a speech. Now, it’s incredibly fast paced, it’s feisty, people are arguing.

Edwin KNEEDLER: With respect to severability, we don’t—
Antonin SCALIA: Mr. Kneedler what happened to the Eighth Amendment? You really want us to go through these 2,700 pages? [laughter]

Sonia SOTOMAYOR: Wow, you’ve been talking a long time. You have two more sentences. [laughter]

SMITH: Jacobi wondered what all of this new talking, and arguing, might tell us about the Court. So she’s been using software to analyze oral-argument transcripts—thousands of pages worth. She’s found that today, justices on average spend nearly 25 percent more time talking than they did in the 1990s. But the reason why is surprising.

JACOBI: During that time there hasn’t been an increase in the number of questions that are asked. It’s all taken up by comments by the justices. But if you look at the Supreme Court website, it says, “The point of an oral argument is for the justices to ask questions and the advocates to inform them through their answers.” But that’s not what we’re seeing. More and more we’re seeing the justices making comments, presenting conclusions, making statements.

One of my favorite examples of this is in a case where Justice Sotomayor was really trying to help out an advocate and she had essentially been making his argument for him, asking leading questions.

SOTOMAYOR: And one day is a day of punishment, isn’t it?
Alan STOLER: No question as to that, Your Honor. Yes.
SOTOMAYOR: Isn’t that your point?
STOLER: Basically it is that.

JACOBI: And he was saying, “Oh yes, your honor. Yes, that’s absolutely right, your honor. I agree your honor, yes that’s correct”—at every point she made. And then later on in the argument the same advocate mistook Justice Kagan for Justice Sotomayor. He said the wrong name. And Justice Kagan said, “She’s Justice Sotomayor; she was the one trying to help you.”

KAGAN: I was the one who wasn’t. [laughter]

JACOBI: And so, there was an admission by a justice. And this is a paper I’ve got forthcoming where we show empirically that the judges are engaging in more advocacy for a particular side. And that’s really interesting to see because you’ve got some pretty great judicial minds who are supposed to be objectively intaking information and making decisions, but instead you see them arguing for a side. And they’re very good at it, a lot of the time, but at the same time they’re sort of pretending that that’s not what they’re doing.

SMITH: Given this new editorializing by the justices, Jacobi and a colleague wondered whether oral arguments can be used to predict how a case will be decided. They’ve created a model that pulls together data about how long each justice talks, how much they interrupt the lawyers and each other, things like that. So far, it’s been pretty successful.

JACOBI: So the clearest example is Justice Gorsuch. If you want to predict his decisions, you can predict him with 86% accuracy by just looking at which side he interrupts more.

SMITH: She and her colleague share their predictions each week on their blog, SCOTUSOA.com (the “OA” standing for oral arguments). But Jacobi acknowledges that all of this analysis raises a big question: If we, WITHOUT knowing much about the case, can predict how a justice is going to vote, does that mean that oral arguments don’t matter?

JACOBI: Now, I don’t believe that, because I think it matters in lots of different ways—the fact that the public get to see the Court operating. And part of the rule of law, and part of due process, is seeing that people have an opportunity to be heard. And I think that’s very important for the legitimacy of the court, and also for a sense that law matters in society, which I think people are having doubts about at the moment.

[music interlude]

SMITH: There is a long history of trying to predict how Supreme Court justices will vote. For example, nearly forty years ago, researchers began scouring newspaper editorials written about nominees to the court. By analyzing the language in these editorials, the researchers measured the ideology of each judge, and then made predictions based on that. It turned out, this strategy worked pretty well. A judge who’s seen as a mild conservative, or a hard­line liberal, will, if confirmed, often go on to vote like one.

SPENKUCH: What’s unclear though is why does that correlation exist?

SMITH: That’s Jörg Spenkuch again. He’s an associate professor at Kellogg.

SPENKUCH: Is it because Justices Scalia and Ruth Bader Ginsburg interpret the law differently? An alternative to that point of view is that Supreme Court justices are essentially politicians in robes. They have an idea of where they want the law to go, in which direction—say, more conservative, or more liberal—and they use their position at the heart of American democracy to shape the law in accordance with their personal views. Our view was, you need data to disentangle what is actually going on.

SMITH: Here was Spenkuch’s theory: If it’s the innocent explanation—that liberal and conservative justices really just interpret the law differently—then justices should be equally ideological all the time, no matter the case. But if it’s the more nefarious explanation, and justices really are actively trying to shape the law, then there will be some situations where they become extra liberal, or extra conservative. And he had a hunch when that would be.

The Supreme Court works like this: After oral arguments, the nine justices lock themselves in a room to deliberate. Then, in order of seniority, they take turns discussing what they think the ruling should be, and why. After each judge makes their case, they say how they’re planning to vote—essentially, taking a “straw poll.”

SPENKUCH: Now it’s important to note that the straw poll is not binding. A justice can always change his or her vote until the Court’s official decision has been announced.

SMITH: Because they do this informal straw poll, every justice knows how the other eight are likely to vote—which means they know when the majority hinges on their vote. In other words, they know when their vote matters most—when they’re casting the pivotal vote—and they can use that information when deciding which side to vote for. For example: Imagine the Court is deciding a divisive case. In the straw poll, five of the nine justices say they’re going to vote for Side A, and the other four say they’re going to vote for Side B.

SPENKUCH: All five justices voting with the majority are pivotal, because if they switched the case would be decided differently. But none of the four justices in the minority are pivotal because if they switched, the outcome of the case would remain the same.

SMITH: So the question is: Do those five justices vote differently than usual, since they’re casting the pivotal vote? To find out, Spenkuch and his coauthors used a huge dataset capturing every justice’s vote on every Supreme Court case going back to 1946.

SPENKUCH: What we find is that a justices’ final vote correlates much more heavily with his or her ideology or perceived ideology when his vote is pivotal than when it is not.

SMITH: This wasn’t just a small change, either. The study found that the average justice becomes roughly twice as ideological when they cast the decisive vote.

SPENKUCH: Take a conservative like Scalia. And, suppose it’s a case that involves certain individual liberties. In cases when Scalia is pivotal, he will almost certainly vote conservative. In cases in which he is not pivotal, he will sometimes vote liberally, writing a dissenting opinion. So when it comes to moving the law in the conservative direction, conservatives can count on someone like Scalia, Gorsuch, John Roberts to a certain extent. And liberals can count on people like RBG or Elena Kagan, et cetera.

SMITH: To Spenkuch, this means that the idea of justices as impartial umpires just doesn’t hold water. But beyond that, it has serious implications for the justice system. After all, a person having their day in court may receive a certain ruling not because of how well their lawyer pleaded their case, but simply because the justices are voting strategically to further their own agendas. And it means that the U.S. president, who gets to appoint new justices, ends up able to shift Constitutional law in ways the country’s Founders never intended.

SPENKUCH: In this day and age, the prospects for a Constitutional amendment are essentially nil. Congress is so polarized that it’s just not going to happen. So who makes Constitutional law in America in the 21st century? It’s the Supreme Court justices. And what our work suggests is that the direction that Constitutional law takes is essentially determined by the ideology of these justices. So this gives the U.S. president an enormous amount of power in moving law further to the left or to the right, and it’s absolutely not what’s intended by our system of checks and balances.

[music interlude]

LOVE: This program was produced by Kevin Bailey, Jessica Love, Fred Schmalz, Jake Smith, Emily Stone, and Michael Spikes. It was written by Jake Smith.

Special thanks to our guests, Tonja Jacobi and Jörg Spenkuch.

As a reminder, you can find us on iTunes, Google Play, or our website. If you like this show, please leave us a review or rating—that helps new listeners find our show. And visit us at insight.kellogg.northwestern.edu, where you can read more about public policy and the impact of elections.

We’ll be back next month with another episode of the Kellogg Insight podcast.