Why We Know So Little about Disparities within the Federal Court System—and How That’s Finally Changing

These may seem like simple questions, but their answers are unknown. In fact, many fundamental questions about the functioning of our legal system remain unresolved—because the data needed to answer them, while technically public, are trapped behind a pricey paywall.

As a consequence, “there is a dearth of systematic studies on how the judiciary operates,” says Adam Pah, a professor of management and organizations at the Kellogg School. “You can ask the most knowledgeable legal scholars in this country, and no one can tell you all the basic facts you would expect.”

It’s rare to see a database painted as a villain, but that’s how many in the legal community regard PACER (Public Access to Electronic Court Records), the online repository for documents filed in federal court. (The federal system is made up of 94 district courts, 90 bankruptcy courts, and 13 circuit courts. It also includes the Supreme Court, though that now operates a separate database. State and local courts are a different animal altogether.) PACER is notoriously difficult to navigate: each federal court operates its own PACER portal, and the system’s user interface is, in Pah’s words, “straight out of 1991.”

Users are also charged an eye-popping ten cents per page for access to most documents in PACER. Although that may not sound like much, with legal briefs routinely hitting 60 pages, and numerous briefs filed in a single case, those fees can add up quickly.

But despite these steep barriers, Pah and colleagues recently managed to conduct a revealing large-scale analysis of the federal courts. He and 17 other scholars from across Northwestern received a grant that allowed them to access an unprecedented amount of PACER data.

Using just a portion of those data, the team uncovered stark disparities in how frequently low-income litigants get their court fees waived. This finding speaks to a serious lack of consistency in how judges—even those within the same district—handle petitions from litigants.

The researchers gleaned these novel insights using just a tiny fraction of the documents in PACER; if they can access more documents in the future, they hope to explore even bigger questions in even greater depth. The team is also working to create an open alternative to PACER, which will allow researchers and citizens to ask questions about how our judicial system works and find answers.

“This is really just the jumping-off point,” Pah says.

Judges Don’t Waive Court Fees Equally

The research was made possible thanks to a $1 million grant from the National Science Foundation that allowed the research team to buy a full year of PACER case filings from all 94 courts.

Pah and his collaborators decided to focus on downloading docket reports—essentially, the table of contents for a given case—which contain a date-stamped record of the filing of important documents and decisions. Because docket reports are short (and therefore cheap), the researchers could get their hands on a lot of them: 359,071 cases filed across all 94 districts in 2016.

The docket reports allowed the researchers to compare how frequently different judges granted motions for a waiver of court fees, such as the $400 required to file a federal lawsuit. These are known as in forma pauperis (IFP) requests.

There is no uniform standard for determining whether to grant IFP status. Instead, courts typically “leave the decision to the judge’s discretion,” the researchers write. “Individual judges thus have considerable power over whether to grant or deny access to the justice system.”

The researchers used the data from the 94 courts to determine how different judges leveraged this power. While the type of cases filed in Chicago will likely differ from cases filed in New York City, for instance, every case is randomly assigned to a judge within the appropriate court, which let the researchers determine how frequently judges were exercising discretion. “This is the key part,” Pah says. “Since the cases are randomly assigned, we would expect judges within the same district to waive fees at about the same rate.”

But that was not what they found. Their analysis showed that there was huge variation in how frequently judges waived fees, despite the fact that all judges in a given district should see similar petitions, on average.

In one district, certain judges approved IFP requests less than 20 percent of the time, while other judges approved the requests more than 80 percent of the time. Nationwide, nearly 40 percent of judges granted waivers at a rate that significantly differed from their peers, far higher than if the variation had been merely due to chance.

Pah and his fellow researchers were surprised to see such extreme variation. They had expected some differences among different judges, he explains, “but not to the point where individuals in the same district are as distinct as all the 94 courts are themselves.”

While the researchers did not examine what made judges more or less willing to grant petitions, Pah can imagine several plausible explanations, which could be tested in future research. “Is it because of differing interpretations of the rules?” Pah wonders. “Or is it related to the timing or volume of petitions that a judge sees across cases—where a petitioner in one case is so destitute it makes another petitioner in the following case seem less qualified?”

More Court Records Will Yield New Discoveries

These findings are already being used to help make the judicial system more equitable. The researchers recently presented their results to a group of federal judges who are responsible for making rules in their districts. “Now that they’ve seen our results, they’re interested in using our data to help improve the process,” Pah says.

Pah says he’s amazed by how much he and his collaborators have been able to learn from docket reports alone—which are, after all, relatively spartan documents. “Just with the docket reports, we were already in the Wild West, finding new information,” he says.

He also hopes the new tool that the team is creating will make the process of understanding the federal courts much simpler. Their ultimate goal is to make an alternative to PACER, an open-knowledge network that links court data to other public data about litigants, judges, lawyers, and courts, enabling researchers to ask questions and find answers.

For now, though, the new research demonstrates how much more could be done with greater access to the public documents in PACER. “Behind most of the docket entries, there’s something that was actually filed: briefs, written motions. That information and that context is going to be important for us to get to the next level of understanding,” Pah says.

And that next level of understanding is crucial for our democracy, he argues. Citizens deserve to know “whether the courts are functioning fairly and efficiently for the public. These questions need to get answered. We need to understand this.”