Scientists Don’t Want to Get Scooped—and It’s Hurting Science

“Academic careers are built on reputation,” says Ryan Hill, a Kellogg assistant professor of strategy who studies the incentives that drive scientific innovation. “If I want credit that I can turn into salary from a university, I need people to recognize that I made novel discoveries.” Being the first to publish a finding is a major way for scientists to establish this recognition.

Still, little is known about the effects that these “priority races” have on scientists’ careers—and on the quality of the science itself. To find out, Hill and Carolyn Stein of the University of California, Berkeley investigated this topic within the field of structural biology, where researchers compete to discover the three-dimensional shapes of individual proteins.

Hill and Stein found that failing to be first does have a measurable cost: second-place projects are nearly 20 percent less likely to be published in a top journal, and the papers receive 21 percent fewer citations from future work.

Furthermore, Hill and Stein found that the more competitive a particular structure-solving race was, the more the researchers rushed their work—resulting in lower-quality findings.

“As economists, we like competition because it can encourage effort and timely disclosure of discoveries,” Hill says. “But it might also create unintended consequences that could have negative effects on science. It’s very useful to understand how scientists behave, based on the incentives they face.”

Shaping science

To understand how competition affects science research, Hill and Stein needed a setting where they could not only track multiple teams racing to solve the same scientific problem, but also keep tracking their work after the race was over.

Hill and Stein found such a setting in the Protein Databank (PDB). It’s a repository of findings by structural biologists, who aim to describe the thousands of precise shapes that proteins fold into within cells. A better understanding of these shapes can drive new medical and pharmaceutical discoveries, so multiple structural-biology teams often compete to “solve” the same proteins.

“Once researchers have a model for how a protein is shaped, they have to upload it into this database,” Hill explains. “That allows us to observe cases where two researchers deposit the same protein discovery at the same time, unbeknownst to each other. Then we observe which paper gets published first and see what happens to the team that got scooped.”

The PDB also gave Hill and Stein a view of how these races affect the research process itself. Certain proteins are known to be more biologically important than others—“maybe they’re linked to a gene that’s related to a disease that we care about,” Hill says. These “high potential” proteins naturally attract more scientific competition to solve them.

Hill and Stein wrote an economic model to describe how this competition would affect the amount of time scientists spend on a protein-structure problem before uploading their findings to the database. In their model, investing more time into research would ensure higher-quality results but would also increase the risk of getting scooped—especially on projects involving high-potential proteins.

The model suggested that scientists racing to solve these important proteins would be more likely to engage in rushed, lower-quality research. Hill and Stein then compared their model’s predictions with how scientists performed in the real world. They did so by analyzing quality metrics reported by structural biologists and validated by the PDB. These include the “resolution” of data created in x-ray crystallography experiments and the goodness-of-fit between structural models and the experimental data.

“We really wanted to consider the execution or the credibility of the research,” Hill explains. “If you open up the PDB, the quality metrics are right there on every protein page. So even if a project gets scooped and a paper doesn’t get published, we can still see the quality of their experimental results.”

Exaggerated fears, rushed results

After analyzing more than 1,600 priority races in the PDB between 1999 and 2017, Hill and Stein discovered that scientists are right to worry about getting scooped. Research that comes in second is almost 20 percent less likely to appear in a top-ten scientific journal and 24 percent less likely to become a “hit” paper that’s highly cited by other scientists in the year of its publication. Scooped papers don’t age as well, either: they receive 21 percent fewer citations than their first-place counterparts in the first five years after publication.

“These are significant effects,” Hill says.

But they don’t imply that scientific discovery is a winner-take-all game: scooped papers are only 2.6 percent less likely to publish. In other words, coming in second doesn’t mean going into the scientific dustbin. “You might still get into a good journal, just not a blockbuster” like Cell
or Nature, says Hill. “That represents real disappointment, but there is still a fair amount of credit sharing.”

Indeed, Hill and Stein surveyed 877 structural biologists and found that they significantly overestimated both the likelihood and costs of getting scooped. Respondents guessed that they had a 27 percent chance of getting scooped by a competitor, when the actual probability was only 3 percent. They also estimated that a scooped research project would get 59 percent fewer citations, when, in reality, the penalty was nearly three times smaller.

“They think that the costs are catastrophic when they’re probably not,” Hill says. “But these results speak to how much reputation matters in science.”

This reputational pressure also affects how research itself is done—at least in structural biology. Hill and Stein’s economic model suggested that in a competitive race, scientists would hedge their fears of getting scooped by rushing their research. When Hill and Stein analyzed real-life projects in the PDB, they found that scientists who were engaged in the most competitive protein races uploaded their findings to the PDB two months faster than scientists did for lower-priority proteins, resulting in significantly lower-quality work.

“The results of these projects are a little sketchier than we might want,” says Hill. “It’s a bit flipped from how we tend to think about competition, which is, ‘This is important, so we’d better get it right.’ Here, the incentive might be to just get something out there first.”

Striking a balance

Does this mean that competition has corrupted the scientific method?

Not so fast, Hill says. While the first solutions in these races may not be the best, scientists often conduct additional work to refine the results down the line. “For high-potential proteins, which are important to science, there do seem to be enough incentives for others to come along and improve the quality,” he says.

But these self-corrections come at a price. Hill and Stein estimate that two to six billion dollars have been spent since 1971 on efforts to improve initial results in structural biology. “In some ways, we’re probably happy as a society to bear that cost,” says Hill. “But you could also think about changing the incentives so that we wouldn’t have to.”

According to Hill, the key is to strike a better balance between incentives that encourage competition and those that relieve it. For example, from 2000 to 2015, the U.S. government funded the Protein Structure Initiative in order to solve protein-folding problems in a systematic manner akin to decoding the human genome.

“They funded a number of labs to work their way through lists of proteins, without being encouraged to write lots of papers,” Hill says. “That’s a more workman-like approach to knowledge creation. It balances out this need to find answers first in order to get credit.”

As new technologies like AI reshape the way science is done, Hill says that it’s a good time to reconsider the institutional incentives driving these discoveries. “We want scientists to have an incentive to publish their results,” he says. “But if everyone is reducing the quality of their science just a little bit, it becomes very hard to fix it later.”