How Well Does COVID Public Policy Align with Science?
Skip to content
Policy Jan 7, 2021

How Well Does COVID Public Policy Align with Science?

In an era of misinformation, policy based on “dubious science” could mean a greater loss of life and economic hardship.

Epidemiologist and Politician give news conference

Yevgenia Nayberg

Based on the research of

Yian Yin

Jian Gao

Benjamin F. Jones

Dashun Wang

The rapid development of effective COVID-19 vaccines is a powerful example of how the business world and the scientific world can successfully work in tandem. But there is a thornier, related question: How well does public policy coevolve with science?

In the context of a major existential threat like the COVID pandemic, that means examining how well national, state, and local policies on social distancing, school and business closures, mask wearing, and the like align with the latest scientific findings on the virus.

The issue is critical because any disconnect between science and policy could mean a much greater loss of life and economic hardship. Indeed, there has been general skepticism about how well the science and policy communities communicate, and concern about how well policymakers can differentiate well-supported scientific ideas from less-established ones.

COVID-19 has presented a unique opportunity to study the link between policy and science. “It has been difficult in the past to quantify the impact of science on policymaking, especially in global and fast-moving contexts,” says Dashun Wang, an associate professor of management and organizations at Kellogg and director of Kellogg’s Center for Science of Science and Innovation (CSSI).

To understand the coevolution of science and policy, Wang and colleague Ben Jones, a Kellogg strategy professor, along with CSSI researchers Jian Gao, a Kellogg postdoctoral research fellow, and Yian Yin, a McCormick doctoral student, analyzed tens of thousands of policy documents worldwide. They studied the scientific research those documents drew on, as well as COVID-related documents from intergovernmental organizations like the World Health Organization (WHO). They were interested in how the type and volume of policy documents was influenced by the number of COVID cases in a country, as well as how closely policymaking relied on scientific research.

Overall, their study revealed good news: “We found that global policy attention to COVID-19 closely follows the number of cases and draws heavily on the rapidly evolving science,” Wang says. “And COVID policies rely on validated science. So there’s great synchrony between policy attention, the realities of the outbreak, and sound research.”

How Policy Evolves

To get at links between COVID case rates, policy, and science, the researchers analyzed over 37,000 policy documents across 114 countries—all major economies except mainland China—from the start of 2020 through May 26. This included publications from government agencies, think tanks, and IGOs. They also scrutinized the scientific research cited in the policy documents.

They started by comparing policy documents focused specifically on COVID with all other policy documents. They found that the share of COVID policy documents represented within the total policy documents followed a nearly identical trajectory to the number of confirmed cases worldwide. As the number of cases went up, so did the level of policy attention.

“In an era of misinformation, some good news is that policy documents emphasize vetted science, including the latest science.”

— Ben Jones

And the volume of policy documents wasn’t the only measure that followed pandemic trends; the focus of policy shifted, as well. As COVID-19 effects shifted from strictly health-related to socioeconomic concerns—business closures and unemployment, for example—the number and share of policies related to those areas increased. For example, policy documents in January and February 2020 focused mostly on medical and public health issues, but the share of economy-related documents rose starting in mid-March, as governments started contemplating and implementing shutdown orders.

Importantly, the team also studied the recency and quality of the scientific research on which policy documents relied.

“Do they actually use good science or dubious science?” Wang says.

One concern has been around scientific-article “preprints,” which are papers that have not yet been peer-reviewed. “There has been a flood of COVID research on preprint servers,” Jones says. “Some of these papers have received massive media attention only to be discredited later.”

The researchers found that pandemic-related policy documents drew on not only extremely recent science but also work that had been peer-reviewed. For example, nearly 20 percent of papers cited in COVID-related policy documents were published in 2020, about 10 times the rate seen for non-COVID policy documents. And the articles tended to be those published in respected medical-research journals like The Lancet.

Moreover, the papers cited by pandemic policy documents were more likely to be those referred to more often by other scientific papers, a typical measure of research quality. Research referenced in policy papers had 40 times more citations, on average, than a broader set of COVID research articles not included in policy documents. “This was important because we know science had a very strong response to COVID-19, as the fast pace of vaccine development suggests,” Wang says. “So it’s good to know that policy kept up.”

IGOs as Policy Conduits

Last, the researchers examined the role of governments and intergovernmental organizations (IGOs) in promoting pandemic-related policies.

The results suggest that the policies and communications of large, well-known IGOs like the WHO are critical in tying cross-border policies together and drawing science into policy.

“IGOs, especially the WHO, are a key conduit in absorbing science and then publishing related policy documents,” Jones says. “Then those policy documents are cited worldwide.” As the authors write, national governments “consume science largely indirectly through the IGOs.”

Moreover, when more countries and regions follow the same IGO guidelines, the result is a more unified, consistent response. But some countries, including the U.S., have been “antagonistic” toward large IGOs like the WHO, Wang notes, which can lead to policies less in line with well-documented science. This could potentially mean fragmented implementation and negative health and economic consequences.

“In an era of misinformation, some good news is that policy documents emphasize vetted science, including the latest science,” Jones says. “But different countries behave differently, and science is heard unevenly.”

That begs an important question: Is there a link between how individual countries and governments use science to inform their COVID-related policy and how well they manage the pandemic?

That’s exactly what the Kellogg team is looking into now, Wang explains. “We’re investigating country-level heterogeneity in leveraging science for policymaking, and what the consequences of that are for critical outcomes like citizens’ health.”

Featured Faculty

Gordon and Llura Gund Family Professor of Entrepreneurship; Professor of Strategy

Professor of Management & Organizations; Professor of Industrial Engineering & Management Sciences (Courtesy), Director, Center for Science of Science and Innovation (CSSI)

About the Writer
Sachin Waikar is a freelance writer based in Evanston, Illinois.
About the Research
Yin, Yian, Jian Gao, Benjamin F. Jones, and Dashun Wang. 2021. “Coevolution of Policy and Science during the Pandemic.” Science.
Most Popular This Week
  1. Sitting Near a High-Performer Can Make You Better at Your Job
    “Spillover” from certain coworkers can boost our productivity—or jeopardize our employment.
    The spillover effect in offices impacts workers in close physical proximity.
  2. Will AI Kill Human Creativity?
    What Fake Drake tells us about what’s ahead.
    Rockstars await a job interview.
  3. Podcast: How to Discuss Poor Performance with Your Employee
    Giving negative feedback is not easy, but such critiques can be meaningful for both parties if you use the right roadmap. Get advice on this episode of The Insightful Leader.
  4. 2 Factors Will Determine How Much AI Transforms Our Economy
    They’ll also dictate how workers stand to fare.
    robot waiter serves couple in restaurant
  5. How Are Black–White Biracial People Perceived in Terms of Race?
    Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
    How are biracial people perceived in terms of race
  6. The Psychological Factor That Helps Shape Our Moral Decision-Making
    We all have a preferred motivation style. When that aligns with how we’re approaching a specific goal, it can impact how ethical we are in sticky situations.
    a person puts donuts into a bag next to a sign that reads "limit one"
  7. Will AI Eventually Replace Doctors?
    Maybe not entirely. But the doctor–patient relationship is likely to change dramatically.
    doctors offices in small nodules
  8. What’s at Stake in the Debt-Ceiling Standoff?
    Defaulting would be an unmitigated disaster, quickly felt by ordinary Americans.
    two groups of politicians negotiate while dangling upside down from the ceiling of a room
  9. 5 Tips for Growing as a Leader without Burning Yourself Out
    A leadership coach and former CEO on how to take a holistic approach to your career.
    father picking up kids from school
  10. How to Manage a Disengaged Employee—and Get Them Excited about Work Again
    Don’t give up on checked-out team members. Try these strategies instead.
    CEO cheering on team with pom-poms
  11. One Key to a Happy Marriage? A Joint Bank Account.
    Merging finances helps newlyweds align their financial goals and avoid scorekeeping.
    married couple standing at bank teller's window
  12. Why Do Some People Succeed after Failing, While Others Continue to Flounder?
    A new study dispels some of the mystery behind success after failure.
    Scientists build a staircase from paper
  13. Which Form of Government Is Best?
    Democracies may not outlast dictatorships, but they adapt better.
    Is democracy the best form of government?
  14. Daughters’ Math Scores Suffer When They Grow Up in a Family That’s Biased Towards Sons
    Parents, your children are taking their cues about gender roles from you.
    Parents' belief in traditional gender roles can affect daughters' math performance.
  15. Take 5: Research-Backed Tips for Scheduling Your Day
    Kellogg faculty offer ideas for working smarter and not harder.
    A to-do list with easy and hard tasks
  16. What Went Wrong at AIG?
    Unpacking the insurance giant's collapse during the 2008 financial crisis.
    What went wrong during the AIG financial crisis?
  17. The Second-Mover Advantage
    A primer on how late-entering companies can compete with pioneers.
  18. Leave My Brand Alone
    What happens when the brands we favor come under attack?
  19. Take 5: Yikes! When Unintended Consequences Strike
    Good intentions don’t always mean good results. Here’s why humility, and a lot of monitoring, are so important when making big changes.
    People pass an e-cigarette billboard
More in Policy