Are Healthcare “Report Cards” Good for Patients?

In grade school, a bad report card might have gotten you grounded, which probably motivated you to do better next time. Or did it just motivate you to take easier classes? In a larger context, “report cards” in sectors such as healthcare and education have received considerable attention from policy makers and researchers. Report cards provide information about the performance of institutions and individuals, where the performance depends both on the skill and effort of the provider (hospitals, physicians or schools), and the characteristics of the patients or students.

In the healthcare sector, proponents of report cards argue that these improve the ability of patients to identify the best physicians and hospitals, while giving providers a powerful incentive to improve quality. Detractors argue that report cards encourage providers to “game” the system by seeking healthier patients, avoiding very sick ones, or both.

A study published in the Journal of Political Economy in 2003 by David Dranove (Kellogg School’s Management & Strategy Department), Daniel Kessler (Stanford University), Mark McClellan (Stanford University) and Mark Satterthwaite (Kellogg School’s Management & Strategy Department) focuses on a well known healthcare sector case.

In the early 1990s, New York and Pennsylvania began publicly reporting the physician and hospital mortality rates for coronary artery bypass graft (CABG) surgeries. Dranove, Satterthwaite, and their co-authors used Medicare claim data on elderly patients and information on U.S. hospital characteristics from the American Hospital Association from 1987 to 1994 to analyze the impact of the introduction of report cards at both the patient and hospital levels. Their empirical strategy uses a “difference-in-difference” framework, which compares the change trends in New York and Pennsylvania before and after the introduction of report cards to the change in trends in the same periods in other states (the “control” states).

The main result of their research is that the introduction of report cards led to a substantial shift in the incidence of intensive cardiac treatment. It found that CABG patients in 1994 (after report cards were instituted) spent less money on healthcare in the year before their surgeries than did CABG patients in 1990 (before report cards were instituted), as shown in Table 1. This is contrary to trends in other states, where CABG patients’ healthcare expenses rose during that same time period. The study concludes that New York and Pennsylvania hospitals began operating on healthier patients—those patients who did not have to spend as much on hospital bills in the year before their surgeries, meaning they likely suffered from fewer health complications.

Table 1: Mean expenses in year prior to admission for AMI or for CABG surgery, elderly Medicare beneficiaries, 1990 and 1994.

Compared to other states that did not have mandatory report cards, doctors in Pennsylvania and New York hospitals became more reluctant to operate on difficult, severely ill patients. Instead, the study suggested that physicians opted to perform the coronary artery bypass graft (CABG) surgery on healthier patients who were more likely to have positive outcomes, possibly to improve their report card scores. The estimated decline in the measure of illness severity in New York and Pennsylvania relative to all other states was statistically significant and ranged from 3.74 to 5.30 percent. This could mean that sicker patients did not receive the treatment that could most benefit them, the researchers determined.

Report cards led to an increase in the number of CABG surgeries performed in those two states. This means that even as they limited the surgeries to healthier patients, doctors chose to perform more of them. “One possible explanation is that operating on healthier patients is a way of boosting your report card score,” said David Dranove, a Kellogg School professor and a co-author of the study.

In terms of aggregate outlays, the authors found that report cards led to higher expenses overall, both because of the increased number of CABG surgeries performed and the increased readmission rates for sick patients. “Poor outcomes are often associated with higher cost, and I think that’s what we’re seeing,” Dranove said.

Another key finding was that the introduction of report cards induced increased sorting of patients to providers on the basis of the severity of their illness. In both states, the heterogeneity of patients within each hospital diminished, with teaching hospitals (presumed to be of higher quality) picking up a larger share of severely ill patients. Supporting their finding that report cards led to increased selection and sorting, the researchers found that the introduction of report cards also led to delayed treatment for both healthy and sick patients. “One thing we know is overall there seemed to be a delay in all cardiac interventions, possibly as the hospitals tried to figure out how the interventions would affect their report card scores,” Dranove said. “But that’s a conjecture.”

To determine the effects of the delays, the researchers analyzed hospital expenditures for heart attack patients in the year after their initial visit and also looked at their rates of mortality and readmission. They found that relatively sicker heart attack patients (those who spent more money on healthcare in the year before their heart attack) experienced higher readmission rates and, in some cases, higher mortality rates after report cards were instituted. “The delays in treatment seemed to be associated with increased complications and increased readmission,” Dranove said, alluding to one of the study’s most alarming findings: sick patients were doubly affected by report cards, first because some might not have received needed CABG surgery and, second, because delays in their treatment led to adverse health affects later on.

But in their analysis, the researchers were hesitant to ring too loud an alarm, emphasizing that what they studied were the short-term effects of report cards. “I think the glass is more full than empty, but we look here at the empty part,” Dranove said. “The hoped-for benefit of report cards is for providers, in addition to the games they’re playing, to actually make an effort to improve patient care. And there are a lot of anecdotal accounts of that happening.”