Hospital evaluations are proliferating, giving patients unprecedented insight into institutions where variations in quality can determine whether they live or die. But those ratings, each using its own methodology, often come to divergent conclusions, sometimes providing as much confusion as clarity for consumers.
Some hospitals rated as outstanding by one group are ignored or panned by another. Even the same group's results can change significantly from year to year.
"I worry a lot about these ratings," said Jerod Loeb, who oversees health-care quality evaluation at the Joint Commission. "They're all justifiable efforts to provide information, but, at the end of the day, every single one of them is flawed in some respect. Rather than enlightening, we may be confusing."
As ratings multiply, more and more hospitals have something they can boast about. A third of hospitals in the United States - more than 1,600 - won at least one distinction from a major rating group or company last year, according to a Kaiser Health News analysis.
In the greater Philadelphia regional hospital market - defined by the Dartmouth Health Atlas as institutions to which patients are referred for certain major procedures in South Jersey and Southeastern Pennsylvania - 24 of 37 hospitals were singled out as exemplary in some way by at least one rating source.
Elsewhere, the proportion of winners was even higher: Around Fort Lauderdale, Fla., 21 of 24 hospitals were singled out as exemplary by at least one group. Around Baltimore, 19 of 22 hospitals won an award.
Many hospitals are eager to trumpet these distinctions. Healthgrades, U.S. News, and Leapfrog not only encourage that, but profit from it by charging licensing fees to hospitals to advertise their awards.
Andrew Brotman, chief clinical officer at NYU Langone Medical Center in Manhattan, said the fees can be substantial.
"Healthgrades, which is one we did well on, charges $145,000 to use this even on the website as a logo, so we don't do that," he said. "U.S. News is in the $50,000 range. Leapfrog is $12,500."
Lankenau buys licensing rights for some, like U.S. News, but not for others, Robinson said. The fee for Healthgrades "is pretty hefty, so we haven't used that one very much."
Healthgrades and Truven Health Analytics also offer consulting services to hospitals that want to improve their overall performances.
Rating groups say the potential for income from the hospitals they judge does not influence them.
"A hospital cannot buy an award, they must achieve it," Healthgrades said in a statement.
Jean Chenoweth, a Truven senior vice president, said the list doesn't earn Truven money but "gives the company a lot of visibility."
John Santa, who directs health ratings for Consumer Reports, which does not sell services to hospitals, said consumers benefit from different vantages, just as they do with cars or electronic devices, and the competition spurs each rating group to get better.
"We think that's consistent with good science," Santa said.
Avery Comarow, health rankings editor for U.S. News, agreed. "People go to hospitals for different reasons and priorities," he said. "I'm not sure there could be a single rating system that can do it all."
The calculations that go into ratings are complex. Most hospital assessments synthesize dozens of sets of data that Medicare publishes for each institution on its Hospital Compare website, including death rates and the results of patient satisfaction surveys. They also examine other sources, and use private surveys to create user-friendly lists or grades on their sites.
The Joint Commission, for example, also looks at how frequently patients received recommended treatments, such as flu shots for those with pneumonia. Consumer Reports examines the numbers of patients who die or are readmitted, infection rates, and Medicare surveys of patient experiences.
Leapfrog looks at data from its surveys of hospitals, the consistency with which hospitals followed safe surgical practices, frequencies of infections, and some types of patient harm. Healthgrades analyzes detailed Medicare records to find death and complication rates for 27 procedures and conditions.
But small changes in methodologies can make a big difference in ratings. Leapfrog gave a "C" to Penn Presbyterian Medical Center in its first safety rating last spring. Six months later, it was an "A" - one of 57 of 126 hospitals in Pennsylvania that got a different grade.
Patrick J. Brennan, chief medical officer for the University of Pennsylvania Health System, said the change had just as much to do with tweaks that Leapfrog made to its methods as with anything the hospital did. "I thought this was a very volatile report," he said.
Leapfrog attributes the change to improvements in Penn's performance. "Patient safety has to be a priority 24-7," said president Leah Binder. "The minute it slips off the priority list, that impacts the rating."
Patient safety rankings are based on overall frequency of correct procedures or errors, so hospitals can get good grades even when they have made some egregious errors. Leapfrog gave a "B" to the University of California, San Francisco, Medical Center, even though state regulators have penalized UCSF eight times for infractions since 2008, most recently for leaving a sponge in one patient and a plastic clip in the skull of another. The fines totaled $425,000.
Josh Adler, UCSF's chief medical officer, said the penalties were partly a result of the hospital's policy to ensure all errors are reported to authorities. In an e-mail, he said: "The key is that we constantly strive to deliver the highest-quality, safest, and most satisfying care, and that we are a learning organization."
National Hospital Ratings Websites
As efforts to rate hospitals have expanded, patients must struggle to make sense of seemingly contradictory assessments. Many of the following organizations measure different things in different ways, even though their sources of data are similar.
When choosing a hospital, it’s important to understand what went into the rating and not to rely on any single group’s judgment as gospel.
Medicare’s Hospital Compare
This government site publishes dozens of quality measures that private ratings groups use for their rankings. It won’t compare more than three hospitals, and it doesn’t synthesize data into one overall rating.
This nonprofit calculates a “safety score” on a 100-point scale based on infection, error, and readmission rates, as well as good communication from doctors and nurses. Data is subscription-only.
This private company rates hospitals in up to two dozen specialties and provides some of Medicare’s Hospital Compare data in a user-friendly format. It also gives awards in various specialties.
The Joint Commission
This accreditation nonprofit tells you whether a hospital has passed its certification reviews. Awards are given in areas such as stroke and heart care. A “top performers” list honors hospitals that follow proper procedures but doesn’t assess their success.
Leapfrog Hospital Safety Score
This nonprofit group uses government data and its own surveys to produce a letter grade for how well hospitals protect patients from unnecessary harm.
Truven Health Analytics 100 Top Hospitals
This private company selects best hospitals in five categories based on their size and whether they are teaching or community hospitals. It also picks 15 top hospital systems and best cardiovascular programs.
U.S. News and World Report
Alone among raters, the media company surveys specialists for their opinions in choosing top national and regional hospitals by specialty. For 12 specialties, the score is one-third reputation, two-thirds data. In four specialties, ratings are based only on reputation. It also ranks best pediatric hospitals. (Children’s Hospital of Philadelphia ties for first nationally.)
— Jordan Rau
Kaiser Health News is an editorially independent program of the Henry J. Kaiser Family Foundation, a nonprofit, nonpartisan health policy research and communication organization not affiliated with Kaiser Permanente. Contact Jordan Rau at firstname.lastname@example.org.