Each year U.S. News takes a fresh look at the methodologies we use to evaluate hospitals. We examine potential changes we believe would improve our measurement of quality and would provide healthcare consumers with better data to support their decision making. Several noteworthy changes to the Best Children’s Hospitals rankings methodology are described here. Additional changes will be described in our methodology report, which will be available to Best Hospitals Dashboard users June 6.
Pediatric Heart Rankings
The ordeal of a family whose child is born with a congenital heart condition can be long and terrifying. But young heart patients have far better odds today than even a decade ago.
The Congenital Heart Surgery Database (CHSD), a clinical registry run by the Society of Thoracic Surgeons, has been a key force behind this improvement. By measuring patient survival after pediatric and congenital heart surgery, the CHSD helps surgeons and their teams benchmark their performance and identify improvement opportunities. The database also represents an invaluable tool for comparing the outcomes of pediatric heart surgery programs.
This year, the methodology for Pediatric Cardiology & Heart Surgery will incorporate a new and important measure from the CHSD: adjusted mortality rate. While surgical survival is not the only measure that matters, many experts — including clinicians and patients’ parents — consider it the most rigorously measured outcome in pediatric heart care. Consequently, we have elected this year to accord substantial weight to this measure, which is new to the rankings, in calculating hospitals’ overall score in that specialty. Adjusted mortality rate will account for 20 percent of the overall score, and the weight of outcomes as a whole — including other CHSD measures and a measure from the Scientific Registry of Transplant Recipients — will rise to 38.3 percent in that specialty, up from 33.3 percent last year.
To accommodate this change, the weight given to a hospital’s reputation will be reduced from 15 percent to 10 percent in Pediatric Cardiology & Heart Surgery. It will remain at 15 percent in other pediatric specialties.
Honor Roll
Last year, U.S. News introduced changes to the methodology of the Best Hospitals Honor Roll, which broadly assesses the quality of complex specialty care that hospitals deliver to adults. This year, we have made changes to the Best Children’s Hospitals honor roll methodology that approximately parallel those made to Best Hospitals last year. Specifically, hospitals will receive up to 25 Honor Roll points per specialty according to their position in a specialty’s rankings. The No. 1-ranked hospital will receive the 25-point maximum, the No. 2 hospital 24 points and so on to No. 20. Hospitals ranked No. 21 through 50 will all receive 5 points. A hospital’s points will be summed across all 10 specialties.
The 10 hospitals with the highest point totals will be designated Best Children’s Honor Roll hospitals. The numerical rankings will be determined by total points. More than 10 hospitals may be included in the Honor Roll in case of ties.
If the new methodology had been in effect last year, the 2016-17 Best Children’s Hospitals Honor Roll would have been minimally different. Nine of the hospitals would have been in the top 10 and most would have earned the same Honor Roll ranking in either scoring methodology.
Identifying Our Advisors
The survey of 1,800-plus items we use to collect data from pediatric hospitals is the core of the Best Children’s Hospitals project, and it would not be possible without the ongoing guidance we and RTI International, our data contractor, receive from 12 working groups of dedicated volunteers. The pediatric researchers, clinicians and hospital coding experts in these working groups meet regularly during the summer and fall to review the survey questions and recommend changes. They also provide advice on how we can make best use of the resulting data we collect, though they do not receive access to the data. Each of the 10 pediatric specialties we evaluate has its own working group, and two additional groups with respective expertise in radiology and infection prevention provide input across all specialties.
We recently asked current members of the working groups for permission to publicly identify them, both to acknowledge their crucial contribution to the project and to increase our transparency. Nearly all agreed. In our forthcoming methodology report, which will become available in the Dashboard June 6 and be publicly available June 27, individuals who have granted us permission will be listed. The institutions of those who declined to be identified will be listed as well.
We are fortunate that so many esteemed experts volunteer so much time and energy to serve on a working group. To fill vacancies when they arise, we make an open call through participating hospitals in late June each year for nominations, due in July, for the next round of working groups to begin meeting in late summer. Selection of working group participants is made solely by U.S. News editors and RTI methodologists.
More from U.S. News
Best Children’s Hospitals 2016-17: Honor Roll and Overview
Risky Children’s Hospitals Highlighted in Surgeons’ Database
Methodology Changes for 2017-18 Best Children’s Hospitals originally appeared on usnews.com
