Today, a commentary was released assessing publicly reported hospital quality rating systems – including The Leapfrog Group’s. “Rating the Raters” appears in the Catalyst section of the New England Journal of Medicine as an opinion piece. It is not designed to offer the evidence and replicability that a traditional study would offer, nor do the authors detail what standards they applied to reach their conclusions about the four ratings programs. That said, the ratings organizations in the piece would not allow themselves the luxury of issuing hospital ratings as random opinions, without basic rigor and transparency.
The authors are entitled to their own opinions and it is valuable to hear their perspectives. However, they are not entitled to their own facts. Rudimentary fact checking would have uncovered serious errors in the description of Leapfrog’s ratings programs in the piece. For instance, the assertion that Leapfrog audits only a handful of hospitals reporting to the Leapfrog Hospital Survey is demonstrably false. As evidence shows, Leapfrog verifies 100% of surveys submitted, first electronically, then by a Survey expert. A random selection of hospitals is required to submit additional documentation, and another selection of hospitals are subject to on-site verification by a team of independent reviewers. Following submission of 2019 Surveys this summer, over 1,000 hospitals are undergoing intensive verification and documentation.
In addition to basic fact-checking, future iterations of this paper would have greater credibility if the majority of authors were not employed at health systems with a history of feuding with one or more of the ratings organizations they analyze. The piece would appear more objective without that conflict.