AHA Review of CMS Hospital Star Ratings Finds Volatility for Smaller Hospitals

In 2020, the Centers for Medicare and Medicaid Services (CMS) reevaluated and revised the Hospital Overall Star Ratings methodology to streamline the ratings process and improve equitable ratings throughout a highly-varied industry. The American Hospital Association (AHA) reviewed the changes and found that they provide some stability for hospitals, but smaller hospitals with fewer reporting metrics are still having a difficult time receiving fair, comparative assessments.

To review the new star ratings system, AHA commissioned KNG Health to compare the old methodology to the new and assess the stability of the new methodology by comparing hypothetical hospital performance under 2020 standards to 2021 actual performance.

One evaluation change, the peer grouping methodology, compares similar hospitals by the number of measures reported. There are now 15 possible combinations of measure groups on which hospitals may be scored, a quarter of all hospitals are split across these combinations. From the AHA report:

This means that a hospital’s star rating may reflect a very different set of measure groups. This can be misleading and make it difficult to assess meaningful differences in quality between hospitals with different ratings. Even within measure groups there is significant variation in the number of measures on which hospitals are scored.

In an AHA Blog Post, the association used the findings to suggest some repairs to the star ratings system. AHA recommends moving away from overall ratings and assessing the peer grouping approach. Furthermore, it recommends using language around star ratings that suggest that the ratings may be a result of the metrics as much as hospital performance. Finally, AHA suggests replacing unstable measures so that hospitals may have sufficient data to be compared on more measures.