The Centers for Medicare and Medicaid Services (CMS) calculates and publicly releases 5-star ratings in a wide variety of domains. Ratings for Medicare Part D plans, for instance, were released in 2006, with ratings for Medicare Part C (“Advantage”) plans and for nursing homes following in 2007 and 2008, respectively. Early in 2014, CMS introduced 5-star ratings for some physician groups. By the middle of 2014, CMS announced its intention to soon release 5-star ratings for dialysis facilities, home health agencies, and hospitals on its Compare websites.
The appeal of a 5-star rating system is obvious, given the ubiquity of rating systems on consumer websites, but the devil is almost always in the details: algorithms to translate a variety of clinical, process, and patient-reported outcomes into a single score are invariably complex and very often sensitive to both data quality and statistical assumptions. Fundamentally, the question is simple: does a 5-star rating for a health care provider have meaning? Analyses in this section suggest that, in the case of dialysis facilities, the answer is far from simple, as it appears that a single rating per facility betrays the complexity of the underlying quality of care.
CMS has proposed that the rating for each dialysis facility be based initially on three domains: standardized outcome measures, process outcomes, and vascular access (Figure 1). The first domain comprises three metrics: the standardized mortality ratio (SMR), the standardized hospitalization ratio (SHR), and the standardized transfusion ratio (STrR). Process outcomes include two metrics: the percentage of patients that received adequate dialysis (as quantified by Kt/V) and the percentage of patients that had hypercalcemia. And the vascular access domain comprises two metrics: the percentage of patients receiving hemodialysis with an arteriovenous fistula access and the percentage receiving hemodialysis with a venous catheter for more than 90 days. All seven of these metrics are currently reported, albeit in a variety of formats, on the consumer-oriented Dialysis Facility Compare website, in datasets at Data.Medicare.gov, and in the Dialysis Facility Reports.
CMS has proposed to combine the three domains and the seven constituent metrics in a specific manner. Each domain will be weighted equally, i.e., standardized outcome measures, process outcomes, and vascular access will each be assigned a weight of one-third (33 percent, Figure 2). Within each domain, the constituent metrics will also be weighted equally, i.e., for the summary of standardized outcome measures, the SMR, SHR, and STrR will each be assigned sub-weights of one-third (33 percent); for the summary of process outcomes, dialysis adequacy and hypercalcemia will each be assigned sub-weights of one-half (50 percent); and for the summary of vascular access, arteriovenous fistula use and long-term venous catheter use will each be assigned sub-weights of one-half (50 percent). Simple multiplication of weights and sub-weights demonstrates that each of the seven metrics is assigned a specific weight, as shown in Figure 2.