Background Despite widespread use of quality indicators, it remains unclear to what extent they can reliably distinguish hospitals on true differences in performance. Rankability measures what part of variation in performance reflects 'true' hospital differences in outcomes versus random noise. Objective This study sought to assess whether combining data into composites or including data from multiple years improves the reliability of ranking quality indicators for hospital care. Methods Using the Dutch National Medical Registration (2007-2012) for stroke, colorectal carcinoma, heart failure, acute myocardial infarction and total hiparthroplasty (THA)/ total knee arthroplasty (TKA) in osteoarthritis (OA), we calculated the rankability for in-hospital mortality, 30-day acute readmission and prolonged length of stay (LOS) for single years and 3-year periods and for a dichotomous and ordinal composite measure in which mortality, readmission and prolonged LOS were combined. Rankability, defined as (betweenhospital variation/between-hospital+within hospital variation)×100% is classified as low (<50%), moderate (50%-75%) and high (>75%). Results Admissions from 555 053 patients treated in 95 hospitals were included. The rankability for mortality was generally low or moderate, varying from less than 1% for patients with OA undergoing THA/TKA in 2011 to 71% for stroke in 2010. Rankability for acute readmission was low, except for acute myocardial infarction in 2009 (51%) and 2012 (62%). Rankability for prolonged LOS was at least moderate. Combining multiple years improved rankability but still remained low in eight cases for both mortality and acute readmission. Combining the individual indicators into the dichotomous composite, all diagnoses had at least moderate rankability (range: 51%-96%). For the ordinal composite, only heart failure had low rankability (46% in 2008) (range: 46%-95%). Conclusion Combining multiple years or into multiple indicators results in more reliable ranking of hospitals, particularly compared with mortality and acute readmission in single years, thereby improving the ability to detect true hospital differences. The composite measures provide more information and more reliable rankings than combining multiple years of individual indicators. ... Read more

Background: Ecological fallacy refers to an erroneous inference about individuals on the basis of findings for the group to which those individuals belong. Suppose analysis of a large database shows that hospitals with a high proportion of long length of stay (LOS) patients also have higher than average in-hospital mortality. This may prompt efforts to reduce mortality among patients with long LOS. But patients with long LOS may not be the ones at higher risk of death. It may be that hospitals with higher mortality (regardless of LOS) also have more long LOS patients-either because of quality problems on both counts or because of unaccounted differences in case mix. To provide more insight how the ecological fallacy influences the evaluation of hospital performance indicators, we assessed whether hospital-level associations between in-hospital mortality, readmission and long LOS reflect patient-level associations. Methods: Patient admissions from the Dutch National Medical Registration (2007-2012) for specific diseases (stroke, colorectal carcinoma, heart failure, acute myocardial infarction and hip/knee replacements in patients with osteoarthritis) were analysed, as well as all admissions. Logistic regression analysis was used to assess patient-level associations. Pearson correlation coefficients were used to quantify hospital-level associations. Results: Overall, we observed 2.2% in-hospital mortality, 8.1% readmissions and a mean LOS of 5.9 days among 8 478 884 admissions in 95 hospitals. Of the 10 disease-specific associations tested, 2 were reversed at hospital-level, 3 were consistent and 5 were only significant at either hospital-level or patient-level. A reversed association was found for stroke: patients with long LOS had 58% lower in-hospital mortality (OR 0.42 (95% CI 0.40 to 0.44)), whereas the hospital-level association was reversed (r=0.30, p0.01). Similar negative patient-level associations were found for each hospital, but LOS varied across hospitals, thereby resulting in a positive hospital-level association. A similar effect was found for long LOS and readmission in patients with heart failure. Conclusions: Hospital-level associations did not reflect the same patient-level associations in 7 of 10 associations, and were even reversed in 2 associations. Ecological fallacy thus potentially influences interpretation of hospital performance when patient-level associations are not taken into account. ... Read more

Conclusions: Although readmission rates are a promising quality indicator, several methodological concerns identified in this study need to be addressed, especially when the indicator is intended for accountability or pay for performance. We recommend investing resources in accurate data registration, improved indicator description, and bundling outcome measures to provide a more complete picture of hospital care.Introduction: Hospital readmission rates are increasingly used for both quality improvement and cost control. However, the validity of readmission rates as a measure of quality of hospital care is not evident. We aimed to give an overview of the different methodological aspects in the definition and measurement of readmission rates that need to be considered when interpreting readmission rates as a reflection of quality of care.Methods: We conducted a systematic literature review, using the bibliographic databases Embase, Medline OvidSP, Web-of- Science, Cochrane central and PubMed for the period of January 2001 to May 2013.Results: The search resulted in 102 included papers. We found that definition of the context in which readmissions are used as a quality indicator is crucial. This context includes the patient group and the specific aspects of care of which the quality is aimed to be assessed. Methodological flaws like unreliable data and insufficient case-mix correction may confound the comparison of readmission rates between hospitals. Another problem occurs when the basic distinction between planned and unplanned readmissions cannot be made. Finally, the multi-faceted nature of quality of care and the correlation between readmissions and other outcomes limit the indicator's validity. ... Read more