Methods sway hospital performance rankings
Hospital readmission rates may provide a yard stick for measuring quality of care, but small permutations in the methods used to calculate those rates can make a big difference in rankings, according to an analysis published Oct. 9 in the Canadian Medical Association Journal.
Hospital readmission rates often are used to measure and compare hospital performance, noted Carl van Walraven, MD, a senior scientist at Ottawa Hospital Research Institute in Canada, and colleagues. But the methods chosen to determine those rates and the patient populations included in the measures vary greatly country by country. Given the lack of consensus on the best approach, they decided to test four different methods to determine their reliability for predicting early death or readmissions based on administrative data.
They pulled discharge data on patients admitted to acute care hospitals in Ontario between 2005 and 2010 as well as death records and records from all emergency departments in the province. They determined whether patients died or were urgently readmitted within 30 days of discharge from the original hospital. With that data, they calculated hospital rates of death or unplanned readmissions within 30 days patient group and admission factors:
Van Walraven and colleagues combined the two patient groups and the two admission factors to create four different measures. The measures were used to calculate rankings based on the ratio of the number of observed events at each hospital over the number of expected events for all hospitals.
They found disagreement among the four measures. Results showed hospital ranking varied greatly and even small changes in the method used to calculate the readmission rates could influence the rankings.
“Our results highlight the caution required when comparing hospital performance using rates of death or urgent readmission within 30 days,” they wrote, adding that the best method for calculating death after discharge or readmissions remained unclear.
“We believe that measures best able to adjust for potential confounders using validated prediction models are preferable. Such measures could use internally developed or previously published models, such as LACE+,” van Walraven and colleagues suggested. “Regardless, the model’s accuracy should be clearly stated to inform readers of its effectiveness for leveling the playing field among hospitals.”
They noted that their study was limited to Ontario and the results might not apply to other regions or to other patient populations.
Hospital readmission rates often are used to measure and compare hospital performance, noted Carl van Walraven, MD, a senior scientist at Ottawa Hospital Research Institute in Canada, and colleagues. But the methods chosen to determine those rates and the patient populations included in the measures vary greatly country by country. Given the lack of consensus on the best approach, they decided to test four different methods to determine their reliability for predicting early death or readmissions based on administrative data.
They pulled discharge data on patients admitted to acute care hospitals in Ontario between 2005 and 2010 as well as death records and records from all emergency departments in the province. They determined whether patients died or were urgently readmitted within 30 days of discharge from the original hospital. With that data, they calculated hospital rates of death or unplanned readmissions within 30 days patient group and admission factors:
- One patient group factor adjusting for age and sex only;
- One patient group factor adjusting for factors in the LACE+ index (length of stay, acute admission, comorbidities and emergency department visits in the previous six months);
- All admissions, where multiple admissions from a single patient would be used; and
- Single admissions, where one admission per patient was selected randomly.
Van Walraven and colleagues combined the two patient groups and the two admission factors to create four different measures. The measures were used to calculate rankings based on the ratio of the number of observed events at each hospital over the number of expected events for all hospitals.
They found disagreement among the four measures. Results showed hospital ranking varied greatly and even small changes in the method used to calculate the readmission rates could influence the rankings.
“Our results highlight the caution required when comparing hospital performance using rates of death or urgent readmission within 30 days,” they wrote, adding that the best method for calculating death after discharge or readmissions remained unclear.
“We believe that measures best able to adjust for potential confounders using validated prediction models are preferable. Such measures could use internally developed or previously published models, such as LACE+,” van Walraven and colleagues suggested. “Regardless, the model’s accuracy should be clearly stated to inform readers of its effectiveness for leveling the playing field among hospitals.”
They noted that their study was limited to Ontario and the results might not apply to other regions or to other patient populations.