Care Transitions Measure Score and Coronary Revascularization Related Readmission: Ready for Primetime Use?

The publication in 2001 of the Institute of Medicine report Crossing the Quality Chasm: A New Health System for the 21st Century focused national attention on the importance of transitional care in enabling safe and efficacious continuity of health care across various settings.¹ As part of this effort to ensure quality care, federally sponsored initiatives such as the Hospital Consumer Assessment of Health Plan Survey (HCAHPS) have become important tools in modern healthcare for establishing performance standards for hospitals.² In 2005, Eric Coleman and colleagues added another measure, the 15-item Care Transitions Measure (CTM), to specifically assess the quality of patients’ care transition experiences.³

Subsequently, in consultation with the National Quality Forum, the CTM-3 was developed as the care transition measure to be used nationally for public reporting. The CTM-3 is a shorter three-item score that retains the major domains of the full CTM questionnaire while still explaining the majority of its variance.^4,5 The original validation cohort of the full CTM consisted of 200 patients (with primary diagnoses of chronic obstructive pulmonary disease, congestive heart failure, stroke or hip fracture) in an integrated health system in the Pacific Northwest.³ Lower CTM scores were associated with recidivism to the emergency room and hospital readmission. In a subsequent validation study with 192 patients aged ≥ 65 years with diabetes or congestive heart failure, CTM-3 scores independently predicted emergency room recidivism at 30 days.⁴

Given the initial promise of the CTM-3, this measure was integrated into the HCAHPS survey in 2013 as the ninth HCAHPS dimension and has been reported on Medicare’s Hospital Compare website since 2014. Beginning in fiscal year 2018, the CTM-3 will be incorporated into the Hospital Value-Based Purchasing Program (VBP). In this role, it will contribute (as part of the HCAHPS domain) to 25 % of the Hospital VBP Total Performance and will impact hospitals’ Value-Based Incentive Payment. As a result, the CTM-3 is poised to assume special significance, as it will soon start impacting hospital reimbursement.⁶ However, before this happens, it seems prudent to take a step back, and consider the strengths, limitations, and the role of this measure in predicting hospital readmissions across a wide range of disease states.

In this issue of JGIM, Goldstein et al. analyze the association of CTM-3 scores with 30-day hospital readmissions in patients undergoing percutaneous coronary intervention (PCI) or coronary artery bypass surgery (CABG) at the Christiana Care Health system between April 2013 and November 2014.⁷ The data were obtained from a quality improvement initiative (the “Bridges”) within the Christiana Care Health system in Delaware, aimed at improving the care transition experience for patients with ischemic heart disease. As part of the Bridges program, 2963 patients with ischemic heart disease (treated with either PCI, CABG or medical therapy) received systematic pre-discharge and post-discharge interventions along with individualized transitional education. CTM-3 items were administered post-discharge in the standard prescribed fashion, along with program-specific questions. A total of 1594 patients (54 %) completed the CTM-3 surveys and were included in the final analyses. Using a non-parsimonious multivariable adjustment model, the key finding was that for every 10-point increase in the CTM-3 score (change in a response of one out of three CTM-3 questions to the next highest response category), there was a 14 % reduction in the odds of a 30-day all-cause readmission.⁷ This finding provides support for the role of the CTM-3 measure in predicting short-term hospital readmissions among patients receiving coronary revascularization, and its potential use in directing payments to high quality hospitals.

However, there are several additional notable findings in Goldstein and colleagues’ paper that raise questions about the validity of the CTM-3 score in this primarily older patient population. One key observation was that several non-modifiable patient level factors, including multiple comorbidities, were independently associated with both CTM-3 scores and hospital readmissions. This finding suggests that the relationship between CTM-3 and hospital readmission is confounded by patient characteristics, and that the CTM-3 does not solely convey the quality of transitional care. If so, with the impending changes to hospital reimbursement practices, it seems possible that hospitals caring for sicker patients could potentially face monetary penalization, despite excellent transitional care programs. This finding suggests that the CTM-3 needs to be risk adjusted.

A second observation is that the CTM-3 did not work equally well in all patient populations. While the CTM-3 was associated with readmissions in the overall study population, and specifically in patients undergoing a PCI, the same association was not demonstrated in patients undergoing CABG. If this finding is upheld in other studies, it would suggest that the CTM-3 may not be an appropriate quality measure in patients hospitalized for CABG. As such, utilizing it to modify hospital reimbursement could unfairly penalize hospitals with a large volume of patients undergoing CABG, irrespective of the quality of transitional care.

Given these two important findings and the potential consequences of broad implementation of CTM-3, it seems appropriate that we pause and consider the unanswered questions that need to be studied before we can be confident in the validity of the CTM-3 score. First, does the CTM-3 have predictive properties across all populations in which it will be used? Second, can hospitals actually change their CTM-3 scores through quality improvement projects, or is the CTM-3 only reflective of patient mix? Third, if hospitals improve their CTM-3 scores, will they observe a reduction in readmissions? Fourth, what is the minimum CTM-3 score improvement required to realize clinically and economically significant reductions in readmissions?

The results of a recent cluster randomized controlled trial provide some answers to these questions, but unfortunately, the results are not encouraging for the continued use of the CTM-3. Researchers at the Oregon Health and Science University prospectively evaluated the impact of a multi-component transitional care improvement program in 382 low-income adults and measured the effect on CTM-3 scores and 30-day hospital and emergency room readmissions. The study found that the transitional care program improved CTM-3 scores without a comparable improvement in hospital and emergency room recidivism.⁸ To our knowledge, this is the only published study where causal inference about changes in CTM-3 scores and hospital readmissions were examined, and it disappointingly showed that improvements in CTM-3 scores did not translate into reduced rates of recidivism. Inasmuch, it seems prudent to reconsider the broad implementation of the CTM-3 outcome measure.

While the results of the present study in patients undergoing coronary revascularization are important, it is not without its limitations. Key limitations include a single-center study population with a modest response rate; however, the response rate achieved in this study is several fold higher than the standard HCAHPS response rate. Additionally, there were some important differences between responders and non-responders, in that non-responders had more comorbidities, were older (more Medicare patients), and had higher rates of readmission. However, the authors make a valid argument that the CTM-3 is probably more predictive among non-responders because they were observed to have higher rates of readmission. Additionally, there remains potential for misclassification of outcomes because readmissions at other hospitals were not captured, and readmission rates were overall lower than expected. Another consideration is that the study population was derived from within a quality improvement cohort. While it is likely that all patients in the cohort received a heightened level of transitional care, patients at higher risk for readmission may have received even more transitional care than those assessed to be at low risk. These limitations potentially alter our understanding of the relationship between CTM-3 scores and hospital readmission.

On the other hand, this study has several notable strengths. First, this is the first contemporary study demonstrating an association between CTM-3 and hospital readmissions among patients with ischemic heart disease. Second, compared with the original validation studies, the study population used by Goldstein et al. consisted of a geographically distinct, demographically diverse patient population that received care at an academic medical center for ischemic heart disease. Third, given its large sample size (five times that of the original validation studies), the study had adequate power to detect meaningful relationships between improvement in CTM-3 scores and hospital readmission.

In conclusion, although the results of Goldstein et al. validate the role of CTM-3 in predicting recidivism in patients undergoing coronary revascularization, they also suggest that there is substantial work yet to be done with the CTM-3 before it is ready for prime time use. Given the planned implementation schedule of the CTM-3, the need for collecting additional data is urgent, particularly given that a recent randomized controlled trial did not establish causality between improved CTM-3 scores and reduced readmission.⁸ Studies remain needed in a broad array of specific populations, with disease-specific transitional care interventions capable of modifying CTM-3 scores (process measure) and effecting changes in readmissions (outcome). Premature adoption could cause hospitals and health systems to initiate burdensome quality improvement initiatives aimed at improving CTM-3 scores (thereby securing value-based incentive payments), without clear evidence that doing so actually improves quality and reduces short-term readmissions.