Opening Pandora’s Box

Background:

Measuring the effectiveness of transitional care interventions has historically relied on health care utilization as the primary outcome. Although the Care Transitions Measure was the first outcome measure specifically developed for transitional care, its applicability beyond the hospital-to-home transition is limited. There is a need for patient-centered outcome measures (PCOMs) to be developed for transitional care settings (ie, TC-PCOMs) to ensure that outcomes are both meaningful to patients and relevant to the particular care transition. The overall objective of this paper is to describe the opportunities and challenges of integrating TC-PCOMs into research and practice.

Methods and Results:

This narrative review was conducted by members of the Patient-Centered Outcomes Research Institute (PCORI) Transitional Care Evidence to Action Network. We define TC-PCOMs as outcomes that matter to patients because they account for their individual experiences, concerns, preferences, needs, and values during the transition period. The cardinal features of TC-PCOMs should be that they are developed following direct input from patients and stakeholders and reflect their lived experience during the transition in question. Although few TC-PCOMs are currently available, existing patient-reported outcome measures could be adapted to become TC-PCOMs if they incorporated input from patients and stakeholders and are validated for the relevant care transition.

Conclusion:

Establishing validated TC-PCOMs is crucial for measuring the responsiveness of transitional care interventions and optimizing care that is meaningful to patients.

TRANSITIONAL CARE—LOOKING BEYOND READMISSIONS

Transitional care has been defined as a set of actions designed to ensure the coordination and continuity of health care as patients transfer between different settings or levels of care.1 Many of the early intervention studies designed to improve transitional care used health care utilization, most commonly hospital readmissions, as a primary outcome measure.2–4 This choice was based on several distinct but interrelated premises, including the notion that readmissions are an important marker of both quality and efficiency of health care, that unnecessary readmissions can and should be prevented, that preventing readmissions saves costs, and that readmissions can be readily defined, measured, and monitored. Importantly, because readmissions are regarded as directly reflecting the quality of care that a patient receives during and immediately after the in-hospital period, readmissions became an appropriate target for interventions designed to improve patient outcomes during the hospital-to-home transition.5,6 From this perspective, value-based purchasing programs including the Centers for Medicare and Medicaid Services (CMS) Hospital Readmission Reduction Program (HRRP),7 were developed with the expressed aim of reducing hospital readmissions. Specifically, the HRRP uses risk-adjusted 30-day readmission rates as performance measures to incentivize hospitals to reduce readmissions, but the degree to which these data reflect differences in the quality of care remains unclear.

The limitations of using health care utilization measures such as readmissions to define the success (or failure) of transitional care interventions have become more apparent.8 First, after early successes of transitional care-based interventions in reducing readmissions,2,4 the results of more recent intervention studies conducted across a wider range of clinical conditions have been more mixed.9–11 Systematic reviews of randomized trials examining a broad array of transitional care interventions, whether applied before hospital discharge (eg, discharge planning; medication reconciliation; patient education) or after [eg, enhanced follow-up procedures (phone calls, clinic visits, home visits); navigators; enhanced self-management programs], confirm that intervention effects are variable and mostly modest in size.5,12–16 A systematic review of clinical trials published between 1990 and 2013 that tested the effects of enhanced discharge procedures on reducing readmissions found that treatment effects were smaller in studies published after 2002.13 Second, a combination of factors makes it difficult for any single intervention strategy or program to achieve meaningful reductions in readmissions. The reliance on all-cause readmissions means that only a minority of readmissions are likely avoidable.17 Furthermore, variation in 30-day readmission rates between-hospitals is typically small, as is the amount of variation explained by differences in hospital care processes.18 These observations suggest that factors outside of the direct control of the hospital (such as patient and community characteristics) are much more influential on observed readmission rates.19 Finally, it would be difficult for any single health care utilization measure, including readmissions, to adequately reflect the wide range of patient experiences during a care transition, which extend beyond immediate medical concerns to include practical, economic, psychosocial, and mental health.20–23

The limitations of using health care utilization to measure the effectiveness of transitional care interventions and programs has led investigators, patients, and other stakeholders to consider additional outcome measures grounded in a patient-centered framework that are relevant to different care transitions. In this report, a patient-centered outcome measure (PCOM) refers to a measure that directly quantifies the impact of disease or treatment on a health outcome that has been identified as being important to patients in a specific context or setting.24

Our overall objective was to describe the opportunities and challenges of integrating PCOMs into transitional care research and practice. Specifically, we conducted a narrative review to: (1) describe conceptual frameworks relevant to transitional care outcomes measurement, and compare and contrast the use of patient-reported outcomes measures (PROMs) and PCOMs; (2) identify PCOMs developed to measure care transitions (ie, TC-PCOMs) and discuss the importance of contextual factors and process measures; (3) discuss practical aspects of TC-PCOM data collection, and (4) suggest future research priorities.

METHODS

The authors of this narrative review were funded by the Patient-Centered Outcomes Research Institute (PCORI) to study transitional care and are active members of PCORI’s transitional care interest group, the Transitional Care Evidence to Action Network (TC-E2AN) (www.pcori.org/topics/transitional-care/network). Through a combination of in-person meetings, webinars, and presentations, the group identified measurement issues and challenges relevant to care transitions. Three areas in need of further exploration emerged: (1) conceptual frameworks; (2) use of existing PCOMs; and (3) practical aspects of PCOM data collection. We expounded each area by conducting targeted literature reviews. Specifically, we searched Medline for relevant articles using combinations of Medical Subject Heading (MeSH) terms (transitional care; care transitions; patient reported outcome measures; patient centered care; reproducibility of results; assessments, outcome health care; psychometrics) and text words (“continuity of care”; “intermediate care”; “patient centered outcomes”). We prioritized systematic reviews and meta-analyses that addressed transitional care and patient outcome measures. We screened the bibliographies of the newly identified relevant publications and used targeted forward searches to identify papers that had cited particularly relevant manuscripts. Our narrative synthesis helped identify future research priorities.

FINDINGS

Conceptual Frameworks for Transitional Care Outcomes Measurement and Distinction Between Patient-centered Outcome Measures and Patient-reported Outcomes Measures

Conceptual frameworks are essential for guiding the development of theories, concepts, and definitions, as well as to define organizational and measurement frameworks. However, we did not find any publications that defined a conceptual framework or model for the measurement of patient outcomes specific to the transitional care setting. Some studies undertook a systematic approach when developing patient-centered measures relevant to care transitions,25,26 while others defined essential components of transitional care,27 characteristics of transitional care models,28 validated measurement tools for patient safety,29 or included a concept analysis of transitional care.30 The complexities of measurement in transitional care research and practice are implicit in the multifaceted nature of the interventions; a report by Naylor et al,27 which identifies 8 essential components of transitional care (Fig. 1), helps illustrate the challenges in developing a comprehensive measurement framework. Although not all 8 components would be relevant to every intervention or setting, each is a complex multidimensional construct that would require a separate set of items (or tools) to accurately quantify. This is further complicated by the fact that the specific nature of transitional care interventions can vary widely within each construct (eg, caregiver engagement, care continuity) which further challenges the selection of outcome measures to assess efficacy. Examination of the transitional care studies funded by PCORI (www.pcori.org/topics/transitional-care) highlights the diversity of measures that have been used as primary outcomes, which include utilization measures (most commonly readmissions), and patient-reported measures of function, health-related quality of life (HR-QOL), satisfaction, experience with care, and activation. Consensus on what outcomes should be measured when quantifying the effect of transitional care interventions is clearly lacking.

FIGURE 1.

Eight core components of comprehensive transitional care as defined by Naylor et al.27

Distinction Between Patient-centered Outcome Measures and Patient-reported Outcomes Measures

The focus of this paper is PCOMs, which are different but related to PROMs. A PROM has been defined as a measure that represents “the status of the patient’s health that comes directly from the patient without interpretation by the clinician or anyone else.”31,32 A PROM reflects how a patient interprets an experience or condition which cannot by definition be observed by others.33 The scope of what constitutes a PROM varies widely and includes self-reported measures of functional status, symptom burden, and HR-QOL or well-being.31,33 PROMs also include measures of patient experience including satisfaction with services.34 The term PROM rose to prominence when the Food and Drug Administration (FDA) used it to define clinically relevant patient outcomes important to receiving regulatory approval of medications and devices.32 Although developed to address product labeling claims by industry, the FDA proposed a comprehensive framework for defining and using PROMs. It emphasized the importance of using a conceptual measurement framework when choosing or developing a PROM which included the assessment of reliability and validity, as well as defining the relevant patient population, disease endpoints, and data collection methods.32 This original measurement framework has now matured to include a broader array of PROM-related definitions as specified in the FDA Roadmap project.35 The Roadmap provides a framework to identify, define, and select (or develop) patient-focused outcome measures that are relevant to the trials’ goals.

It is important to note that absent from the original FDA guidance was any mention of how patients should be involved in the definition (or selection) of the PROM.32 Indeed, Rothman et al33 noted that most PROM measurement models reflected the judgement of professionals regarding what is important to patients, rather than seeking this information from patients themselves. Thus, if a PROM measure was developed without direct input from patients or stakeholders then it does not meet the definition of being patient-centered. Extending this to the selection of measures for transitional care, even if a PROM was originally developed following patient input, if this measure has not been adapted after seeking input from patients and stakeholders who have experienced the specific transition of interest, followed by testing in the same transitional care setting, then it should not be assumed to be valid for use. This is particularly true of transitional care measures given the wide range of challenges and needs that are unique to the individual patient experience during transitions.20,21,23 We believe that the use of PROMs in transitional care studies that have not undergone a process of adaption and validation in the relevant setting is a major limitation of many of the existing transitional care intervention studies, and likely contributes to the variation in their effectiveness.

To make the distinction between outcome measures that have been adapted and tested in the context of transitional care and those that have not, we introduce the concept of PCOMs developed for transitional care—referred to as TC-PCOMs. Building off definitions of patient-centered care used by the Institute of Medicine, PCORI, and others,24,36 we define a TC-PCOM as an outcome that matters to patients because it accounts for their individual experiences, concerns, preferences, needs, and values during the transition period. All TC-PCOMs should be developed following direct input from representative samples of patients and stakeholders who have experienced the transition, in a process that applies rigorous mixed methods research and engagement practices.24,31 Because TC-PCOMs are developed directly from patient and stakeholder input and reflect how patients feel, function, and live during the transition, they are “fit-for-purpose,”24 and are therefore more likely to demonstrate the clinical effectiveness of transitional care interventions.

Our desire to develop a broad array of TC-PCOMs creates a potential problem; the confusing definitions of PROMs and PCOMs, along with uncertainties regarding their validity and applicability to care transitions, opens a Pandora’s box of measure delineation (Fig. 2) that runs the risk of greatly increasing complexity. Our conceptualization depicts how it is possible for a particular measure to be a PCOM only, a PROM only, both a PCOM and PROM, or neither (Fig. 2). However, we draw an important distinction between PCOMs and PROMs in that despite their degree of overlap the 2 terms they are not synonymous or necessarily interchangeable.

FIGURE 2.

The Pandora’s box of PCOMs illustrating the overlap between PCOM and PROM. *Utilization measures can be regarded as a PCO if identified by patients as being important to them, else they may be neither PCO or PRO. PCOM indicates patient-centered outcome measure; PROM, patient-reported outcomes measure.

Transitional Care Patient-Centered Outcome Measures Designed to Measure Quality of Care Transitions

It is well recognized that patients and caregivers have individual transitional care experiences and needs that cannot be addressed with a universal or single transitional care service or intervention, even among patients with the same condition. The wide range of transitional care intervention components used to address patient needs include care coordination, continuity, and care quality, and are often setting-specific, for example, hospital, primary care. Although these dimensions are interrelated, they are difficult to assess across the full continuum of transitional care as experienced by a patient and their family. Likewise, many different outcomes, including HR-QOL, depression, functional status, and social support, have been used in transitional care studies but often relate only to specific intervention components, and most instruments were not developed and validated for transitional care. As outcomes, they do not meet the definition of a validated TC-PCOM.

Our review identified 2 instruments designed to measure the quality of care transitions: the Care Transition Measure (CTM) and the Partners at Care Transitions Measure (PACT-M). Both instruments utilized robust patient-centered methodology for development and validation within the context of a specific care transition, thus strengthening their content validity and meeting the definition of a TC-PCOM. Table 1 summarizes the initial validation work and psychometric testing conducted on each instrument. CTM-15,39 a unidimensional TC-PCOM administered 6–12 weeks posthospital discharge, is the most widely used. Its 15 items cover information transfer, patient and caregiver preparation, support for self-management, and empowerment to assert preferences. The CTM-15 has been shown to discriminate between patients who have a subsequent emergency department visit or rehospitalization.25 A 3-item version, which explains 90% of the variance of the CTM-15 measure and demonstrates a similar ability to detect group differences,37 is used in the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Hospital Survey. Although both CTM versions showed good validity when initially developed, a recent study reported limitations including acquiescence bias and limited score variability.40 Also, because the CTM was developed in the United States to measure how well hospitals provide care to patients returning to the community, it does not apply to all care transitions and may not be generalizable to other countries without further validation (as conducted, eg, in Japan41 and Sweden42). Finally, despite the popularity of the CTM, there is limited information on its relationship to other patient outcomes, its performance among different patient populations, and its ability to detect intervention effects. Thus, researchers have called for the development of measures that reflect broader patient-centered domains. The PACT-M26 is an example of a new TC-PCOM. The PACT-M measures 8 components of care transitions, including patient experience, quality, safety, and adverse events or problems associated with managing care at home (Table 1). There are 2 versions depending on when data are collected; PACT-M1 measures preparation for the care transition and is administered immediately postdischarge, while PACT-M2 measures the experience of managing care and health at home and is administered 1 and 3 months after discharge. Initial validation conducted among elderly patients discharged from hospital to home in the UK showed high internal validity.38

TABLE 1.

Development and Validation of TC-PCOMs: CTM, PACT-M

Instrument	Publication Year and Location	Instrument Structure	Validation Study: Setting; Assessment Timepoint; Collection Method	Validation Study: Population	Themes From Factor Analysis	Internal Reliability (Cronbach α)	Strengths	Limitations
CTM-1525	2005, United States	15-item unidimensional	Hospital to home or skilled nursing facility; 6–12 wk postdischarge; data collected via phone	N=200, mean age: 67 y (18–90 y) from 3 hospitals representing COPD and congestive heart failure	(1) Critical understanding (2) Preferences important (3) Management preparation (4) Care plan	0.93	Valid and reliable TC-PCOM instrument Patient-centered development process Multithematic	Focused on immediate discharge period Excludes TC safety themes
CTM-337	2008, United States	3-item	Hospital to home or skilled nursing facility; within 6 mo postdischarge; data collected via phone	N=225, underserved population (minority or rural-dwelling), mean age: 67 y (18–90 y) recruited from national sample	See CTM-15	0.93–0.96	See CTM-15 CTM-3 implemented in CAHPS	See CTM-15
PACT-M126	2019, UK	9-item unidimensional	Hospital to home; 1 wk postdischarge; data collected via phone and mail	N=138, mean age: 79 y (65–95 y) from a single-teaching hospital primarily cardiovascular, women’s center, and surgical emergency wards	(1) Patient involvement (2) Medication management (3) Discharge arrangements (4) Coordination with other providers (5) Providing information and guidance to patient/family (6) Anticipation and preparation for emergencies/deterioration (7) Feeling safe	0.84	Valid and reliable TC-PCOM instrument Patient-centered development process Multithematic Addresses multiple TC timepoints Includes 7 items for adverse events/problems (1. nonhealing sores, 2. infection, 3. falls, 4. unable to schedule appointments, 5. medication problems, 6. delay receiving health care supplies, 7. ED or hospital readmission)	New measure Potentially limited generalizability—validation study population was White, cognitively intact, and most living with a spouse/partner
PACT-M238	2020, UK	8-item unidimensional	Hospital to home: 1 and 3 mo postdischarge; phone and mail	N=110—similar population characteristics as PACT-M1	See PACT-M1	0.92	See PACT-M1	See PACT-M1

CAHPS indicates Consumer Assessment of Healthcare Providers and Systems; COPD, chronic obstructive pulmonary disease; CTM, Care Transitions Measure; ED, emergency department; PACT-M, Partners at Care Transitions Measure; TC, transitional care; TC-PCOM, Transitional Care Patient-Centered Outcome Measure.

Understanding the Importance of Context and Process Measures in Care Transitions

Developing new TC-PCOMs that measure the quality of transitional care experiences, patient outcomes, and intervention effects is challenging due to the large number of contextual factors associated with care transitions that need to be considered. Relevant contextual factors can include the specifics of the target population, transition setting, and patient-specific biopsychosocial needs, goals, preferences, and capacities. Comparatively little is known about the patient and caregiver experience during care transitions, the services they need, or the outcomes they value.21,43 Recent work on care transitions involving patient and caregiver focus groups identified an array of biopsychosocial needs, including limited financial resources, education, and social support, that require additional services not covered by typical medical care delivery models.44 Moreover, findings from a recent large qualitative study of patients and stakeholders who experienced transitions,21 as well as a review of the transition experience of underserved populations,20 serve to illustrate that the typical components of transitional care—discharge planning, individualized care, effective communication, and coordination of care—are unlikely to capture the full-spectrum of patient-defined transitional care experiences and desired outcomes. Importantly, themes such as therapeutic relationships (ie, trust), social fragility, access failures, and the need for unambiguous responsibility and accountability on behalf of providers, are absent in the CTM and PACT-M instruments. The different needs of patients recovering from a new acute illness and exacerbation of chronic disease are also not covered by these existing measures.

In addition to contextual factors, collecting robust process measures relevant to transitional care is also critical for advancing our understanding of the underlying mechanisms of transitional care processes, and to develop more contextually appropriate TC-PCOMs. Process measures include the specific details of the intervention components, service delivery, workforce, and uptake by the target population. Such measures help determine what TC-PCOMs should be measured for whom and at what time during the transition experience.

Transitional Care Patient-centered Outcome Measure Data Collection and the Critical Role of Stakeholders

Different types of data require different approaches to data collection. Typically, a standard method is established when an instrument or a measure is developed, however, the study question, design, and the population of study influence this standard and can introduce new limitations. For example, the cost and timeliness of using claims data to measure health care utilization leads some study teams to consider self-reported data instead. The accuracy of patient self-report to recall health care encounters varies by many factors including timing and specificity of event details.45 To overcome these challenges, alternative approaches to data collection should be considered for PCOMs (ie, computer-assisted survey instruments, short-message service (SMS/text) responses, and observations of patient behavior). Development of alternative approaches to collect PCOMs are still in their infancy. More studies are needed to evaluate the influence of newer modes of administration (ie, online or device based) on data reliability and usability.

Transitional care studies have included surrogate or proxy respondents to minimize the risk of selection bias in patient samples and missing data. Although common for outcomes associated with patient function or activity limitations, research suggests that proxies report greater levels of impairment than patients, and the value of proxy reports for more subjective measures, such as quality of life and depressive symptoms, remains unclear.46 The use of proxies and alternative approaches to data collection can reduce selection bias and missing data but can also introduce measurement error. Researchers can learn from pediatrics where the use of proxies for PCOM is well-established,47 to consider the advantages and disadvantages of the different approaches to data collection and sources of data. When >1 approach to data collection is used, the team should conduct sensitivity analyses to understand the impact on study findings.

To be truly patient-centered, the process of developing and choosing TC-PCOMs must involve all relevant stakeholders who will be either administering or receiving the PCOMs. The POWER-tool (Patient participation in Outcome measure WEighing for Rare diseases) model provides a process to choose clinical outcomes and measurement instruments during the design stage of a clinical trial based on the input and perspective of patient and caregiver stakeholders.48 The POWER-tool approach has the advantage that stakeholder and researcher interactions can be customized to the population of interest. This approach needs to be applied and tested in transitional care research.

Any selection process must always balance the availability of validated PCOMs that address the domains of interest and the practicalities of administration and delivery. A long PCOM tool that will take many minutes to administer will be unrealistic for use and burdensome for patients. The PCORI-funded study, ST3P-UP, that evaluated the effectiveness of structured education to improve transition outcomes among young adults with sickle cell disease (SCD), involved stakeholder input. They opted to use the PedsQL-SCD module because it was a well-established and validated quality of life measure in this population of interest. The investigators also chose the Medical Outcomes Study Social Support Scale,49 despite that it was not validated in SCD, because it had wide applicability in the populations experiencing chronic illness, is simple to administer, and covered the domains of interest for the project. Before a final decision on which TC-PCOMs to use, all available measures and tools were reviewed, discussed, and vetted for applicability and practicality by individuals living with SCD, as well as members of the study’s community advisory board. It was important to find a balance between stakeholder input and instrument validity. While stakeholder involvement in research processes is not the easiest or quickest approach, engagement can have many advantages.50 Stakeholders involved in the study development phase can improve long-term collaboration, highlight the limitations of alternative approaches, support reliability and validity assessments before study implementation, and increase the generalizability and patient-centeredness of chosen PCOMs.

Future Research Directions for Transitional Care Patient-centered Outcome Measures

TC-PCOMs are crucial to measuring and addressing transitional care experiences. The availability of validated instruments is currently limited, and measure development is challenging. Additional research is warranted in several areas. Although several transitional care models exist, a comprehensive consensus-based TC-PCOM measurement framework is needed. This could involve the evolution of previous transitional care or care coordination frameworks following stakeholder engagement and input. As a practical matter, the field would benefit from developing generic TC-PCOMs applicable to most transition settings and populations. An ideal TC-PCOM tool set could include a menu of items each designed to measure the specific components common to transitional care (eg, care coordination, patient engagement),27 as well as a menu of TC-PCOMs shown to be relevant and valid for most transitions (eg, patient function, satisfaction, experience).

To achieve this, continued investment is needed to understand the complexity of patient and stakeholder needs and preferences. The development of valid TC-PCOMs holds great potential to identify interventions that are better tailored to the needs and experiences of patients and caregivers. In doing so, transitional care could be better optimized in this new era of value-based care.