The study protocol to evaluate implementation of the transitional care model in four U.S. healthcare systems during the Covid-19 pandemic

Abstract

This study protocol describes the conceptual framework, design, and methods being employed to evaluate the implementation of the Transitional Care Model (TCM) as part of a randomized controlled trial. The trial, designed to examine the health and cost outcomes of at-risk hospitalized older adults, is being conducted in the context of the COVID-19 pandemic. This parallel study is guided by the Practical, Robust, Implementation and Sustainability Model (PRISM) and uses a fixed, mixed methods convergent parallel design to identify challenges encountered by participating hospitals and post-acute and community-based providers that impact the implementation of the TCM with fidelity, strategies implemented to address those challenges and the relationships between challenges, strategies, and rates of fidelity to TCM's core components over time. Prior to the study's launch and throughout its implementation, qualitative and quantitative data related to COVID and non-COVID challenges are being collected via surveys and meetings with healthcare system staff. Strategies implemented to address challenges and fidelity to TCM's core components are also being assessed. Analyses of quantitative (established metrics to evaluate TCM's core components) and qualitative data (barriers and facilitators to implementation) are being conducted independently. These datasets are then merged and interpreted together. General linear and mixed effects modeling using all merged data and patients’ socio-demographic and social determinants of health characteristics, will be used to examine relationships between key variables and fidelity rates. Implications of study findings in the context of COVID-19 and future research opportunities are suggested.

Trial registration: ClinicalTrials.gov Identifier: NCT04212962

1. Introduction

The outbreak of COVID-19 changed the healthcare landscape for older adults living with multiple chronic conditions (MCCs). Prior to the pandemic, these patients (compared to all Medicare beneficiaries 65 years and older) experienced significantly higher rates of hospitalizations as well as physician and emergency department visits (Anderson, 2010; Berenson & Horvat, 2002; Pham et al., 2009) and, in 2019, were responsible for a substantial proportion of the $782 billion spent for the Medicare program (MedPAC, 2021). Numerous studies revealed that poor management of the complex care needs of older adults with MCCs throughout vulnerable healthcare transitions was the norm, often with harmful human and economic consequences (Casado et al., 2011; Desai & Stevenson, 2012; Health Quality Ontario, 2017; Kansagara et al., 2011; Krumholz, 2013; Lloren et al., 2019; Naylor et al., 2011).

Throughout this public health crisis, the unprecedented response by healthcare systems and government agencies to stem the spread of the virus has further disrupted the delivery of transitional care (hospital-to-home) services to this at-risk patient group. Some older adults delayed seeking urgent care because of their fear of contracting the virus (Hsieh, 2020; Wong et al., 2020). Others who were hospitalized and required intensive post-discharge services were sent directly home, the result of some skilled nursing facilities’ (SNFs) refusals to admit them (Grabowski & Mor, 2020). Due to hospital visitation restrictions, many family caregivers were not adequately prepared for the critical roles they assumed once patients returned home (Naylor et al., 2020). Given the increasingly complex challenges experienced by older adults with MCCs during the pandemic, some healthcare systems have accelerated their use of evidence-based solutions. For example, the research-based Hospital at Home intervention was approved for use in the U.S. during this period to care for select Medicare populations who otherwise would have been hospitalized (Clarke et al., 2021). More commonly, however, massive staffing disruptions in post-acute and other healthcare facilities (Ochieng et al., 2022) likely had a negative impact on the use of evidence-based transitional care by hospitals and other providers.

1.1. Intervention

The Transitional Care Model (TCM) is an advanced practice registered nurse (APRN)-led, team-based, care management strategy demonstrated to improve the outcomes of older adults with complex health and social needs throughout transitions from hospital to home. A key role of the APRN is to foster communication across hospital, post-acute, and community-based providers. Many methods of communication—attending visits with patients, phone, secure text, etc.—are encouraged to exchange information about and on behalf of older adults. In multiple National Institutes of Health funded randomized controlled trials (RCTs) conducted by a multidisciplinary team of clinical scholars and health service researchers based at the University of Pennsylvania (Penn), the TCM has consistently been shown to enhance the care experience of this patient group and improve their health and quality of life, while decreasing total health care costs (Naylor et al., 1994; Naylor et al., 1999; Naylor et al., 2004; Naylor et al., 2014).

In February 2020, Arnold Ventures funded a study (Multisite Replication of a Randomized Controlled Trial - Transitional Care Model, hereafter referred to as MIRROR-TCM) designed to assess the outcomes achieved by replicating the TCM in geographically dispersed U.S. healthcare systems. In brief, the MIRROR-TCM study is a prospective, intent-to-treat, replication RCT designed to compare the health and cost outcomes demonstrated by at-risk older adults living in the community who are hospitalized with common medical conditions and then randomized to receive standard transitional care services from the admitting hospital and post-acute and community providers (control group) or the TCM (intervention group). MIRROR-TCM was launched with nine hospitals that are part of four diverse healthcare systems: Swedish Health Services in Washington (Swedish Health); Trinity Health-Michigan and IHA (Trinity Health), University of California San Francisco (UCSF) Health in California, and the Veterans Health Administration (VHA). The original plan was to enroll 1600 older adults admitted to the hospital from home with heart failure, chronic obstructive pulmonary disease, or pneumonia who have other specified health and social risks, are expected to return home following discharge, and consent to participate in this RCT. Penn team members, the architects of the TCM, are coordinating this replication effort in collaboration with partnering healthcare systems. Mathematica, Inc. (Mathematica) is independently evaluating the RCT outcomes. The protocol for MIRROR-TCM and ethical approvals are described elsewhere (Naylor et al., 2022).

Prior to the onset of COVID-19, several studies had shown that effective implementation and sustainability of evidence-based solutions are related to how well these interventions are integrated into local contexts (Leonard et al., 2017; Shelton et al., 2018; Wiltsey Stirman et al., 2015). Findings from a systematic review of 35 studies that examined contextual factors across diverse healthcare systems revealed that consideration of context is needed to enhance the design and delivery of quality improvement initiatives (Coles et al., 2020). Øvretveit defined context as factors that are not part of the intervention but are likely to influence its implementation and outcomes (Øvretveit, 2011). Examples of such factors include staffing, regulatory, and payment policies at national levels and organizational leadership, climate, and culture at local levels; these factors are relevant to all organizations, including health care.

In response to COVID-19, clinicians and staff in acute, post-acute, primary care and community-based settings who support older adults with MCCs throughout transitions from hospital to home have modified many aspects of care delivery. Thus, the launch of MIRROR-TCM presented a unique opportunity to advance knowledge regarding the contextual challenges associated with COVID-19 that impact the delivery of an evidence-based transitional care model. Due to this unprecedented situation, MIRROR-TCM team members designed and launched a parallel study to rigorously evaluate the implementation of the TCM in diverse U.S. healthcare systems during this pandemic. The Penn team hypothesized that local contextual factors associated with COVID-19 would pose new risks to effective integration of the TCM at participating sites and that timely, site-specific strategies would be essential to mitigate these risks and advance fidelity to the intervention's core components over time.

1.2. Objectives

The objectives of this study are to: 1) identify contextual challenges encountered by participating hospitals and local post-acute and community-based providers during COVID-19 that impacted the implementation of the TCM with fidelity and the strategies implemented in response to these challenges over time; and 2) to examine and interpret the relationships between identified challenges, strategies to address them and rates of fidelity to TCM's core components over time. When data collection is completed, the relationships between independent variables including selected system factors, patients’ socio-demographic and social determinants of health characteristics, intervention dose, and fidelity rates over time will be explored. This paper describes the protocol developed to evaluate the implementation of the TCM in four U.S. healthcare systems in the context of COVID-19. Specifically, the conceptual framework, design, and methodological approach being employed to achieve study aims and the rationale for their selection are presented.

2. Methods

2.1. Design

A fixed, mixed methods convergent parallel design [QUAL+QUANT→convergent interpretation (Creswell & Plano Clark, 2018)] is being employed to rigorously describe what is needed to move a diverse set of participating hospitals and their post-acute and community-based providers to maximal fidelity to core components of the TCM during the conduct of MIRROR-TCM and within the context of each participating site's responses to COVID-19.

2.2. Conceptual model

The Practical, Robust, Implementation and Sustainability Model (PRISM) was selected to guide this evaluation. PRISM centers on the identification of contextual factors that influence effective implementation of evidence-based interventions as well as their desired outcomes (McCreight et al., 2019). PRISM integrates key concepts from the Diffusion of Innovations Chronic Care Model and Institute for Health Care Improvement frameworks (Feldstein & Glasgow, 2008). PRISM also extends the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework to better explain the influence of context (McCreight et al., 2019). Assessment of key factors in the external environment (e.g., policy changes made in response to COVID-19) and internal environment (e.g., healthcare delivery modifications made in response to COVID-19 and healthcare teams’ responses to these changes) is central to PRISM. Thus, PRISM was selected to enable timely responses to ongoing contextual changes with the goal of implementing the TCM as designed. Fig. 1 depicts how PRISM is guiding assessment of key contextual factors, strategies to address these factors, and their effects on fidelity to TCM's core components. Based on this framework, the intervention (TCM) influences and is influenced by internal implementation and sustainability factors (e.g., changes in delivery of transitional care, staffing changes, visitor policies) and external factors (e.g., CMS telehealth waivers) as well as the responses of the clinicians responsible for implementing the TCM. In turn, all of these factors both influence and are influenced by implementation outcomes (e.g., rates of fidelity to TCM's core components).

Fig. 1.

PRISM Framework to guide assessment of COVID-19 challenges and strategies to achieving TCM implementation fidelity.

Adaptation of The Practical Robust Implementation Sustainability Model (PRISM) (Feldstein & Glasgow, 2008)

2.3. Study setting

The MIRROR-TCM study was launched in four diverse healthcare systems in five states: Swedish Health (Washington), Trinity Health (Michigan), UCSF Health (California), and the VHA (Ohio and Missouri). Based on data from the year prior to the grant submission (2018), the number of older adults discharged annually at each participating healthcare system with heart failure, chronic obstructive pulmonary disease, or pneumonia, ranged between 1572 to 2476. The participating healthcare systems are located outside the East coast of the U.S. where the original TCM trials were conducted. VHA and Trinity Health hospitals are in Central U.S.; Swedish Health and USCF are on the West coast. Overall, the percentage of at-risk older adults is similar across these regions. In addition to different geographic regions, MIRROR-TCM expanded the testing of the model to healthcare systems with more diverse patients (e.g., 25% of UCSF's patients are non-English speaking).

2.4. Data collection and management

2.4.1. Overview

Guided by PRISM, quantitative and qualitative contextual data are being collected in two phases: pre-implementation and implementation (see Fig. 2 ). In both phases, data related to patient, provider, and system challenges affecting implementation of the TCM are being collected via surveys and facilitated meetings with participating healthcare system staff. In the implementation phase, data on strategies identified to address these challenges are being collected via facilitated meetings and fidelity to TCM's core components is being assessed via APRN documentation. In the section below, we describe the sources and rationale for data collected in each phase and how this information is managed (see Table 1 ).

Fig. 2.

Cyclical phases of implementation of the TCM in the context of COVID-19.

Adapted from Leonard et al. (2017) “Cyclical Phases of Implementation and Evaluation”.

Table 1.

Data collection tools to assess implementation of the TCM in the Context of COVID-19.

Tool	Type	Phase / Timing	Source
Discharge Planning/ Transitional Care Services survey	Quantitative & Qualitative	Pre-implementation (Annually thereafter)	Healthcare system discharge planning experts
Changes in Practice survey	Qualitative	Implementation (Quarterly)	Clinical Coordinators
Fidelity to TCM Core Components	Quantitative	Implementation (Daily/ongoing)	APRNs
Time Spent Delivering TCM	Quantitative	Implementation (Daily/ongoing)	APRNs
TCM Case Conference /Fidelity Meetings	Qualitative	Implementation (Monthly)	APRNs
Leadership Meetings	Qualitative	Pre-implementation (Monthly) Implementation (Bimonthly)	Healthcare system, APRNs, Clinical Coordinators
Site Visit Guide/Key Informant Interviews	Qualitative	Year 3 of Study Evaluation and Next Steps Phases	Healthcare system, APRNs, post-acute care and community-based organizational leaders, clinicians and staff

2.4.1.1. Pre-implementation phase data

This phase represents the planning period prior to the start of patient enrollment at each study site (February-June 2020). The primary source of quantitative contextual data was the Discharge Planning/Transitional Care Services survey, an instrument designed by the Penn team and used in prior studies to assess the baseline status of services available to at-risk older adults at participating sites. A second source of quantitative data was findings from the Quarterly Change Survey, also developed by the Penn team and completed every three months by each site's clinical coordinator. This survey seeks periodic updates on leadership and healthcare delivery changes affecting transitional care, including those directly related to the pandemic.

Qualitative contextual data regarding participating sites’ challenges in response to COVID-19 (e.g., staffing shortages) were elicited during monthly organizational leadership calls. These calls included key system leaders and clinical coordinators. Agendas distributed in advance of each meeting identified key questions that would be asked of respondents related to broad system changes associated with the pandemic (e.g., visitation policies, staffing) as well as those specific to transitional care (e.g., patients’ access to SNFs, restrictions on in-person home services). With participants’ permission, calls were recorded and transcribed. Summaries were provided to all participants to review for accuracy. Analyses of these data (described under Statistical Methods) took place near the end of this phase. Findings were used to guide the implementation phase.

2.4.1.2. Implementation phase data

This phase represents the period from the beginning of patient enrollment and ending when the intervention protocol for the last enrollee is completed (July 2020-March 2023). In this phase, the collection of quantitative and qualitative contextual data described under pre-implementation phase continued with some refinements. For example, modifications were made to the Discharge Planning/Transitional Care Services survey to explicitly capture changes in healthcare system practices that have occurred in response to the pandemic (e.g., visitation policies, staffing of discharge planning services, post-acute care challenges); this quantitative survey is deployed annually. New sources of qualitative contextual data were also added. Using a similar procedure to that outlined above (agendas with guided questions), calls were held with leaders of key post-acute care (e.g., SNF, home health) and community providers (e.g., primary care physicians of enrolled patients and leaders of community service organizations) to identify how changes in these contexts in response to COVID-19 may be affecting transitional care; outreach to sites’ post-acute care and community providers will be repeated in the six months prior to the end of the RCT. While all stakeholders are given the opportunity to discuss barriers and facilitators across the entire scope of topics (patient, provider, system), the focus of directed discussion is tailored to each stakeholder's expertise. For example, discussions with APRNs emphasize patient and provider issues, while conversations with organizational leaders focus on system-level issues.

The other focus of data collection during the implementation phase relates to the assessment of fidelity to the TCM and the time devoted by APRNs to the intervention. The primary data source to assess fidelity to TCM's core components is documentation provided by APRNs during and following the completion of the protocol with each MIRROR-TCM intervention patient. In a prior study funded by the Robert Wood Johnson Foundation and guided by a national advisory committee of experts in implementation science, the team operationally defined TCM's 10 core components and developed and tested metrics to assess their use (see Table 2 ) (Naylor et al., 2018). A standardized data collection form was developed to document implementation of each core component. The form also captures APRNs’ perceptions regarding the impact of COVID-19 on their ability to deliver TCM components as intended.

Table 2.

Transitional care model core components and definitions and elements.

Component	Definition (Naylor et al., 2018)	TCM Component Elements
Delivering Services from Hospital to Home	Transitional care begins during a hospital admission, with visits occurring in the hospital, in SNF (if referred), and at home for approximately 2 months following hospital discharge, with 7 days per week availability throughout this period.	•Visited within 24 hours of enrollment at the hospital •Minimum 1 visit during the index hospitalization •Daily visits in the hospital (for first 4 days) and at least 1 visit in the hospital within 48 hours of discharge •Visited within 24 hours of index hospital discharge to home •Visited within 24 hours of index hospital discharge to SNF, if applicable •Visit at home within 24 hours of SNF/rehab discharge, if applicable •Visit in hospital within 24 hours of readmission, if applicable •Minimum 1 visit clinician (e.g., primary care, specialist) •Minimum of 1 visit or telephonic contact each week through an average of two months post-discharge •Visited in person once a week for the first four weeks •Minimum total visits = 9 All visits in-person by APRN unless specified otherwise
Screening At-Risk Older Adults*	Uses a standardized protocol to target hospitalized older adults who are at risk for poor outcomes.	•Consistent use of a standardized screening tool
Staffing*	Relies on APRNs to assume primary responsibility for transitional care services focused on the management of older adults throughout episodes of acute illness.	•Use of APRNs to coordinate and deliver the intervention
Promoting Continuity	Services are designed to prevent breakdowns in care from hospital to home by having the same clinician involved across the illness episode; the clinician uses evidence-based strategies to elicit and communicate older adults’ priority needs, goals, and plans of care.	•Older adults’ needs, goals, and plans of care are communicated within and across care sites •Ensured SNF/Rehab staff had updated plan, if applicable
Coordinating Care	Promotes communication and connections between health care and community-based practitioners.	•Assessment of health and/or social service needs is documented and referral (if needed) for needed services initiated •Ensured community-based organization understands health needs •Assessment that community-based organization's services meet patients’ needs •Priority needs and plans to address them are communicated to the entire care team
Collaborating with Patients, Caregivers & Team	Promotes consensus on the plan of care between older adults, family caregivers, and members of the care team.	•Documented communication between the hospital, health care team, and community agencies team to get them to work together •Collaborative design, implementation, and evaluation of care plans •Care plan updated with providers
Maintaining Relationships with Patients & Caregivers	Establishes and maintains a trusting relationship with the older adult and family caregivers involved in their care.	•Documentation of APRNs advocacy in enabling patients and caregivers to achieve their goals •Coach patient about preparation for visits and conversations with providers •Advocacy for patients’ goals and preferences with the multidisciplinary team
Engaging Patients & Caregivers	Engages older adults in the design and implementation of the plan of care aligned with their preferences, values, and goals.	•Patient's goals and preferences identified •Patient's goals and preferences updated or reevaluated •Caregiver's goals and preferences identified, if applicable •Caregiver's goals and preferences updated or reevaluated, if applicable •Advance care planning, care preference discussion, or end-of-life care conversations
Managing Symptoms & Risks	Identifies and addresses patients’ priority risk factors and symptoms.	•Patients assessed and addressed for symptom management needs •Completion of clinical assessment •Urgent/emergent care plan developed, updated, or reinforced •Routine symptom management plan updated and/or reinforced
Educating/ Promoting Self-Management	Prepares older adults and family caregivers to identify and respond quickly to worsening symptoms.	•Education about the connection between the plan of care and achieving goals •Education to recognize symptoms •Preparation re: symptom management skills •Education about how to implement the emergency plan •Use of teach-back, coaching, and problem solving •Educated about self-care management

^⁎Component not included in fidelity score.

The methodology used to calculate fidelity to the TCM components is described below (see Fidelity to TCM Core Components). Rates of fidelity to TCM components are shared with clinical coordinators and APRNs at monthly quality improvement calls hosted by Penn-based TCM clinical experts. On these calls, facilitators of and barriers to established performance expectations for each TCM component are discussed and strategies to maintain or improve fidelity in the context of sites’ responses to COVID-19 are collaboratively established. Routinely, clinical coordinators and Penn clinical experts review APRNs documentation for completeness and accuracy. If lack of adherence to fidelity is detected, the clinical coordinators work with the APRNs to correct documentation errors. The Penn clinical experts, clinical coordinators and APRNs collaborate to address patient, provider, and system barriers to fidelity.

At the completion of the implementation phase, the “dose” of the TCM that each intervention group patient receives will be calculated using APRN documentation. Specifically, the total number of TCM contacts (in-person, telehealth, telephonic) that occur with or on behalf of each patient (e.g., calls to community-based services) and the total time (minutes) spent by each APRN in delivering the TCM to each patient will be determined. The length of intervention (i.e., days from the date of enrollment to completion of the intervention) also will be calculated. Data on the “dose” of intervention and baseline findings on selected socio-demographic (e.g., age, sex, education, economic status) and social determinants of health (e.g., housing, food insecurity, transportation) characteristics of intervention group patients will be used to examine relationships between these factors, time devoted to the TCM and fidelity rates.

2.4.2. Data management

Patient-level socio-demographic and social determinants of health data are collected by trained research staff at the time of patient enrollment and recorded in the Random Assignment, Participant Tracking, Enrollment and Reporting (RAPTER®) online data entry system as part of the RCT. RAPTER®-is a cloud-based proprietary enrollment software system developed and securely managed by Mathematica (Naylor et al., 2022). Survey data are recorded in Qualtrics™, fidelity to TCM core components is documented by APRNs in REDCap (Research Electronic Data Capture) and summary notes or transcripts of meetings are managed using NVivo. Qualtrics™ is a web-based survey tool used to capture and store data from the Discharge Planning/Transitional Care Services and Quarterly Change surveys (Qualtrics, 2020). REDCap is a web application with a back-end database designed to support research data collection (Harris et al., 2009). The Penn team built a set of tools that allow APRNs and clinical coordinators at participating sites to enter intervention patient information directly into REDCap including dates essential for monitoring fidelity (e.g., dates of first hospital and home visits) and encounter information (e.g., time and actions related to goal setting, symptom management, etc.); these data are stored on a secure, password-protected server at Penn's School of Nursing. Finally, summary notes and transcripts from meeting recordings are being managed using NVivo, a software program used for qualitative and mixed-methods research (QSR International Pty Ltd., 2018). Recordings are stored on a secure, password-protected cloud-based platform and then moved to the same server at Penn's School of Nursing as the REDCap data. Recordings will be kept for the duration of the RCT and destroyed after the final recording is transcribed and a summary of the meeting is created.

2.5. Ethics approval and consent to participate

IRB approval for the conduct of the MIRROR-TCM study was obtained at all participating sites (ClinicalTrials.gov Identifier: NCT04212962), Penn and Mathematica. This study to evaluate the implementation of the intervention portion of the MIRROR-TCM RCT has been reviewed and approved by the University of Pennsylvania Institutional Review Board (Protocol # 842744).

2.6. Statistical methods

2.6.1. Overview

Consistent with a fixed, mixed methods parallel convergent design, the quantitative and qualitative data described above are first analyzed independently, then these datasets are merged and interpreted together (see Fig. 3 ). In the section below, the analyses of quantitative and qualitative data and the method used to calculate fidelity are described, followed by a review of the approach used to merge data (interim analyses). Consistent with the evaluation phase of PRISM, these ongoing analyses are being conducted to inform the concurrent implementation phase (see Fig. 2). At the conclusion of this section, the final analysis of the study findings is described.

Fig. 3.

MIRROR-TCM parallel, convergent mixed methods implementation evaluation design.

Adapted from Creswell & Plano Figure 3.4 (Creswell & Plano Clark, 2018); TCM = Transitional Care Model; APRN = Advanced Practice Registered Nurse.

2.6.2. Quantitative analyses

Descriptive statistics (number and percent; mean, standard deviation, median; interquartile ranges and ranges) are used to summarize findings generated from the Discharge Planning/Transitional Care Services and Quarterly Change surveys. Differences in findings over time are assessed with particular attention to changes that may be due to COVID-19.

2.6.3. Qualitative analyses

Monthly, qualitative data are coded using the TCM components as a structure to capture categories of challenges to the implementation of the intervention as intended and strategies collaboratively developed to address them. Data are being coded by two trained research assistants (BM, MM) independently using directed content analysis (Elo & Kyngas, 2008; Hsieh & Shannon, 2005; Vaismoradi et al., 2013). Directed content analysis (Elo & Kyngas, 2008; Hsieh & Shannon, 2005; Vaismoradi et al., 2013) is an ideal approach when seeking to extend knowledge through a combination of inductive and deductive techniques (Hsieh & Shannon, 2005). Under a study team member's supervision (KBH), these research assistants listen to recorded calls with key stakeholders and document both COVID-19-related and overall challenges to each TCM component as well as strategies implemented to address them. Monthly, qualitative challenges and strategies are then transformed into quantitative counts and merged with quantitative intervention fidelity data. Perspectival triangulation is being used to come to a broad understanding of implementation challenges, weighting coded data from various sources (e.g., organizational leaders, clinical coordinators, APRNs) equally while noting similarities and differences across stakeholders (Ravitch & Carl, 2016)

2.6.4. Fidelity to TCM core components

The TCM has 10 core components. In MIRROR-TCM, however, all study participants are screened in a standardized manner as part of the RCT and all patients assigned to the intervention group receive the TCM from an APRN. Therefore, these two core components are not included in the fidelity score in the MIRROR-TCM study. Established metrics for each of the eight remaining TCM components, identified and tested in an earlier study (Naylor et al., 2018), are scored as Yes or No, indicating whether or not each component was implemented as intended.

2.6.4.1. Patient level fidelity scores

Assessment of fidelity to the eight core components of interest in MIRROR-TCM is based on APRNs documentation of each component's elements during contacts with and on behalf of patients and their family caregivers. In Table 2, each component and related element is listed. There are 38 possible elements, each equally weighted. Once a patient has completed the intervention, an independent review of the APRN documentation is completed by the site's clinical coordinator and Penn clinical experts. A set of rules are applied to the data to determine if each element did (rating of 1) or did not (rating of 0) occur. If the element is not relevant to a given patient, that element is not included in the denominator. For example, if a patient is not sent to a SNF following hospital discharge, that patient's denominator would be 36 out of a possible 38. In this example, the denominator is reduced by two as this patient is not expected to be visited at the SNF (elements not included: visited within 24 hours of admission to SNF; visited at home within 24 hours of discharge from SNF). To make scores with different denominators comparable across patients, the fidelity score is multiplied by the fraction of the total of all possible elements (38) and divided by the total elements relevant for a specific patient. For example, if a patient who did not go to a SNF earned a score of 33 (out of a possible 36), the final fidelity score for this patient is [33X(38/36)] 35.

2.6.4.2. Overall fidelity rates

Thresholds to evaluate rates of fidelity at the site- and overall-study levels for each core component were identified a priori. APRNs are expected to have achieved most metrics with at least 90% of intervention patients (e.g., patients visited within 24 hours post-hospital discharge; patients’ needs, goals, and plans of care communicated with team members within and across care sites). The specific thresholds for each component are listed in Table 2. Site- and study-level fidelity rates are summarized using descriptive statistics. Monthly, fidelity rates are reviewed with APRNs and clinical coordinators at each site and used to guide the identification of quality improvement strategies by these providers.

2.6.5. Interim analyses

Quarterly, the quantitative and qualitative data are merged and analyzed together to better understand challenges to TCM fidelity through rigorous identification of convergent and divergent findings (Curry et al., 2013; O'Cathain et al., 2010). To facilitate these mixed analyses, multiple methods are used. First, rates of fidelity are reviewed to identify areas where performance thresholds are not being met. Then, challenges to meeting fidelity expectations are identified from the qualitative data. Exemplar quotes that help to explain low fidelity or diverge from the fidelity findings are examined. Similarly, the challenges and strategies noted in the qualitative data are further explored for patterns. Data are combined in both a side-by-side convergent table as well as through visualization plots. These approaches are designed to depict the challenges and strategies implemented relevant to each component over time and the corresponding rate of fidelity for that component over time. Data are then presented to the Penn team for discussion of the interpretation with consensus agreement on the interpretation. These mixed findings in the Evaluation Phase are the basis of quarterly presentations at organizational leadership meetings. Findings are used to support system-level reinforcement or modification of strategies implemented to maintain or improve fidelity.

2.6.6. Final analyses

All analyses described in this section are being conducted using data from the entire sample, including variables specific to each participating healthcare system (e.g., site, changes in transitional care services, staffing, leadership), patient socio-demographic (e.g., age, sex, education, economic status) and social determinants of health (e.g., housing, food insecurity, transportation) characteristics collected as part of RCT, and dose of intervention (e.g., minutes spent in contact with APRN, length of intervention). The goal is to characterize patients who have higher fidelity scores in TCM implementation based on socio-demographic characteristics, social determinants of health, the dose of intervention, and system-level challenges faced during their intervention and strategies implemented to overcome these challenges. System-level variables (e.g., challenges and strategies from the qualitative data), along with local COVID-19 rates from an outside data source, will be mapped to the patient-level data by matching the month and site of the system-level variable to the site and month in which the patient was enrolled.

Recognizing there will likely be imbalances in some patient demographics across participating sites which could introduce analytical challenges, particularly concerning confounding and identifying direct effects associated with fidelity, our analytic strategy is multifaceted. The approach includes general linear modeling using the combined dataset to estimate effect sizes for each of the aforementioned predictors of interest while controlling for potential confounders. When feasible, two-way interactions will be examined to adjust for potential effect modifiers (e.g., site by gender). Model assumptions will be evaluated both quantitatively and through visual inspection, with normality of the residuals examined using Q-Q plots and histograms. Should the residuals suggest a non-Gaussian distribution, the data will be transformed accordingly to achieve approximate normality. Additionally, a linear mixed effects model will be specified with site as a random effect and predictors of interest included as fixed effects. Again, model assumptions will be checked. Finally, sensitivity analyses based on subgroups defined by specific site and sociodemographic variables of interest will be generated. The summarization of descriptive statistics will allow for a characterization of the population to which generalizations of the study findings can be made.

3. Discussion

This study is capitalizing on a unique opportunity, the conduct of the MIRROR-TCM study, to rigorously describe changes in response to COVID-19 made by participating hospital and community-based organizations as well as clinicians and staff that affect the transitional care of at-risk older adults. In the section below, the implications of this study related to advances in knowledge and support of quality clinical care are examined. Potential areas for future research are also suggested.

3.1. Advances in knowledge

There are several ways by which this study will advance knowledge. First, this study was launched right as COVID-19 was identified in the U.S. and continues through multiple surges at each site. Given the nature of TCM, a cross-site intervention, examination of the challenges experienced by patients and their providers throughout multiple phases of a public health crisis that affected virtually all segments of our health and social system (e.g., hospitals, post-acute, and community-based agencies) was needed. Thus, this study provides a unique lens on significant challenges associated with a crisis that impacts multiple sectors responsible for the care of older adults as they transition from hospital to home. Importantly, knowledge gleaned from this work has implications for the healthcare system that extends beyond COVID-19. Healthcare systems and their community providers frequently confront internal and external crises for which knowledge gained from this study has implications. Second, the use of PRISM allows for the examination of specific challenges across all phases of implementing the TCM across diverse healthcare systems over time—pre-implementation, implementation, and evaluation. Notably, PRISM's cyclical approach guides the use of knowledge gleaned in each phase to inform other phases. Third, the design selected for this study enables the collection of multi-level (e.g., patient, provider, and system) quantitative and qualitative data which, when merged, inform ongoing interpretation of fidelity rates and MIRROR-TCM outcomes.

3.2. Support of quality clinical care

The selection of the study's conceptual model and design was motivated by the study team's desire to foster an iterative approach to knowledge development and allow for continual changes in clinical care that are responsive to the challenges posed by COVID-19. Specifically, the potential for improvement in clinical care in the context of COVID-19 by implementing a proven care management is fostered by three strategies. First, multiple data collection points and strategies (e.g., surveys, facilitated meetings) position site project teams with ongoing information about the complex care needs of those hospitalized with or without COVID-19 and the unique challenges patients and family caregivers are confronting throughout their transitional care journeys to home during this pandemic. The study team anticipated that this information may activate timely system changes to transitional care. Second, by providing APRNs and clinical coordinators with data about the issues patients and systems are confronting, coupled with information on their fidelity performance, the team anticipated that such data may motivate changes in provider behaviors to improve fidelity. Third, by directly engaging APRNs, clinical coordinators, and Penn clinical experts in collaborative decision-making regarding strategies to promote fidelity to evidence-based core components, the expectation is that the quality of clinical care and, ultimately, patient outcomes will be enhanced.

3.3. Future research

Through rigorous evaluation of the implementation of the TCM, this study advances knowledge regarding challenges posed by the pandemic to transitional care of older adults and diverse healthcare systems’ responses. While it remains to be seen if the approach used in this study improves fidelity over time and positively impacts MIRROR-TCM outcomes, information gleaned from this study to date reinforces the importance of studying contextual factors to interpret study outcomes both overall and by site. Hence, the use of a similar approach to study the implementation of the TCM under “more normal” circumstances is recommended. Additional insights regarding the relative importance of specific patient, provider, and system characteristics and challenges are also worthy of further investigation.

3.4. Study protocol limitations

Important limitations to the study protocol need to be acknowledged. First, while the study funding began in February 2020, the start of the implementation phase was delayed until July 2020, possibly missing an opportunity to implement the TCM during the crucial early phase of the COVID-19 pandemic in the U.S. (i.e., March through June 2020). However, data collected during the pre-implementation phase captured valuable information from these early months of the pandemic (e.g., COVID-19 peak patient volumes, healthcare systems’ responses to meet care needs, restrictions in hospital visitation policies) through regular meetings with healthcare system leaders and site team members, offer valuable insights regarding each system's preparedness for crises. Second, the perspectives of primary care, post-acute, and community-based clinicians and staff were not directly elicited during the early COVID-19 period (pre-implementation) and are being captured only through annual interviews during the implementation phase. The views of payers will not be elicited until the “Next Steps” phase of this work. In retrospect, this study would have benefited from the direct input of individuals representing all parts of a system essential for effective transitions throughout all study phases. Third, the perspectives of older adults throughout this study were reported indirectly by the APRNs involved in their care. Once again, the direct voices of patients would have strengthened the study findings. Fourth, the primary source of fidelity data is APRNs documentation in REDCap. Despite ongoing reviews and quality improvement efforts, documentation may not be entirely accurate or complete. Finally, as with any observational study, findings are only generalizable to populations with similar patient demographics. The multifaceted analytical approach described earlier will consider potential observed and unobserved confounders and facilitate a comprehensive examination of the variables most strongly associated with fidelity.

3.5. Trial status

Data collection and ongoing data analyses will continue through March 2023, coinciding with the completion of MIRROR-TCM. Three of the four partnering healthcare systems will complete the parallel RCT and continue to provide data for this study. Due to substantial challenges associated with COVID-19, Swedish Health withdrew from MIRROR-TCM. Thus, data from this healthcare system will only include data from the pre-implementation phase and for the first seven months of implementation, the point at which the system withdrew. Analyses will include Swedish Health data throughout their participation in the study. Final data analyses will be completed by September 2023.

Consistent with PRISM's Next Steps Phase, a preliminary description of the impact of COVID-19-related challenges and strategies on fidelity to TCM's core components are being discussed at individual site visits conducted with key stakeholders (e.g., participating hospitals, post-acute, primary care, community agencies, and payers) in the Fall of 2022. Using the qualitative method described above, notes taken at these meetings will be analyzed. A summary of the final study findings (e.g., relationships between challenges, strategies, and rates of fidelity over time) will be provided to the Mathematica team to guide their interpretation of MIRROR-TCM's outcomes. Similarly, final site-specific findings will be shared with healthcare system leaders to guide their decision-making regarding the continued spread and sustainability of the TCM following the completion of MIRROR-TCM.

Funding

The primary funder of MIRROR-TCM is Arnold Ventures (Penn: 19-02984; Mathematica: 19-02999), with co-sponsorship from the Missouri Foundation for Health (#20-0006-OF-20) and the Health Services Research and Development Service, Department of Veteran Affairs. BM and MM were supported by the National Institute of Nursing Research of the National Institutes of Health under Award Number T32NR009356 and BM was supported by Award Number F31NR020140. The funders have no role or authority over study implementation, interpretation of results, report writing, or decisions regarding publication.

Dissemination

To date, information related to the design of this study and preliminary pre-implementation findings were presented at the Gerontological Society of America's 72nd Annual Scientific Meeting in November 2021. Findings from interim and final analyses will be submitted for publication in peer-reviewed research journals and presented at national and international conferences.

Availability of data and materials

Information about the Transitional Care Model and the MIRROR-TCM trial can be located at https://www.nursing.upenn.edu/ncth/mirror-tcm/. Deidentified study data for analyses may be made available on request following study closure.

CRediT authorship contribution statement

Mary D. Naylor: Conceptualization, Methodology, Writing – review & editing, Funding acquisition. Karen B. Hirschman: Conceptualization, Methodology, Writing – review & editing. Brianna Morgan: Writing – review & editing. Molly McHugh: Writing – review & editing. Alexandra L. Hanlon: Conceptualization, Methodology, Writing – review & editing. Monica Ahrens: Writing – review & editing. Kathleen McCauley: Conceptualization, Writing – review & editing. Elizabeth C. Shaid: Writing – review & editing. Mark V. Pauly: Conceptualization, Methodology, Writing – review & editing.

Declaration of Competing Interest

None.