Sustaining innovations in complex healthcare environments: A multiple-case study of rapid response teams

Abstract

Objectives

Rapid response teams are one innovation previously deployed in U.S. hospitals with the goal to improve the quality of care. Sustaining rapid response teams is important to achieve the desired implementation outcomes, reduce the risk of program investments losses, and prevent employee disillusionment and dissatisfaction. This study sought to examine factors that do and do not support the sustainability of Rapid Response Teams.

Methods

The study was conceptually guided by an adapted version of the Planning Model of Sustainability. A multiple-case study was conducted using a purposive sample of two hospitals with high RRT sustainability scores and two hospitals with low RRT sustainability scores. Data collection methods included: (a) a hospital questionnaire that was completed by a nurse administrator at each hospital; (b) semi-structured interviews with leaders, RRT members, and those activating RRT calls; and, (c) review of internal documents. Quantitative data were analyzed using descriptive statistics; qualitative data were analyzed using content analysis.

Results

Few descriptive differences were found between hospitals. However, there were notable differences in the operationalization of certain factors between high- and low-sustainability hospitals. Additional sustainability factors other than those captured by the Planning Model of Sustainability were also identified.

Conclusions

The sustainability of rapid response teams is optimized through effective operationalization of organizational and project design and implementation factors. Two additional factors—individual and team characteristics—should be included in the Planning Model of Sustainability and considered as potential facilitators (or inhibitors) of RRT sustainability.

INTRODUCTION

Understanding how innovations are sustained is a research imperative. The implementation of innovations requires the use of substantial organizational resources [1], however, successful implementation of an innovation in organizations may not necessarily result in its sustainability [2, 3, 4]. Failure to sustain innovations may result in investment losses and the waste of organizational resources [5]. Furthermore, when, in future, new innovations are adopted and implemented in an organization, buy-in and the achievement of goals behind implementation of the innovation (i.e., improving quality and patient outcomes) may not occur [1,6, 7]. In contrast, the effects of sustained innovations may be long-lasting and may enhance an organization’s strategic position and competitive edge, while attracting clients and healthcare workers, and benefitting hospitals, their employees, and patients [8, 9].

The sustainability of innovations in healthcare organizations is particularly difficult for several reasons: complexity of the environment, the patients cared for, and services provided; a workforce that encompasses different professions; and existing organizational cultures [10]. Until recently, research has predominantly focused on innovation implementation rather than innovation sustainability [11–14], and the adoption and implementation of innovations in community settings rather than in acute care hospitals [15–19]. Various factors have been associated with sustainability [18, 19, 20]. However, research has failed to explain the how and why of innovation sustainability in complex healthcare environments, which makes the building of knowledge around organizational sustainability elusive. Therefore, the purpose of this study was to examine the sustainability of an innovation—rapid response teams. Rapid response teams, implemented widely across U.S. hospitals, consist of the deployment of specialists who intervene quickly to bring critical care expertise and resources to the bedside of patients in crisis on acute care units.

Rapid response teams (RRTs) have the potential to improve patient outcomes by instituting prompt interventions and patient treatments to possibly prevent cardiac arrest and death [23]. Rapid response teams were selected as the organizational innovation through which to examine sustainability for several reasons. First, when RRTs were initially adopted and implemented in hospitals, they were a new innovative process for taking care of patients in crisis on acute care units. Second, despite their adoption and implementation, the research suggests that there is underutilization and delayed use of RRTs [24, 25]. Third, barriers to calling RRTs in hospitals remain despite their widespread adoption. [26] For both of these reasons, RRT sustainability is unlikely. Finally, in an earlier study examining the sustainability of RRTs in a group of hospitals four years after their participation in a statewide collaborative to implement and sustain RRTs (n=56), the authors found wide variability in RRT sustainability scores (1.0 – 5.98 (Mean = 3.78, Median = 3.98, SD=1.06) [27]. Thus, despite participation in a state-wide collaborative to implement and sustain RRTs, RRT sustainability was lacking in many of the study hospitals.

In summary, there is limited research on the how and why sustainability of innovations is achieved in complex healthcare organizations. RRTs were one innovation adopted and implemented in these organizations with the goal to improve patient outcomes. Yet, evidence of their lack of sustainability in some organizations remains. A multiple-case study that, guided by a conceptual model, provides detailed information on why and how the sustainability of innovations can be achieved, has the potential to guide future sustainability efforts and, potentially, facilitate long-term effects of sustained innovations. Therefore, the purpose of the study was to examine the sustainability of one innovation that was adopted and implemented in complex healthcare organizations. The specific aim of the study was to examine factors that do and do not support RRT sustainability.

CONCEPTUAL FRAMEWORK

An adapted version of The Planning Model of Sustainability (PMOS) [28] was used in this study (Figure 1). The original model incorporates three general categories of sustainability factors: the broader community, project design and implementation, and the organizational setting. However, because examining factors in the broader community was beyond the scope of this study, community factors were not included in the adapted model. (Please see Additional File A for variable definitions).

Figure 1.

Adapted Planning Model of Sustainability

METHODS

Design, Setting, and Sample

A multiple-case study design was used to conduct the study. [29, 30, 31] Four hospitals were purposively selected from a sample of hospitals in a southeastern state that participated in a larger study of RRT sustainability [27]. Hospital inclusion criteria for this study were hospitals that (a) participated in a statewide RRT collaborative to implement and sustain RRTs; (b) responded to a previous survey in which RRT sustainability scores were reported [27]; and, (c) ranked in either the top or bottom 25% of the group of hospitals that participated in the larger study. Exclusion criteria included hospitals that (a) participated in the collaborative but did not participate in the larger study; and, (b) indicated during the earlier study that they did not want to be considered for inclusion in the multiple-case study. To reduce sample variability, only community hospitals of similar size were selected for participation.

Purposive and snowball sampling was used to identify potential participants for interviews. Individuals from three key groups that would likely be involved in the adoption and implementation of RRTs were interviewed: organizational leaders (such as directors and quality improvement specialists), RRT members (RRTMs) who previously responded or currently respond to RRT calls, and RRT end-users (RRT-EU) (i.e., front-line registered nurse clinicians) who were affiliated with patient care areas from which RRT calls could originate.

Data Collection

Three data collection approaches were used. First, a Hospital RRT Questionnaire (HRRT-Q) was used to gather descriptive information on the RRT at each case hospital. Second, semi-structured interviews were conducted to gather information and depth about the presence of sustainability factors in organizations. Factors were considered to be present in a hospital if participants from at least two of the three key groups sampled verified their existence. Every effort was made to conduct in-person interviews; however, when this was not possible, telephone interviews were conducted. Interviews lasted 30–45 minutes and were audiotaped and transcribed. Finally, organizational documents were reviewed to gather additional descriptions and explanations when issues were mentioned during data collection that needed further examination or clarification. Data gathered from these three sources were augmented by handwritten field notes and interview notes.

Data Analysis

Transcribed interviews were analyzed using the qualitative data analysis software program Atlas ti. (Version 4.2) analyzed through content analysis [30, 31]. A provisional list of codes was developed prior to fieldwork based on the study’s conceptual framework and research questions [30–32]. Following content analysis, in an effort to present the study data in an organized, succinct way and enable the researcher to draw conclusions and verify information, data were displayed in tabular format. Cross-case analysis using pattern-matching logic (i.e., looking for patterns in the data based on the patterns expected by the study’s conceptual framework) [29] was used to identify factors that might have facilitated or prevented RRT sustainability in high-sustainability hospitals (SUS-H) or low sustainability hospitals (SUS-L).

Ethical Considerations

The institutional review boards at the University of North Carolina at Chapel Hill and each of the four hospitals approved this study. Chief Nursing Officers’ at all four hospitals gave approval for their organization to participate in the study. An informed consent form was also given to each participant and written and/or oral consent was obtained prior to the interviews.

RESULTS

Sample Characteristics

Descriptive statistics of hospitals and their RRT programs are reported in Table 1. Because one SUS-H hospital and both SUS-L hospitals, initially selected for participation, declined to participate and thus necessitated the selection of additional hospitals, some variation in hospital size resulted (Table 1). The four community hospitals reported sustainability scores that ranged between 2.94 and 4.67. Their RRT programs had been in place for 5–6 years and were similar in characteristics. Interviews were conducted with 19 hospital leaders, 11 RRTMs, and 20 RRT-EUs (see Additional File A for a detailed breakdown of interviewees by hospital). For ease of reporting, the following abbreviations are used to refer to hospitals in the study: High-1 and High-2 represent the hospitals with the highest and second highest sustainability score of participating hospitals; and, Low-1 and Low-2 represent the hospitals with the highest and second highest sustainability score in the group of SUS-L participating hospitals.

Table 1.

Hospital and RRT Program Characteristics

Characteristics	High-1	High-2	Low-1	Low-2
Hospital
Sustainability Scorea	4.67	4.55	2.97	2.94
Sizeb	160	258	312	101
Number of direct care RNs	233	428	476	195
Number of years RRT in place	5	6	6	6
External funding	No	No	No	No
Current budgeted internal funding	Yes	Yes	Yes	Yes
Collaboration with other hospitals and organizations	Yes	Yes	Yes	No
Collaboration with individuals outside of the hospital	No	Yes	No	Yes
RRT pilot/duration (months)	Yes/ ≤ 12	Yes/ ≤ 6	Yes/ ≤ 3	No
Expansion to specialty areas	Yes	Yes	No	Yes
RRT Program
RRT model	RN-led	RN-led	RN-led	RN-led
RRT calling criteria stipulated	Yes	Yes	Yes	Yes
RRT order set	Yes	Yes	No	Yes
Number of RRT calls/patient outcomes tracked	Yes	Yes	Yes	Yes
RRT end-user involved in call	Yes	Yes	Yes	Yes
RRT information received by patients	Yes	Yes	Yes	Yes
Staff training provided prior to RRT launch	Yes	Yes	Yes	No
Annual staff training	Yes	Yes	Yes	Yes

Note. RN = registered nurse; DNT = hospital did not track.

^a= sustainability scores ranged from 1.0 to 5.98 with 5.98 representing highest sustainability score;

^b= Number of beds licensed and staffed.

Individual case hospitals are distinguished as: High-1= hospital with the highest sustainability score; High-2= hospital with the second highest sustainability score; Low-1= hospital with the highest sustainability score in the low-sustainability group; Low-2= hospital with the second highest sustainability score in the low-sustainability group.

Sustainability Factors

Case hospitals were examined for the presence of factors from two categories proposed in the PMOS to facilitate sustainability, namely organizational factors and project design and implementation factors. Other additional factors that were also identified are described below.

Organizational factors

Two organizational factors were examined: institutional strength and program champion. Aspects of each factor will be described next.

Institutional strength

All four case hospitals reflected some level of institutional strength. Participants reported goal alignment between the RRT program and the hospital’s mission and/or vision. Leaders created the necessary structures, lines of authority, and mechanisms to evaluate and support the RRT program. RRT programs were integrated into existing structures and lines of authority. For example, existing critical care nurses and respiratory therapists staffed the RRT and existing committees provided oversight. Additional structures were also developed including RRT policies, procedures and/or protocols and RRT order sets. Individuals were assigned to provide program oversight, including collecting, analyzing, and reporting RRT data and outcomes.

At all four hospitals, RRTMs’ performance was evaluated through the annual review process. At two hospitals, High-1 and Low-1, a formalized process existed for RRT-EUs to provide feedback on RRTMs and RRT call experiences.

The key aspects of organizational strength that differentiated SUS-H hospitals from SUS-L hospitals were strong nursing department involvement with RRT-related performance improvement (PI) activities, staff received feedback on the RRT program, and the person responsible for PI activities being perceived as passionate about the RRT program. SUS-H hospitals also reported a lower percentage of direct care nurses with an Associate Degree in Nursing.

Program champions

Champions were present in all study hospitals, but there was far greater awareness of champions at the SUS-H hospitals. Initial champions, or those involved in conceiving and/or launching the RRT, were all in leadership roles (i.e., a nurse manager or a nurse educator). Participants at all the hospitals identified RRTMs as being current champions. The actions and behaviors (i.e., a positive attitude, enthusiasm and passion about RRT program) of initial and current champions were perceived as more important than their position in the organization (i.e., leader, expert clinician, or RRTM). Fewer participants in the SUS-L hospitals agreed that a champion was currently present in their hospital.

Additional Organizational factors

Additional factors were identified as key differences between the SUS-H and SUS-L hospitals. Leadership support of nurses activating RRT calls and nurse autonomy being valued and fostered were mentioned only by participants in the SUS-H hospitals. In the SUS-L hospitals resource limitations (i.e., an insufficient number of skilled, experienced critical care nurses to staff the RRT) translated into delayed responses by RRT members and a reduction in RRT-EUs’ confidence in the team and their feelings of being supported by the team. Both groups of hospitals reported the lack of a dedicated RRT nurse as an inhibiting factor. Participants perceived that a dedicated RRT nurse could round on units, collaborate with, and provide RRT-related education to frontline staff.

Project design and implementation factors

Three project design and implementation factors were examine: project negotiation process, project effectiveness, and training.

Project negotiation process

At all four hospitals, staff members were engaged in the RRT implementation process. However, RRT implementation was perceived as a “ground-up” approach in SUS-H hospitals, compared to a top-down approach in SUS-L hospitals. Also, SUS-H hospitals used larger, multi-disciplinary teams and a higher number of nurse leaders were involved in implementation processes. At the SUS-L hospitals, RRT implementation was instead described as being limited to a “select few” staff members. RRT-EUs’ involvement in RRT implementation was either absent (High-1, Low-1, and Low-2) or very limited (High-2). At High-2, charge nurses from the acute care units where RRT calls could originate from, participated in the implementation process.

Project effectiveness

At all four hospitals, participants reported similar perceived RRT program benefits for patients, staff, and the organization. Patient benefits included early recognition of a crisis and early intervention, enhanced patient safety, and patient and family experiencing a sense of security. Staff benefits included reduced RRT-EU work load and work stress. Organizational benefits reported included promoting positive community perceptions of the hospital and improving patient and family confidence in care.

Participants at the SUS-H hospitals placed a greater emphasis on the learning aspect of RRT calls. Specifically, they mentioned the knowledge, skills, and expertise that both RRT-EUs and RRTMs gained from RRT call experiences. On the other hand, SUS-L hospital participants highlighted getting expert help. Nearly half of the participants at the SUS-H hospitals perceived RRTs to improve nurse, patient, and family satisfaction, while very few participants at the SUS-L hospitals perceived this benefit.

Training

Except at Low-2, initial training was provided prior to the launch of the RRT program to nurses, physicians, and other hospital staff. Training modalities (such as posters, in-services, and fliers) and content (such as defining the RRT program, stipulating RRT calling criteria, and indicating ways to activate a RRT call) were similar across hospitals. At Low-2, training consisted of only a review of the RRT policy prior to launching the team. At all four hospitals, ongoing annual RRT-specific training was provided. RRTMs’ clinical skills in caring for patients in crisis were largely assumed at all hospitals. At the SUS-H hospitals, RRTMs received training on ideal RRTM behaviors. This was not reported at the SUS-L hospitals.

The need for improvements—such as clarifying role expectations of RRTMs and RRT-EUs during calls—in RRT training was reported by participants’ at all four hospitals. Other suggested improvements included providing a refresher course for RRTMs on RRT documentation and RRT order sets, and incorporating interactive scenarios of various types of calls. It should be noted that no additional project design and implementation factors were identified in this study.

Additional Sustainability Factors

In addition to organizational and project design and implementation factors, two additional factors were also identified as important to the sustainability of RRTs: individual and team factors. Details on each of these factors will be described next.

Individual factors

Nurses’ experiences with RRT calls and their overall clinical knowledge, skills, and nursing experience were identified as individual factors that might impact their decision to call the RRT. Because nurses shared their RRT experiences with colleagues, positive RRT experiences were viewed as fostering future RRT calls by nurse and unit colleagues, whereas negative experiences were believed to inhibit future calling. However, participants at all four hospitals reported that nurses on acute care units lacked general knowledge, skills, and experiences needed to know when an RRT call was warranted. At High-2, RRT use was viewed as being higher when nurses were afraid to call patients’ physicians. This was not reported at the other hospitals.

Team factors

Team factors identified as important to RRT sustainability included the perception of immediately observable RRT effectiveness, and the general attitudes and behaviors exhibited by RRTMs. RRTs were utilized because nurses perceived the RRT as effective in providing care to patients who were experiencing a medical crisis and expediting patient transfers to higher levels of care if needed. RRTMs’ attitudes and behaviors were also perceived as an important factor in RRT sustainability: positive attitudes and behaviors facilitated nurses’ calling the RRT, while negative, derogatory attitudes and behaviors inhibited calling. In SUS-H hospitals, RRTMs were perceived to foster collegiality and teamwork with RRT-EUs, which, in turn, reduced RRT-EUs fear of calling.

RRT Sustainability

Two questions, used by Mancini and Marek,[19] to measure sustainability were adapted and used to measure sustainability in this study (Table 2). Participants at SUS-H (n=23, 85%) and SUS-L (n=20, 87%) were nearly equally very confident that the RRT at their organization would still be active in 5 years. The reasons that participants were only somewhat confident that RRTs would still be active in 5 years included the continually changing healthcare system and the development of new programs that might nullify the RRT program (SUS-H), and a poor economic climate and limited funding (SUS-L).

Table 2.

Views of the Continuation of RRTs

	High-Sustainability N=27 N (%)	Low-Sustainability N=23 N (%)
How confident are you that RRTs will still be active in 5 years?
Very	23 (85)	20 (87)
Somewhat	4 (15)	3 (13)
Not at all	0	0
In your opinion, to what extent do RRTs meet the needs of patients?
Fully	16 (59)	16 (70)
Moderately	11 (41)	5 (22)
Somewhat	0	2 (9)
Not at all	0	0

Compared to the SUS-L hospitals, fewer participants in the SUS-H hospitals perceived that RRTs fully met the needs of patients (n=16, 59% vs. n=16, 70%) while a higher number perceived RRTs as moderately meeting patients’ needs f (n=11, 41% vs. n=5, 22%). Nine percent of participants in the SUS-L hospitals reported that RRTs met the needs of patients only somewhat. Reasons why participants perceived RRTs to only somewhat or moderately meet the needs of patients included a limited RRT order set restricting the care of patients with certain conditions (SUS-H), and nurses’ lack of awareness of the RRT program (SUS-L).

Two questions were also used to examine the sustainability of the perceived and actual benefits by RRTs. First, participants were asked whether the benefits they perceived from RRT implementation had persisted in the organization since the initial implementation of the program. All participants at the SUS-H hospitals reported that these benefits had persisted and the majority of participants at the SUS-L hospitals reported that the benefits had persisted. Second, at each hospital, staff members who collected and analyzed the RRT program data were asked if patient outcomes associated with RRTs had been maintained in the hospital. These persons were also asked to report the maintenance of specific patient outcomes that had been used to measure RRT effectiveness, including acute care inpatient mortality rates, unplanned intensive care unit transfers, codes outside of the intensive care unit, and codes per 1000 discharges. Both the SUS-H and SUS-L hospitals reported the maintenance of a decrease in three of these patient outcomes since the RRT was implemented: unplanned intensive care unit transfers, codes outside of the intensive care unit, and codes per 1000 discharges. In all of the organizations, however, acute care inpatient mortality rates were reported as unchanged.

Factor Rankings and Relationships

Participants were asked to rank the five factors in the PMOS (i.e., project negotiation process, project effectiveness, training, institutional strength, and program champion) in order of importance, from most to least important. Overall, participants rated a program champion as the most important factor, followed by the project negotiation process, training, and institutional strength. Project effectiveness was ranked as the least important factor.

As shown in Table 3, there were differences between the SUS-H and SUS-L hospitals. In the SUS-H hospitals, participants ranked the presence of a program champion as most important, whereas participants in the SUS-L hospitals ranked training as most important, and the presence of a program champion was ranked third. The only factor that was ranked the same in both the SUS-H and SUS-L hospitals was the project negotiation process, which was ranked second. Finally, participants were also asked whether the factors in the adapted PMOS were inter-related; 44% (n=22) of participants responded “yes”. Eight (29%) participants at the SUS-H hospitals and 14 (61%) participants at the SUS-L hospitals said that the factors were inter-related, as shown in

Table 3.

A Comparison of Factor Rankings

Rank	Overall	High-Sustainability Hospitals	Low-Sustainability Hospitals
1	Program Champion	Program Champion	Training
2	Project Negotiation Process	Project Negotiation Process	Project Negotiation Process
3	Training	Training	Program Champion
4	Institutional Strength	Institutional Strength	Project Effectiveness
5	Project Effectiveness	Project Effectiveness	Institutional Strength

Note. 1 = most important and 5= least important

DISCUSSION

Findings from this study provide support for the PMOS, developed by Shediack-Rizkallah and Bone [29], but also suggest that the PMOS may need to be expanded. That is, findings suggest that the organizational factors (i.e., institutional strength and program champion) and project design and implementation factors (i.e., project negotiation process, project effectiveness, training, and project funding) were important for RRT sustainability. All of the proposed factors were present in each of the case study hospitals, and all four of the hospitals achieved some degree of RRT sustainability.

The research findings reported in this article also extend the work of Shediack-Rizkallah and Bone in several ways. First, findings indicate that the degree to which factors are present in hospitals may determine the level of RRT sustainability that a hospital will achieve. Second, the ways in which organizational leaders operationalized the various factors represented in the PMOS in hospitals may determine if high levels of sustainability are achieved. SUS-H hospitals were characterized by strong nursing department involvement in PI activities, staff receiving feedback on outcomes, and visible program champions who were enthusiastic about RRTs and who were respected by clinicians. In these hospitals, nurse leaders and a multi-disciplinary team were actively involved in designing and implementing the RRT program. Ongoing training was important, but providing the training prior to the launching of programs was more important for sustainability. This is likely because training prepared organizational members for upcoming changes associated with the RRT, such as revisions to policies for managing patients with acute, avoidable deterioration on acute care units.

Shediack-Rizkallah and Bone also did not clarify project effectiveness to include both actual and perceived benefits. However, several other researchers have suggested that, even in the absence of actual benefits, the perception of benefits may facilitate innovation sustainability [18, 20, 21, 26]. The current study supports the notion that project effectiveness could be operationalized as perceived benefits, and that the perception of benefits, even if different than actual benefits, could facilitate sustainability.

This study also demonstrated that some additional organizational factors should be included in the PMOS. The importance of leadership support aligns with the findings of other studies [9, 19, 38] in which leaders are reported to play an important role in the implementation of innovations because they set organizational goals, provide direction, create the necessary structures to support programs, and manage the resources that are crucial to program success. Because nurse leaders encouraged nurses to practice autonomously and call the RRT when they identified the need, nurses felt that they were supported should their decision to activate an RRT call be questioned. Nurses did not fear or hesitate to call the RRT, particularly when they encountered physician resistance to RRT calling. Thus, leadership support translated into autonomous practice that fostered nurses’ sense of security and trust to use the RRT. Another PMOS organizational factor not included in the study’s conceptual framework, namely the availability of funding and resources, was also important to RRT sustainability. The lack of skilled, experienced critical care nurses who could serve as RRT members was an inhibiting factor at SUS-L hospitals because critical care nurses with advanced assessment skills are essential to an RRT program’s success [36–38]. The lack of experienced ICU nurses made staffing the RRT out of the ICU very difficult and increased the risk that an experienced ICU nurse trained as an RRT member was unavailable to respond to the RRT call.

In addition to PMOS factors, this study also demonstrated the important role that individuals and teams play in sustainability. The more knowledgeable individual RRT-EUs were with the functions and role of the RRT, the more likely they were to, in future, activate a RRT call. When RRT members’ attitudes and behaviors during and following RRT calls fostered teamwork and collegiality with RRT-EUs, confidence in the team and future calling behavior were positively impacted. Training that facilitates positive RRTM behavior is therefore essential for RRT sustainability because RRTMs play a critical role in facilitating RRT-EUs experiences with RRT calls. Nurses’ experiences with RRT calls can be both a facilitator and inhibitor to RRT calling. When RRT members were positive, supportive, and encouraging with RRT end-users, end-users’ experiences with the team were seen as positive and facilitated future calling. However, demeaning, resentful or rude behavior was inhibiting. This aligns with similar reports of the important role that RRTMs play in facilitating RRT calling. [39, 40, 41] These variables should be included in future sustainability models and studies examining determinants of sustainability.

Limitations and Recommendations for Future Research

Limitations of the multiple-case study design include common issues that arise from sampling and the data collection strategies employed. [29] The generalizability of this study’s findings may be limited. However, according to the Proximal Similarity Model (PSM) [42], generalizability can be justified by theorizing about the different contexts to which a study’s findings may be applicable. In this regard, the generalizability of the current study’s findings to other hospitals may be possible. The rationale for this approach is that the sample consisted of community hospitals in one state that were situated in rural locales and were relatively small (i.e., less than 350 licensed and staffed beds). Staff characteristics included a higher proportion of ADN nurses compared to baccalaureate prepared nurses, which is consistent with the educational level of U.S. employed nurses, in general [43]. Thus, based on the PSM, the findings of this study may potentially be applicable to community hospitals in other U.S. states, to non-community hospitals located in rural areas or that are similar in size to hospitals in this study, and U.S. hospitals that have a similar proportion of ADN and baccalaureate prepared nursing staff. Future studies of RRT sustainability should be conducted in these settings to examine the applicability of the sustainability factors identified in this study in these contexts.

The use of a semi-structured interview guide, a neutral, professional tone in voice and behavior during questioning, and digital recordings of participants’ exact words reduced the risk of interviewer bias in this study [44]. Close collaboration with faculty advisors and qualitative experts at a research institute ensured the trustworthiness of data [44]. Ensuring participant confidentiality, carefully wording questions, and providing an environment where participants could safely share their thoughts and feelings, reduced the risk of response bias [44]. Although the researcher collaborated closely with the research team on the coding of data, independent inter-rater reliability of the coding of interviews by other team members was not established. However, the researcher discussed the coding of terms and provided rationale for the coding of data to the research team.

Because of the limited number of RRTM and physician participants, study findings may not adequately reflect the views of these two groups. Thus, future studies should include targeted strategies to increase the participation of and seek viewpoints of various multidisciplinary team members. Also, larger-scale, empirical studies are needed to examine the relationships among organizational factors, project design and implementation factors, and the characteristics of RRTs. Potential future studies should, for example, examine the relationship between RRT models (such as RN-driven versus physician-driven RRT models) and RRT sustainability.

CONCLUSIONS

This study examined various sustainability factors included in the PMOS. We found support for the model as well as rationale to extend the model to include additional factors. Specifically, the study identified the organizational factors of leadership support and nurse autonomy as important facilitators of sustainability. In contrast, resource limitations inhibited sustainability. Furthermore, individual and team factors should be included in future studies on sustainability. Other studies have previously highlighted the role of various factors in sustainability. However, this study identified that the operationalization of these factors was what differentiated high- and low-sustainability hospitals that had implemented RRTs. Specifically, strong nursing department involvement with RRT-related PI activities, leaders who are passionate about the RRT and provided feedback to staff on the RRT program, frontline staff involvement in implementation activities, training prior to launching innovations in organizations, and am emphasis on learning during RRT calls differentiated SUS-H and SUS-L hospitals. Also, program champions’ actions and behaviors are more important for sustainability than their positions in organizations.

Abbreviations

RRTs

Rapid response teams

RRTM

Rapid response team members who respond to RRT calls

RRT-EUs

Clinicians who work on units where RRT calls may be activated for patients in need

PMOS

Planning Model of Sustainability

SUS-H

Hospitals in the highest quadrant of sustainability scores reported

SUS-L

Hospitals in the lowest quadrant of sustainability scores reported

High-1

SUS-H hospital with the highest sustainability score in the study

High-2

SUS-H hospital with the second highest sustainability score in the study

Low-1

SUS-L hospital with the highest low-sustainability scoring hospital in the study

Low-2

SUS-L hospital with the second highest low-sustainability scoring hospital in the study (also the hospital in the study with the lowest sustainability score)