Longitudinal Outcomes of an Institutionally Developed Nurse Residency Program

Abstract

Nurse residency programs are widely implemented to enhance integration of new graduate nurses entering the workforce. This article presents a retrospective analysis of 10 years of residency data from an internally developed residency program that utilized the Casey-Fink Graduate Nurse Experience Survey. Outcomes of this program were similar to those from studies using commercially available products, suggesting an internally developed residency curricula may be equally beneficial to the development of new graduate nurses.

Nurse residency programs are an evidence-based approach to support newly-graduated nurses upon entry to the nursing workforce. The Institute of Medicine recommends the development and implementation of residency programs to support nurses’ transition to practice (1). Many institutions utilize commercially available residency program curricula, the results of which have been widely reported in the literature (2–4). These articles frequently utilize the Casey-Fink Graduate Nurse Experience Survey© (5) (Casey-Fink Survey) to measure outcomes related to professional comfort and confidence, specifically of new graduate nurses (6–9). In addition, retention data is used as a benchmark for the success of residency programs. Few articles have presented longitudinal results related to an internally developed residency curricula, particularly in a specialty care setting. Evaluating the outcomes of such a program can further inform the use of individually developed residency curricula, particularly across specialty care settings.

Building upon longitudinal evidence related to commercially available residency curricula this article presents outcomes from an institutionally developed residency program at a National Cancer Institute (NCI)-designated comprehensive cancer center. Data from the Casey-Fink Survey (5), administered at the beginning and completion of the 12 month residency program at this institution, is analyzed along with participant retention rates at 1, 3, and 5-years post hire. This data suggests that an internally developed residency curricula may be equally effective in supporting the growth, development, and retention of new graduates.

Overview of Nurse Residency Programs

A nurse residency program is defined as evidence-based curricula that deliver content on leadership, patient safety, and professional development as new nurses enter into practice (10). Programs range from 4 to 12 months and may be presented using web-based, commercially available, or internally developed programs (6, 11). Residency programs traditionally consist of 2 focal areas, clinical orientation, which addresses the training of the new nurse in the clinical practice setting, and entry to practice content, which addresses professional development, concepts, and skills. The fundamental components of a residency program include socialization, reflection, delegation, quality, evidence-based practice, and conflict management (12). Accreditation for residency programs is available through the Commission on Collegiate Nursing Education (CCNE) or the American Nurses Credentialing Center® (ANCC). Few residency programs are currently accredited which allows for the variation in program structure, content, and outcomes (13). Residency programs may also serve as complementary to the comprehensive clinical orientation program and include components of leadership, workplace resources, and socialization (8).

Residency programs have been linked with enhanced retention rates at 1 year (4, 14). According to nursing literature, the cost of hiring and training a new nurse ranges from $60,000 to $ 96,000 (15) making the retention of the successfully trained nurse a fiscal and operational priority. According to the Texas Center for Nursing Workforce Studies, the Texas median facility turnover is 23.6% for registered nurses (RNs) (16). Most of the existing literature focuses on the 1 or 2 year retention rate for nurse residents suggesting short-term retention rate improvement (17–19). One year retention rates of > 90% following the implementation of standardized residency curricula are reported in the literature (2). Another 10-year study reports a 60 month (5 year) retention rate of 60.2% versus 92.9% at 12 months (3).

Literature describing residency programs and program implementation is readily available, yet there is limited research that evaluates longitudinal outcomes of internally developed programs. As new graduates enter the nursing workforce in increasingly diverse and often highly specialized practice settings, the ability to provide effective, evidence-based curricula for their clinical and professional development, tailored to the practice setting, is important to nurse satisfaction and retention outcomes. While the fundamental components of a residency program identified above are development skills consistent across practice settings, how these skills are acquired and reinforced can be personalized to the distinct patient population and culture of an individual institution. An opportunity was identified to evaluate 10 years of Casey--Fink Survey (5) data from new graduate nurse participants in an institutionally developed residency program.

Institutional Background

At this institution, a residency program for new graduate nurses was developed to be a separate but complementary component to the formal clinical nursing orientation program. Established in 2005, the residency program offers comprehensive curricula including leadership skills and support designed to build confidence, enhance professional relationships, and to assist in the transition into the role of RN. At the time of the program development, there were limited commercial programs available, therefore a customized program was created (8). The Chief Nursing Officer (CNO) along with program leadership, selected the Casey-Fink Survey (5) as one of several evaluation tools for the residency program. The program defines a graduate nurse as an individual with < 12 months experience as an RN or a nurse who has never worked in a U. S. hospital. These individuals are assigned to cohorts, ranging in size from 19 to 71 participants, based on hire date. They attend an 8 hour, paid education day once a month over 12-months, focused on supporting transition to professional nursing, the original curriculum design for which was presented by Keller et al (8). The CNO led an evaluation to ensure the program is aligned to National Council of State Boards of Nursing (NCSBN) Transition to Practice Model (12) and the CCNE Standards for Accreditation of Entry to Practice Nurse Residency Programs (20), which emerged since the program’s original development. The program was enhanced to include concepts for the patient population and professional practice environment of a comprehensive cancer center. Unique elements include oncology specific content, such as oncologic emergencies and end-of-life care, and an emphasis on compassion fatigue. Additionally, simulation with patient volunteers treated at this institution provides residents real time feedback on patient-nurse interactions.

Purpose

The purpose of this article is to present a 10 year retrospective review of outcomes from an internally developed nurse residency program. Data collected using the Casey-Fink Survey (5) and institutional retention metrics from over 1000 participants were analyzed, and reviewed in comparison with published longitudinal outcomes of studies using commercially available residency curricula.

Methods

The request for data collection and retrospective analysis was reviewed and approved by the institutional Quality Improvement Assessment Board (QIAB), which is responsible for the review and approval of all quality focused initiatives and data analysis as a complement to the institutional review board process. With appropriate permission obtained, the Casey-Fink Survey (5) was administered to nurses participating in the residency program beginning in 2005 through 2015. The survey was administered twice, at the beginning of the program to provide baseline data for each cohort, and at the completion of the program. The survey was initially administered on paper, and later through an electronic survey program, and was anonymous and voluntary. Retention data were collected in collaboration with the human resources department.

Instrument

The Casey-Fink Graduate Nurse Experience Survey© consists of 41 questions divided into 5 sections: demographics, skills and procedure performance, work environment and role transition, job satisfaction, and comfort and confidence (5). Two sections consist of open-ended questions, 1 addressing skills and procedure performance (3 open-ended questions) and 1 addressing work-environment and difficulties in role transition (4 open-ended questions). One additional section addresses job satisfaction (9 items).

The section on comfort and confidence (24–items with Likert scale response with 1 = strongly disagree and 4 = strongly agree) has been tested for and demonstrates internal consistency reliability (Cronbach’s alpha =.78). This section consists of 5 factors, namely support (α =.90), patient safety (α =.79), stress (α =.71), communication/leadership (α =.75), and professional satisfaction (α =.83). Higher scores are positive for all domains except stress, where lower scores reflect lower reported stress. The instrument may be administered on paper or electronically and takes approximately 15 to 20 minutes to complete.

Sample

The sample consisted of individuals with < 12 months experience as a RN in a hospital in the U.S. who participated in the institutional nurse residency program. Participants represent a convenience sample of individuals who participated in the residency program and completed the surveys.

Setting

All participants were enrolled in the residency program at a comprehensive cancer center. Participants were assigned to a diversity of practice settings, including, but not limited to hematologic, surgical and medical oncology units, intensive care perioperative units, and the emergency center.

Data Analysis

Data were prepared in an electronic data management spreadsheet program. Frequencies and percentages were calculated for demographic characteristics and described for the pre and post cohorts separately. Summary statistics were calculated for the Casey-Fink Survey subscales (5) over time. Graphical plots were used to display possible trends over time. Cronbach's alpha was calculated for Casey-Fink Survey (5) items to assess reliability of the instrument. Frequencies and percentages were calculated for the Casey-Fink Survey (5) satisfaction items over time. Pre- and post- scores were compared as a whole using a 2-sample t-test. Graphical plots were used to display retention trends over time. Data were descriptively evaluated to identify trends in responses across cohorts. In order to assess the effect of Casey-Fink Survey (5) scores on retention rates, linear regression models were conducted with the dependent variable being the retention rate at the post assessment and the independent variable being the Casey-Fink Survey (5) scores at pre-assessment. Stata v14.1 (College Station, TX) (21) was used to conduct statistical analysis.

Results

A total of 1,638 completed surveys were included for analysis. A potential pool of 1,267 residents could have taken the initial survey and up to 1,118 residents could have taken the final survey. The difference of potential participants at baseline and program completion represents attrition due to NCLEX fail after actual employment start date (n=11) and individuals who left the organization before residency program completion (n=138). Since completion was anonymous data was analyzed in aggregate and pre-test scores were compared to post-test scores at the cohort, rather than the individual level. Results were reported across 31 cohorts from Summer 2005 to November 2014. Demographic data, collected for descriptive purposes only, is presented in Table 1.

Table 1.

Participant Demographics

Demographic	N	%
Age
20 – 25	313	56.6%
26 – 30	120	21.7%
31 – 40	83	15.0%
41 – 50	33	6.0%
51 +	4	0.7%
Ethnicity
Asian	140	26.3%
Black	115	21.6%
Caucasian	192	36.0%
Hispanic	65	12.2%
Other	21	3.9%
Gender
Female	504	91.1%
Male	49	8.9%
Nursing Program Completed
Local ADN	30	2.9%
Local Or Outside area BSN	1019	97.1%

Outcomes of the Casey-Fink Survey (5) comparing pre and post-participation scores across the 5 domains, specifically, support, patient safety, stress, communication/leadership, and professional satisfaction for this institution and as reported across longitudinal studies of commercially available products are presented in Supplemental Digital Content 1–3. Pre- and post-participation scores from this institution were compared across these domains using a 2 sample t-test (P-values < 0.05). Results indicate statistically significant change in scores in all domains except stress (p = 0.05). These results and those reported across longitudinal studies of commercially available products are presented in presented in Supplemental Digital Content 4. Retention rates at 1, 3, and 5-years post-hire date are presented for each of the cohorts over this 10 year period (Figure 1).

Figure 1.

Retention Rates by Cohort Across Time

Discussion

Demographics (Table 1) of the residents that completed the survey are representative of nursing staff at the institution. Of note, there is a high level of diversity and baccalaureate prepared nurses and less than 10% are age 41 plus. The lack of age variability is directly related to the focus on the new graduates of nursing programs and those with less than 1 year experience.

Casey-Fink Survey (5) data demonstrate a statistically significant change in score across all domains with the exception of stress, which approaches significance at 0.05. Communication/Leadership and Patient Safety scores demonstrated the most favorable improvement with an increase in mean score from 2.88 to 3.24 (p<0.001), indicating increased confidence and comfort in communicating with interprofessional colleagues, patients and families, and in delegating. The Patient Safety domain increased from 2.77 to 3.16 (p<0.001), indicating improved confidence in organizing and prioritizing patient care and comfort with safely completing the components of the patient care assignment. Communication and Patient Safety are key components of institutional and clinical orientation, as well as the residency program curricula, and the emphasis on content related to these domains is consistent with these findings.

Conversely, scores in the Support and Professional Satisfaction domains showed moderate, but statistically significant decline over the course of the residency program. Mean scores in the Support domain decreased from 3.36 to 3.29 (p=0.002) and in the Professional Satisfaction domain from 3.53 to 3.41 (p<0.001). One indication for the decreased score may be related to the transition to independent practice (without a preceptor) over the course of the 1st year of practice. Of note, the Support scores have been consistently higher on the post-scores since 2014 when the patient experience simulation was added, engaging actual patient volunteers in the simulation lab experience for residency participants.

The Stress domain of the Casey-Fink Survey (5) primarily addresses external stressors, including finances, personal life/relationships, student loans, living situation and childcare, aspects which the residency program and work environment may not necessarily influence, but certainly may compound. Stress responses ranged from 0.00 to 1.00, reflecting a low level of stress among residency participants. Changes in the mean scores of from 0.15 to 0.13 suggest a small decrease in stress that approached significance (p=0.05). In an effort to identify which of the areas of stress was most strongly influencing the residents an additional review of data was completed. The pre-data identified student loans and finances to be the highest stressors and the post-data identified personal relationships and finances to be the highest. Stress intervention programs may improve retention rates (22) but the data from this retrospective review does not clearly find a link between stress and retention rates. Job performance was identified as having the largest change from baseline (16.8%) to program completion (7.8%), indicating a more than a 50% decrease in stress related to job performance.

For the purposes of this analysis, retention was defined as the percent of residency participants who were retained to the institution, not necessarily the role. A regression model utilized to assess the effect of Casey-Fink Survey (5) scores on retention rates demonstrated one statistically significant relationship, that between responses on the Support domain and 1-year retention data. For every additional unit in support, 1-yr retention increased on average of 0.21 (p = 0.041), suggesting that increased perception of support may be related to improved retention at 1-year post-hire. Findings of decreased retention at 3 and 5-years may be reflective of trends related to mobility of the nursing workforce, as well as return to school for career advancement. Notably, this institution provides prepayment of educational expenses, and 18.4% of the nursing workforce is enrolled in advanced degree programs. Of the residency participants eligible for this financial support, up to 86% of graduates from each eligible cohort (> 18 months since hire date) were enrolled in academic programs for fiscal year 2016. These findings align with other reported long-term outcomes regarding residents pursuing advanced education (4, 23).

Of the articles identified that measured longitudinal outcomes from residency programs, 1 utilized the Versant® RN residency (3) and 2 utilized the University Health System Consortium/American Association of Colleges of Nursing (AACN) Residency (2,4). Programs were evaluated utilizing the Casey-Fink Survey (2), Nursing Competencies Rating Scale and the Nurse Job Satisfaction Scale (3), and McCloskey/Mueller Satisfaction Scale(4), as outcome measures respectively. Of these, trends in Casey-Fink scores were similar to those reported by Goode (2), however the stress scores on this data showed a remarkably low level of stress among residents at this institution. These findings suggest that an internally developed residency program, emphasizing entry to practice content as a distinct track from clinical orientation, may be equally effective to commercially available residency programs in supporting new graduates’ comfort and confidence in a new role and is consistent with findings from other reviews (19).

Outcomes related to retention are consistent with findings in the literature reflecting a high retention rate (> 90%) at 1 year (2, 3, 4). Similar to findings on the Casey-Fink Survey data (5), these outcomes suggest that an internally developed residency program may be equally effective as pre-packaged residency programs in supporting new graduates’ retention.

Limitations

There are several limitations to this data and the presented findings. Data were collected over a 10 year period with some variability in the questions asked. In addition, no pre-data were collected for the 1st residency cohort in summer 2005, and no post-data were collected for the January and February 2014 cohort. Therefore responses for every item were not reported across all cohort groups. Further, the residency program curricula evolved overtime to meet the changing parameters of the NCSBN guidelines, as well as emerging needs of the institution and its newly hired staff. Therefore each cohort experienced slightly different curricula making it difficult to draw conclusions about what aspects of the curricula were more or less impactful on respondents’ Casey-Fink Survey (5) scores. Data were collected in aggregate due to the anonymity required by the quality initiative project so pre and post scores were not linked. Pre- and post- scores were treated as independent cohorts and the paired nature of the data was not taken into account for the t-test. This limits the ability to report change in individual’s scores over the course of the residency program. In addition, Casey-Fink Surveys (5) were originally completed on paper and subsequently completed via an electronic survey link. Responses on paper were manually entered into an electronic data collection spreadsheet and could have been affected by human error, though multiple audits of the data by the 1st and 2nd author were conducted to reduce the likelihood of such error.

Implications for Practice

Results suggest that internally developed residency content may be equally effective as pre-packaged programming in supporting the comfort, confidence, and retention of new graduate nurses. While further research is necessary to more formally compare outcomes of commercially available versus internally developed programs, the data suggests the possibility of developing an effective program within an institution that can be further specialized to the practice setting. This may be particularly beneficial for specialty care settings that desire to integrate content about their unique population into both the clinical and entry-to-practice content of the residency program.

The evaluation of an internally develop program compared to a commercially developed program must be done thoughtfully. At minimum, the internal resources, costs, volume of graduate nurses and program sustainability must be considered when nurse administrators are determining the most effective path for their particular organization. While the internally developed program provides for greater flexibility and customization, that benefit must be weighed against the resources required for program maintenance.

Whether utilizing a commercially available or institutionally developed program, there is demonstrated benefit related to new graduate comfort and confidence in practice as well as retention. Ultimately, the nursing profession benefits as a whole from a highly educated and well-trained workforce. Implementing a residency program founded on the standardized guidelines of the NCSBN and CCNE, and enhanced with content and objectives specific to the population and practice setting of individual institutions contributes to strong retention at 1 year post-hire, and may contribute to improved institutional retention of residency program participants further in their career. This translates to cost-savings to and a more robust nursing workforce within institutions with well-established residency programs.

Nurse leaders exploring novel and innovative approaches to address the evolving needs of new graduate nurses may benefit from consideration of an internally developed program. These benefits include specializing training content to meet the needs of the practice setting, while remaining consistent with the fundamental requirements of the NCSBN and the CCE. Further, this paper provides a template for both content development, as well as measurement of outcomes. Future directions for consideration by nurse leaders include how to revitalize nurse residency programs to meet the competing demands of staff engagement, staffing, and fiscal responsibility.