A Statewide Consortium’s Adoption of a Unified Nursing Curriculum: Evaluation of the First Two Years

Abstract

AIM

This article provides an evaluation of the first two years of implementation of a statewide nursing consortium (SNC) curriculum on nursing faculty work life, teaching productivity, and quality of education.

BACKGROUND

In response to the call for nursing education reform, the SNC incorporated new approaches to competency-based, student-centered learning and clinical education.

METHOD

Faculty and two cohorts of students were measured at three points over the first two years of the curriculum implementation.

RESULTS

The expected positive impact of the SNC was documented at the start of the first year, but not sustained. Students reported having more confidence in their clinical skills at the start of the first year, yet demonstrated significantly less confidence in their ability after two years.

CONCLUSION

Faculty indicated that the SNC allowed greater opportunity for collaboration, but that the experience did not alter their classroom performance or satisfaction beyond the first year.

Several landmark calls to action led a large flagship public university system, the University of Hawaii System, to transform the educational model for nursing in the state of Hawaii. A single baccalaureate nursing curriculum was implemented for university and community college students throughout the state to bolster faculty teaching productivity, improve faculty work life, increase student enrollment through an intervention to address workforce shortages, increase access to BSN education in the state, and enhance clinical learning. This statewide nursing consortium (SNC) sought to transform the educational environment based on an understanding of learning. Using the approach taken by the Oregon Consortium for Nursing Education (Herinckx, Munkvold, & Tanner, 2014; Munkvold, Tanner, & Herinckx, 2012; Tanner, Gubrud-Howe, & Shores, 2008), the SNC consisted of a common curriculum taught on all campuses (community college and flagship campus) for the first two years of a three-year BSN program and provided mechanisms for the seamless transfer of students to the university for the final year of the curriculum.

The SNC curriculum offers an innovative, integrated learning opportunity for nursing students throughout the state, regardless of their island of residence. The curriculum incorporates a new, competency-based pedagogy organized around foci of care, including concept-based and student-centered learning, spiraling curriculum, and case-based clinical educational redesign. The development of the SNC is detailed by Magnussen et al. (2013) and Niederhauser, Schoessler, Gubrud-Howe, Magnussen, and Codier (2012).

BACKGROUND

“A better use and preparation of nurses, including a mandate that 80% of all nurses should have baccalaureate preparation by 2020” was put forth as a goal in the Institute of Medicine’s Future of Nursing report (IOM, 2011). The Carnegie Foundation for the Advancement of Teaching (Benner, Sutphen, Leonard, & Day, 2010) also indicated that schools of nursing were lagging behind in responding to scientific and technological changes in the practice setting. In 2003, the IOM proposed a set of core competencies for inclusion in all health care professionals’ education in the 21st century. In addition, the Pew Commission urged health care professionals to take a critical look at what constituted a competent health care professional by using its 21 Competencies for Health Education in the 21st Century (O’Neil & Pew Health Professions Commission, 1998).

The purpose of this evaluation was to examine the overarching evaluation question: How does the SNC intervention enhance faculty work life (thus sustaining recruitment, retention, and productivity) and quality of education and promote productivity? The theoretical rationale for the productivity-job satisfaction relationship was grounded in expectancy-based and self-determination theories of motivation (Judge, Bono, Thoresen, & Patton, 2001).

In response to the demands for nursing education reform, this article explores: a) the impact of the SNC curriculum on faculty work life, teaching productivity, and quality of education among faculty members; b) the impact of the SNC curriculum on students’ confidence in performing nursing functions; c) the differential impact of the SNC program on work life, teaching productivity, and quality of education between faculty in programs that adopted this curriculum (the intervention group) versus those in programs that did not adopt it (the comparison group); d) the differential impact of the SNC curriculum on intervention and comparison group students’ confidence in performing nursing functions; and e) differences in how intervention and comparison program faculty spend time in their work-related professional activities.

METHOD

Design

A quasi-experimental design involving both quantitative and qualitative methodologies (mixed methods) was used to evaluate the impact of the SNC intervention. The quantitative analysis used a nonequivalent comparison group design with a baseline, end-of-year 1, and end-of-year 2 posttest. A qualitative evaluation involved key informant interviews with purposefully selected faculty participants from both groups to enrich interpretation of the outcome data.

Data Source and Sample

This evaluation is an analysis of faculty and student responses from two community colleges and one four-year university that admitted students into the SNC curriculum (the intervention). These responses were compared with responses from two other community colleges and one other four-year university that maintained their traditional curricular approaches (the comparison). This project was approved by the University of Hawaii Institutional Review Board; informed consent was obtained from all participants and codes were assigned to sites and participants.

For inclusion in the intervention group, each participant had to be teaching a course within the SNC curriculum (the faculty) or enrolled in SNC curriculum courses (a student). The inclusion criteria for faculty and students of the comparison schools were that they agreed to participate.

SURVEYS

Each fall, from academic year 2010–2011 (cohort 1) through 2011–2012 (cohort 2), all first-year students enrolled at the intervention and comparison schools were recruited on a rolling basis to participate in the student surveys. A total of 422 student surveys (intervention n = 296; comparison n = 126) were obtained over the two years. The students participated at baseline, at the end of year 1, and one year later.

Faculty at the intervention and comparison schools were invited to participate in the faculty surveys. A total of 44 intervention and 26 comparison faculty completed surveys during the same three data collection times. Table 1 shows the survey response rates for the faculty and student groups. The total student response rate ranged between 38 percent and 42 percent. The total faculty response rate ranged between 48 percent and 56 percent.

Table 1.

Participants and Survey Response Rates

	Baseline (N)*	Baseline Cohort 2 (N)**	End Year 1 (N)*	End Year 1 Cohort 2 (N)**	End Year 2 (N)*	End Year 2 Cohort 2 (N)**	Total Observations (N)
Faculty
Intervention Group
Total	26		26		26		78
Completed Surveys	15		13		16		44
Response Rate (%)	58%		50%		62%		56%
Comparison Group
Total	18		18		18		54
Completed Surveys	13		9		4		26
Response Rate (%)	72%		50%		22%		48%
Students
Intervention Group ^
Total	131	103	131	103	131	103	702
Completed Surveys	65	65	56	47	47	16	296
Response Rate (%)	50%	63%	43%	46%	36%	16%	42%
Comparison Group ^, #
Total	84	27	94	27	94	–	336
Completed Surveys	48	18	34	10	16	–	126
Response Rate (%)	51%	67%	36%	37%	17%	–	38%

^*Data collection: Baseline (Fall 2010); end of Year 1 (Spring 2011), and one year later posttest (end of Year 2, Spring 2012)

^**Data Collection: Cohort 2 admitted in Spring of each academic year. Baseline (Spring 2011); end of Year 1 (Fall 2011), and one year later posttest (end of Year 2, Fall 2012)

^{^}No Spring admission for 1 university and 1 community college (intervention group) and 1 community college (comparison group)

^#Surveys not administered per school’s request (program undergoing accreditation preparation)

The data were collected by administering hard-copy surveys to students in a classroom and to faculty at a meeting. Participants were given a survey packet at baseline. Subsequent data collection was done with the same methodology at the end of years 1 and 2. A customized electronic survey link provided reminders for nonrespondents or absentee participants. At least three reminders were emailed to nonrespondents to improve response rates (Dillman, 2007). Faculty who consented to participate received a $50 gift card; students who consented received a $10 gift card.

All but one of the faculty (n = 43) and 77 percent of the students (n = 324) were female. Faculty averaged 52 years of age (SD = 9.5), with a range from 28 to 67 years; students averaged 29 years of age (SD = 8.6), with a range from 19 to 65 years. Asians, overall, were the largest racial group, constituting 16 percent of the faculty and 42 percent of the students, followed by Caucasians (71 percent faculty, 24 percent students), two or more races (3 percent faculty, 16 percent students), and Native Hawaiian and Pacific Islanders (6 percent faculty, 13 percent students). Fifty-seven percent of the student respondents were enrolled in community colleges; 43 percent were enrolled in four-year universities.

KEY INFORMANT INTERVIEWS

Interviews with faculty were conducted with two randomly selected subsamples from the intervention (n = 7) and comparison (n = 5) groups. The selected faculty were engaged in individual interviews at baseline and at the year 1 follow-up. Data collection consisted of digitally recorded telephone or Skype open-ended and semistructured interviews lasting 30 to 90 minutes and addressing faculty work life, teaching productivity, and quality of education.

TIME ANALYSIS

Using state-of-the-art technologies that were tested in a landmark nursing time-and-motion study (Hendrich, Chow, Skierczynski, & Lu, 2008), this evaluation assessed a subsample of intervention (n = 15) and comparison (n = 14) faculty’s activities during their working day. Three periods over one semester were sampled: at the start of the semester, at mid-semester, and at the end of the semester. Activities and their superordinate categories were determined through interviews and focus groups involving participating faculty. (See Table 2.)

Table 2.

Time Study Activity Categories and Examples

Category and Example(s)	Description
PREPARATION:	Any activity that is considered to be preparation for didactic teaching and/or clinical teaching or supervision activities.
Examples:	Curriculum and course development, online course preparation, simulation case study prep, preparing cases developed for illustration of concepts, concept-based learning activities, video “pods,” clinical preparation, supervision and related meetings, verifying clinical site requirements, or course group meetings
TEACHING	The act of providing instruction in a variety of contexts.
Examples:	Conducting classes, case discussions, student group work; developing teamwork; proctoring exams; didactic material/mini-lectures
PROFESSIONAL DEVELOPMENT	Any activity related to maintaining standards of the nursing profession.
Examples:	Keeping up-to-date with training, journals, or professional news; maintaining licensure/certification; contract evaluations; continuing education
FACULTY PRACTICE	The performance of clinical practice that is NOT a part of the faculty role.
Examples:	Providing patient care, functioning as a patient educator, providing administrative services, functioning as consultant or researcher
CLINICAL INSTRUCTION	The performance of the faculty instructional role at a health care setting.
Examples:	Pre- and postconferences, clinical assignments, onsite reports and consultation with staff, orientation/student integration to site, content expert/facilitator for staff, troubleshooting problems, monitoring students, and evaluating/signing charts
COLLEGE/UNIVERSITY SERVICE	Activities that contribute to a high-quality nursing program, department, and university as well as the betterment of the community and professional nursing community.
Examples	Engaging in new faculty recruitment, mentoring new faculty, peer evaluations, student advising, thesis and dissertation committee work, meetings, community board services, volunteer services, community-based student/faculty volunteering
RESEARCH	Any activity connected with research other than those done primarily for teaching purposes.
Examples	Reading research/literature reviews, thinking/reflecting, developing proposals or grants, collecting data, analyzing data, reporting research results, implementing a study, administering experiments, grant management
EVALUATION OF STUDENT PROGRESS	Evaluating student progress, curriculum, and programs.
Examples	Analyzing finished or graded assignments, using computerized systems for test or exam scoring and analysis, clinical observation, nd standardized level testing
STUDENT SUPPORT	Facilitation of/for students.
Examples:	Writing letters of recommendation, facilitating struggling students, reducing student stressors, providing accommodations to students with special needs, remediation
TECHNOLOGY	Using technology to accomplish curricular activities.
Examples:	Learning technology foundations, using classroom tools (PowerPoint, Skype, Typhon, Clinical Decisions Support, Clickers, Collaborate, YouTube), podcasting or blogging, clinical setting tools (Simulation, Second Life, Standardized Patients, or Web-based videoconferencing), adjusting to IT system changes or teaching with technology
PAPERWORK/CLERICAL	Clerical activities in general.
Examples	Writing thank-you notes, getting gifts for clinical site personnel, general clerical activities, emailing, and other communication strategies
PERSONAL ACTIVITIES	All activities not related to professional role(s) or responsibilities. If work is stopped for longer than 15 minutes, especially to attend to non-work-related needs (other than illness or accident), then personal time should be recorded.
Examples:	Sleeping, eating, self-care
OTHER ACTIVITIES	Any activity that cannot be classified within the other nine categories.

Faculty participants were provided with an iPod Touch to record their activities at 24 randomly determined points during their waking day. When the alarm sounded, participants used a form developed with FileMaker Pro 11 to report their activity, location, and the expected duration of the activity. Participants received an additional $100 gift card or were given the iPod Touch upon completion of the data collection.

Variables and Measures

The measures were designed to collect information regarding the impact of the SNC curriculum on expanding the teaching capacity of nursing faculty. Table 3 depicts the three domains addressed, the variables, and the associated measures.

Table 3.

Specific Domains, Variables, and Measures

Domain	Variables	Measure and Description
Faculty Work Life	Role satisfaction	NLN Faculty Role Satisfaction Survey (NLN, 2005) : measure of individual, institutional, and leadership factors affecting nursing faculty work satisfaction and productivity.
	Collaboration and collegiality	Quantitative: two questions written specifically for this evaluation, asking about faculty satisfaction with these aspects of the program (Niederhauser, 2010). Qualitative: open-ended questions in the key informant interviews to uncover faculty perceptions of the ways in which innovative or standard educational practices are being enacted, how perceptions impact engagement in and commitment to nursing education, and topics/issues not covered by the quantitative measures (Niederhauser, 2010).
	Burnout	Oldenburg Burnout Inventory (Demerouti, Bakker, Nachreiner, & Schaufeli, 2000): 14-item measure of work-related burnout with two core dimensions of burnout: exhaustion and disengagement (from work). Inventory has been used extensively with nurses and health professionals.
	Life satisfaction	Satisfaction with Life Scale (Diener, Emmons, Larsen, & Griffin, 1985): 5-item instrument designed to measure global cognitive judgments of satisfaction with one’s life.
Teaching Productivity	Allocation of faculty time	Time Spent on Student Learning Activities and Course Preparation (Time Study): amount of time spent on student-centered learning activities and preparing for courses; allocation of faculty time to student-centric tasks and learning versus administrative, course development, or management time.
Quality of Education	Student confidence	Breadth of Education Survey (Evaluating Innovations in Nursing Education, 2010): measure of the confidence students have in performing 43 care tasks selected to exemplify competencies acknowledged as central to nursing education, based on the AACN Essentials of Baccalaureate Education (2008), the National Council of State Boards of Nursing, and Nurse Executive Center.

FACULTY WORK LIFE

A key factor in saving faculty time is the development of instructional materials that are shared among consortium faculty and students (e.g., student-ready case studies, simulation scenarios, group discussion guides, reference lists with study guides, and clinical learning activities). Increased access to shared campus resources allows faculty to spend time on significant learning activities. Faculty work life was measured by role satisfaction, collaboration and collegiality, burnout, and life satisfaction.

TEACHING PRODUCTIVITY

The SNC is engaging in a change that is transformative in nature and represents a high degree of collaboration among associate- and baccalaureate-degree nursing faculty. It requires the overcoming of geographic barriers and learning tool limitations experienced by smaller or rural campuses. The curricular changes may ensure increased teaching productivity and a better work life for faculty. Sharing expertise and teaching tools and methods across campuses allows faculty to spend more time on student-centered learning activities rather than course development activities. Teaching productivity was measured by the allocation of faculty time to student-centered learning activities and preparation for courses and student-centric tasks and learning versus administrative, course development, or management time.

QUALITY OF EDUCATION

As nurse faculty adopt a learner/outcome-centered education model in place of the outdated teaching/content-centered model, it is important to assess whether the teaching and learning activities resulting from collaboration improve student outcomes. Quality of education was measured as student confidence by using the Breadth of Education Survey (Evaluating Innovations in Nursing Education, 2010).

Data Analysis

At the time of this analysis, data were available for the first two years of the curriculum implementation.

SURVEYS

The SPSS version 19 statistical package was employed. Propensity score matching was used to approach equivalence between the intervention and comparison groups (Rosenbaum & Rubin, 1985). Propensity scores were computed using demographic and work-related (for faculty participants) characteristics (e.g., age, sex, race, degree, years of teaching, years in nursing). When practical, propensity scores were used to create five hierarchically ordered participant groups, or strata, of equal size. These propensity score-derived groups were then matched between the intervention and comparison groups, while outliers with no cross-group matches were discarded.

Responses of faculty in the intervention group were compared using repeated-measures analyses of variance, with baseline and one-year follow-up administrations as the repeated measures. This allowed for inspection of changes resulting from the new curriculum’s implementation. In the following analyses, eta squared (η²) was used to estimate effect sizes for multivariate and univariate analyses of variance, interpreting values of .01 as small, .059 as medium, and .138 as large. Missing values were replaced with the mean of the remaining items when there were four or fewer missing values. Responses of students in the intervention group were similarly analyzed. In cases where several items were individually analyzed with repeated-measures analysis, a Bonferroni correction for experiment-wise error was used to reduce the conventional level of significance (p < .05).

The differential impact of the SNC program on faculty members in programs that have and have not adopted this curriculum was analyzed using a one-way repeated-measures multivariate analysis of covariance (MANCOVA). Group assignment (intervention and comparison) was the main factor tested, with time of survey administration (baseline and follow-up) being the repeated measure. Propensity scores, used as a covariate, were obtained through logistic regression using demographic and work-related characteristics of faculty as predictors and group assignment (dummy coded with 0 for comparison and 1 for intervention group membership) as the dependent variable. Predictors with insufficient variability, redundancy with other predictors, and inability to predict group membership at p < .50 were eliminated from the final model solutions.

The analytic model for determining the existence of a differential impact on intervention and comparison program students’ confidence in performing nursing functions was analyzed using a two-way repeated-measures analysis of variance (ANOVA). Group assignment and propensity score strata were the main factors tested, with time of survey administration (baseline and follow-up) as the repeated measure. This allowed for inspection of any changes occurring for the intervention and comparison students over the year following implementation of the new curriculum.

KEY INFORMANT INTERVIEWS

Interviews were transcribed verbatim. Qualitative data analysis software NVivo 9 assisted with data management. The interview guide addressed faculty work life, teaching productivity, and quality of education. Saturation was reached after completing the 12 interviews. The process of inductive content analysis was used. Constant comparative analysis with no a priori codes was used. Text analysis was conducted, with an independent coder providing secondary verification of saturation. The interviews were recoded into descriptive themes. A final summary of themes was sent to all participants to verify their experiences (Lincoln & Guba, 1985).

TIME ANALYSIS

SPSS version 19 was used to compare a subsample of the intervention and comparison faculty activity categories during the working day. Chi-square effect sizes were estimated post hoc using phi (ϕ) and interpreting values of .10 as small, .39 as medium, and .50 as large.

RESULTS

Impact of the SNC Curriculum on Faculty

For the intervention faculty (n = 44), repeated-measures analyses demonstrated that burnout, life satisfaction, and collaboration and collegiality showed significant changes from baseline to follow-up. More burnout was reported on the Oldenburg Burnout Inventory (F [1,35] = 13.76, p < .001, η² = .282) and less collaboration and collegiality both within their programs (collaboration: F [1,35] = 6.75, p < .05, η² = .162; collegiality: F [1,35] = 12.1, p < .001, η² = .257) and across other entities in their colleges or universities (collaboration: F [1,35] = 4.21, p < .05, η² = .107; collegiality: F [1,35] = 4.12, p < .05, η² = .105). Satisfaction with life did not show any differences over time.

On the National League for Nursing (NLN) survey (2005), only one of 55 items achieved an adjusted significance level. Intervention faculty reported in the first-year follow-up that they felt less confidence in the direction in which the profession of nursing was headed than at baseline (F [1,36] = 13.01, p < .0001, η² = .266).

Impact on Students’ Confidence in Performing Nursing Functions

Responses of intervention students (n = 440; follow-up data were only available for 162 participants) to the Breadth of Education Survey (BOE) were analyzed using repeated-measures analyses of variance. The BOE revealed a sharp increase in students’ confidence in performing nursing tasks from baseline to the first-year follow-up (F [1,161] = 3785.01, p < .0001, η² = .959). A repeated-measures multivariate analysis of variance (without replacement of missing values) was then performed on all 43 BOE items to determine which individual items showed the most change. The change from baseline to follow-up was significant (F [43,97] = 6.1, p < .0001, η² = .730). Subsequent univariate analyses showed that students’ confidence in all 43 items showed a significant increase at the p < .001 level or better.

Differential Impact on Faculty Work Life

Eleven demographic and work-related characteristics were entered using backward regression. Using the propensity score as the predicted probability of membership in the intervention group for each individual, predictors with insufficient variability, redundancy with other predictors, and inability to predict group membership at p < .50 were eliminated from the final model solution. The five final predictors that met these criteria were older age, fewer years as a nurse faculty member, not having a doctoral degree, having an advanced practice nurse certification, and not having a plan for achieving academic career goals. It was impractical due to a small comparison group to create five strata using propensity scores; thus, propensity scores were used as covariates in these analyses.

The Oldenburg Burnout and Satisfaction with Life scales and the collaboration and collegiality measures, separately, were analyzed using multivariate, repeated-measure analyses of covariance. None of these measures reached multivariate or univariate statistical significance for their main effects or interactions. Across survey administrations, there were no differences on these measures between intervention and comparison faculty.

Thirteen MANCOVAs were performed on the 55 NLN survey items. NLN items were grouped for MANCOVAs based on their conceptual similarity. No main effects for group or time reached significance. One analysis reached significance for a group-by-time interaction (F [3,45] = 3.48, p < .05, η² = .188). The analysis for “I have confidence in the current direction in which the following are headed” had the “profession of nursing,” “my college or university,” and “my department or school of nursing within my college or university.” Univariate analyses on each item showed a significant group-by-time interaction for “my college or university” (F [1,47] = 5.07, p < .05, η² = .097), with comparison faculty showing a sharp decline in confidence while intervention faculty showed no substantive change from baseline to follow-up.

Key Informant Interviews

Overall, the intervention faculty participants felt productive in their teaching, were pleased with the quality of the educational product delivered, and were satisfied with their work lives. They felt that the planned approaches to team teaching and other activities created the perfect environment for increased collaboration and collegiality. In fact, this was a hallmark of the program, bounded not just by a department or college, but across the university system. Given the longer period of time in which they were able to implement the new curriculum and teaching modules, they were more comfortable with the program at the follow-up interview. Concern over an increased workload was not a major finding, as it had been at the baseline interviews. At both interview periods, all intervention interviewees were satisfied with the demonstrable increase in active and critical student learning. Some expressed delight at the level of their own learning as well.

The comparison group faculty also felt productive in their teaching, were pleased with the quality of the educational product they were delivering, and were satisfied with their work lives. They perceived their colleagues to be collegial and provided examples of their collaborative projects, including instances of team teaching. All comparison group faculty participants were satisfied with their current work situations. Naturally, given their independent statuses, their collaboration and collegiality resonated within their department or college. Some collaboration and collegiality were noted with peers across the university system, but most examples were department or college specific.

Differential Impact of SNC On Students’ Confidence in Performing Nursing Functions

Responses of student participants in the SNC curriculum adoption group (intervention, n = 440) and in the comparison group (n = 249) to the BOE were analyzed using repeated-measures analyses of variance with baseline and one-year follow-up administrations as the repeated measures. (Follow-up data were only available for 162 intervention participants and 89 comparison participants.) Propensity scores were obtained through logistic regression as previously described. Sixteen demographic and work-related characteristics were entered using backward regression. The 12 final predictors that met these criteria were younger age, being female, not intending to continue toward the BSN degree, and being a mixture of eight different races or ethnicities (being black or white; not being native Hawaiian, Hispanic, American Indian, Japanese, Korean, or Vietnamese).

Five strata, from lowest to highest propensity scores, were created. The BOE revealed a sharp increase in students’ confidence in performing nursing tasks for both groups and across all strata from baseline to the first-year follow-up (F [1,236] = 338.99, p < .0001, η² = .590). There were no significant main effects for group (F [1,236] = .053, ns) or strata (F [4,236] = .091, ns) nor their interaction (F [4,236] = .824, ns). There were significant effects for the group by time (F [1,236] = 16.97, p < .0001, η² = .067) and strata-by-time (F [4,236] = 2.51, p < .05, η² = .041) interactions, but their effect sizes were quite small. The group by strata by time interaction was not significant (F [4,236] = .332, ns). The time-by-group interaction increase for the comparison group was sharper than for the intervention group.

Subsequent univariate analyses showed that comparison students had significantly sharper increases in confidence over time than the intervention students for all 43 items. Generally, the first and second strata showed sharper increases than the fifth stratum for those 19 of 43 items that reached significance.

Work-Related Faculty Professional Activities

In wave 1 (semester start), the two groups differed significantly in activities (χ² [10] = 68.6, p < .0001, ϕ = .306). Intervention group participants were more likely to report engaging in college and university service, faculty clinical practice, teaching, and other educational activities than did those in the comparison group. Conversely, the comparison group reported more engagement in clinical instruction, paperwork, personal, teaching preparation, and research and professional development activities than their counterparts. The groups also differed with regard to the locations for their activities (χ² [2] = 32.8, p < .0001, ϕ = .213), with intervention participants more likely to be in school or some other place and comparison participants more likely to be at home at the time of the event.

In wave 2 (mid-semester), the two groups differed significantly in activities (χ² [10] = 56.1, p < .0001, ϕ = .286). Intervention group participants were more likely to report engaging in paperwork, teaching preparation, and teaching than did those in the comparison group. Conversely, those in the comparison group reported more engagement in clinical instruction, personal, research and professional development, and other educational activities than their counterparts. The groups also differed with regard to the locations for their activities (χ² [2] = 32.9, p < .0001, ϕ = .222), with intervention participants more likely to be in school or some other place and comparison participants more likely to be at home at the time of the event.

In wave 3 (semester end), the two groups differed significantly in activities (χ² [10] = 39.9, p < .0001, ϕ = .235). Intervention group participants were more likely to report engaging in paperwork, student support, and other educational activities than were those in the comparison group. Conversely, those in the comparison group reported more engagement in preparation and research and professional development activities than their counterparts. The groups did not differ with regard to the locations for their activities (χ² [2] = 4.3, ns).

DISCUSSION

Intervention faculty reported increased burnout and decreased collaboration and collegiality at the one-year follow-up assessment. They also expressed less confidence in the direction in which the profession of nursing was headed. While these statistically significant results had small to medium effect sizes, they should not be minimized. Systemic change brings with it a period of disturbance, and these findings may well be an expression of that disturbance. In addition, at each time period, there were no differences between the intervention and comparison faculty groups in burnout or collegiality and collaboration, suggesting that the implementation of the SNC curriculum did not significantly impact burnout, collegiality, and collaboration among the intervention faculty members.

An increase in confidence for the comparison group students was sharper than that for the intervention group over the first two years. Comparison students had more experience with specific nursing tasks. These findings are suggestive of the Dunning-Kruger effect (Dunning, Johnson, Ehrlinger, & Kruger, 2003), which is a cognitive bias wherein inexperienced individuals experience an illusory superiority, mistakenly rating their ability higher because of the inability of the inexperienced to recognize their mistakes. Implications call for addressing a student’s self-confidence as a component of effective nursing education. Effect sizes for these analyses were remarkably strong. As expected, beginning students gained confidence when performing tasks.

No substantive differences were found from baseline to first-year follow-up between the intervention and comparison groups in analyses of the major study survey measures. The only statistically significant group-by-time effect was that the comparison group had a sharper decline than the intervention group in their confidence in the direction their nursing program was taking. The effect size for this analysis was quite small. These findings do suggest that there is no immediate positive impact of the curriculum change.

Other nursing education programs contemplating such changes might be warned by these findings not to promise that there will be an immediately noticeable positive impact for faculty upon adoption of curricular revisions. In the interviews, intervention faculty admitted to some initial dissatisfaction, but then proclaimed increasing comfort with the new curriculum and the enhancement of collaboration among their colleagues. Comparison faculty, in contrast, reported great satisfaction with their work lives as well as rewarding colleague collaborations. One explanation for this discrepancy may well be that the follow-up interviews were conducted toward the end of the second year of the evaluation while the surveys for the first cohort (the largest of all four) were administered closer to the start of the second year. Perhaps these interviews, at least for the intervention faculty, reflect a settling in with the curriculum revisions.

Analyses showed that both groups of students showed increased confidence in performing nursing tasks from baseline to first-year follow-up, but that the increase for the comparison group was stronger. However, the effect sizes for the intervention versus comparison differences are small. This finding is consistent with the differences between the SNC and traditional curricula. The SNC curriculum emphasizes problem solving and meta-skill building, while the more traditional curriculum focuses on the development of specific, demonstrable skills. It would be expected for comparison students to outpace intervention students with regard to this measure at the initial stages of training, as they likely have had more direct, hands-on experiences.

Limitations

First, as noted above, the data presented here are incomplete. There are data from two cohorts that have not yet been fully integrated into the evaluation dataset. While the addition of data may change some of the conclusions drawn here, it is unlikely to do so because somewhat over 75 percent of the potentially available data for the follow-ups have already been collected.

Second, the intervention under study, implementation of a revised curriculum and didactic plan, is extraordinarily complex and required a radical shift by most participating faculty from the usual ways of educating nursing students. Any such systemic change brings with it a period of uncertainty, confusion, dissatisfaction, and adaptation. It is likely that this colors some of the short-term findings. A truer picture of the impact of the SNC curriculum will likely be obtained in later follow-up assessments.

Third, the management of the faculty and student participants and in-person administration of the self-report instruments was far more complex and costly than had been anticipated because the study sample was dispersed over four islands comprising the SNC. Fourth, aside from basing the selection of the measures on the reliability and validity values reported in the literature, no psychometric analysis was performed. Independent evaluation of content validity by members of the project and evaluation teams was used as a part of the measures selection processes. Lastly, the effect sizes were not calculated.

CONCLUSIONS

Collectively, the findings from this evaluation reveal a complex picture of change promoted by the implementation of the SNC curriculum. SNC faculty reported increased burnout and decreased collaboration and collegiality from baseline to follow-up in their survey responses, but key informant interviews with a small sample of these faculty members indicated the opposite effect. Analyses comparing intervention and comparison faculty showed no meaningful differential change in the study measures over time, including burnout and collegiality and collaboration.

While both groups of students increased their confidence over time (BOE), increases in comparison students’ confidence were stronger. The implementation validation study findings were equally complex, showing that differences between intervention and comparison faculty in their activities were not consistent over the course of the semester. The expected impact of the SNC implementation in increasing intervention faculty involvement in students’ educational and training activities was found at the start of the semester, but not at later points in time.

One limiting factor to drawing conclusions from these findings is that complete study data were only available for the first year follow-up. Continued follow-up of both faculty and student participants through the fifth year is important to determine the ultimate trajectory of this innovation and document its sustainability.