Addressing Core Competencies Through Hospital Quality Improvement Activities: Attitudes and Engagement

Abstract

Background

Hospital quality improvement initiatives are becoming increasingly common. Little is known about the influence of these initiatives on resident learning and attitudes. Our objective was to assess whether training in a hospital committed to involving residents in hospital-initiated, continuous quality improvement (CQI), and to participation in such activities, would influence residents' attitudes toward CQI and engagement in the hospital community.

Methods

We surveyed Seattle Children's Hospital pediatric residents, from residency graduation years 2002–2009. We included questions about participation in quality improvement activities during residency and measures of attitude toward CQI and of workplace engagement. We used descriptive statistics to assess trends in resident participation in hospital CQI activities, attitudes toward CQI and workplace engagement.

Results

The overall response rate was 84% (162 of 194). Among graduated residents, there was a significant trend toward increased participation in CQI activities (P  =  .03). We found no difference in attitude toward CQI between those who had and those who had not participated in such activities nor between residents who began training before and those who began after the hospital formally committed to CQI. Sixty-three percent of residents (25 of 40) who participated in CQI activities were engaged in the hospital community compared with 53% (57 of 107) who did not participate in CQI activities (P  =  .21).

Conclusions

Training in a hospital committed to involving residents in CQI was associated with a high rate of participation in CQI activities. Although such training and participation in CQI were not associated with resident attitudes toward CQI or hospital engagement, it may allow residents to learn skills for practice-based learning and improvement and systems-based practice.

Background

The release of the Institute of Medicine's report Crossing the Quality Chasm: A New Health System for the 21 Century¹ in 2001 increased the focus of hospitals and health care systems on efforts to improve the safety and quality of health care. In 2002, the Accreditation Council for Graduate Medical Education (ACGME) initiated the Outcome Project to delineate 6 competencies expected of graduates of accredited residency programs.² Two competencies, practice-based learning and improvement and systems-based practice, required programs to enhance resident education and assessment in the areas of quality improvement and patient safety. The ACGME Common Program Requirements stipulate that “Residents must demonstrate the ability … to continuously improve patient care … [and] to systematically analyze practice using quality improvement methods, and implement changes with the goal of practice improvement.”^3(p1) In the systems-based practice competency, the standards require residents to “demonstrate an awareness of and responsiveness to the larger context and system of health care … [and] work in interprofessional teams to enhance patient safety and improve patient care quality.”^3(p3)

Residency programs across the country have implemented curricula to aid residents in achieving these competencies. A systematic review⁴ revealed that many curricula emphasize lectures and small-group work that lead to project implementation on a limited basis, if at all. Although one of the most common models of teaching quality improvement is rotations that include elements of continuous quality improvement (CQI) training,^4–6 there are significant barriers to defining quality problems and to designing and implementing CQI projects in residency programs, given the limitations of resources and time and the existing block-rotation model.

In 2003, the leadership at Seattle Children's Hospital (SCH) asked the pediatric residency program to become involved with the hospital's CQI work. The SCH had recently adopted the Toyota Production System as a primary method for improving efficiency and eliminating errors in their health care system.⁷ Residents were asked to participate in a series of events to assess and improve the delivery of care to patients and their families. Because these events were time-intensive, participation was encouraged, but voluntary.

Events included 5-day rapid-process improvement workshops, 1- to 3-day “design events,” development of standard order sets, and other CQI activities. These events were interprofessional and included attending physicians, nurses, residents, pharmacists, nutritionists, and families, among others. After a brief introduction to CQI principles, teams worked to solve various problems, such as implementing family-centered rounds as standard practice on the inpatient medicine service, reducing errors in total parenteral nutrition orders, reducing central-line infections, and increasing the rate of morning discharges from the inpatient medicine service. Resident participants were asked to disseminate the resulting changes through presentations at resident meetings and through informal conversations with colleagues.

We were interested in understanding how training in such an environment would affect a resident's attitude toward CQI and toward his or her engagement in the hospital community. Specifically, we hypothesized that training in a hospital committed to involving residents in hospital-initiated CQI would be associated with a more-positive attitude toward CQI and with greater levels of engagement in the hospital community.

Methods

In February 2007, we conducted an anonymous online survey of SCH pediatric residents with graduation dates of 2002–2009. This study was exempt from review by the SCH Institutional Review Board.

Recruitment

Graduates (2002–2006) were notified of the study by e-mail and by letters mailed to home or work addresses. Information was not available for 8 graduates. Current residents were notified of the study by e-mail and by a letter in their residency mailboxes. A $5 gift card was offered to all potential participants. The survey remained open for 1 month, and 3 e-mail reminders were sent.

Data Collection

The survey was an anonymous, web-based instrument, administered by Gallup, Inc (Washington, DC). Questions included graduation year, extent of participation in CQI activities during residency, the attitude scale from the Continuous Quality Improvement Questionnaire (CQIQ),⁸ and the Gallup Q¹² employee participation survey.⁹ We defined participation in hospital CQI activities as involvement in a rapid-process improvement workshop, a clinical-guidelines development, or a vision or design event. The CQIQ scale was composed of 5 statements about CQI and used a 3-point Likert scale (agree, undecided, disagree), with 3 points given for the response that indicates the most positive attitude.⁸ The Gallup Q¹² survey asks 12 questions that are specifically designed to measure workplace engagement.⁹ We asked graduates to answer questions based on how he or she felt during residency. Current residents answered based on current, ongoing experiences. The Gallup Q¹² is scored on a scale from actively disengaged to engaged, based on a proprietary algorithm. As such, scoring was completed for us by Gallup, Inc.

Data Analysis

We used descriptive statistics to assess trends in resident participation in hospital CQI activities, attitudes toward CQI, and workplace engagement. We defined acculturated residents as those who began their residency during or after 2003, the year SCH implemented a formal CQI method. Acculturated residents graduated between 2006 and 2009, whereas unacculturated residents graduated between 2002 and 2005. Participants who did not report their graduation year (n  =  5) were excluded from analyses requiring that data. Our sample size gave us greater than 80% power to detect a one-half standard-deviation difference between acculturated and unacculturated residents in the CQIQ scale.

We used χ² tests of trends to assess the changes in resident CQI participation over time. The χ² tests were also used to examine the association between CQI participation and hospital engagement. Wilcoxon rank sum tests were used to compare CQIQ scores between acculturated and unacculturated residents, as well as between residents who had and those who had not participated in CQI activities during residency. SAS version 9 software (SAS Institute Inc., Cary, NC) was used for all analyses.

Results

The overall response rate was 84% (162 of 194) but differed significantly between acculturated and unacculturated residents (91% [96 of 105] versus 74% [66 of 89]; P value =  .001).

Residents from the class of 2006 were the only acculturated residents to have completed residency at the time of the survey, and they had the highest rate of participation in CQI activities. The lowest rates of participation were found among the class of 2002 (20%, 2 of 10) and the class of 2009 (the first-year residents at 4%, 1 of 26) (table 1). Among the classes (2002–2006) that had completed residency at the time of the survey, there was a statistically significant trend toward increased participation in CQI activities (P  =  .03).

Table 1.

Survey and Continuous Quality Improvement (CQI) Participation Rates by Class Year

Twenty-five percent of respondents (39 of 157) had participated in at least one CQI activity during residency. Among respondents from the classes that had completed at least 2 years of residency since SCH began using CQI methodology (2005–2007), 42% (26 of 62) had participated in at least one CQI activity during residency, and more than 25% (16 of 62) had participated in more than one CQI activity. Additionally, 26% of all graduates (2002–2006; 21 of 81) reported participating in CQI activities since completing residency.

We found no statistically significant difference in the total scores on CQIQ attitude between residents who had and those who had not participated in CQI activities during residency. Similarly, there was no difference between acculturated and unacculturated residents in total score on the CQIQ (P  =  .92) or on any individual question (table 2). Most residents, whether or not they participated in CQI during residency, were highly engaged (63% [25 of 40] versus 53% [57 of 107], P  =  .21]. Engagement also did not differ significantly between acculturated and unacculturated residents.

Table 2.

Continuous Quality Improvement Questionnaire (CQIQ) Attitude Scale Responses by Acculturation Status

Discussion

Our study showed that training in a hospital committed to involving residents in CQI activities was associated with an increasing rate of CQI participation by residents. The peak of 50% participation by the class of 2006 is high for an optional residency activity, and most residents elected to participate in these CQI events during their second or third year when their schedules have more elective time. The low rate of participation in CQI activities among the class of 2009 likely reflects the relative lack of elective time available to first-year residents.

We expected to find that acculturated residents and those who had participated in hospital CQI activities to have more positive attitudes toward CQI and to be more engaged in the hospital community but found only a slight trend toward each. We believe the lack of significant differences likely reflects a ceiling effect from our overall positive CQI attitudes and high degree of engagement in the hospital community, compared with established means (D. Witters, MS, written communication, 2007).⁸

Interestingly, although both groups scored above the mean, the CQIQ attitude scale results were not overwhelmingly positive. Most notably, many acculturated and unacculturated residents were undecided or had negative attitudes about the degree to which CQI is driven by a desire to incorporate patient preferences. Similarly, many residents from both groups were undecided or had negative attitudes about the degree to which CQI affects physician autonomy. This may reflect actual concerns about CQI or a knowledge deficit regarding its key concepts. It also is possible that the concerns about patient preferences relate to training in a pediatric environment where patient preferences and parent preferences are not always synonymous or easy to distinguish. Future research, focused on understanding residents' attitudes in greater detail, would assist CQI advocates and educators to construct curricular interventions addressing these concerns.

Participation in CQI activities by the residents in our study helped meet the requirement for competency in practice-based learning and improvement. Specifically, participants worked together to delineate the steps of the process being analyzed (process-mapping) to find areas of waste and opportunities for improvement. They then developed a plan for implementing, evaluating, and refining their improvement using a plan-do-study-act method.¹⁰ Uniquely, involving residents in the hospital CQI activities allowed them to gain an understanding of the role of CQI in systems-based practice by working with an interprofessional team to improve care, a perspective that may not exist when each resident creates his or her own de novo project or focuses on his or her own continuity clinic panel.¹¹

Our study has several limitations. Acculturated residents had a higher response rate than unacculturated residents did. Although we suspect this reflects different time pressures and priorities, particularly because most of those acculturated were current residents, there may be a respondent bias with respondents having more interest in CQI, more positive attitudes toward CQI, or being more engaged in the SCH community than were nonrespondents. This may result in a misestimate of the differences between acculturated and unacculturated residents. Also, neither the CQIQ nor the Gallup Q¹² has been validated for retrospective use, and there is a risk of recall bias as the survey asked graduates to recall the extent of their CQI participation and the feelings they had as a resident.

Conclusions

This study adds to the existing literature by assessing resident CQI attitudes using previously published measures. It illustrates what we believe is an uncommon approach to resident CQI participation, namely voluntary participation in hospital-initiated activities. Finally, unlike previous reports of resident CQI activities, all of the projects undertaken were actually implemented, many within the week of the workshop.

Although our study was limited to pediatric residents at a single institution, we believe the results would be replicable by other hospitals committed to CQI and resident participation in CQI activities. The positive trends in this study indicate that resident participation in hospital CQI activities is a feasible approach to learning the skills necessary for practice-based learning and improvement and for systems-based practice. Given the extent of participation in CQI by the end of residency, residents clearly find value in such participation. Residents who participate in hospital-based CQI activities are able to have a tangible impact on their daily work. Although other researchers have measured resident CQI competency among residents participating in a formal curriculum¹² or developing their own CQI proposals,¹³ further studies are needed to assess whether participation in hospital-initiated CQI activities allows residents to achieve competency.