Evaluation of a Case-Based Primary Care Pediatric Conference Curriculum

Abstract

Background

Our goal was to assess the impact of a standardized residency curriculum in ambulatory pediatrics on residents' participation, satisfaction, and confidence.

Methods

A case-based curriculum for weekly primary care conference was developed to replace the existing free-form review of topics at the Yale Pediatrics Residency Program. Before the curricular switch, faculty preceptors and members of the academic year 2005–2006 intern class completed surveys designed to measure conference occurrence and resident attendance, participation, satisfaction, and confidence in clinical skills. One year after the curricular switch, identical surveys were completed by faculty preceptors and members of the academic year 2006–2007 intern class.

Results

Faculty surveys indicated that conferences took place significantly more often after the curricular switch. The number of residents at conference each day (3.18 vs 4.50; P < .01) and the percentage who actually spoke during conference (45% vs 82%, P < .01) significantly increased. There were 18 demographically similar interns in each of the 2 classes. Members of the academic year 2006–2007 intern class, having trained exclusively with the standardized curriculum, were significantly more likely to respond favorably to survey items about participation, satisfaction, and confidence. In addition, they were more likely to endorse survey items that reflected explicit goals of the standardized curriculum and the Accreditation Council for Graduate Medical Education core competencies.

Conclusion

Implementation of a structured curriculum for ambulatory care improved interns' self-reported participation, satisfaction, and confidence. The primary care conference occurred more dependably after the curricular change, and improvements in attendance and participation were documented. Pediatric residency programs may make better use of conference time in the ambulatory setting through the use of structured, case-based educational material.

Editor's Note: The curriculum can be found at pcpc.yale.edu. Two sample chapters (Week 5 ^{(440KB, pdf)} , Week 41 ^{(905.8KB, pdf)} ), the intern survey ^{(170.5KB, doc)} , a supplemental table ^{(82.5KB, doc)} that includes the responses to all of the items on the intern survey, and a supplemental table ^{(33KB, doc)} that shows the data for each demographic variable are available on the JGME.org website. Interested parties may contact the authors for information related to access to all current and future curricular material.

Background

The Accreditation Council for Graduate Medical Education (ACGME) requires pediatrics programs to provide training across the breadth of “conditions commonly encountered in primary care practice.”¹ Residents gain knowledge and experience in inpatient and outpatient settings through caring for patients, participating in conferences, and self-directed learning. Over time, resident education has been shifting away from ambulatory medical practice toward hospital-based care.^2,3 Pediatric residents spend the bulk of their time in the hospital, although most eventually will work in office-based primary care settings.^4,5 Surveys of residents and recent graduates reveal deficiencies in content areas commonly encountered in ambulatory care, and document that during training, residents are exposed to information they will rarely need in practice.^6–13

Novel curricula can improve residents' sense of preparedness for general pediatrics practice.^14,15 One such innovation is the primary care conference, a short discussion focusing on ambulatory pediatric care before continuity clinic. Many pediatric training programs use an unstructured format to guide primary care conference sessions, and teaching methods, quality, content, and accuracy can vary widely from day to day.

Case-based learning is an effective small-group teaching method that provides opportunity for clinical problem solving while encouraging debate and discussion among learners. Studies show that this method uses time well and has been received favorably by learners and faculty.^16,17

There are no previous reports of an evaluation of case-based, comprehensive curricula in general ambulatory pediatrics. The purpose of this study, therefore, was to assess the impact of a case-based standardized curriculum as compared to the previously used unstructured format on (1) conference occurrence and attendance, (2) resident participation in learning, (3) resident satisfaction, and (4) residents' report of their levels of confidence in clinical skills. We hypothesized that improvements would be seen in all of these outcomes after the curricular change.

Methods

Needs Assessment

The unstructured format for primary care conference had been in place at our institution for more than 2 decades. Under the unstructured format, a resident was charged with preparing a 30-minute talk on a topic for the primary care conference. No guidelines existed regarding learning objectives or teaching methods to be used. During interviews with the program's leadership in 2004, many residents in the Pediatrics Residency Program of Yale-New Haven Children's Hospital revealed dissatisfaction with the unstructured format. Although the 30-minute conference session accounted for approximately 10% of the total protected educational time during an intern's average work week, conference attendance was not a high priority for interns because the conference did not reliably take place and the information presented was of variable quality. The conference also often started late, and residents perceived it made them late for subsequent patient appointments. Some expressed a desire to have a more structured curriculum in place.

Curricular Development

We developed a structured, case-based primary care curriculum to address common topics in ambulatory care, as well as issues relating to the ethical, business, and legal aspects of pediatrics, entitled “Yale Primary Care Pediatrics Curriculum.” A selection committee that included residents and faculty reviewed the list of topics discussed over the previous 3 years and selected topics based on the perceived needs of residents, the Ambulatory Pediatric Association Educational Guidelines for Pediatric Residency,¹⁸ and the American Board of Pediatrics General Pediatrics Content Outline.¹⁹ All members of the general pediatrics faculty received a preliminary list of 52 topics for review and comment. The final curriculum consisted of 52 chapters, 1 for each week of the academic year (table 1).

TABLE 1.

Yale Primary Care Pediatrics Curriculum Topics List 2006–2007^a

Starting in September 2005, residents and faculty were invited to author chapters. Authors received detailed instructions, which included suggestions on writing learning objectives, case vignettes, case-based questions, answers for discussion moderators, as well as tips for conducting an effective literature search. After submission, all chapters went through an editorial process, including 2 to 5 sets of revisions. Ultimately, each chapter was arranged in a standardized format: directed learning objectives followed by case vignettes with questions, and references to required reading material. In addition, a moderator's guide contained suggested answers, teaching “pearls,” and recommendations for group exercises to reinforce key points.²⁰

Implementation

The standardized curriculum was distributed in both paper and CD-ROM versions and replaced the existing unstructured format in the pediatric continuity clinic in July 2006. In both the earlier unstructured format and new standardized curriculum, a given topic was discussed at the primary care conference each day for a week. Residents were provided the topic in advance. In the standardized curriculum, residents and faculty were provided curricular materials in advance and asked to read a designated reference article each week. A resident was preassigned to facilitate the discussion and was provided with the moderator's guide for that week. Faculty received the entire 52-chapter moderator's guide at the start of the program. Once in clinic, residents and faculty used the case vignettes and questions in each chapter as the focus of a 30-minute discussion.

Participants in this study included categorical pediatrics residents and faculty preceptors in the continuity clinic of the Yale-New Haven Children's Hospital. The program has 18 residents in each class of the 3-year training program and a pool of 23 part-time clinic preceptors, all of whom are either academic or community general pediatrics faculty. On average, 3 to 4 interns, 1 to 2 residents, and 3 preceptors are scheduled for each afternoon clinic session. Maximum attendance at primary care conference on any given day would be 5 residents, as the clinic space precluded a greater number of residents seeing patients.

Evaluation

An evaluation of the curriculum was conducted through surveys of clinic preceptors and residents, performed both before and after the change. The survey instruments were developed and modified on the basis of discussions with the Department of Pediatrics and Section of General Pediatrics research groups. The Yale School of Medicine's Institutional Review Board granted exemption from review and informed consent for all phases of the evaluation process owing to the educational nature of the project.

Preintervention

In the spring of 2006, faculty preceptors present during the primary care conference completed a brief, anonymous questionnaire daily for 5 weeks. Preceptors documented conference occurrence, attendance, participation, and duration, estimated resident interest and knowledge, and rated resident preparedness as moderators and as learners by using a 5-point Likert scale. Only 1 questionnaire was completed per clinic session and faculty identity was not tracked to maintain anonymity. All clinic faculty received instruction on how to complete the questionnaire.

During the same time period, each member of the intern class received a 42-item survey. Responses used a 5-point Likert scale ranging from strongly agree (5) to strongly disagree (1). Survey items fit into 1 of 3 prespecified categories: self-reported satisfaction (7 items), participation in learning (18 items), and confidence in clinical skills (17 items). Several items addressed the ACGME core competencies. Three reminders were sent by E-mail to nonresponders over the course of 1 month.

Postintervention

In the spring of 2007, after the change to the structured curriculum, we surveyed the academic year (AY) 2006–2007 intern class and the faculty, using the same survey instruments used for the preintervention survey. These interns, who were trained exclusively with the standardized curriculum, also received an additional 6-item survey related to the reading material. Demographic data for both the preintervention and postintervention cohort of interns were collected from their files.

Data Analysis

Data were compiled and analyzed with Excel (Microsoft Inc, Redmond, WA) by using Mann-Whitney U test to compare Likert scale responses, χ² test to compare dichotomous variables, and Student t test to compare means. Significance was defined as P < .05.

Results

Faculty Questionnaire

Faculty questionnaires were submitted on 72% (18/25) of clinic days during the AY 2005–2006 preintervention phase, and on 76% (19/25) of clinic days during the AY 2006–2007 postintervention period (table 2). Faculty indicated that conference had taken place on 56% of days in AY 2005–2006 and 100% of days in AY 2006–2007 (P  =  .001). The means of (1) number of residents at conference, (2) number of residents speaking in conference, (3) number of residents who appeared to have read before conference each day, and (4) percentage of residents at conference who actually spoke were all significantly greater postintervention. The mean number of faculty members present at the conference did not change from AY 2005–2006 (1.91) to AY 2006–2007 (1.95) (P  =  .90). Faculty responded more favorably on the 5-point Likert scale to the statement “the moderator was well prepared” in AY 2006–2007 than AY 2005–2006 (P < .05) and indicated that neither the structured nor the unstructured curriculum caused conference participants to be late to see patients. On average, the postintervention conference was 4 minutes longer (P  =  .03). This improvement was achieved primarily by starting 5 minutes closer to the scheduled start time (P < .01).

TABLE 2.

Conference Data Based on Daily Faculty Questionnaires

Intern Survey

There were no statistical differences between the 2 intern groups in age, sex, underrepresented minority status, United States Medical Licensing Examination Step 1 and 2 scores, Alpha Omega Alpha membership, other advanced degrees, or eventual entry into primary care practice. Surveys were completed by 61% (11/18) of the AY 2005–2006 (unstructured format) cohort and by 89% (16/18) of the 2006–2007 (standardized curriculum) cohort.

Both cohorts responded favorably to items designed to measure general attitudes about primary care conference and continuity clinic. Members of the AY 2006–2007 (postintervention) cohort were significantly more likely to agree overall with items about satisfaction, participation, and confidence (table 3). Members of the AY 2006–2007 intern class were more likely to endorse several survey items that reflected explicit goals of the standardized curriculum and the ACGME core competencies¹ (table 4). These included items related to satisfaction (eg, “conference is well organized”), participation in learning (eg, “my intern colleagues consistently have good attendance”), and confidence in clinical skills (eg, “conference improves my motivational interviewing skills”).

TABLE 3.

Pooled Responses of Interns to Survey Items^a in Predetermined Categories

TABLE 4.

Responses to Selected Items^a on Survey of Interns

Discussion

We designed a novel standardized curriculum to encompass elements from the 6 ACGME competencies¹ and found favorable responses to survey items related to 5 of the competencies after the curricular change.

Our curriculum improved conference occurrence, attendance, and participation; increased interns' self-report of satisfaction, participation in learning, and confidence in clinical skills; and was well received by interns and faculty preceptors. While structured, case-based curricula for specific topics in ambulatory pediatric care have been developed and evaluated previously,^21–24 this is the first description of a comprehensive, longitudinal, primary care curriculum in general ambulatory pediatrics. Descriptions of similar educational interventions exist in the internal medicine literature, and our findings are in keeping with previously published reports,^25–28 including the use of ambulatory care curricula to meet ACGME competency and program requirements.²⁹

Our case-based learning techniques allowed for optimal use of limited protected educational time. Case-based learning enables learners to focus efficiently on key points related to a given topic, while providing room for mature learners to explore more advanced topics and areas of personal curiosity.¹⁶ Well-constructed cases serve to engage learners effectively in discussion, and by simulating complexities encountered in actual practice, case-based learning mimics experiential learning.³⁰ These factors make case-based learning particularly well suited for primary care conference.

Physicians across disciplines, but particularly pediatricians who recently completed training, have expressed the desire to have more exposure to ambulatory care during training.^6,31 The disproportionate focus on training of inpatient care extends to conference topics as well, with less than half of residents in a multiprogram study reporting that they regularly attended didactic sessions or received handouts or articles as part of their continuity clinic experience.³² In a related study, receiving an “appropriate amount of teaching” was associated with resident satisfaction with continuity clinic (odds ratio, 3.25).³³ In our study, 88% of interns reported that the standardized curriculum is a “valuable learning experience.” We believe the interns recognized that the standardized curriculum provided information through a reliable and organized approach, making their limited protected educational time more rewarding. Another factor in improved satisfaction may have been that the primary care conference was twice as likely to actually take place.

There are a variety of possible explanations for the improved confidence in clinical skills seen after our curricular change. Interns valued the material taught, and 94% reported that they found the reading material useful. The curriculum reinforced evidence and practice guidelines and provided ready access to current literature on common topics in ambulatory pediatrics. Other studies have demonstrated improvements in residents' knowledge and confidence after the introduction of novel pediatric curricula in ambulatory settings.^21–24 Faculty also may have gained knowledge by reviewing the literature on basic topics in general pediatrics. This newfound confidence in preceptors' own knowledge may have improved subsequent precepting interactions, translating to improved resident knowledge and skills.

Attendance and participation in primary care conference also improved after the change. Residents may have been more inclined to participate because they found the conference to be a valuable learning experience. Embi and colleagues²⁶ described a similar increase in participation after the implementation of an improved primary care curriculum in internal medicine. In our program many upper-level residents had participated in the curricular change as chapter authors, and conference attendance may have increased owing to their personal investment in the curriculum. Previous work suggests that resident involvement in curricular reform can enhance curricular success, group identity, and educational ownership,³⁴ consistent with established principles of adult learning theory.³⁵

Given the high levels of resident satisfaction with the standardized curriculum, we are hopeful that with time, the programmatic culture will change such that participation will further improve. To optimize future participation, residents have remained involved in curricular revisions. In addition, we have developed a web-based version of the standardized curriculum to allow residents easy access to curricular material, an effective delivery strategy for other curricular innovations.^26–28 We have continued to add chapters to meet the changing needs and interests of our residents and those at other programs using the curriculum.

Our study has 4 limitations. First, the sample size was restricted to the size of the intern classes at a single institution during the 2 years of the study. Despite the small sample size, there were significant differences found in attendance, satisfaction, participation, and confidence in clinical skills, in addition to favorable reception of specific sections of the standardized curriculum. Second, there may have been differences between the 2 intern classes that were not evident from the demographic information available. The response rate to the intern survey was lower for the AY 2005–2006 intern class, which may be a reflection of apathy toward the primary care conference. Third, while there were no major changes in the training program during the time of the study, we could not control for subtle changes that may have affected our results. Finally, our study was not blinded. Interns may have responded more favorably after the curricular change because they knew they were using a novel curriculum. Additionally, since members of the faculty were not blinded to the group status, they might have provided more favorable responses after the curricular change. This problem seems unlikely, because our significant findings on the faculty questionnaire were largely objective measures.

Since its development, the Yale Primary Care Pediatrics Curriculum has been implemented by nearly 50 training programs. Future studies should examine the effects of the curricular change on multiple institutions, on directly observed changes in patient care practices, on long-term retention of the information presented, or on the impact on patient outcomes.

Conclusion

Our implementation of a standardized curriculum for ambulatory care was associated with improvement in interns' self-reported satisfaction, participation in learning, and confidence in clinical skills. The primary care conference occurred more dependably after the curricular change, and there were improvements in attendance, advance preparation, and participation. Pediatric residency programs may make better use of protected educational time in the ambulatory setting through the development or adoption of structured, case-based educational material. Given the favorable reception of our curriculum, we believe that this approach can be useful for other pediatric training programs.