Abstract
Introduction
Evidence-based medicine (EBM) teaching is most successful when integrated with patient care, but finding time for teaching on inpatient rotations is difficult. Obstetrics (OB)-Newborn TEACH (Teaching Evidence-based medicine And Clinical topics in the Hospital) Cards is a curricular tool for efficient teaching sessions on maternity care rotations. We evaluated the impact of OB-Newborn TEACH Cards on resident EBM attitudes and skills, exposure to clinical topics, and patient management.
Methods
OB-Newborn TEACH Cards includes 56 cards on obstetrics and newborn topics with background and foreground discussion questions. Residents on a family medicine maternal-child service completed pre- and postrotation surveys to assess the cards’ impact on EBM attitudes and skills. Faculty and residents also completed point-of-care surveys to assess the self-reported influence on management decisions.
Results
Of 24 potential resident participants, 58% completed pre- and postrotation surveys, which showed significant increase in perceived EBM skills like applying evidence to a clinical scenario (P=.04), but a decrease in reported attitudes that EBM is realistic (P=.028) and useful (P=.025). Residents agreed the cards exposed them to a variety of topics. Point-of-care surveys (n=58) indicated that 57% of the time respondents used a card to learn about a topic not related to a specific patient. When used to learn about specific patients, the cards influenced self-reported patient care 44% of the time.
Conclusion
OB-Newborn TEACH Cards are a promising inpatient teaching tool for improving perceived EBM clinical application, exposing residents to maternal-child topics, and influencing patient care decisions.
Introduction
Effective use of evidence-based medicine (EBM) is critical on busy inpatient services, and learners prefer EBM education integrated into clinical care using miniteaching sessions.1,2 Finding time for inpatient teaching, however, is challenging.3,4
To address these EBM and inpatient teaching needs, we previously developed TEACH (Teaching Evidence-based medicine And Clinical topics in the Hospital) Cards to guide adult inpatient teaching sessions and practice point-of-care EBM.5 An initial study found that TEACH Cards improve exposure to inpatient topics, confidence with EBM skills, and efficiency in answering clinical questions.
When our residency started a maternal-child teaching service, we created Obstetrics (OB)-Newborn TEACH Cards as a way to help residents meet the rotation objective of learning the breadth of pregnancy care and newborn topics. Teaching on OB services frequently occurs with drills, simulations, and team-based learning.6–8 OB-Newborn TEACH Cards supplement resident didactic seminars by providing an interactive, point-of-care clinical learning opportunity. We evaluated whether OB-Newborn TEACH Cards improve resident EBM attitudes and skills, increase exposure to maternal and newborn topics, and influence patient care decisions.
Methods
Residents and faculty from a university-based family medicine residency participated in this study. The University Institutional Review Board exempted the study from review.
Educational Tool
OB-Newborn TEACH Cards are 56 cards (41 obstetrics, 15 newborn) with topics selected from the American Academy of Family Physicians Family Medicine Residency Curriculum Guidelines.9,10 The front of each card contains background basic science questions; a foreground Patient, Intervention, Comparison, Outcome (PICO) question; and a learning pearl question. The back contains instructions and resources (Figure 1). OB-Newborn TEACH Cards are commonly used to guide miniteaching sessions where small groups discuss the background questions, develop PICO questions, and research answers to those questions. The cards can also be used to guide independent study. Answers are intentionally not provided in order to encourage self-directed learning and practice with point-of-care evidence-based medicine.
Figure 1.
OB-Newborn TEACH Cards Example Card Showing the Front of a Card With Discussion Questions and the Back of a Card With Resources and Instructions
Study Participants and Protocol
Study participants were postgraduate year (PGY)-1 and PGY-2 residents and faculty rotating on an inpatient maternal-child service at a community hospital over 6 months. Resident rotations were 3 or 4-weeks. Faculty rotations were 1-week, and some faculty did multiple rotations. We evaluated the cards with multiple surveys: resident pre- and postrotation surveys, faculty postrotation survey, and point-of-care survey.
Residents received an email and electronic prerotation survey before their rotation and an electronic postrotation survey following their rotation. Residents could use the cards without completing the surveys. The pre- and postrotation surveys asked questions about EBM attitudes and skills adapted from a previously validated tool using a 5-point Likert scale (1=strongly agree, 5=strongly disagree).11 The postrotation survey asked about the educational value of OB-Newborn TEACH Cards. Resident participants were eligible to receive monetary compensation from a drawing.
Faculty participants completed an electronic postrotation survey evaluating the instructional value of the cards on a 5-point Likert scale (1=strongly disagree, 5=strongly agree) and reported the number of cards used weekly.
Faculty and residents could complete a point-of-care survey after using a card to assess (1) if the card pertained to a patient they were caring for, and if so, if it influenced medical decisions; (2) if using the card would influence future patient care; and (3) if they had improved perceived knowledge. Point-of-care surveys were paper surveys physically located adjacent to the displayed OB-Newborn TEACH Cards in the team room. Completed surveys were deposited into a secure drop box.
Data Analysis
We compared resident pre- and postrotation survey responses with a one-sample, paired t-test comparisons of means and performed subgroup analysis for PGY-1s and PGY-2s. We analyzed categorical data on the postrotation and faculty surveys with χ2 tests. We described point-of-care surveys using n (%) for the binary response question (question 1) and mean (SD) for Likert questions. We assessed resident and faculty differences with logistic regression for the binary response question and with Mann-Whitney-Wilcoxon tests for Likert questions.
Results
There were 24 potential resident participants (9 PGY-1s, 15 PGY-2s), and 58% (6 PGY-1s, 8 PGY-2s) completed both pre- and postrotation surveys. Of 15 faculty who rotated over 25 weeks, there was a 60% weekly survey completion rate (10 individuals during 15 weeks).
Combined resident data showed statistically significant changes for 4 of 11 EBM attitudes and skills (Table 1). Respondents reported increased agreement in ability to evaluate evidence quality and apply evidence to clinical scenarios and decreased agreement that EBM is realistic and useful on a daily basis. Subgroup analysis revealed PGY-2s had significant increases in ability to use EBM resources to answer questions and attitude that asking consultants is more efficient than EBM.
Table 1.
Prerotation vs Postrotation Survey Comparisons for Resident EBM Attitudes (Questions 1–6) and Skills (Questions 7–11)
| Questions | N of Pretest | Pretest Scores Mean (SD) | N of Posttest | Posttest Scores Mean (SD) | Significancea |
|---|---|---|---|---|---|
| Q1: EBM is realistic to practice in routine patient care. | N(c) = 14 | 4.79 (0.43) | 14 | 4.36 (0.63) | P=.028* |
| PGY-1 | N = 6 | 4.67 (0.52) | 6 | 4.17 (0.41) | P=.076 |
| PGY-2 | N = 8 | 4.88 (0.35) | 8 | 4.50 (0.76) | P=.197 |
| Q2: EBM is useful on a daily basis. | N(c) = 13 | 4.85 (0.38) | 13 | 4.23 (0.83) | P=.025* |
| PGY-1 | N = 5 | 4.60 (0.55) | 5 | 4.20 (0.45) | P=.178 |
| PGY-2 | N = 8 | 5.00 (0.00) | 8 | 4.25 (1.04) | P =.080 |
| Q3: Literature searches are too time consuming to do while caring for patients in the hospital. | N(c) = 14 | 2.86 (1.17) | 14 | 2.64 (0.84) | P =.487 |
| PGY1 | N = 6 | 2.33 (1.03) | 6 | 2.67 (0.52) | P =.465 |
| PGY2 | N = 8 | 3.25 (1.16) | 8 | 2.63 (1.06) | P =.140 |
| Q4: I rarely form questions about patients seen in the hospital. | N(c) = 14 | 1.71 (0.61) | 14 | 1.71 (0.73) | P =1.000 |
| PGY1 | N = 6 | 1.50 (0.55) | 6 | 1.83 (0.75) | P =.175 |
| PGY2 | N = 8 | 1.88 (0.64) | 8 | 1.63 (0.74) | P =.170 |
| Q5: EBM is important to practice on a regular basis. | N(c) = 14 | 4.71 (0.47) | 14 | 4.50 (0.52) | P =.189 |
| PGY1 | N = 6 | 4.67 (0.52) | 6 | 4.33 (0.52) | P =.363 |
| PGY-2 | N = 8 | 4.75 (0.46) | 8 | 4.63 (0.52) | P =.351 |
| Q6: Questions can more quickly be answered with consultants rather than with EBM. | N(c) = 14 | 2.50 (0.86) | 14 | 2.79 (0.70) | P =.165 |
| PGY1 | N = 6 | 3.17 (0.98) | 6 | 3.17 (0.75) | P =1.000 |
| PGY2 | N = 8 | 2.00 (0.00) | 8 | 2.50 (0.54) | P =.033* |
| Q7: I can form a searchable clinical question. | N(c) = 14 | 4.07 (0.62) | 14 | 4.36 (0.63) | P =.165 |
| PGY1 | N = 6 | 4.33 (0.52) | 6 | 4.33 (0.52) | P =1.000 |
| PGY2 | N = 8 | 3.88 (0.64) | 8 | 4.38 (0.74) | P =.104 |
| Q8: I can use evidence-based resources to find answers to clinical questions. | N(c) = 14 | 4.21 (0.58) | 14 | 4.43 (0.51) | P =.189 |
| PGY1 | N = 6 | 4.33 (0.52) | 6 | 4.17 (0.41) | P =.363 |
| PGY2 | N = 8 | 4.13 (0.64) | 8 | 4.63 (0.52) | P =.033* |
| Q9: I can find an evidence-based answer to a clinical question in less than 5 minutes. | N(c) = 14 | 3.21 (0.98) | 14 | 3.64 (0.63) | P =.111 |
| PGY1 | N = 6 | 3.00 (1.09) | 6 | 3.50 (0.84) | P =.296 |
| PGY2 | N = 8 | 3.38 (0.92) | 8 | 3.75 (0.46) | P =.285 |
| Q10: I can evaluate the quality of evidence that is found in a point-of-care search. | N(c) = 14 | 3.21 (0.80) | 14 | 3.86 (0.66) | P =.007** |
| PG1 | N = 6 | 3.57 (0.83) | 6 | 3.83 (0.78) | P =.363 |
| PG2 | N = 8 | 3.00 (0.76) | 8 | 3.88 (0.64) | P =.006** |
| Q11: I can apply evidence from a point-of-care search to a clinical scenario. | N(c) = 14 | 4.07 (0.27) | 14 | 4.36 (0.50) | P =.040* |
| PGY2 | N = 8 | 4.12 (0.35) | 8 | 4.50 (0.54) | P =.080 |
| PGY1 | N = 6 | 4.00 (0.00) | 6 | 4.17 (0.41) | P =.363 |
N=14
Scale: 1= strongly disagree; 5=strongly agree
N(c)= full sample
P<.05
P<.01
One-sample, paired t test
Residents agreed that OB-Newborn TEACH Cards are beneficial for exploring topics not encountered through patient care (Mean [SD]=4.50[0.52]), practicing point-of-care searches (Mean[SD]=4.17[0.39]), improving knowledge (Mean[SD]=4.50[0.52]), and impacting patient care (Mean[SD]=3.58[1.08]). Residents spent 15 minutes per card on average.
Faculty respondents used about two cards weekly (Mean[SD]=2.13[1.73]) and agreed that OB-Newborn TEACH Cards is a valuable teaching tool (Mean[SD]=4.2 [0.68]) and increases teaching (Mean[SD]=3.6[0.91]).
Point-of-care surveys (N=25 with 58 total surveys) showed that over half the time (57%), users did not use the card related to a specific patient. When a card was used to learn about a specific patient, it influenced self-reported patient care decisions 44% of the time. There were no statistically significant differences between residents and faculty regarding TEACH Cards’ self-reported influence on patient care (Table 2). Respondents agreed that using the card would impact future patient care (weighted mean (SD)=4.37 [0.50]) and increased their knowledge (weighted mean [SD]=4.56 [0.55]).
Table 2.
Point-of-Care Survey Data Showing Impact of OB-Newborn TEACH Cards on Patient Care and Medical Knowledge (N=58 Observations, n=25 Individuals)
| Full Sample | Faculty/Fellows | PGY1 | PGY2 | ||
|---|---|---|---|---|---|
|
| |||||
| Number or responses | 58 | 20 | 22 | 16 | |
| Number of participants | 25 | 10 | 8 | 7 | |
|
| |||||
| Q1. If you used this OB-Newborn TEACH Card to review a topic that pertains to a patient you or someone on the team is caring for, did using this card impact the care of this patient? | Responses | ||||
| Yes, n (%) | 11 (44%) | 4 (40%) | 3 (38%) | 4 (57%) | |
| No, n (%) | 14 (56%) | 6 (60%) | 5 (62%) | 3 (43%) | |
| Not applicable (not used for a specific patient), n | 33 | 10 | 14 | 9 | |
|
| |||||
| Participants | |||||
| With applicable responses, n | 16 | 6 | 5 | 5 | |
| With at least one Yes, n (%) | 8 (50%) | 3 (50%) | 2 (40%) | 3 (60%) | |
| Adj. odds ratioa | Ref | 0.90 | 2.00 | ||
| P valuea | -- | Ref | 0.914 | 0.488 | |
|
| |||||
| Q2. Using this OB-Newborn TEACH Card will impact my care of patients in the future. (1 strongly disagree-5 strongly agree) | Mean (SD) | 4.41 (0.59) | 4.30 (0.57) | 4.68 (0.48) | 4.19 (0.66) |
| Weighted meanb (SD) | 4.37 (0.50) | 4.30 (0.45) | 4.68 (0.46) | 4.11 (0.50) | |
| P valuec | -- | Ref | 0.089 | 0.477 | |
|
| |||||
| Q3. Using this OB-Newborn TEACH Card increased my knowledge of this topic. (1 strongly disagree-5 strongly agree) | Mean (SD) | 4.59 (0.62) | 4.30 (0.73) | 4.95 (0.21) | 4.44 (0.63) |
| Weighted meanb (SD) | 4.56 (0.55) | 4.48 (0.57) | 4.88 (0.35) | 4.43 (0.61) | |
| P valuec | -- | Ref | 0.052 | 0.917 | |
Adjusted odds ratios and the corresponding p-values estimate the association of “yes” responses with level of training by a logistic regression mixed model with random per-participant intercepts.
Weighted means are calculated by inversely weighting by the number of times the participant responded.
P values are from Mann-Wilcoxon tests on the average response per participant.
Conclusions
Summary and Significance
Consistent with our prior TEACH Cards study, use of OB-Newborn TEACH Cards helped to expose residents to a variety of topics. Perceived EBM skills like applying evidence to clinical scenarios increased. While residents agreed on the postsurvey that using EBM is realistic and useful, there was a small but significant decrease in agreement compared to the presurvey. This discrepancy could be explained by fatigue at the end of a rotation and realization that EBM is not always straightforward, which aligns with findings that clinicians only pursue about half of clinical questions due to time constraints and doubt that answers exist.12 The decrease in perceived utility of EBM may also be related to the finding that PGY-2s on the postrotation survey reported a significant increase in the attitude that asking consultants is more efficient that EBM. The perceptions that consultants are more useful and EBM is less useful may reflect the significant role that local practice approaches and expert opinion play in guiding patient care, especially in situations where there is more than one evidence-based answer or where evidence is lacking.
When OB-Newborn TEACH Cards were used to learn about a topic related to a specific patient, almost half the time there was a self-reported influence on patient care. Few EBM education tools have been shown to impact patient care decisions, although use of one EBM tool influenced treatment in 18% of cases.13 OB-Newborn TEACH Cards serve as a guide for topic discussion and self-directed EBM learning; however, they are not a reference with answers, which may explain why they did not influence patient care most of the time. OB-Newborn TEACH Cards provide an opportunity for residents to experience firsthand how EBM can influence medical decision-making.
Study Limitations
There was no control group, so we cannot determine the full effect of OB-Newborn TEACH Cards versus the rotation. This study was performed at a single institution with a small sample, so results may not be generalizable.
Conclusion
OB-Newborn TEACH Cards is a novel teaching tool for maternal-child rotations that may improve resident EBM application, expose residents to clinical topics, and influence patient care decisions.
Future Directions
Future directions include further evaluating OB-Newborn TEACH Cards in a controlled environment and at other institutions to better determine the generalizability of the results, utility of the teaching tool, and influence on patient care.
Acknowledgments
The authors thank Emily Torell, MD, for assistance with conceptual development and creation of OB-Newborn TEACH Cards; David Feldstein, MD, for guidance with conceptual development; and Mindy Smith, MD, MS, for guidance with the manuscript.
Presentations: This study was presented as follows:
Hahn T, Regner C, Metzger E. OB-Newborn TEACH Cards: An innovative curricular tool for maternal-child rotations. 2019 Society of Teachers of Family Medicine Annual Spring Conference. April 2019. Toronto, ON, Canada. Poster presentation.
Regner C, Metzger E. Hahn T. Obstetrics and Newborn TEACH Cards: An Innovative Curricular Tool for Family Medicine Learners. Family Medicine Midwest Conference. November 2018. Madison, WI. Oral seminar presentation.
Hahn T, Torell E, Metzger E, Regner C, Carlson J. OB TEACH Cards: A Curricular Tool for Inpatient OB Rotations and EBM Skills. May 2018. Society of Teachers of Family Medicine Annual Spring Conference. Washington, DC. Poster presentation.
References
- 1. Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ. 2004;329(7473):1017. doi: 10.1136/bmj.329.7473.1017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Torre DM, Simpson D, Sebastian JL, Elnicki DM. Learning/feedback activities and high-quality teaching: perceptions of third-year medical students during an inpatient rotation. Acad Med. 2005;80(10):950–954. doi: 10.1097/00001888-200510000-00016. [DOI] [PubMed] [Google Scholar]
- 3. Arabshahi KS, Haghani F, Bigdeli S, Omid A, Adibi P. Challenges of the ward round teaching based on the experiences of medical clinical teachers. J Res Med Sci. 2015;20(3):273–280. [PMC free article] [PubMed] [Google Scholar]
- 4. Beigzadeh A, Bahaadinbeigy K, Adibi P, Yamani N. Identifying the challenges to good clinical rounds: A focus-group study of medical teachers. J Adv Med Educ Prof. 2019;7(2):62–73. doi: 10.30476/JAMP.2019.44710. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Hahn TW, D’Agata C, Edgoose J, Mastrocola J, Zakletskaia L, White M. TEACH Cards: Teaching Evidence-Based Medicine and Clinical Topics in the Hospital. PRiMER Peer-Rev Rep Med Educ Res. 2018;2:25. doi: 10.22454/PRiMER.2018.479320. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Hampton BS, Raker CA, Sung VW. Implementation and evaluation of a novel operating room curriculum for the obstetrics and gynecology clerkship. J Surg Educ. 2014;71(4):521–529. doi: 10.1016/j.jsurg.2013.12.008. [DOI] [PubMed] [Google Scholar]
- 7. Nitsche J, Morris D, Shumard K, Akoma U. Vaginal delivery simulation in the Obstetrics and Gynaecology clerkship. Clin Teach. 2016;13(5):343–347. doi: 10.1111/tct.12458. [DOI] [PubMed] [Google Scholar]
- 8. Mody SK, Kiley J, Gawron L, Garcia P, Hammond C. Team-based learning: a novel approach to medical student education in family planning. Contraception. 2013;88(2):239–242. doi: 10.1016/j.contraception.2012.07.012. [DOI] [PubMed] [Google Scholar]
- 9.American Academy of Family Physicians. Recommended curriculum guidelines for family medicine residents: Maternity care. [Accessed February 10, 2022]. Reprint No AAFP 261. Published July 1980. Revised August 2018. https://www.aafp.org/dam/AAFP/documents/medical_education_residency/program_directors/Reprint261_Maternity.pdf .
- 10.American Academy of Family Physicians. Recommended curriculum guidelines for family medicine residents: Care of infants and children. [Accessed February 10, 2022]. Reprint No AAFP 260. Published July 1998. Revised August 2018. https://www.aafp.org/dam/AAFP/documents/medical_education_residency/program_directors/Reprint260_InfantChild.pdf .
- 11. Baum KD. The impact of an evidence-based medicine workshop on residents’ attitudes towards and self-reported ability in evidence-based practice. Med Educ Online. 2003;8(1):4329. doi: 10.3402/meo.v8i.4329. [DOI] [PubMed] [Google Scholar]
- 12. Del Fiol G, Workman TE, Gorman PN. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA Intern Med. 2014;174(5):710–718. doi: 10.1001/jamainternmed.2014.368. [DOI] [PubMed] [Google Scholar]
- 13. Umscheid CA, Maenner MJ, Mull N, et al. Using educational prescriptions to teach medical students evidence-based medicine. Med Teach. 2016;38(11):1112–1117. doi: 10.3109/0142159X.2016.1170775. [DOI] [PMC free article] [PubMed] [Google Scholar]