Assessing the Effect of E-Learning on Perineal Repair Knowledge and Skill Acquisition

Introduction

Residency training is the critical time to learn and develop procedural skills. However, exposure to obstetrical injuries is limited for family medicine residents, due a shorter training program. Open-access e-learning provides an opportunity to close this gap. Various online and simulation training programs have been developed for perineal repair^1,2 as well as other skills^3–6 across many surgical specialties.^7,8 However, very few programs have been studied in family medicine. The objectives of our small study were to create a novel, open-access e-learning program⁹ on the diagnosis and management of second degree obstetrical perineal injury for family medicine trainees, and to evaluate the effectiveness of our tool through blinded assessments of knowledge recall and technical skill at multiple points in time.

Methods

In 2019, we recruited nearly all first-year family medicine residents from the University of Toronto Michael Garron Hospital teaching site with approval from the hospital Research Ethics Board. The study design consisted of a pre/postintervention protocol. The intervention was an online, interactive e-module designed by study investigators.⁹ We assessed baseline knowledge and skill. Afterwards, the module was released and could be viewed without limitation. The second assessment was completed 1 week later. Unfortunately, the 6-month follow up coincided with the outbreak of COVID-19 in March 2020 and was cancelled.

The assessments and main outcome measures were a written, multiple-choice knowledge test and technical performance of a simulated second-degree perineal repair. Performance was assessed via video recording participants performing second-degree perineal laceration repairs on a simulated bench-top model. We anonymized participants by using black, long-sleeved shirts and gloves, and sound was removed from the videos to ensure blinding. Performance was marked by two obstetrical providers using a task-specific checklist that was modified from Siddiqui et al.⁸

We calculated frequencies and proportions for nominal variables including mean, median, and measures of distribution. We used the Wilcoxon signed-rank test to analyze the primary outcomes. We used Cronbach’s α to assess correlation between the two assessors.

Results

In total, there were 11 participants included in the knowledge recall and 10 participants in the performance assessment.

The mean baseline score for knowledge recall was 6.7/10 (SD 1.8), compared with 9.0/10 (SD 1.3) at follow up (Table 1). Six participants’ knowledge test score improved, one declined, and four remained the same. The number of participants improving their score was statistically significant (P=.031).

Table 1.

Knowledge Test Results at Baseline and 1 Week Follow Up

	Baseline	Follow-up
Number of participants	15	11
Mean	6.7	9.0
Median	7.0	10.0
Standard deviation	1.8	1.3
Range	7.0	3.0
Minimum	3.0	7.0
Maximum	10.0	10.0
25th %	6.0	8.0
50th %	7.0	10.0
75th %	8.0	10.0

Abbreviation: %, percentile.

A maximum score of 11 was possible on the performance assessment. Participants obtained a mean baseline score of 4.5/11 (SD 3.2) and 3.7/11 (SD 3.3), respectively for evaluators one and two. At follow up, the mean score for evaluators one and two were 6.2 (SD 3.2) and 5.5 (SD 3.3), respectively (Table 2). After the intervention, nine participants improved, one participant obtained the same score, and no participants’ performance declined. The number of participants improving their performance was statistically significant (evaluator one P=.004 and evaluator two P=.004). There was also a very high intraclass correlation coefficient between evaluators (0.940, P<.001).

Table 2.

Performance Checklist Score by Evaluator at Baseline and 1 Week Follow Up

	Mean (SD)	Min	Max	25th %	50th %	75th %
Evaluator 1 baseline	2.9 (2.2)	0	7	1.0	3.0	4.3
Evaluator 2 baseline	3.5 (1.7)	1.0	6.0	2.0	4.0	5.0
Evaluator 1 follow up	6.1 (2.6)	2.0	9.0	4.0	6.0	9.0
Evaluator 2 follow up	6.7 (2.4)	3.0	10.0	5.0	6.0	9.0

Abbreviations: SD, standard deviation, %, percentile

Discussion

This exploratory study evaluated the impact of an open access, obstetrical perineal injury e-learning program among family medicine residents. This is the first study of its kind to focus on family practice, as well as implement blinded assessments using multiple assessors at multiple points in time. Based on our results, we can conclude that our tool was effective in improving both knowledge and technical skill of second-degree perineal repair. Moreover, there was a statistically significant agreement between evaluators suggesting that the intervention, not the evaluator, influenced performance.

One key strength of the training module is portability. In our constantly-changing environment, the flexibility to choose on-demand, interactive learning and access to reliable information is critical to the successful training of learners.

Despite a 73% participation rate, one key limitation to our study was small sample size. This was due to the small number of residents in our training program. Another limitation includes the loss of our 6-month follow-up assessment that coincided with the outbreak of COVID-19 in March 2020. Additionally, because this study was a residency research project (for authors K.B. and A.I.), it required completion by a predetermined deadline and could not be extended.

In summary, this study demonstrated the effectiveness of teaching obstetrical surgical skills to family medicine trainees using an e-learning platform. It also demonstrated the feasibility to conduct this study on a larger scale. Further research is needed to assess knowledge/skill retention at greater time intervals and whether benchtop skills are transferred to the clinical environment. Data from this study will be used as a foundation to launch a larger, longer study using a more robust study design.