Supporting self-regulated-learning in colonoscopy training—A comparison cohort trial

Abstract

Aim

This study aimed to demonstrate that using a self-regulated learning (SRL) approach can improve colonoscopy performance skills.

Background

Colonoscopy is the gold standard for detecting colorectal cancer and removing its precursors: polyps. Acquiring proficiency in colonoscopy is challenging, requiring completion of several hundred procedures. SRL seems to be beneficial to help trainees acquire competencies in regulating their future learning processes and enhance the outcomes of current learning situations. SRL is a learner-centred approach that refers to a trainee's ability to understand and control their learning environment, including cognitions, motivations and emotions. The key abilities include self- and situational awareness, task analysis, and strategic planning. This study is the first to use an SRL approach for workplace-based colonoscopy training.

Methods

In this comparison cohort trial, participants used two SRL supports: a self-review of videotaped performance, and an online learning platform with procedural and conceptual knowledge about colonoscopy. In the control cohort, participants performed patient-based colonoscopy as usual in their departments. Improvement was monitored via three video-based ratings (study start, end of the study period, and follow-up) using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT). Outcomes were analysed using two-way analysis of variance with repeated measurements.

Results

This study recruited 21 participants (12, intervention cohort; nine, control cohort); 58 videos were recorded. The intraclass correlation coefficient was 0.88 (95% CI 0.61–0.98; p < 0.001). The global rating scale (GRS) and checklist (CL) in GiECAT were analysed separately. No statistically significant main effects of cohort (GRS: F(1,16) = 2.84, p = 0.11; CL: F(1,16) = 1.06, p = 0.32), test (GRS: F(2,32) = 2.56, p = 0.09; CL: F(2,32) = 0.76, p = 0.48), or interactions between cohort and test were observed (GRS: F(2,32) = 1.16, p = 0.33; CL: F(2,32) = 1.01, p = 0.37).

Conclusions

SRL in patient-based colonoscopy is feasible; however, no clear effect on performance scores was observed.

Highlights

• •Self-regulated learning in patient-based colonoscopy is feasible.
• •Self-regulated learning in patient-based colonoscopy is valued by participants.
• •Self-regulated learning supports are necessary to increase participants' engagement.
• •Protected training time might increase engagement.

1. Introduction

Colonoscopy is a demanding procedure that requires considerable technical and non-technical skills. Generally, trainees must perform several hundred procedures to gain proficiency [[1], [2], [3], [4]]. Unsurprisingly, the skills of practising endoscopists vary widely owing to numerous factors, including the total number of colonoscopies performed, training quality and quantity, previous experience with other endoscopy modalities, and the endoscopist's specialty (physician, surgeon, or nurse endoscopist) [5]. Research has yet to clearly establish an efficient approach to gain proficiency in colonoscopy via workplace-based learning. Given that the amount of practice required to master colonoscopy varies considerably among trainees, we hypothesised that implementing a learner-centred educational approach, based on the needs of individual trainees, would be beneficial.

Self-regulated learning (SRL) arises from an evidence-based, theory-oriented literature that prioritizes learner agency in academic environments [6,7]. SRL refers to a trainee's ability to understand and control their learning environment, including their cognitions, motivations, behaviours, and emotions. Hence, by self-regulating their learning, trainees activate cognitive and metacognitive processes that trigger behaviours that transform their mental abilities into observable skills. Evidence suggests that SRL can be supported through guided practice and feedback [8]. Several SRL models have been described in the literature [9,10]. Arguably, the most prominent Zimmerman and colleagues' cyclical model involves three main phases: forethought, performance, and self-reflection (Fig. 1) [11]. Certain aspects of SRL, such as self-diagnosis, self-reflection, and self-improvement, have often been studied during surgical and colonoscopy training [[12], [13], [14]]. However, research has shown that up to 15% of surgical trainees continue to experience difficulty reaching proficiency in surgical technical skills [15], and 23% feel unprepared for independent practice [16].

Fig. 1.

Self-regulated learning: Cyclical phases model. Adapted from Zimmerman and Moylan (2009).

To maximise the available time trainees can dedicate to self-regulate their learning, supportive measures can be built into the training environment to provide guidance [17]. Based on a systematic review [18] and a subsequent randomised controlled trial [19], researchers have categorised educational support for SRL into three types: content-related support (i.e. to help learners develop an integrated knowledge base), SRL support (e.g. list of goals to pursue), and practice scheduling support (i.e. to help learners organise their time and resources during training). Even though SRL support is well established in the literature, further research should continue to examine how educational initiatives may enhance learners’ SRL outcomes [19]. Consequently, the present study explored the impacts of these two supports: 1) content-related support via a web-based platform with procedural and conceptual knowledge about colonoscopy, and 2) support of core SRL processes via self-review of videotaped performances.

This study is the first to use an SRL approach for workplace-based colonoscopy training. We aimed to establish an SRL-supportive workplace-based colonoscopy training environment. We targeted the time when the trainees reached basic competencies and were allowed to perform the procedures independently. Specifically, our intervention used video recordings of real-life colonoscopies and a web-based platform that taught SRL processes to stimulate the trainees’ self-regulatory processes. We assessed the impact of our intervention on the trainees' competency in colonoscopy skills for real-world patients.

2. Material and methods

2.1. Study design

This study was a comparison cohort trial performed in four high-volume endoscopy departments in central Denmark. To minimize interaction between participants in the intervention and control cohorts, those in the intervention cohort were recruited from two regional hospitals, while those in the control cohort were recruited from two comparable regional hospitals. (Fig. 2).

Fig. 2.

Study flowchart: Self-regulated learning in colonoscopy training.

2.2. Participants

Participants were residents in surgery or gastroenterology who had recently completed a simulation and workplace-based colonoscopy training program and had the privileges to perform colonoscopy independently. All participants completed a questionnaire regarding demographic data, including sex, age, and the number of colonoscopies performed prior to participating in this study.

2.3. General procedure

Over 4 weeks, participants in both cohorts were scheduled to perform 20 colonoscopies, during which they had access to help and supervision as needed for patient safety. We collected videos #1 and #20, which were sent to blinded raters for procedural competency assessment. All other videos recorded in the intervention cohort were dedicated for the participants' personal use only.

2.3.1. Intervention cohort

Initially, participants in the intervention cohort attended a theoretical session (60 min) on the principles and framework of SRL. In this session, the primary author (CJ) presented Zimmerman et al.‘s [11] three-phase cyclical model: forethought, performance, and self-reflection phases, as shown in Fig. 1. As an illustration, three phases were set in the context of colonoscopy, with one complete circle representing one performed procedure (including preparation before and conclusion after the procedure).

Following this session, each participant received individual password-protected access to an internet platform designed for the purpose of this study (Fig. 3), where the participant's login dates and the time spent on each site could be traced and logged. Information on the internet platform included pictures, schematics, and further information on two general topics: 1) procedural and conceptual knowledge about colonoscopy (i.e. setup in the endoscopy suite, scope handling and ergonomics, patient positioning, step-by-step procedural instructions, loop handling, problem solving, use of magnetic endoscopic imaging (MEI), use of water, and challenges in patient anatomy) and 2) more detailed information about SRL based on a systematic review of core SRL processes [10].

Fig. 3.

Screenshots of the specially designed learning platform for participants in the intervention cohort (the text is written in Danish language).

During the 4-week study period, the participants had access to their videotaped colonoscopy performances immediately following each procedure. The participants were encouraged to review the videos and discuss certain self-selected segments with a colonoscopy expert in their respective departments. At the 4-week mark and during the 4-month follow-up, participants completed a questionnaire concerning their experiences and thoughts about SRL in the present study.

2.3.2. Control cohort

Participants in the control cohort were informed that the aim of the research was to ‘observe and rate their competencies during colonoscopy across different departments’. They were informed about being included in the control cohort of a comparison cohort study and that the intervention contained ‘a new approach to training colonoscopy’.

Participants in this cohort had no access to the internet platform and had no opportunity to review their colonoscopy performance videos. At the 4-week mark and at the 4-month follow-up, participants filled out a questionnaire regarding their learning/training habits and the number of hours of leisure time spent on colonoscopy-related activities (e.g. reading relevant literature, simulation-based training, and discussions with colleagues).

2.3.3. Colonoscopies

All colonoscopies were performed on patients who were newly referred to the hospital or from a surveillance list. Patients were excluded if (i) they presented with stenosis, prior rectal and/or colonic resection, or severe diverticular disease and active diverticulitis; (ii) they required colonoscopy without the intention of reaching the caecum; or (iii) the MEI could not be used. All videotaped colonoscopies were performed either without sedation or under conscious sedation with Midazolam and Fentanyl.

3. Video recordings

Two video sources were used to assess colonoscopy competency; one recorded the endoscopy suite with the patient and endoscopist, and the other recorded the endoscopic view. The videos contained recordings of the endoscopist handling the scope, the endoscopic view on the monitor, and images from the MEI (Fig. 4).

Fig. 4.

Screenshot of a video recording. Left side: setting in the endoscopy suite, centre: magnetic endoscopic imaging, right side: endoscopic view of the colon.

3.1. Baseline, immediate post-test, and retention test

We video-recorded participants performing three colonoscopies during the course of this study (at the start of the study, at the end of the study period after 20 colonoscopies, and at follow-up after 4 months). These three video recordings were used for the competency rating. The procedure, including audio, was recorded from insertion until removal of the colonoscope. Videotapes of the endoscopists were recorded using a digital video camera (LEGRIA HF G26 Canon; Tokyo, Japan). Endoscopic view and ME imaging were combined with the videotapes of the endoscopist and retrieved by an export manager (AIDA™ KARL STORZ, Tuttlingen, Germany). The patients and endoscopists were blinded to all video recordings. Video and audio sections identifying either the patient or endoscopist were edited using Adobe Premiere Pro (Adobe Systems, San Jose, California, USA). All videos were assigned a unique identifier.

3.2. Outcome measures

As blinded, independent raters, we recruited three expert endoscopists and trained supervisors who had performed more than 3000 colonoscopies. They were randomly assigned allotments for videotaped performances, which were assessed using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) [[20], [21], [22]]. Before the videos were rated, all expert endoscopists participated in a training session, in which the assessment tool and its items were studied, possible answers were discussed, and two sample videos were reviewed.

As our primary outcome, the GiECAT has supportive scoring and extrapolation validity evidence supporting its use for video-based assessments [23]. The seven global rating items contained in the GiECAT were rated from 1 (unable to achieve tasks despite significant verbal and/or hands-on guidance) to 5 (highly skilled performance of all tasks). The 19 checklist items in the GiECAT are rated as 0 (not done or done incorrectly) or 1 (done correctly). The possible ratings ranged from 7 to 54 points.

The secondary outcome measures were the inter-rater reliability of video-based assessment, participants’ self-reported experiences and thoughts about SRL, and login data from the internet platform.

3.2.1. Statistical methods

In a previous study [23], novices had a median GiECAT score of 29.2%, with an interquartile range of 10.1. In the present study, a difference of 10 percent point in GiECAT scores was considered clinically relevant. Standard deviation (SD) was estimated as 8. To show the desired difference in scores with a two-sided test significance of 5% and a power of 80%, a sample size of 22 participants was required, with 11 participants in each of the two cohorts.

Data were checked for normality (Shapiro–Wilk test) and homogeneity of variance (Levene's test). We analysed the global rating scale (GRS) and checklist (CL) components of GiECAT separately using two separate analyses of variance (ANOVA). Specifically, we conducted a two (group: intervention vs. control) by three assessment (timing: baseline, immediate post-test, and retention test) with repeated measures of the timing factor. Inter-rater reliability was investigated utilising the intraclass correlation coefficient (ICC). ICC estimates and their 95% confidence intervals (CI) were calculated based on individual ratings, absolute agreement, and two-way mixed-effects model.

Statistical analysis was performed using Stata/MP 17.0 for windows (StataCorp LLC, Texas, USA). Differences were considered statistically significant at p-values<0.05.

4. Results

4.1. Demographics

Between November 2020 and January 2022, 21 participants (11 women and 10 men) were recruited and allocated to the intervention or control cohort depending on their place of employment; 12 were allocated to the intervention cohort and nine to the control cohort. All participants were residents at various stages of postgraduate training; however, 76% were in postgraduate year II. Before the start of the study, all participants had performed a median of 112 colonoscopies (range, 42–300). The mean time from the first video recording at the start of the study to the second recording at end of the study period was 35.1 ± 10 days. The mean time from the second video recording at the end of the study period to the third recording at follow-up was 4.4 ± 1.0 months.

There were two dropouts in the intervention cohort (one owing to maternity leave and the other because of a change in employment), and one participant in the control cohort could not participate in follow-up owing to pregnancy-related sick leave. The detailed results are listed in Table 1.

Table 1.

Demographic data.

	Intervention Cohort (n = 12)	Control Cohort (n = 9)	P-value
Age, y, mean ± SD	33.2 ± 5.7	32.6 ± 4.7	0.604*
Sex, n (%)			0.130†
Female	8 (66.7)	3 (33.3)
Male	4 (33.3)	6 (66.7)
Background
Postgraduate training, n (%)  1st year  2 nd year  3rd-5th year	8 (66.7) 2 (16.7) 2 (16.7)	8 (88.9) 0 1 (11.1)	0.378†
Completed colonoscopies prior to study start, n, mean ± SD	132.4 ± 78.3 (missing: n = 2)	110.4 ± 46.6	0.763*
Weekly training, hrs, mean ± SD		1.2 ± 0.8
Drop out, n (%)	2 (16.7)	1 (11.1)	0.719*

Demographic data for all participants. SD= Standard Deviation. P-values calculated by means of: * students t-test, † chi2 test.

Table 1: Demographic data of the study participants.

4.1.1. Colonoscopies

In 34 (59%) of the 58 recorded procedures, an intervention (biopsy and/or polypectomy) was performed. Twenty-one (66%) interventions were performed in the intervention cohort (32 colonoscopies) and 13 (50%) in the control cohort (26 colonoscopies). There was no statistically significant difference in the incidence rate ratio between the two cohorts: 0.76 (CI: 0.38–1.52; p = 0.441).

4.1.2. Inter-rater reliability

A total of 58 videos were recorded for the competency ratings. Six videos were rated by all three raters, showing a ‘good’ to ‘excellent’ interrater reliability with an ICC of 0.88 (95% CI 0.61; 0.98), p < 0.001.

4.1.3. GiECAT scores

The GRS and CL scores were normally distributed within the groups, and each combination of groups had equal homogeneity of variance. All mean values and SD for the intervention and control cohorts at the start of the study, post-test (after 20 colonoscopies), and retention test are reported in Table 2. We did not observe any statistically significant main effects of cohort (GRS: F(1,16) = 2.84, p = 0.11; CL: F(1,16) = 1.06, p = 0.32), test (GRS: F(2,32) = 2.56, p = 0.09; CL: F(2,32) = 0.76, p = 0.48), or interactions between group and test (GRS: F(2,32) = 1.16, p = 0.33; CL: F(2,32) = 1.01, p = 0.37).

Table 2.

GiECAT competency scores.

		Intervention (n = 10)	Control (n = 8)
GiECAT scores in mean (SD)
Study start	GRS CL	17.80 (4.94) 9.10 (2.64)	16.46 (3.72) 8.88 (3.70)
Post-test	GRS CL	23.17 (4.97) 11.40 (2.22)	17.54 (6.72) 8.75 (3.35)
Retention test	GRS CL	20.70 (4.95) 10.10 (3.25)	18.88 (5.67) 9.63 (4.14)

GiECAT competency scores at study start, post-test (after 20 colonoscopies) and at retention test. GiECAT: Gastrointestinal Endoscopy Competency Assessment Tool, SD: Standard Deviation, GRS: Global Rating Score, CL: Checklist Score.

Table 2: GiECAT competency scores of the study participants.

4.2. Login data on the internet platform

The internet platform created for the intervention cohort was used by 11 out of 12 participants from the start of the study to the second rating. One of the two participants who did not use the website dropped out shortly after the initiation of the study. Number of logins to the website ranged from 0 to 6 with a mean ± SD of 2.75 ± 2. In average, 23.75 ± 8.4 min were spent on the website during the 4-week study period, ranging from 0 to 66 min. Twenty out of 33 logins (60.6%) were made on the day before or the same morning a video recording was scheduled. Only one participant logged onto the internet platform after the study period ended and prior to follow-up.

4.3. Questionnaire

All participants in the intervention cohort reported using SRL concepts into their training mindset. On a 5-point Likert scale, they rated the SRL-concept ‘effective’ (3.8), the internet platform ‘relevant’ (4.5), and the video-recordings ‘useful’ (3.6). Six of the participants did review ‘some’ of the video-recordings of their 20 colonoscopies, and the other four participants reviewed ‘at least one’ of the videos. All the participants reported choosing certain segments of the videos to work with. Several participants stated that viewing themselves on tape helped them recognise the problems they faced. Although there was an opportunity to discuss the videos with a supervisor, only three participants chose to do so. The reported main reason for not viewing more videos and discussing parts of the videos with a supervisor was ‘lack of time.’

The participants described what they felt in their own words and what they gained through SRL. Six out of ten described that they became more conscious of their actions and movements during colonoscopy. Two participants stated that they improved their skills owing to an increased focus on problem-solving and common issues in colonoscopy. However, two other participants experienced performance anxiety when the video camera was turned on.

Four months later, at follow-up, the participants were once again questioned about their use of SRL. Most participants had used SRL ‘occasionally’ but did not revisit the internet platform. They found the learning concept as ‘instructive’ and suggested that they might have benefitted more from the setup with additional ‘time to immersion’ in the subject. When asked whether they improved their colonoscopy skills using SRL, eight answered yes and two were unsure.

Participants in the control cohort expressed satisfaction with participating in the present study but were not overly affected by the video recordings. They reported to spend an average of 1.2 ± 0.8 h (range, 0–2 h) a week of their leisure time on colonoscopy related topics.

5. Discussion

This study investigated the effect of an SRL theory-informed approach on colonoscopy training skills. To enhance the ecological validity, we implemented an intervention with relatively experienced trainees to ensure that the workplace-based intervention was safe for patients and efficient for healthcare teams. While we observed a pattern of relative improvement in the SRL intervention cohort, the GiECAT competency scores did not differ statistically from those of the control cohort at any time in the study. In the following sections, we have discussed the merits of conducting this type of study in authentic settings while also considering the limitations that likely reduced the expected impacts of our intervention.

The participants’ varying levels of prior knowledge and skills influenced the impact of the intervention. Notably, our participants had a wide range of colonoscopy experience, and the 20 colonoscopies performed in the study represented 20% of their prior collective experience. It is possible that such an intermediate group benefitted less from our intervention and that the control cohort was sufficiently experienced to implement their own effective learning strategies. The consequence of one or both of these possibilities is a tendency toward the null hypothesis, resulting in a lower impact of our educational manipulations.

The adherence to an intervention inevitably dictates its impact [24]. Participants in the intervention cohort rated the learning platform ‘relevant’ with 4.5 on a 5-point Likert scale. Despite this fact, they each spent approximately 24 min on the website over a 4-5-week period. Judging by the participants' statements in the questionnaire, the problem was a lack of time rather than a lack of interest. In previous studies, trainees declared lack of time, general workload, and competing interests, such as training for other surgical skills, as major barriers to effective colonoscopy training. These issues are well known [25,26] and had certainly exacerbated during the COVID-19 pandemic, as many trainees experienced reduced training times and deployment to other departments [27]. Only few participants reviewed more than three videos, and three of the twelve participants discussed one video with an expert. When inquired about why participants did not discuss their videos with an expert, one participant responded, ‘I did not believe it would contribute to my improvement’. This raises the question of how to encourage participants to engage in (and consequently benefit from) the intervention [28]. On one hand, careful support to engage and nudging are crucial, but on the other hand, too much control might counteract self-regulation. Hence, peer learning as a support may be considered in future studies.

Two-thirds of the participants in the intervention cohort used the internet platform the day before or on the same morning as the next video recording was scheduled. This might be interpreted in two ways: participants either diligently prepared for the upcoming colonoscopies, or they were in need of last-minute knowledge after having procrastinated. In either case, reviewing several hours of videos and reading additional information would take time. Scheduling protected training times may help increase participants' engagement in time-consuming activities. Furthermore, the findings of the present study highlight the need for regularly scheduled follow-up meetings, where learners discuss the progression of colonoscopy skills with their supervisor. These follow-up meetings should consider their point of departure from the recorded videos and how they adhere to the information on the internet platform. This initiative is similar to the recommendations of Grossmann and Salas on the transfer of training, in which the work environment plays a crucial role in the development of trainees’ competencies [29].

This study had some limitations. Participants in our study scored higher than those in the study used for our power calculation, suggesting potential underpowering, and highlighting the need for enrolment of additional participants. However, the logistics of workplace-based interventions hindered our ability to address this limitation. Furthermore, selecting participants with relatively advanced experience in our study may have reduced the impact of our intervention compared to an intervention involving a novice population. In future studies, researchers will need to consider this choice to strike a balance that ensures the safe and efficient implementation of the intervention with the intended impacts. In this study, only one video was recorded for each participant at each of the three study points, which made the competency scores prone to fluctuation. It is easier to obtain higher scores in straightforward procedures, whereas the same operator may obtain lower scores in more complex procedures. Several factors, such as bowel preparation, patient anxiety, and previous operations, are independent of the competency of the operator, yet have a bearing on the difficulty of the case. Although it would be more time-consuming, the option to base each rating on several procedures exists. Considering the inherent differences in patients involved in clinical research, we opted to exclude patients from the study when potential difficulties were anticipated, such as those with a history of previous operations, to minimize score fluctuations.

6. Conclusion

SRL in patient-based colonoscopy was feasible and valued by the participants. Nonetheless, a clear effect on performance scores could not be observed. This could be attributed to a lack of adherence to study methods. Based on weak adherence to SRL support, it is recommended to initiate scheduled follow-up meetings where learners discuss the progression of their colonoscopy skills with their supervisor. Future research should continue to examine ways to increase engagement in SRL interventions.

ACGME competency

Practice Based Learning and Improvement.

Funding

This study did not receive any specific grants from funding agencies in the public, commercial, or non-profit sectors.

Data availability statement

Data supporting the findings of this study are available from the corresponding author upon request.

Ethics declarations

All patients and personnel in the endoscopy suite were informed of the video recording before the procedure and informed consent was obtained, in conformity with the Danish penal code §264. No changes were observed in patient care.

Participants were included in the study after receiving verbal and written information and after providing written consent. Participation in this study was voluntary, and participants could withdraw their written consent at any time.

This study was conducted in accordance with the principles of the Declaration of Helsinki. The Central Denmark Region Committee on Health Research Ethics (1-10-72-181-20) waived the requirement for approval.

CRediT authorship contribution statement

Claudia Jaensch: Writing – review & editing, Writing – original draft, Visualization, Validation, Resources, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Rune Dall Jensen: Writing – review & editing, Writing – original draft, Validation, Supervision, Methodology, Formal analysis, Conceptualization. Ryan Brydges: Writing – original draft, Validation, Supervision, Formal analysis. Charlotte Paltved: Validation, Supervision, Resources, Funding acquisition, Conceptualization. Anders Husted Madsen: Writing – original draft, Validation, Supervision, Methodology, Conceptualization.

Declaration of AI and AI-assisted technologies in the writing process

During the preparation of this study, the authors (s) did not use artificial intelligence (AI) or AI-assisted technology.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Abbreviations

ANOVA

Analysis of Variance

CL

Checklist (part of the GiECAT)

GiECAT

Gastrointestinal Endoscopy Competency Assessment Tool

GRS

Global rating scale (part of the GiECAT)

ICC

Intraclass correlation coefficient

MEI

Magnetic Endoscopic Imager

SD

Standard Deviation

SRL

Self-Regulated Learning

Appendix A. Supplementary data

The following are the Supplementary data to this article: