An effective procedure skills training programme for GIM registrars

ABSTRACT

Background

The Royal College of Physicians’ Acute care toolkit 8 recommends procedural training for medical registrars at all hospitals. We aimed to determine the interest and need, and to pilot the delivery of such training in the procedures outlined by the Joint Royal Colleges of Physicians Training Board (2017).

Methods

An online survey was sent to general internal medicine (GIM) trainees within the Thames Valley Deanery in January 2019. This identified a need for procedure skills training. Ninety per cent of trainees felt simulation training would improve their confidence in the outlined procedures.

We trialled a simulation programme for GIM registrars between September 2019 and October 2019. Sessions lasted 3–3.5 hours and trainees rotated through four stations. Feedback was obtained from trainees and trainers during each pilot session.

Results

Thirty-two trainees attended across both sites. Excellent feedback was obtained and trainee confidence improved by visual analogue scale scoring post-training for all procedures. Almost 90% of trainees felt the sessions would improve safety on GIM on calls.

Conclusion

Simulation training is an effective way to improve trainee confidence in procedural skills and this pilot shows such training is desired and necessitated in higher specialty training. Further work will assess its impact on maintaining trainee skillsets and impact on patient safety.

Background

Introduction of Modernising Medical Careers and the European Working Time Directive (EWTD) has resulted in a decrease in procedure exposure.^1,2 Despite this, there is an expectation set by the Joint Royal Colleges of Physicians Training Board (JRCPTB) that a general internal medicine (GIM) registrar is competent in procedural skills.³

Current evidence suggests confidence in the listed skills can be enhanced, both within the laboratory and clinical setting, through simulation training.⁴ Simulation training improves patient safety and may therefore play an important role in an era where procedural exposure is limited and clinical practice varies significantly.^5,6

Skill retention in trainees is a recognised issue with conflicting literature on the duration of retainment after initial training. Retainment seems to depend on the procedure, with retainment up to 1 year for central venous catheter (CVC) insertion.⁷ Regardless, we can extrapolate that skill degradation with time means that procedural competence attained at internal medicine training (IMT) level does not infer maintained competence at registrar level.

The Royal College of Physicians (RCP) Acute care toolkit 8 recommends procedure training be available for medical registrars at all hospitals to ensure patient safety out of hours and recommends simulation training.⁸ Simulation training is now mandated for foundation and internal medicine trainees and is well delivered nationally and locally.⁹ There is no similar training for GIM registrars within our deanery.

Aim

We aimed to explore the interest and need for simulation training for procedure skills for GIM Registrars and subsequently pilot a procedure simulation training programme.

Methods

An online survey collecting categorical, ordinal and free text data about procedural experience and confidence was sent to all GIM registrars within our deanery in January 2019. The survey was closed after 4 weeks. To maximise response rates, a pre-alert email was sent 1 week prior to the survey link, and a follow up email was sent 1 week later. The survey questions were reviewed by colleagues to ensure they were neutrally worded and easy to understand to minimise potential bias.

Quantitative data was analysed using descriptive statistics and free text comments underwent emergent and a priori coding and subsequent thematic analysis.

A pilot simulation training programme was introduced between September 2019 and October 2019 and feedback collected as part of a service evaluation strategy.

Results of online survey

Quantitative data

Seventy-one responses (47% of total GIM trainees in the region) were obtained from the online survey. Respondents spanned a variety of grades and specialties, all of which take part in the general medical rota. Fig 1 shows the demographics of the respondents.

Fig 1.

Demographics of respondents to course development survey to an online course development survey, n=71. a) Higher specialty training grade. b) Gender. c) Full-time training status. d) Training specialty. AIM=acute internal medicine; GIM = general internal medicine; ICM = intensive care medicine; ID = infectious diseases; LTFT = less than full time; ST = specialty trainee year.

Trainees had limited exposure to procedures (Fig 2a). Fewer than 50% of trainees had performed CVC or chest drain insertion in the past 12 months. For all procedures, the majority of trainees had performed fewer than five procedures in the preceding year. Trainee confidence varied significantly between procedures, with the majority of respondents confident in only direct current (DC) cardioversion and abdominal paracentesis, the most common procedures performed (Fig 2b). Less than half of trainees were confident to perform a chest drain for pneumothorax, and less than a third of trainees were confident to perform a chest drain for effusion or at performing a CVC insertion.

Fig 2.

Procedure experience, competence and perceived utility. a) Number of ward-based procedures performed by trainees in the past year. b) Proportion of respondents confident to perform ward-based procedures without supervision and manage potential complications. c) Proportion of respondents who considered each ward-based procedure as important for the safe management of the acute medical take or inpatient ward cover, n=71. CVC = central venous catheter; PTx = pneumothorax; DC = direct current.

Thirty-five (49.3%) trainees had sought out additional opportunities for procedure skills training in the past 12 months. Sixty-three (90%) trainees felt simulation training would help improve confidence with GIM procedure skills and 53 (75%) trainees were willing to be part of the faculty delivering simulation training.

Although the listed procedures are all needed for GIM sign off, there was a discordance in trainees’ perception of the utility of some of these procedures (Fig 2c). The insertion of a chest drain out of hours is a scenario that trainees are often anxious about, and 92% of trainees felt chest drain insertion for a pneumothorax was an important skill to possess for the safe management of the medical take. Almost 70% of trainees reported current procedural training within the deanery to be inadequate.

Thematic analysis

Thematic analysis identified 20 codes, and six themes (see supplementary material S1 for supporting quotes).

Theme 1: Factors affecting confidence

Trainee confidence was reduced by the infrequent need to perform many of these procedures and although some felt competent in the procedures, their confidence level was low. Extended periods of time out of training resulted in a decrease in confidence in independent practice.

Theme 2: Impact of inadequate procedure training on patients

Lack of confidence to perform a procedure can result in patient harm, both from inability to relieve a patient’s symptoms (such as breathlessness) and, equally, procedures performed by inexperienced clinicians can be ‘dangerous’.

Theme 3: Impact of inadequate procedure training on doctors

Medico-legal implications were mentioned with one trainee stating it as ‘indefensible’ to perform a procedure when not appropriately skilled. It was recognised that confidence at a point in time does not infer competency at a later date. Furthermore, competency sign-offs were hindered by lack of appropriate supervision.

Theme 4: Barriers to seeking procedure training during on calls

The busy nature of the acute medical take, lack of appropriate supervision and delegation of procedures to other specialties often limits procedure training opportunities during on calls.

Theme 5: Barriers to seeking other procedural training opportunities (other than on calls)

Lack of time and difficulty balancing clinical commitments with training opportunities act as barriers. Trainee perception of the relevance of these skills can also be a contributing factor. Furthermore, there is competition among trainees to book onto what are already limited training opportunities.

Theme 6: Factors influencing a ‘good’ simulation session

Small group size, approachable trainers, a supportive environment, protected simulation training time, regular sessions and appropriate feedback were deemed to be important factors for a good simulation session.

Implementation of a pilot simulation programme for GIM registrars

We used the results to design and implement a simulation programme for GIM registrars, with an aim of improving trainee confidence and providing evidence of basic competency training. We used feedback from the first pilot to make changes to the second and third pilot sessions.

Issues considered during implementation phase

Duration of sessions

Each session lasted 3–3.5 hours. The length of time was determined by availability of the simulation suite at the two trusts involved.

Trainees rotated through four stations.

• Station 1: abdominal paracentesis/knee aspiration.

• Station 2: chest drain insertion.

• Station 3: DC cardioversion/pacing.

• Station 4: central line insertion.

Station set-up

Each station utilised simulation models provided by the simulation suite. The models and consumables were bought from specialist supplier Limbs & Things.¹⁰ Part-task trainer pads were used, which allowed multiple needle insertions. The approximate costings for each station are outlined in Table 1. This excludes the cost of the models, each of which costs £2,000–£3,000 depending on the supplier.

Table 1.

Approximate costings for each station per session

Station 1: Knee aspiration	Fluid: £5; part task trainer pad: £25
Station 1: Abdominal paracentesis	Part task trainer pad: £15; Bonanno catheter: £30
Station 2: Chest drain insertion	Part task trainer pad: £38; Seldinger chest drain kit: £40
Station 3: DC cardioversion	Pads: £50; defibrillator (from Sim suite)
Station 4: CVC insertion	CVC catheter: £150

DC = direct current; CVC = central venous catheter.

Number of trainees per station

The first pilot (nine trainees in total) accommodated two to three trainees per station. Feedback suggested there was room to accommodate more trainees, so the second and third pilots were expanded to three trainees per station. Trainees were allocated to groups beforehand. Significant flexibility was required on the day to manage trainee or trainer tardiness (due to clinical duties or car parking issues), or trainee absence.

Session coordination

During the first pilot, one of the trainers also took on the role of coordinating the overall running of the session. This proved difficult and resulted in delays. The subsequent two pilots had an independent coordinator who helped address issues, like accommodating trainees who were late, ensuring the stations ran to time and ensuring trainees and trainers were provided refreshments as required.

Trainers

Trainers were recruited through word of mouth and emails. Although the initial survey showed 75% of GIM registrars were willing to be part of the faculty, recruitment was more difficult than anticipated due to rotas and on-call commitments.

Trainers were specialty registrars who had extensive experience in performing the procedures they were teaching. Trainers were briefed about the objectives for the session as per the JRCPTB curriculum. Some trainers taught on all three pilot programmes allowing standardisation of training at those stations. Ideally, only trainers with adequate simulation teaching experience (for instance, a formal training course on simulation provision) would have been recruited but this was not always possible due to faculty recruitment difficulties. Feedback for trainers was collected at the end of the session to allow trainers to develop as teachers.

Certificates

A certificate of attendance was given to all trainees. This was approved by the GIM Training Programme Director (TPD) and provided sufficient evidence for formative competency. A certificate of thanks was given to all trainers, which could be used as evidence of teaching.

Food and breaks

Refreshments were provided for trainees and trainers. For the first pilot, pastries and beverages were provided before the session. For the subsequent pilots, beverages and biscuits were provided throughout the day, and a sponsored lunch was provided between the morning and afternoon session.

Feedback

Feedback was received from both trainers and trainees from each session. Feedback forms were altered between the first pilot and the second based on trainee feedback.

Trainee feedback

Visual analogue scale (VAS) score assessments showed improvement in confidence levels post-simulation training in all of the procedural skills.¹¹ Fig 3 shows the number of trainees who felt ‘competent to perform the procedure independently and deal with complications’ before and after the course. The greatest change was noted for chest drain insertion, knee aspiration and CVC insertion.

Fig 3.

Number of course participants confident to perform ward-based procedures without supervision and manage potential complications before and after the course, n=32. CVC = central venous catheter; DC = direct current.

All trainees felt the procedures were relevant to their clinical practice. Ninety-four per cent of trainees felt the sessions improved their knowledge of the potential complications of the procedures and how to deal with them safely. Eighty-eight per cent felt simulation training would improve patient safety on GIM on-calls and 97% of trainees would recommend the simulation programme to fellow GIM trainees.

Supporting comments from trainees included:

Great to go through procedures as exposure limited in clinical practice. Excellent facilitators. Well organized.

Good to have a measured way of going through procedures.

Excellent – filling gap in training. Thank you.

Excellent to have practice at less commonly performed procedures.

Suggested improvements included:

• more time on management of complications

• providing a ‘rest’ station at the end to allow trainees to catch up on stations they may have missed or felt they needed more hands-on practice with

• 5-minute breaks between sessions.

A word cloud of further comments made by trainees after the session can be found in supplemental material S2.

Trainer feedback

Informal feedback from trainers highlighted the need for sufficient breaks between stations. For the second and third pilots, the majority of trainers helped with the morning and afternoon sessions. Most trainers found this extremely tiring, despite voluntary participation.

Discussion

A simulation programme for GIM registrars is an effective way of improving confidence in the procedural skills outlined in the JRCTPB curriculum. With diminishing opportunities to perform these procedures on actual patients, simulation training may offer the best alternative in maintaining confidence.

While in certain hospitals, the opportunity to ask other specialties to help with procedures may exist, all medical registrars should feel confident in performing these procedures when there is a critical need. We would maintain that chest drain insertion using the Seldinger approach for a pneumothorax is perhaps the procedure most likely required to be performed by a GIM registrar out of hours.

Ideally, trainees should strive for a state of unconscious competence with these procedures, but given the training climate, it is likely trainees will fall between ‘conscious incompetence’ or ‘conscious competence’. Trusts should make available contingency processes to make trainees feel supported and not work beyond their level of competence and confidence. Although confidence and competence cannot be used interchangeably, with maturity and experience, we would assume that GIM registrars can recognise their own limitations.

While procedure skill training can be enhanced through simulation, an effort should be made to address other aspects of the procedure process, such as taking consent, post-procedure care and management of procedure-related complications. Aspects of this were covered during our sessions but not in great detail. One way of addressing this is to run the course over a full-day, with a didactic, lecture-based approach combined with practical skills training. This model is used for IMT training within our deanery but was not utilised for our programme due to time constraints and the assumption that GIM registrars will not require much revision of the theories. However, it was clear during the sessions that many GIM registrars could have benefited from teaching of these aspects in greater detail. To cater for different needs and different levels of experience, while working within time constraints, pre-reading material or pre-recorded webinars could have been made available for trainees to access in their own time. Furthermore, grouping trainees according to assumed level of experience (as determined by grade and specialty) may help address heterogeneity in learner requirements.

This article does not focus on theory-based teaching methods used for skills training. A well-known model is Peyton’s four-step approach.¹² The model can be adapted to trainees of varying experience and will help trainees progress through the levels of Miller’s pyramid, from ‘knows’ to ‘does’.¹³

The small number of trainees at each station allowed sufficient time for directed feedback, which we feel contributed to the positive feedback we received. Simulation training offers a safe environment to practice, make mistakes and learn, as long as good group dynamics are fostered and effective facilitation is maintained. By ensuring the environment was supportive and providing adequate food and beverages, we attempted to address the physiological and safety needs outlined in Maslow’s hierarchy of needs thus facilitating learning from the simulation sessions.¹⁴ In future, we will factor in more breaks as suggested by the feedback.

Recruitment of trainers was more difficult than expected. Generating a database with the contact details of potential trainers who are keen to be contacted for future sessions (while ensuring compliance with 2018 General Data Protection Regulations) may be useful in the future.

Standardisation of training between sessions is important. In order to ensure this, we tried to encourage trainers to volunteer on multiple sessions. This was not always possible due to availability. For future sessions, we will ensure trainers undergo formal training 1–2 weeks prior to the session to allow adequate time for planning. Furthermore, a handout outlining the indications and possible complications and management strategies can be given to trainees to ensure the objectives are adequately met, the procedure is taught in context and the teaching delivered is constructively aligned to the outcome measures stated on the direct observed procedural skills (DOPS) tool.

Trainees greatly appreciated the attendance certificate they received which provided evidence for training. We felt a formative DOPS assessment would add too much to the workload of trainers and was therefore not implemented.

Using an online survey was an efficient way of demonstrating the ‘need’ for a change in training provision. Evaluation of the pilot programme was performed using paper-based feedback forms to facilitate immediate response and prevent recall bias. While this method allowed collection of invaluable information, analysis was more time-consuming than if an online method had been used. Our evaluation strategy only assessed the ‘reaction’ level of Kirkpatrick’s evaluation model and captured trainees’ feelings and levels of satisfaction about the programme.¹⁵ While this provided us with good insights into the quality of our programme, and the ability of a trainee to perform the procedure in the simulation laboratory helped determine whether learning had happened, we cannot assume that the trainees’ confidence/competence in a laboratory setting extends to clinical competence. Offering the programme at specific time intervals during specialty training may help with skill retainment. This has not been formally studied in the literature with reference to the JRCPTB procedural skills in specialty registrars but will likely be dependent on trainees’ parent specialty and the frequency of procedure exposure. The trainees who attended our pilot programmes requested the course be repeated annually or biennially (53% and 31%, respectively) but budgetary and time constraints may prevent this.

Checklist when setting up a simulation programme

The following checklist may be useful for colleagues who are keen to set up simulation programmes. It is by no means a complete list but may help provide guidance.

• Survey trainees to establish a need for change.

• Establish appropriate recruitment methods for trainers.

• Establish good rapport with your local medical education department, simulation teams and relevant training programme directors.

• Advertise the course.

• Provide certificates for trainees.

• Provide certificates for trainers.

• Train (or at least brief) the trainers beforehand.

• Develop an evaluation strategy.

• Reflect and make changes based on the feedback.

• Other factors to consider: group sizes, refreshments and combining didactics with practical skills teaching.

Limitations

The results only represent the views of trainees in one deanery. As the programme was organised by registrar colleagues, the feedback results may be subject to leniency bias. Many trainees did not answer the free-text questions on the feedback forms. This may have been due to time constraints, questionnaire fatigue (as there were several forms to complete) and mental fatigue after a long session. Furthermore, although we assumed that trainees were intrinsically motivated to improve their procedural skills, it is possible that some were motivated by the proof of attendance only, with less than anticipated interest in improving the teaching programme, thus reducing the quality of the feedback given.

Trainee grade, specialty and full-time status may have affected the feedback and change in confidence levels due to prior experience and training. Stratifying responses according to these variables may help address this bias.

As this is the first time the session was being run and is a programme greatly sought after within the deanery, trainees may have given favourable feedback to ensure the programme continues.

Interpretation of the change in confidence levels has to be made carefully, and competence not assumed. Ideally, trainees will be directed to clinical opportunities to carry out the procedure on a patient. As this may not be possible, simulation training should be performed at regular intervals to limit skill degradation with time.

Conclusion

While the need for or success of simulation training cannot be generalised to other deaneries, our simulation pilot shows convincing evidence that simulation training can be both successful and useful for trainees if correctly implemented. We would advocate that simulation training for procedural skills be made available for all higher specialty trainees, ideally on an annual or bi-annual basis.

Supplementary material

Additional supplementary material may be found in the online version of this article at www.rcpjournals.org/clinmedicine:

S1 – Online survey and supporting quotes.

S2 – Word cloud of course participant free-text feedback following the course.