Lasting impact: insights from a surgical mission-based mentoring training programme in the Republic of Congo

Abstract

The global shortage of surgeons, anaesthetists and obstetricians is significant, especially in low and middle income countries (LMICs). A significant amount of LMIC surgical volume is provided by surgical missions and non-governmental organisations (NGOs) who are often well resourced, making them ideal environments for training. However, there are few publications addressing how to train in this setting, or the long-term impact of such training. Mercy Ships operates the largest non-governmental hospital ship in the world, the Africa Mercy, serving LMICs at the invitation of their President by providing free surgery and training for the surgical workforce. Mercy Ships developed and offered a comprehensive training programme across surgical specialties and disciplines in the Republic of Congo, 2013–2014. In this analysis paper, we present our experiences in developing and implementing the training portion of the programme. We also present the findings of an evaluation of the programme, which show a sustained positive impact and lasting change on personal and organisational practice 12–18 months post-training. We also make recommendations to NGOs and surgical mission organisations seeking to augment the impact of surgical missions with effective surgical training programmes.

Key questions.

What is already known about this topic?

• Five billion people worldwide lack access to safe, affordable surgery and 143 million surgical procedures are needed each year to meet this need.

• Expansion of the surgical volume must be accompanied by doubling the global surgical workforce within the next 15 years, with a focus on quality, safety and sustainability.

• Training programmes in low-income settings can produce positive immediate outcomes.

What are the new findings?

• A positive long-term impact on personal and organisational practice can result from a mentoring programme offered by a surgical non-governmental organisation (NGO).

• We identified inhibitors and facilitators of personal and organisational change in low-resource settings.

• The programme provides a strategy of incorporating mentoring-based training alongside high-volume surgical missions that could be implemented by other NGOs.

Recommendations for policy

• Surgical NGOs should make volunteers aware of the impact that mentoring (including on non-technical skills) can have and consider making it a part of standard practice during surgical missions.

• Collaborative planning and preparation with all stakeholders (individual doctors, hospital management and national organisations/government administration) is key to creating a lasting impact in such programmes.

• Foreign language considerations and the availability of only short time frames in which to train should not be viewed as absolute barriers.

The need

There is a severe global shortage of surgeons, anaesthetists and obstetricians; at least 1.2 million need to be trained by 2030 to reach the target workforce density of 20 providers per 100 000 population.¹ Workforce deficits in low and middle income countries (LMICs) are compounded by an unequal distribution of providers to the population, as the majority of providers practise in urban areas. A significant amount of surgical and anaesthesia care in LMICs is provided by surgical non-governmental organisations (NGOs) and visiting teams.¹ These teams are often well-resourced and can offer high case volume, making them ideal environments for training. It has been argued that ‘NGOs should have a training component hardwired into their programmes to ensure the durability of their effect’.¹ And that ‘in an ideal situation, governments would work in partnership with NGOs’.¹ However, the knowledge and skills acquired in training must then be translated into clinical practice to ensure sustainability and improvement in patient outcome.² Yet documented success in implementation and impact evaluation of NGO training programmes in LMICs is lacking.³

Mercy Ships operates the world's largest non-governmental hospital ship, the Africa Mercy, and serves LMICs at the invitation of their President. The ship usually serves for 10 months in a given country and provides free surgery for those in greatest need, as well as training for the surgical workforce. From August 2013 to June 2014, the Africa Mercy served in the Republic of Congo. This analysis paper presents our experience of developing a mentoring programme and the findings of its impact evaluation. We offer these as an example to other NGOs and mission-based surgical organisations, and suggest that lasting change in personal and institutional practice can be achieved through mentoring projects of any length.

The programme

Initial assessment and design

In October 2012, an educational needs assessment was conducted, consisting of hospital surveys and interviews with key personnel and a training programme was collaboratively created with the Ministry of Health of the Republic of Congo. Local Ministry of Health representatives and Chiefs of Surgery, Anaesthesia and Nursing in the Republic of Congo were involved and engaged in the design phase of this programme. The programme aligned Mercy Ships’ surgical, anaesthesia and nursing expertise to local needs, offering mentoring opportunities for individual surgeons, anaesthetists and nursing staff, and educational training courses designed to accommodate larger numbers of participants.

Mentoring followed the formal apprentice-type approach.⁴ This approach allowed for the greatest flexibility among participants, as availability for training differed greatly between surgeons, anaesthetists and nurses. Participants also needed to balance competing demands of their own hospital practice and service delivery with their desire for training. The mentoring training programme was designed to address knowledge, skills and attitudes, with careful consideration of the practical capacity of knowledge translation to implementation for each participant in their working environment. Therefore, the programme also included donation of equipment to aid implementation and sustainability of learnt skills, and a structured plan for monitoring and evaluation using the Kirkpatrick model as shown in table 1.⁵

Table 1.

Kirkpatrick model for evaluating educational courses

Level 1: reaction	Participants' perception of the course (enjoyment, relevance and engagement)
Level 2: learning	Acquired knowledge, skills, attitude, confidence and commitment
Level 3: behaviour	Translation of knowledge and skills into routine personal practice
Level 4: results	The ultimate goal; improved patient outcome

Selection of participants

Application forms were distributed to the Ministry of Health and local Hospital Directors, and to the representative of the World Federation of Societies of Anaesthesiology (WFSA). Various selection methods were used according to logistical and pragmatic restraints: nursing participants were interviewed; anaesthetists were accepted on the recommendation of their Hospital Directors and the WFSA representative; and surgeons were accepted because they were either recommended by the Ministry of Health or they were the only surgeon of that specialty in the area.

Four hospitals (A, B, C, D) were represented in total. Hospitals A, B and C were in the port city of Pointe Noire and hospital D was in the capital city, Brazzaville (400 km away).

Length of mentoring programme

Mentoring took place over a 35-week period, from September 2013 to May 2014.

• Surgeon mentoring varied from a total of five consecutive days to a total of 1–2 days per week over a 30-week period. The duration depended on the availability of the surgeon and the duration of the surgical specialty programme on the ship.

• Anaesthesia mentoring was 2 days per week for 8 weeks.

• Ward nurse mentoring was 5 days per week for 6 weeks for 10 participants. In addition, two ward nurses came for 1 week, accompanying their surgeon who was mentored simultaneously.

• Operating room (OR) nurses were mentored continuously throughout the entire 35-week surgical programme. The OR nurses spoke English and simultaneously worked as translators for the OR while participating in the mentoring programme.

Anaesthesia, ward and OR nurse mentoring programmes all included some formal teaching sessions; surgeon training was entirely a hands-on, practical apprenticeship.

Language considerations

The Republic of Congo is a French-speaking country but Mercy Ships’ working language is English. Therefore, for training purposes, translators were used when needed but none were professional translators nor had any medical experience.

The evaluation

Design

In May 2015, we returned to the Republic of Congo, and used a qualitative research design to assess the impact of the training programme at Kirkpatrick model levels 2–4. Structured interviews, focus groups and questionnaires were used to collect data, which were manually coded and themed for analysis.⁶ Our goal for this evaluation was to measure impact, validate the effectiveness, quality and value of our training programmes, and to build theories and recommendations for future programme development.⁷ The Mercy Ships Institutional Review Board approved the evaluation, and the Republic of Congo Ministry of Health approved the education programme in 2013 and the evaluation in 2015.

Participants were contacted by telephone and invited to take part in the evaluation. Most interviews took place at their hospital; two were in a café. Structured interviews were carried out by the authors, MW and KC. MW is an anaesthetist and KC is a non-clinician who had coordinated the mentoring programme and knew the participants. KC conducted all the anaesthetist interviews as MW was involved in anaesthesia mentoring. Notably, data collection for this analysis was undertaken by people known to the participants (even if the authors did not personally deliver all the training), and therefore the reported results are subject to a positive responder bias. Each participant was interviewed once; interviews were conducted in French using a translator where necessary and lasted for 15–30 min. Focus group discussions were also conducted among ward nurses and lasted for 30–60 min. The responses were not recorded and transcribed due to budget constraints but were handwritten contemporaneously on paper by an observer, and this may have led to selection bias. Some interviews were conducted using a translator, and therefore it is possible that information may have been lost in translation. Most participants completed questionnaires in French which asked open, free text questions that were similar to the structured interview, but anonymously. This questionnaire also included a question on the potential negative consequences of the training.

Structured interviews and discussion groups centred on the following questions:

1. What were the most important things you learnt from participation in the mentoring programme?

   • This measured the value of the programme, identifying learning that was still relevant 12–18 months later, Kirkpatrick level 2.
2. Have you made any changes in your personal practice as a result of/since the mentoring programme?

   • This measured the effectiveness of the programme and behaviour change, Kirkpatrick level 3.
3. Have you been able to make any changes in your hospital (institution) as a result of the mentoring programme?

   • This measured the effectiveness of the programme, institutional change and patient impact, Kirkpatrick level 4.
4. What things helped or hindered you in making changes to your personal and/or hospital's practice?

   • This was to aid future development and recommendations.

The impact

Twenty-seven individuals participated in the mentoring programme between September 2013 and May 2014. Two participants were asked to leave during training due to misconduct, and were excluded from the evaluation. We attempted to contact all the remaining 25 participants and 20 (80%) were interviewed in May 2015 (12–18 months after participation in the programme). Details are given in table 2.

Table 2.

Numbers of participants in the mentoring programme and number followed up

	Number who completed the programme	Number followed up at 12–18 months	Reason for lack of follow-up
Surgeons	4*	4
Anaesthetists	4	3	Moved to work in France
OR nurses	5*	4	Unable to make contact via mobile phone or email
Ward nurses	12	9	1 had moved abroad; 2 were unavailable but working in a health centre and a private clinic
Total	25	20

*One surgeon and one OR nurse were asked to leave the programme due to misconduct, and so were excluded from the study.

OR nurses, operating room nurses.

Kirkpatrick level 2 evaluation—learning

Participant responses to the first question were themed and classified into two topics: specialty specific and organisational. Organisational factors were then coded and subdivided into four categories: teamwork, communication, organisation and infection control. The number of positive responses in each of these categories is shown in table 3.

Table 3.

Number of responses (grouped by theme) to the question: what were the most important things you learnt in the mentoring programme?

	Surgeons n=4	Anaesthetists n=3	Nurses (OR and ward) n=13	Totals (%)
Specialty-specific techniques	3	3	5	11 (55)
Teamwork	4	2	4	10 (50)
Communication	2	2	4	8 (40)
Organisation/preparation and safety*	2	3	3	8 (40)
Infection control	1		11	12 (60)

*Organisation/preparation and safety: this included use of the WHO Surgical Safety Checklist.⁸

OR nurses, operating room nurses.

Most participants indicated improvement in specialty-specific knowledge, teamwork and communication; notably, all surgeons indicated that teamwork was the most important thing they had learnt. These results show that learning acquired during a mentoring programme, regardless of length, can be retained and recalled 12–18 months post-training.

Kirkpatrick level 3 evaluation—behaviour

Answers to the second question, concerning sustained changes in personal practice, are shown in table 4.

Table 4.

Changes in personal practice

Surgeons	Early feeding and mobilisation Valuing and respecting other staff Taking time to talk to patients Delegating certain tasks to nurses because they now trusted them as the nurses had also received training Having the courage to say ‘no’ to surgery for patients with a poor prognosis so that the patients would not have unnecessary financial expenditure
Anaesthetists	Using anaesthesia checklists to help improve their preparation and organisation Attempted to use the WHO Surgical Safety Checklist but encountered resistance, which made it difficult even though they wanted to use it Hand washing and stricter asepsis during spinal anaesthesia Using laryngeal mask airway
OR nurses	Better sterile technique Hand washing Attempted to use the WHO Surgical Safety Checklist but encountered resistance, which made it difficult even though they wanted to use it Attempting to count swabs and instruments whenever the surgeons would let them
Ward nurses	11/13 said they had changed their personal practice with regard to hand washing and infection control on the wards Taking more care with medication (check calculations and doses with 2 people when possible) Recording the dose and time when medication is given in the patients’ chart (this was not carried out before) Early feeding and mobilisation after surgery

OR nurses, operating room nurses.

Many participants reported implementing changes in their personal practice. The most significant impact was among nursing participants, where 60% reported sustained improvements in infection control practice. Participants also mentioned changes in overall teamwork and communication; key patient safety practices, including nurses verifying and recording medications and initiating a formal handover process at shift change.

Mentoring of anaesthesia providers by surgical missions can be challenging, as visiting teams often use different drugs and equipment than are available locally. However, in these settings, a focus on non-technical skills such as organisation, teamwork, communication and infection control can have a lasting impact on trainees and their practice even in very different environments.

Kirkpatrick level 4—results

Changes in institutional practice and patient outcome were grouped by hospital and are listed below.

Hospital A

• Nurses from two specialties (maxillofacial and orthopaedics), who were mentored simultaneously with their surgeons, have changed the practice for dressing changes in their department. Prior to training, surgeons would do the dressing changes with little help from the nurses. Now the nurses prepare everything with close attention to infection control procedures and in most cases are allowed to change the dressings themselves. They reported that surgeons still handled difficult cases, but now the nurses prepare everything beforehand and assist, making it faster and easier to maintain sterility, thereby reducing infections. The nurses reported that this change made them feel valued and respected by the surgeons, and the surgeons reported having more time for other things.

• These nurses also reported that they felt empowered and no longer fearful to make suggestions about patient care. They said that they would remind surgeons to wash their hands before and after examining patient dressings and would make suggestions about analgesia for patients in pain. The surgeons independently corroborated these examples stating that they found nurses’ suggestions helpful and that they preferred this team approach to patient care.

• One OR nurse had taken the Mercy Ships’ teaching materials and written 15 new OR policies which were awaiting approval from the Hospital Director.

Hospital B

• The ward nurses had learnt the importance of documenting the medication given (dose and timing) and this was now standard practice across the ward. They were also trying to have two nurses check the dose of medication; this was difficult as often they had only 3 nurses for up to 48 patients, so it was not always possible.

• The ward nurses had instituted ‘handover’ at the change of nursing shift, verbally discussing each patient's status with each other. Before the training, they tended to view each shift as a separate entity and not a continuous process, and the addition of these two new practices notably affected patient care.

• One OR nurse had been asked by the Hospital Director to become the ‘trainer for the operating room’. He reported that they were trying to focus on patient safety specifically with regard to hand washing; using the WHO Surgical Safety Checklist; skin preparation of the surgical site before incision; using indicators of sterility on the autoclave machines; and counting swabs and instruments presurgery and postsurgery.

Hospital C

• One anaesthetist participated in the mentoring programme from this hospital. He reported that he had personally found the mentoring experience beneficial but was unable to implement any changes in practice in his hospital after he had returned.

Hospital D

• One surgeon and two ward nurses participated together in 1 week of mentoring in gynaecology and obstetrics fistula surgery. They excitedly reported how they learnt the importance of early feeding and mobilisation after surgery, and had instituted this in their hospital. Prior to training, it was standard practice for postoperative patients to lie in bed starved for 3 days (to prevent bowel movements). Now they feed patients immediately and get them out of bed; they have noticed that patients tend to go home 1 day earlier. They started this change with only this one surgeon's patients, but once the other surgeons saw the difference in outcome they started doing the same thing. Now 1 year later, it has become standard practice in this large hospital.

Facilitators and inhibitors to change

The top five responses to participant identified ‘facilitators’ and ‘inhibitors’ to change are identified in table 5.

Table 5.

Participant identified ‘facilitators’ and ‘inhibitors’ to personal and organisational change

Facilitators

Participants themselves and others seeing the benefit to patients (in the Mercy Ships hospital) as opposed to just being told about a new technique or new practice Being trained as part of a team, ie, not just on your own but with someone else from your institution—ideally a surgeon and a nurse Supportive leadership (from a surgeon or hospital director) Courage to persist and keep going Credibility as being trained by Mercy Ships as an external organisation

Inhibitors

Resistance to change from colleagues due to lack of belief in/understanding of new techniques and practices Lack of materials Lack of staff Lack of support from ‘superiors’ Changing long-standing habits and attitudes is hard work

Participants indicated that being able to see techniques and behaviours (teamwork and good communication) in practice and their subsequent benefit to patients was a powerful facilitator for change. Participating in training as a ‘hospital team’ (having more than one person from the same hospital department) and having a supportive hospital director were also valuable. Team-based learning in non-surgical contexts is known to be important for sustainability in low-income settings.⁹ Several of our participants came from one local ward and trained together; once back in their hospital setting, they helped each other put new skills into practice and persuaded others to follow. They also reported that training with Mercy Ships brought some additional credibility that aided instituting change. Furthermore, in most cases, the participants had been selected by their chief of nursing or hospital director, so there was an expectation of ‘bringing something back to share’ when they returned to work. This emphasises the value of partnerships between the training institution and the receiving institution, where objectives are shared and there is joint ownership of the programme.

Resistance to change among older staff was the most common inhibitor identified. Lack of sufficient staff and equipment were also identified as problems. Budget constraints, especially in LMICs, make this a formidable challenge. Without support from hospital directors or NGOs lobbying the Ministry of Health, improvements in this area are likely to be slow. However, quality improvement initiatives such as the WHO Surgical Safety Checklist can save lives simply through improved communication, teamwork and organisation.⁸ Therefore, if mentoring participants can lead to benefit in terms of these so-called ‘soft’ or non-technical skills, then patient outcomes can improve despite continuing staff and equipment shortages. Donations of suitable equipment should also be considered as a way to augment training programmes, so that acquired skills can actually be put into practice.

No participant reported any negative consequences of the training programme, and none had any additional training since the Mercy Ships’ programme.

Our response

This evaluation showed that a mentoring programme for surgeons, anaesthetists and nurses can have a measurable positive impact on improved personal and institutional practice 12–18 months after the programme finished. This is the first long-term impact evaluation to describe changes after short-term NGO training for surgical teams. Mercy Ships uses the established train-the-trainer model in the courses offered to local medical professionals,^{10 11} and we are in the process of developing the train-the-trainer model for the mentoring programme as well. In response to the identified facilitators and inhibitors to change, we developed a theory of mentoring as part of surgical training. Referred to as the Partner Unit Mentoring Program (PUMP), this initiative has three components: (1) mentor-based training, (2) equipment donations and (3) infrastructure assessment and renovations where necessary. Mentoring is surgical-team focused; a surgeon, anaesthetist and several nurses from one specialty participate together. The Mercy Ships Hospital Director now engages with the local directors to facilitate integration of new initiatives into hospital policy, and to help overcome the resistance of older staff. PUMP was designed to increase accountability, and to increase coordination of the training with equipment and infrastructure needs. We have started using this model in Madagascar and evaluation is underway.

Conclusion

Our evaluation supports Meara's suggestion that “NGOs should have a training component hardwired into their programmes”,¹ and provide evidence that sustainable, significant change is possible. After just 1 week of mentoring, one team witnessed the benefit of early feeding and mobilisation after surgery, and after implementing this change in their hospital, they noted that patients were being discharged a day earlier. This is a financially significant outcome for patients, because globally an estimated 33 million individuals face catastrophic health expenditure as a result of payment for surgery and anaesthesia.^{1 12} We therefore make the following recommendations for other organisations seeking to introduce a training component to their programmes:

1. Apprentice-type mentoring should be a feasible first step for most NGOs: Mentoring is based on shared experience and relationship, which requires less preparation and organisation than formal lecture style teaching. Mentoring also offers greater opportunity to model non-technical skills known to improve patient outcome. Therefore, while large group lectures and training courses can have a lasting impact,^{10 11 13 14} mentoring is likely to be easier to incorporate into routine NGO practice.

2. Collaborative planning and preparation is essential: Mentoring involves complex, dynamic, interpersonal relationships. A successful mentoring programme requires time, interest and commitment of mentors, participants and organisational leadership. As a result of cooperation with the Ministry of Health, national and international professional organisations, and hospital directors, we were able to include suitable participants from supportive organisations/hospitals. The participants completed an application form to assess motivations and we only used experienced mentors who had offered to train.

3. Language and time available should not be viewed as absolute barriers to training: We successfully used non-medical, non-professional translators when needed and mentoring was of various durations (1 week to 35 weeks). A long-term positive impact can occur even after 1 week of mentoring.

4. Robust data collection to better quantify impact will be a necessary next step: Future NGO implementers should ensure the time and resources necessary for recording, transcription and professional translation of interviews. A more in-depth assessment should include questions of long-term impact on satisfaction in work, ability to get work overseas, reduction in stress levels, unintended negative consequences and improved understanding of patient safety. In addition, future NGO implementers should aim to systematically track long-term quantifiable outcomes such as perioperative mortality rates, surgical site infection rates and hand washing compliance rates among nursing staff.

In our opinion, the time has come for NGOs to consider how best to incorporate training into their programme culture and strategy. Training, even if only for a short duration, can produce a lasting impact, and the value of non-technical skills should not be overlooked.