African relevance
-
•
Emerging research has focused on the implementation of programmes to improve Emergency Medical care in low- and middle-income Countries.
-
•
Emergency conditions contribute to a higher burden of disease in Low-and-Middle-Income Countries when compared to high-income Countries.
-
•
Emergency Medicine exposure and education is required during medical internship to improve emergency medical care provision in South Africa.
-
•
Improvement of Emergency Medicine training programmes during internship may result in better patient care provided by South African doctors during their Community Service and Independent Practice years.
Keywords: Modified Delphi study, Emergency medicine, Assessment tool, South Africa, Medical intern, Junior doctors
Abstract
Introduction
Emergency Medicine is a relatively new specialty in South Africa. Limited data is available regarding junior doctors’ competence in managing emergencies, however previous surveys have identified limited teaching and supervision of junior doctors in Emergency Medicine. Currently there is no formal standardised assessment tool to assess an intern's Emergency Medicine competence. The aim of the study was to, through expert opinion and consensus, develop an Emergency Medicine assessment tool to identify a level of appropriate Emergency Medicine knowledge at the end of internship.
Methods
The Modified Delphi Methodology was used to create an assessment tool via interaction with a panel of experts and took place over 4 rounds via an online survey platform. The initial round identified the high-priority topics within each intern domain. A questionnaire was created based on these topics and was presented to the panel for consensus during the following round/s. Rounds continued until each question met consensus of 75 %.
Results
A total of 35 panellists consented to participate, representing 6 provinces. The majority were Emergency Medicine specialists. High-priority topics included acute respiratory distress, polytrauma, dehydration and shock in children, airway management, and the agitated patient. A 40-question, multiple choice questionnaire was created with all questions reaching consensus.
Conclusion
This study highlighted the core high-priority Emergency Medicine topics that interns should be exposed to during their internship and created a questionnaire aimed at evaluating them. The study findings provide a novel contribution to identifying gaps in Emergency Medicine knowledge during intern training, allowing for potential interventions to be implemented to improve intern EM training. The addition of a clinical skills component and increasing the question database is suggested to further develop this tool. Larger iterative studies involving the HPCSA, and health education experts provide avenues for future research.
Introduction
Access to Emergency Care is a constitutional right within South Africa (SA) [1]. In district hospitals, a large percentage of this care is administered by doctors performing their compulsory community service year [2].
Emergency Medicine (EM) is a relatively new speciality in SA, having first been registered in 2003 [3]. There is a global emphasis on improving emergency care in Low-and-Middle-Income Countries (LMIC), including SA [4]. A paucity of data exists surrounding the application of EM in SA and a disconnect between the professional skills and service delivery has already been noted [4]. Wallis et al. emphasises that key areas to address include the appropriate training of junior doctors in EM [3].
In SA, after completing a 6-year undergraduate medical degree, junior doctors complete 2 years of internship in the public sector. The aim of internship is to “complete their medical training under supervision” to ensure that “competent and safe medical practitioners” are produced [5]. The Health Professions Council of South Africa (HPCSA) delineates 8 major medical domains to which interns must be exposed. Within each domain, emergency knowledge and skills are mandated, but EM is not included as a domain on its own [5].
Recently, the HPCSA has placed a larger emphasis on primary health care training during internship. Current recommendations include a one month district/tertiary hospital Emergency Department (ED) placement, under supervision [5,6]. However, previous reports have highlighted concerns with regards to supervision by junior doctors with less than 3 years’ of experience in the specialty, as well as expressing infrequent ED teaching [6]. Multiple emergency procedures are also reportedly performed without supervision, and not only does this pose a potential risk to patient safety, but poor knowledge and skill learnt may not be corrected [6].
The community service year was implemented as part of the national strategy to improve health services in under-served and rural areas [2,7]. Community service doctors practise in areas with limited supervision [2,7]. EM competence are of utmost importance in these settings – case mix data of rural EDs reflect high numbers of trauma, followed by infectious disease and non-communicable disease [8]. However, community service doctors express that triage and emergency procedures are areas in which they feel the least prepared [7,9].
Community service doctors are often expected to supervise and teach interns. This may be exacerbating a cycle of “confident incompetence” [6,10]. Competence can be defined as the “degree to which the individual can use knowledge, skills and judgement associated with the profession to perform effectively in the domain of possible encounters defining the scope of professional practice” [11]. The literature contains limited data regarding junior doctors’ competence in managing emergencies within the SA context. While there is published literature investigating internship and community service in terms of adequate preparation for independent practice, none of these cover EM specifically. This may result from the fact that EM is a relatively new specialty in SA, with undergraduate and intern EM exposure and training still in its developmental phases [3].
A need exists to assess and determine the EM competence of junior doctors. The aim of a standardised assessment tool is not to assess individual competencies or delay career progression, but to assess areas where training in emergencies may be improved. In order to effect training interventions to improve clinical outcome, a standardised system to measure knowledge must first be created [12]. Formative testing including simulation and direct observation are impractical and difficult to standardise across sites. An objective, single best answer (SBA) assessment is practical, can be performed online, assesses a range of topics across multiple EM domains, and can be utilised to assess problem-solving [13].
The aim of the study was to, through expert opinion and consensus, develop an EM assessment tool for medical interns training in SA - identifying a level of appropriate EM knowledge that interns should have acquired during internship. This was done through the implementation of the Modified Delphi technique.
Methods
The use of Delphi methodology was utilised, as it allowed for expert group consensus over single human opinion [14]. Delphi methodology employs a consensus building technique, consisting of iterative surveys to a panel, anonymous to one another, with feedback on responses shared in subsequent rounds [14], and termed "Modified" if web based technology is used [15]. This method has gained popularity over recent years in the healthcare setting as a method to develop assessment criteria and improve quality of care provided in areas where there are standardisation gaps [14,16,17]. We chose this approach to allow for diversity in geography, increased validity due to anonymity and prevention of bias associated with group discussion, and representation of diverse views based on experience and expertise [18].
The Modified Delphi study took place over 4 rounds via email and an online survey platform, Survey Monkey (Momentive Al. Copyright 1999 - 2022; Momentive) from July 2022 to October 2022. All responses remained anonymous and individual responses were weighted equally.
Panellist identification, selection, and recruitment
A purposive and snowball sampling technique was utilised to identify potential panellists, with the aim of selecting an expert panel diverse in gender, race, and geography. The snowball technique may introduce sampling bias by including a group of experts well known and similar to each other. To improve heterogeneity, we identified panellists from university faculty websites, the HPCSA register, and hospital EM/FM faculty websites, ensuring invites were sent to experts in all 9 provinces. 108 potential panellists were invited via two emails sent one week apart. Panellists who signed informed consent were sent a link to Round 1. Although there is no standardised panel size for Delphi studies, most panels range from 10 to 100 [14]. We set a minimum panel number of 20 - aiming for as large a panel as possible to ensure heterogeneity, balancing the practicality of risking non-consensus, additional rounds, and potential high attrition rates.
Inclusion criteria into the panel included
-
•
Healthcare Practitioners registered with the HPCSA.
-
•
Senior doctors/specialists with at least 5 years’ experience in EM or Family Medicine (FM), practising in the public sector.
-
•
Experience/current involvement with medical intern training.
-
•
Access to an electronic device that can operate email and Survey Monkey (Momentive AI. Copyright 1999 - 2022; Momentive).
Initiation of study
We aimed to develop a 40-question SBA multiple choice questionnaire. 40 questions were chosen to optimise expert participation and retention and were allocated according to time spent in the domain, a representation of the current HPCSA internship training priorities. We agreed that this number would provide enough questions to test each domain's EM knowledge without risking too much panellist attrition, while acknowledging that this is not a comprehensive audit of intern EM knowledge. General Medicine, General Surgery, Obstetrics & Gynaecology and Paediatrics were assigned 5 questions each. Anaesthetics, Psychiatry and Orthopaedics were assigned 3 questions each. FM (including the one month of EM) was assigned 11 questions (See Table 4).
Table 4.
Description of the results from Rounds 2-4.
| Total | General Medicine | General Surgery | Paediatrics | Obstetrics and Gynaecology | Anaesthetics | Orthopaedics | Psychiatry | Family medicine | |
|---|---|---|---|---|---|---|---|---|---|
| Round 1 | |||||||||
| Number of months spent in each domain during internship | 24 | 3 | 3 | 3 | 3 | 2 | 2 | 2 | 6 |
| Total topics per domain | 40 | 5 | 5 | 5 | 5 | 3 | 3 | 3 | 11 |
| Round 2 | |||||||||
| Questions > 85 % consensus | 29 | 1 | 2 | 3 | 3 | 3 | 3 | 3 | 11 |
| Questions 75 – 85 % consensus | 8 | 3 | 2 | 2 | 1 | 0 | 0 | 0 | 0 |
| Did not reach consensus | 3 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| Round 3 | |||||||||
| Questions > 85 % consensus | 10 | 3 | 3 | 2 | 2 | 0 | 0 | 0 | 0 |
| Questions 75 – 85 % consensus | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Did not reach consensus | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Round 4 | |||||||||
| Questions > 85 % consensus | 2* | 1 | 0 | 0 | 1* | 0 | 0 | 0 | 0 |
| Questions 75 – 85 % consensus | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Did not reach consensus | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
One Obstetrics & Gynaecology Question in Round 3 was reposed as a new question in Round 4 to ensure consensus.
The panellists were allocated two weeks to complete each round with a reminder email sent halfway through the allocated period and a final reminder sent 24 h prior to the deadline. Incomplete responders received a reminder 24 h prior to the deadline. The completed portion of the survey was included in the analysis, however incomplete responders were subsequently excluded from the following rounds. The authors used the following two weeks to critique the responses and formulate the new/modified, and then initiated the next round. We planned 3 rounds as sufficient for consensus while still aiming to limit attrition rates, with the possibility of additional rounds until 40 questions had met consensus.
Description of rounds
The aim of the first round was to identify important EM topics from each of the 8 major intern domains. Within each domain, a list of the core emergencies were identified from the HPCSA intern booklet and coalesced into a condensed version which was presented to the panellists during Round 1. The panellists selected priority topics from each domain to be used in creating the multiple-choice questions (MCQs).
After Round 1 identified the high-priority topics, 40 SBA MCQs were formulated for each topic by the researchers, guided by the National Board of Medical Examiners in constructing written-test questions for the basic and clinical sciences [19].
These questions, together with the suggested answers, were presented to the panellists throughout multiple rounds with the following potential answers:
“Agree” (the questions can be included into the final questionnaire unchanged.)
“Agree with the following modifications” (the question tests the correct knowledge and can be included into the final questionnaire but requires slight modifications to improve the question quality.)
A free form text box was provided for suggestions, comments, and modifications from the panellist.
“Disagree” (the question must be discarded, and a new question created.)
In subsequent rounds, panellists were presented with modified questions of the previous round's questions meeting consensus for “Agree with the following modifications”, as well as newly developed questions for topics that did not meet consensus. After each domain, the panellists were given the opportunity to provide general comments on the questions created.
At the start of each new round, a summary of the results from the previous round was presented to the panellists, including the suggested modifications and comments from the panel. All engagement with the panellists was anonymous, via email and the online survey platform.
Consensus
While there is no consensus in Delphi methodology as to what cut-off should be used for consensus, 75 % appears to be the most widely utilised [20]. Consensus was therefore set a priori at 75 % for inclusion of questions to which the panellists responded "agree". For those questions reaching 75–85 % with a combination of “agree” and “agree with modifications”, the suggested modifications from the panellists were incorporated into the original question or a new question was created. This modified or newly created question was presented to the panellists in the subsequent round/s until a consensus of 75 % was reached. Questions reaching less than 75 % consensus were deleted entirely, and a new question created under the topic. New questions were presented to the panellists in the subsequent rounds until consensus was met.
If a question was only modified, the panellists could “agree” or “disagree” to the modified question in subsequent rounds (the question having already met consensus, but an attempt at improving the question was made). Any new questions were posed with the 3 original optional answers of “agree”, “agree with modifications” and “disagree” and the cycle repeated.
Results
Expert panel characteristics
A total of 35 panellists met inclusion criteria and consented to participate in the study (32 % of the initially identified 108 potential panel). The demographics of the panel are shown in Table 1. Most panellists were between 31 and 40 years old, with over 80 % being between the age of 31 and 50. Approximately 75 % of the panellists were working in EM. At least 33 % of the panel are involved in teaching medical students/interns in EM. Six provinces were represented, with the top 3 being Western Cape, Eastern Cape and Gauteng.
Table 1.
Panellist demographics.
| Number of Panellists (Total = 35) |
Percentage (%) | |
|---|---|---|
| Age (Years) | ||
| 30 or younger | 0 | 0,00 |
| 31–40 | 18 | 51,43 |
| 41–50 | 11 | 31,43 |
| 51–60 | 5 | 14,29 |
| 61–70 | 1 | 2,86 |
| 70 or older | 0 | 0,00 |
| Gender | ||
|---|---|---|
| Female | 10 | 28,57 |
| Male | 24 | 68,57 |
| Gender Fluid | 1 | 2,86 |
| Province | ||
|---|---|---|
| Western Cape | 13 | 37,14 |
| Eastern Cape | 8 | 22,86 |
| Gauteng | 7 | 20,00 |
| Kwazulu Natal | 5 | 14,29 |
| Northern Cape | 1 | 2,86 |
| North West | 1 | 2,86 |
| Free State | 0 | 0,00 |
| Limpopo | 0 | 0,00 |
| Mpumalanga | 0 | 0,00 |
| Work Experience | ||
|---|---|---|
| EM Specialist | 21 | 60,00 |
| 5 years or more experience as an EM Specialist: 4 | ||
| FM Specialist | 9 | 25,71 |
| 5 years or more experience as an FM Specialist: 3 | ||
| Other: 5 years or more experience in EM as a medical officer/registrar | 5 | 14,29 |
| Academic Involvement | ||
|---|---|---|
| Involvement in training final year medical students in EM | 12 | 34,29 |
| Involvement in training interns in EM | 12 | 34,29 |
| Current or previous intern curator | 0 | 0,00 |
Delphi round outcomes
Fig. 1 provides a summary of the rounds and results of the Delphi process. Each subsequent round was met with some expected attrition. 30 (28 complete and 2 incomplete) responses were received during Round 2 with an attrition rate of 14,29 %. Round 3 met the highest attrition rate of 20,00 %, however the incomplete responses remained minimal at 1 incomplete response and a total of 24 complete responses. Round 4 consisted of a total of 23 complete responses, 0 incomplete responses and 1 panellist attrition (4,16 %).
Fig. 1.
Description of the Modified Delphi Rounds / Delphi process.
A 100 % completion by the panellists occurred during Round 1 Table 2 displays emergency topics identified as high-priority within each domain during Round 1, and Table 3 lists topics excluded based on the number of votes received.
Table 2.
The topics that were presented to the panellists during Round 1 and their corresponding votes.
| Topics chosen during Round 1 | Number of Votes (Total = 35) | Percentage of Votes (%) |
|---|---|---|
| GENERAL MEDICINE | ||
| Approach to acute respiratory distress | 32 | 91,43 |
| Approach to different types of shock | 30 | 85,71 |
| Approach to acute cardiac arrhythmias | 27 | 77,14 |
| Approach to seizures | 20 | 57,14 |
| Approach to altered mental state | 20 | 57,14 |
| GENERAL SURGERY | ||
| Resuscitation of a polytrauma patient (ATLS principles) | 35 | 100,00 |
| Approach and management of an acute abdomen | 29 | 82,86 |
| Approach and management of a head injury | 25 | 71,43 |
| Approach to upper and lower GIT bleeding | 23 | 65,71 |
| Approach to trauma related shock | 21 | 60,00 |
| PAEDIATRICS | ||
| Approach to acute respiratory disorders (Paediatrics) | 35 | 100,00 |
| Approach to acute gastroenteritis and hypovolemic shock in children | 35 | 100,00 |
| Resuscitation principles in children (PALS/APLS) | 33 | 94,29 |
| Approach to and management of seizures in children | 28 | 80,00 |
| Resuscitation principles in neonates | 23 | 65,71 |
| OBSTETRICS & GYNAECOLOGY | ||
| ESMOE: Shoulder dystocia, cord prolapse, breech delivery, PPH | 32 | 91,43 |
| Identification of gynaecological acute abdomen (ectopic pregnancy, ovarian torsion etc.) | 30 | 85,71 |
| Approach to obstetric haemorrhage in early pregnancy | 28 | 80,00 |
| Identification and management of different types of miscarriage | 22 | 62,86 |
| Approach to maternal sepsis | 19 | 54,29 |
| ANAESTHETICS | ||
| Medication, equipment, procedural skills related to airway management | 35 | 100,00 |
| Use of resuscitation equipment and medication (defibrillation, pacing, central venous access) | 27 | 77,14 |
| Use and optimisation of respiratory system using ventilator | 21 | 60,00 |
| ORTHOPAEDICS | ||
| Management principles of open fractures | 26 | 74,29 |
| Approach to and management of joint injuries and dislocations | 25 | 71,43 |
| Management principles of closed fractures | 19 | 54,29 |
| PSYCHIATRY | ||
| Approach to the agitated/aggressive patient | 35 | 100,00 |
| Differentiating between delirium, dementia, and acute psychosis | 26 | 74,29 |
| Suicidal risk assessment | 20 | 57,14 |
| FAMILY MEDICINE | ||
| Interpretation of ECGs | 34 | 97,14 |
| Approach to an undifferentiated medical patient | 30 | 85,71 |
| Approach to an undifferentiated trauma patient | 28 | 80,00 |
| Interpretation of imaging modalities | 27 | 77,14 |
| Approach to and management of cardiac arrest | 26 | 74,29 |
| Approach to anaphylaxis | 24 | 68,57 |
| Interpretation of the arterial blood gas | 23 | 65,71 |
| Approach to triage and SATS | 21 | 60,00 |
| Acute wound care and debridement | 20 | 57,14 |
| Primary, secondary and tertiary surveys | 19 | 54,29 |
| Types, problems and pitfalls of transport of the unstable patient | 16 | 45,71 |
Table 3.
The topics that were excluded from the high-priority topics based on the number of votes received during the Delphi Process.
| Topics Excluded dueing Round 1 | Number of Votes (Total = 35) | Percentage of Votes (%) |
|---|---|---|
| GENERAL MEDICINE | ||
| Management of toxin ingestion / exposure | 16 | 45,71 |
| Approach to hypertensive emergencies | 11 | 31,43 |
| Management of acute electrolyte disorders | 11 | 31,43 |
| Approach to acute focal neurological fallout | 8 | 22,86 |
| Approach to acute haematological disorders | 0 | 0,00 |
| Approach to acute rheumatological emergencies | 0 | 0,00 |
| GENERAL SURGERY | ||
| Management of penetrating injuries | 16 | 45,71 |
| Approach to paediatric surgical emergencies | 10 | 28,57 |
| Identification and management of blunt trauma | 7 | 20,00 |
| Identification and management of peripheral vascular emergencies | 3 | 8,57 |
| Identification and management of urological emergencies | 3 | 8,57 |
| Approach to neurosurgical emergencies | 3 | 8,57 |
| PAEDIATRICS | ||
| Approach to metabolic emergencies in children | 6 | 17,14 |
| Approach to altered mental status in children | 6 | 17,14 |
| Approach to toxins in children | 5 | 14,29 |
| Identification and management of cardiac emergencies | 4 | 11,43 |
| OBSTETRICS & GYNAECOLOGY | ||
| Maternal complications during third trimester | 18 | 51,43 |
| Maternal resuscitation of the unconscious patient | 15 | 42,86 |
| Foetal and maternal monitoring during labour | 11 | 31,43 |
| ANAESTHETICS | ||
| Different types of anaesthesia and analgesia and their complications | 15 | 42,86 |
| Preparation for anaesthesia in “special populations” (obese patients, geriatric patients, paediatrics, etc.) | 4 | 11,43 |
| Optimization of metabolic and cardiovascular derangements | 3 | 8,57 |
| ORTHOPAEDICS | ||
| Approach to cervical spine protection and cervical spine injury | 14 | 40,00 |
| Imaging interpretation and description of orthopaedic fractures | 7 | 20,00 |
| Management principles of paediatric orthopaedic emergencies | 6 | 17,14 |
| Approach to spinal cord injury including neurogenic shock | 4 | 11,43 |
| Identification and management of compartment syndrome | 4 | 11,43 |
| PSYCHIATRY | ||
| Mental Health Care Act and completion of Mental Health Care User (MHCU) forms | 19 | 54,29 |
| Complications or side effects of antipsychotics and other psychiatric medications | 5 | 14,29 |
| FAMILY MEDICINE | ||
| Approach to HIV associated emergencies | 15 | 42,86 |
| Approach to epistaxis | 15 | 42,86 |
| Approach to infectious diseases | 14 | 40,00 |
| Removal of foreign body | 14 | 40,00 |
| Approach to a rash | 14 | 40,00 |
| Approach to acute visual loss | 12 | 34,29 |
| Approach to otolaryngeal emergencies | 9 | 25,71 |
| Approach to acute red eye | 6 | 17,14 |
| Approach to environmental emergencies | 5 | 14,29 |
| Identification and management of Stevens-Johnson Syndrome (SJS) and Toxic Epidermal Necrolysis (TENS) | 5 | 14,29 |
| Approach to otological emergencies | 4 | 11,43 |
| Petechiae and purpuric emergencies | 2 | 5,71 |
| Approach to acute eye pain | 2 | 5,71 |
Table 4 describes each round's results and the number of questions that met consensus within each round. Most questions met consensus during Round 2, with a total of 37 questions meeting 75 % or more consensus. Of these, 29 questions were automatically included into the final questionnaire (85 % or more consensus) and 8 questions were modified according to the previously described methodology. Only 3 questions did not meet consensus and were replaced by new questions which subsequently met consensus within the following two rounds.
Discussion
Final year interns are likely not fully prepared to manage emergencies due to a lack of rigorous national guidelines for gaining EM experience during internship [7,9]. This modified Delphi study is, to the best of our knowledge, a first attempt to obtain consensus on an EM assessment tool for interns. High-priority topics identified from this study include, acute respiratory distress, polytrauma, dehydration and shock in children, airway management, and management of the agitated patient.
Triage and the use of the South African Triage Scale(SATS) also reached consensus as an important topic for intern learning. Triage has been shown to improve ED overcrowding and identify sick patients to allow for timely intervention [21,22]. The SATS was designed specifically for implementation in LMIC [23], and Nkabinde et al. noted that community service doctors feel underprepared to perform triage [7].
Interestingly, questions using emergency point-of-care ultrasound (PoCUS) reached consensus, however comments in the free-form text included: “eFAST is not formally taught … so [ultrasound interpretation] will be unfair”. Emergency PoCUS is not formally taught in undergraduate and internship training. However, evidence regarding its utility in resource limited settings as well as for emergencies and cardiac arrest is emerging [24], [25], [26]. This tool may be used to identify those that cannot interpret PoCUS, however it cannot determine if PoCUS is performed correctly, a recognized limitation of this study.
Deliberate self-poisoning is the most common cause for suicide in LMIC [27], accounting for 90 % of non-fatal suicides [27]. These presentations place a significant burden on SA's already financially constrained healthcare system [28]. It is therefore interesting that toxicology was not one of the high-priority topics. Had we allowed for a higher number of high-priority topics within each domain, adult toxicology would have been included. We recognize this as a limitation of the study, and do not discount the importance of emergency toxicology management in SA.
Limitations of study
An expected limitation was panellist attrition, mitigated by limiting questionnaire size. This limits the number of priority topics and may unintentionally exclude topics, further emphasising the need for larger iterative studies to be performed. A high attrition rate affects the heterogeneity of the results resulting in potential bias. Sampling bias via the snowball technique is also a potential limitation.
Although significant effort went into identifying EM and FM stakeholders, another limitation includes the absence of input from health professions education experts and policymakers. Their input on optimal ways to provide medical education and practical ways to rehaul the current EM curricula would be invaluable. This provides a potential avenue for future involvement.
Further review, validation, potential standard setting and expansion and refinement of a sustainable question bank need to be considered. As a next step, a pilot study is being planned for interns in two tertiary-level hospitals in the Eastern Cape, South Africa. Options for expansion and refinement of the question bank include inclusion of potential “test questions” that are not scored, but then retrospectively assessed for validity and reliability and future inclusion in later tests. Validity testing utilising item analysis as well as expert opinion from a broader range of experts, and reliability assessment utilising Cronbach's alpha can be applied to future tests. We aim to grow a sustainable and valid question bank that can be utilised to effectively assess intern training in South Africa.
This tool represents a summative assessment, although not planned for high stakes decision making, it also has little educational impact on the test takers. The aim of the assessment is to assist with creating a picture of the overall strengths and gaps in intern emergency training. A longer term, formative assessment methodology would have the advantages of strengthening and improving gaps in individual doctor's knowledge but may not be best suited to the milieu of South African internship training.
As an assessment tool, this project is limited in that it does not reflect clear learning outcomes or blueprinting for the 2-year internship, but its validity is driven by the HPCSA-stipulated time in each domain as well as the expert consensus on high-value topics. This information may be valuable in retrospectively driving training in these high value topics, as well as advocating for a longitudinal EM training programme during internship.
The development and ongoing refinement of this tool remains an iterative process. While MCQs are not validated for an assessment of skills, a self-assessment of skill preparedness will be added to the pilot intern study, with potential for a mixed-methods study including qualitative assessment of emergency preparedness amongst interns.
We hypothesise that these further steps will highlight a need for a lengthened EM training block and clear curriculum, likely arising from engagement with the HPCSA and individual intern training sites. It is our hope that this limited, summative assessment may represent a first step in these engagements.
Conclusion
To date, this is the first known study of this nature within Africa and has the potential to lay foundations for a significant impact on EM training amongst junior doctors. This assessment tool may be used to identify current EM knowledge at the end of internship and identify training gaps for potential interventions with the overall objective of improving medical education. These gaps may be used to advocate for a longer EM rotation during internship, create a better structured EM curricula in collaboration with the HPCSA, policymakers and medical education experts, as well as potentially be incorporated into undergraduate training. Ultimately this will lead to well prepared and competent junior doctors who will provide a better level of care that will result in improved patient outcomes.
This represents a first step in evaluating the current EM intern training, and external validation, additional questions, and inclusion of practical components by a wide stakeholder engagement may lead to a significant impact on the training of junior doctors in South Africa.
Ethics
Ethical clearance for this study was approved by the Frere and Cecilia Makiwane Hospitals Research Ethics Committee. FCMHREC/A0118/2022.
Funding
All costs incurred during the study were covered personally by the researchers.
Data dissemination
A summary of the previous round's results/comments was presented to the panel at the start of each new round. Access to the final completed questionnaire was also provided to the panellists upon their request.
Author's contribution
Authors contributed as follows to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting the work or revising it critically for important intellectual content: LA contributed 60 %; LT contributed 17,5 %; SG contributed 17,5 % and RM contributed 5 %. All authors approved the version to be published and agreed to be accountable for all aspects of the work.
Declaration of Competing Interest
We have no conflict of interest to declare.
References
- 1.Kramer E.B. No one may be refused emergency medical treatment' - ethical dilemmas in South African emergency medicine. S. Afr. J. Bioeth. Law. 2008;1(2):53. https://hdl.handle.net/10520/EJC64473 [Google Scholar]
- 2.Reid S.J., Peacocke J., Kornik S., Wolvaardt G. Compulsory community service for doctors in South Africa: a 15-year review. S Afr Med J. 2018;108(9):741. doi: 10.7196/samj.2018.v108i9.13070. [DOI] [PubMed] [Google Scholar]
- 3.Wallis L.A., Garach S.R., Kropman A. State of emergency medicine in South Africa. Int J Emerg Med. 2008;1(2):69–71. doi: 10.1007/s12245-008-0033-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Reid C., Habig K., Hsu R., Coombes S. Emergency medicine in South Africa – time to catch up. S Afr Med J. 2012;102(11):836. doi: 10.7196/samj.5913. 10.71962FSAMJ.5913. [DOI] [PubMed] [Google Scholar]
- 5.Health professions council of South Africa. Handbook on internship training. https://www.hpcsa.co.za/Uploads/Registration_Forms/MDB%20FORM/2020%20Forms/2021_IN_HANDBOOK_PART_I.pdfPretoria: HPCSA, 2021 (Accessed 8/12/21).
- 6.Bola S., Trollip E., Parkinson F. The state of South African internships: a national survey against HPCSA guidelines. S Afr Med J. 2015;105(7):535. doi: 10.7196/samjnew.7923. [DOI] [PubMed] [Google Scholar]
- 7.Nkabinde T.C., Ross A., Reid S., Nkwanyana N.M. Internship training adequately prepares South African medical graduates for community service – with exceptions. S Afr Med J. 2013;103(12):930. doi: 10.7196/samj.6702. 10.7196/samj.6702. [DOI] [PubMed] [Google Scholar]
- 8.Meyer N.T., Meyer G.D., Gaunt C.B. What presents to a rural district emergency department: a case mix. Afr J Prim Health Care Fam Med. 2020;12(1):2275. doi: 10.4102/phcfm.v12i1.2275. 10.41022Fphcfm.v12i1.2275. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Cameron B.J. Teaching young docs old tricks – was Aristotle right? An assessment of the skills training needs and transformation of interns and community service doctors working at a district hospital. S Afr Med J. 2002;92(4):276–278. [PubMed] [Google Scholar]
- 10.Marteau T.M., Wynne G., Kaye W., Evans T.R. Resuscitation: experience without feedback increases confidence but not skill. Br. Med. J. 1990;300(6728):849–850. doi: 10.1136/bmj.300.6728.849. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kane M.T. The assessment of professional competence. Eval Health Prof. 1992;15(2):163–182. doi: 10.1177/016327879201500203. [DOI] [PubMed] [Google Scholar]
- 12.Schmutz J., Eppich W.J., Hoffmann F., Heimberg E., Manser T. Five steps to develop checklists for evaluating clinical performance: an integrative approach. Acad Med. 2014;89(7):996–1005. doi: 10.1097/acm.0000000000000289. [DOI] [PubMed] [Google Scholar]
- 13.Sherbino J., Bandiera G., Frank J.R. Assessing competence in emergency medicine trainees: an overview of effective methodologies. Can. J. Emerg. Med. 2008;10(4):365–371. doi: 10.1017/s1481803500010381. [DOI] [PubMed] [Google Scholar]
- 14.Nasa P., Jain R., Juneja D. Delphi methodology in healthcare research: how to decide its appropriateness. World J Methodol. 2021;11(4):116–129. doi: 10.5662/wjm.v11.i4.116. 10.56622Fwjm.v11.i4.116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.von der Gracht H.A. Consensus measurement in Delphi studies: review and implications for future quality assurance. Technol Forecast Soc Change. 2012;79(8):1525–1536. doi: 10.1016/j.techfore.2012.04.013. [DOI] [Google Scholar]
- 16.Singer M., Deutschman C.S., Seymour C.W., Shankar-Hari M., Annane D., Bauer M., et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3) JAMA. 2016;315(8):801–810. doi: 10.1001/jama.2016.0287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Boulkedid R., Abdoul H., Loustau M., Sibony O Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS ONE. 2011;6(6):e20476. doi: 10.1371/journal.pone.0020476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Christie C.A., Barela E. The Delphi technique as a method for increasing inclusion in the evaluation process. Can. J. Program Eval. Vol. 2005;20(1):105–122. doi: 10.3138/cjpe.020.005. [DOI] [Google Scholar]
- 19.Paniagua M., Swygert K. 4th editor. Philadelphia: National Board of Medical Examiners; 2016. Constructing written test questions for the basic and clinical sciences; pp. 5–19.https://www.bumc.bu.edu/busm/files/2018/10/NBME-Constructing-Written-Test-Questions.pdf Available at. (Accessed: 20/02/23) [Google Scholar]
- 20.Diamond I.R., Grant R.C., Feldman B.M., Pencharz P.B., Ling S.C., Moore A.M., et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J. Clin. Epidemiol. 2014;67(4):401–409. doi: 10.1016/j.jclinepi.2013.12.002. [DOI] [PubMed] [Google Scholar]
- 21.Aacharya R.P., Gastmans C., Denier Y. Emergency department triage: an ethical analysis. BMC Emerg Med. 2011;11:16. doi: 10.1186/1471-227x-11-16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Meyer G.D., Meyer T.N., Gaunt C.B. Validity of the South African Triage Scale in a rural district hospital. Afr J Emerg Med. 2018;8(4):145–149. doi: 10.1016/j.afjem.2018.07.004. 10/1016/j.afjem.2017.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Conti A., Sacchetto D., Putoto G., Mazzotta M., De Meneghi G., De Vivo E., et al. Implementation of the South African Triage Scale (SATS) in a new ambulance system in Beira, Mozambique: a retrospective observational study. Int J Environ Res Public Health. 2022;19(16):10298. doi: 10.3390/ijerph191610298. 10.33902Fijerph191610298. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Kizito P.M., Bagonza K.D., Odakha J.A., Nalugya L.G., Opejo P., Muyingo A., et al. Diagnostic performance of Point of Care Ultrasound compared to chest X-Ray in patients with hypoxia at a teaching hospital emergency department in Uganda. Afr J Emerg Med. 2023;13(2):61–67. doi: 10.1016/j.afjem.2023.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Shaddock L., Smith T. Potential for use of portable ultrasound devices in rural and remote settings in Australia and other developed countries: a systematic review. J Multidiscip Healthc. 2022;15:605–625. doi: 10.2147/jmdh.s359084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Gottlieb M., Alerhand S. Managing cardiac arrest using ultrasound. Ann Emerg Med. 2023;81(5):532–542. doi: 10.1016/j.annemergmed.2022.09.016. [DOI] [PubMed] [Google Scholar]
- 27.Laher A.E., Motara F., Gihwala R., Moolla M. The profile of patients presenting with intentional self-poisoning to the Charlotte Maxeke Johannesburg Academic Hospital emergency department, South Africa. S Afr Med J. 2022;112(5):347–351. [PubMed] [Google Scholar]
- 28.Benedict M.A., Mofolo N., Adefuye A.O. The economic burden of deliberate self-poisoning: insight from a tertiary hospital in the Free State Province, South Africa. Pan Afr Med J. 2020;36:35. doi: 10.11604/pamj.2020.36.35.22346. 10.116042Fpamj.2020.36.35.22346. [DOI] [PMC free article] [PubMed] [Google Scholar]

