Evaluation of a point of care ultrasound curriculum for Indonesian physicians taught by first-year medical students

Abstract

BACKGROUND:

The purpose of this study was to assess the short-term efficacy of a 4-week ultrasound curriculum taught by American first-year medical students to general practitioners working in public health care clinics, or puskesmas, in Bandung, Indonesia.

METHODS:

We performed a prospective, observational study of Indonesian health care practitioners from public clinics in Bandung, Indonesia. These practitioners were enrolled in a 4-week ultrasound training course taught by first-year American medical students. A total of six sessions were held comprising of 38 ultrasound milestones. A pre-course and post-course written exam and practical exam was taken by each participant.

RESULTS:

We enrolled 41 clinicians in the course. The average pre-course exam score was 35.2% with a 2.4% pass rate, whereas the average post-course exam score was 82.0% with a 92.7% pass rate. The average practical score at the completion of the course was 83.2% (SD=0.145) with 82.9% of the class passing (score above 75.0%).

CONCLUSION:

Our data suggests that first-year medical students can effectively teach ultrasound to physicians in Indonesia using a 4-week intensive ultrasound training course. Future studies are needed to determine the amount of training required for proficiency and to evaluate the physicians’ perceptions of the student-instructors’ depth of knowledge and skill in point of care ultrasound.

INTRODUCTION

Indonesia is the fourth most populous country in the world with a population of 250 million and an annual growth rate of 2.1%.[1] Currently, there is a lack of access to health care with only 0.11 general practitioners per 1 000 individuals.[2] According to 2011 data polling Indonesian primary care physicians, sonography is offered by less than 50% of general practitioners in the country.[3,4] In contrast, in the United States, point of care ultrasound (POCUS) is used clinically and now being incorporated into American medical school curriculums.[5] As a portable and radiation-free imaging modality, POCUS has been shown to extend the scope of care to facilitate diagnosis.[6] POCUS has been implicated as a cost effective tool in the absence of expensive and non-portable radiological services to aid in diagnosing patients.[7–9] In developing countries where there is lack of access to health care and a growing need for medical screening, POCUS may prove to be a method to help identify pathology in these settings.[10]

According to the World Health Organization, Indonesia not only has a high prevalence of HIV, but it also has one of the highest rates of TB in the world.[11–13] Thus, there is potential benefit to not only increasing basic ultrasound education for Indonesian physicians, but also in expanding physicians’ capability to apply ultrasound to this large subset of patients. Possible imaging includes Focused Assessment with Sonography for HIV-associated Tuberculosis (FASH) exam as well as the Focused Assessment with Sonography in trauma (FAST). Recent evidence has shown that ultrasound education programs established by foreign physicians in developing countries have been proven to be successful and sustainable.[14,15] However, given the limited availability of skilled physicians able to conduct such ultrasound education programs abroad, few studies have evaluated medical students’ ability to effectively teach ultrasound practical skills and theoretical knowledge. Recent studies have shown that ultrasound is a teachable skill that may be applicable in the clinical setting after a short course of intensive training.[16–19]

We hypothesized that first-year American medical students are capable of teaching a 4-week, intensive ultrasonography course to Indonesian primary care physicians. The aim of this study was to assess the short-term efficacy of a 4-week ultrasound curriculum taught by American first-year medical students to general practitioners working in public health care clinics, or puskesmas, in Bandung, Indonesia.

METHODS

Study design and setting

This was a prospective, observational study utilizing a convenience sample of Indonesian health care practitioners from a government health office that oversees public clinics in Bandung, Indonesia. The study was approved by the Institutional Review Board. Written informed consent was obtained from all participants prior to enrollment.

Study protocol

We approached primary care clinical practitioners at public clinics in Indonesia for enrollment in the ultrasound training course. This course was voluntary and offered to any clinical practitioners that agreed to the time commitment of the course. The course itself was four weeks in duration, divided into 6 sessions. In addition, three hands-on, hour long sessions were arranged in didactic format for pupils to practice sonography. Prior to any instruction, clinicians were asked to take a pre-course examination which assessed their prior ultrasound knowledge, familiarity with sonography, their ability to access an ultrasound machine, and their primary language. All participants were asked to rate their comfort level using ultrasound on a scale from 0 to 5. Next, clinicians were given a 32-question multiple-choice exam designed to test their knowledge of the topics of the course. While the course was primarily taught in English, course materials were provided in both English and Indonesian to enhance comprehension.

The ultrasound training course was organized and taught by American medical students. These students had completed an entire first year of medical education which included human physiology, human anatomy, cardiac anatomy and basic ultrasound education. Their ultrasound education required ten hours of supervised hands-on training in addition to pre-session podcast on eight separate organ systems of medical ultrasound. These systems included cardiac, biliary, hepatic, pulmonary, renal, vascular, integumentary and genitourinary examinations. All teaching and testing was performed using SonoSite Nanomaxx (SonoSite Inc, Bothell WA) ultrasound machines. Each ultrasound machine was equipped with a linear, curvilinear, and phased-array probe.

We structured the ultrasound training course curriculum in such a way that each session began with a lecture presentation on a given topic (30 minutes), followed by hands-on ultrasound training (2 hours). The subjects were given a 10-minute break in the middle of hands on training. Lastly, a multiple choice quiz was administered to assess comprehension. A total of six sessions were held comprising of 38 ultrasound milestones. Initial teaching session covered basic ultrasound including physics, knobology, probe manipulation and scanning techniques. The second session focused on pulmonary ultrasound, evaluation of pneumothorax, pleural effusions and pneumonia. The third session taught cardiac ultrasound in the parasternal long axis view, parasternal short axis view, apical four chamber view and subxiphoid view. In the fourth session, practitioners were taught abdominal ultrasound which included hepatic, biliary, renal, intestinal, aorta and inferior vena cava. The fifth session consisted of obstetric and gynecologic ultrasound including identification of intrauterine pregnancy, estimated gestational age, visualization of the ovaries and bladder. The final teaching session consisted of a combination of the previous sessions to aid in Focused Assessment with Sonography in Trauma (FAST) and the Focused Assessment with Sonography for HIV-Associated Tuberculosis (FASH).

At the conclusion of the final session of the course, we gave participants a post-test, which was identical to the pre-test. Passing score for the exam was 75%. We also administered a practical exam, a post-course survey, as well as an “Intent to Use” survey. The practical exam required the subjects to perform ultrasound scans from each teaching session. Grading was based on both image acquisition and image interpretation. For each question, the clinician could earn up to 3 points– 1 point for correct probe orientation and indicator positioning, 1 point for placing the probe in the correct anatomical position, and 1 point for producing the correct image and correctly identifying the structures on the image. Additionally, participants were provided a post-course survey to measure their confidence in performing the scans and allow them to provide feedback to the course directors. The feedback portion included a section that asked the participants to provide ratings, on a scale of 1 to 5, of the aspects of the course that they found most helpful.

Data collection and statistics

Data was collected into Microsoft Excel from both the pre-test and post-test examinations as well as the practical examination. R was used for two-sample t-tests to determine statistical significance.

RESULTS

We approached 52 clinic practitioners for enrollment in the study. Of the 52 clinicians, 41 completed the course, resulting in a 78.9% retention rate (Figure 1). Out of the 41 physicians who completed the course, 38 attended all 6 class sessions (Figure 2). Twenty-two (53.7%) physicians had no prior ultrasound experience, 18 (43.9%) physicians had only observed the use of ultrasound, and 1 (2.4%) physician had taken a prior ultrasound course.

Figure 1.

Comparison of pre-course and post-course final exam score. There was a statistically significant difference (two-sample t-test, t=–16.5, P<0.001).

Figure 2.

Physicians indicated in the intent-to-use survey that they would perform cardiac and OB scans the most often.

The average pre-course exam score was 35.7% with a 2.4% pass rate, whereas the average post-course exam score was 82.1% with a 92.7% pass rate. There was a statically significant difference between the pre-course final exam scores (35.6%±13.9%) and post-course final exam scores (82.1%±11.5%) (two-sample t-test, t=–16.5, P<0.001) (Figure 1). The average practical score at the completion of the course was 83.2%±14.5% with 82.9% of the class passing (score above 75.0%).

The post-course survey indicated that the physicians had an overall increased level of comfort in all scans. There was a highly significant difference between all pre-course and post-course comfort levels performing the various scans (two-sample t-test, P<0.001) (Table 1). Post-survey feedback indicated that the physicians found that watching the scans being performed was most helpful (4.60±0.63), followed by practicing the scans (4.28±1.21), attending class lecture (4.23±0.80), studying online materials (3.54±0.98), using other study tools (3.50±0.92), and using the study guides (3.42±1.36). On average, the physicians independently studied 2.57±3.37 hours per week, and spent 0.66±1.49 hours per week in office hours.

Table 1.

The mean comfort level difference between all pre-course and post-course scans

Participants indicated that, given unlimited access to ultrasound machines, they would perform cardiac scans most often per week (8.4±16.7), followed by OB (7.8±12.9), bladder (5.8±12.1), renal (5.7±11.9), lung (5.5±12.2), FASH (5.5±12.5), gallbladder (4.3±11.2), and FAST (4.3±11.1). Single-factor ANOVA analysis results in a P-value of 0.794, indicating that the variation in values among different scans is not significant. Given their actual expected access to ultrasound machines, physicians indicated that they intended to perform OB scans most often per week (6.0±9.1), followed by cardiac (5.8±13.0), bladder (4.5±7.7), lung (3.9±6.9), renal (3.6±5.8), FASH (3.3±7.8), gallbladder (2.5±4.2), and FAST (2.1±3.5) (Figure 2).

DISCUSSION

Our data indicates that first-year medical students are able to teach POCUS to primary care clinicians in Indonesia. Following a month long intensive ultrasound teaching course, clinicians were able to identify basic anatomy as well as a breadth of pathology with the use of POCUS[20,21] The incorporation of ultrasound courses in medical education has fostered positive peer-led instruction in ultrasound in which student-teachers are positively evaluated as competent physician instructors.[22,23] Though numerous international ultrasound education programs have been conducted by physicians, studies have shown merit in medical student led peer-to-peer ultrasonography courses.[24,25] Given the limited information on the efficacy of medical students instructing international physicians in ultrasound, this study aimed to assess whether first-year medical students could effectively teach physicians POCUS techniques. The efficacy of a medical student run ultrasound training course for physicians is shown directly through the data collected for this study. We showed a statistically significant difference between pre-course and post-course final exam scores, having the post-course exam yield a 92.7% pass rate, indicating that the physicians’ acquired adequate ultrasound knowledge after completing the curriculum. Furthermore, there was a significant positive correlation between practical exam rank and post-course final exam rank, indicating that the curriculum successfully carried a theoretical understanding of ultrasound over to practical usage.

Prior to the training course, the majority of physicians did not feel comfortable performing POCUS. However, post-course comfort levels were significantly higher, indicating that the physicians perceived the curriculum as successful in aiding their ultrasonography abilities. This increase in the skills of ultrasound image acquisition can be readily seen by the 89.2% practical pass rate and the 92.7% post-course final exam pass rate achieved by physicians with minimal prior ultrasound experience, providing evidence to support the efficacy of this curriculum. At zero cost to the Indonesian public health clinics, or puskesmas, this ultrasound training course serves as a cost-effective method to increase exposure of general practitioners to POCUS. For more resource-restricted countries and regions, this methodology of utilizing a well-informed medical student population to teach POCUS may address deficits in health care professionals’ training in ultrasound. In addition, both parties involved are given the unique opportunity to learn medical practices that may differ internationally.

Currently, the World Health Organization’s published guidelines for training physicians in diagnostic ultrasound recommend that physicians undergo training over 3–6 months including 300–500 ultrasound examinations tailored to the local epidemiology.[14] Although our training course performed in this study does not meet WHO’s guidelines, Shah et al[18] and Maru et al[26] argues that focused applications of ultrasound may not require as much training as implicated by the WHO’s guidelines and that “some degree of flexibility should be allowed to ensure that training does not interfere with treatment access”. Furthermore, there is strong evidence that short term, intensive training can be both successful and sustainable for foreign physicians in developing countries.

For the population in Indonesia, the potential for POCUS to be used clinically has significant advantages due to the lack of clinicians per capita as well as the high rates of HIV and TB. The correlation between HIV and TB has been strongly characterized in research literature, evidencing that approximately 10% of people with HIV and latent TB infections will develop active TB each year.[27] Developing cities, such as Bandung, Indonesia, are especially prone to TB co-infections due to the high prevalence of HIV.[12,13,28] Some studies recommend the use of ultrasound as a supplementary diagnostic tool in the diagnosis of TB in HIV infected patients, particularly in regions with a high prevalence of TB and HIV co-infections.[29] In limited resource areas, POCUS could allow providers to make diagnoses in patients using the FASH exam.[8,30] The teaching material of this course was modeled off the guidelines and recommendations outlined by Dr. Tom Heller.[30,31] Our study showed that if given unlimited access to ultrasound machines, the physicians would spend about 5.5 hours per week performing the scan. However, due to limited access, they predicted that they’d actually only use the FASH scan about 3.3 hours per week, indicating true intentions to perform this scan to supplement their clinical decisions.

A one-year follow up is scheduled to re-evaluate our promising results and to survey how frequently the physicians who participated in the course utilize POCUS in their current practice. Repeat examination can be performed to reassess the physicians’ performance on the post-course exam to measure long-term retention of their ultrasound skills. Follow-up prospective studies are necessary to determine not only whether physicians have access to ultrasound in their outpatient clinics, but also if POCUS is accurately identifying pathology.

Limitations

This study has several limitations. In its present form, our data can only assess the short-term efficacy of the student-led ultrasound curriculum. No conclusions can be made regarding long term knowledge acquisition and future performance of POCUS. The sample size for our study is small and represents the abilities of Indonesian clinicians with minimal exposure to ultrasound. It is unclear if these results can be generalized to clinicians in other countries. Furthermore, the entire course was presented in English. This could have potentially created a learning environment that was less than ideal for the clinicians and limited the effectiveness of instruction. There was also the challenge of finding models with pertinent pathological findings as the models we used were healthy and generally did not have any pathological findings. On the practical grading rubric, image quality was a subjective measurement. Due to the limited timeframe of the study, a pre-course practical exam was not administered, inhibiting our ability to directly test the improvement of the participants’ clinical ultrasound skills. The pre-course written examination and survey served as our only assessments of prior knowledge and skill.

CONCLUSIONS

Our data suggests that first-year medical students can effectively teach ultrasound to physicians in Indonesia using a 4-week intensive ultrasound training course. Future studies are needed to determine the amount of training required for proficiency and to evaluate the physicians’ perceptions of the student-instructors’ depth of knowledge and skill in POCUS. Re-evaluation of physicians at a one year follow up must be performed to characterize the long term retention of ultrasound knowledge as well as to determine the accuracy with which the physicians are able to diagnose pathology in clinical practice after taking the course.