First-year medical students use of ultrasound or physical examination to diagnose hepatomegaly and ascites: a randomized controlled trial

Abstract

Purpose

To compare point-of-care ultrasound and physical examination (PEx), each performed by first-year medical students after brief teaching, for assessing ascites and hepatomegaly. Ultrasound and PEx were compared on: (1) reliability, validity and performance, (2) diagnostic confidence, ease of use, utility, and applicability.

Methods

A single-center, randomized controlled trial was performed at a tertiary centre. First-year medical students were randomized to use ultrasound or PEx to assess for ascites and hepatomegaly. Cohen’s kappa and interclass coefficient (ICC) were used to measure interrater reliability between trainee assessments and the reference standard (a same day ultrasound by a radiologist). Sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) were compared. A ten-point Likert scale was used to assess trainee diagnostic confidence and perceptions of utility.

Results

There were no significant differences in interobserver reliability, sensitivity, specificity, accuracy, PPV, or NPV between the ultrasound and PEx groups. However, students in the ultrasound group provided higher scores for perceived utility (ascites 8.38 ± 1.35 vs 7.08 ± 1.86, p = 0.008; hepatomegaly 7.68 ± 1.52 vs 5.36 ± 2.48, p < 0.001) and likelihood of adoption (ascites 8.67 ± 1.61 vs 7.46 ± 1.79, p = 0.02; hepatomegaly 8.12 ± 1.90 vs 5.92 ± 2.32, p = 0.001).

Conclusions

When performed by first-year medical students, the validity and reliability of ultrasound is comparable to PEx, but with greater perceived utility and likelihood of adoption. With similarly brief instruction, point-of-care ultrasonography can be as effectively learned and performed as PEx, with a high degree of interest from trainees.

Electronic supplementary material

The online version of this article (doi:10.1007/s40477-017-0261-6) contains supplementary material, which is available to authorized users.

Background

Point-of-care ultrasound (POCUS), defined as ultrasound performed at the patient’s bedside by the clinician providing care, has been shown to improve the quality and safety of patient care through increased diagnostic accuracy and procedural safety [1–7]. In addition, POCUS instruction as early as first and second year medical school has been shown to enhance knowledge acquisition and retention [8–10]. Surveys have shown that 50% of Canadian and 62% of US medical schools have integrated ultrasound into their curriculum, with 60% beginning in first-year [11, 12]. These findings are part of a worldwide trend [13, 14].

As an example of curricular content, a consensus statement from leading POCUS medical educators suggests that graduating medical students should be competent in abdominal POCUS, including liver, spleen, and gallbladder visualization, recognition of cholelithiasis, the sonographic Murphy’s sign, and peritoneal free fluid [15]. However, effective instruction in the short time available remains a challenge [12]. The majority of Canadian medical schools teaching ultrasound were able to devote only 1−5 h per year in instruction time [11]. Studies comparing targeted, practical instruction in POCUS to the current pedagogical gold standard, physical examination (PEx), could aid educators in deciding on time allocations for each, as well as help to establish evidence-based curriculum design. Given that there are few educational studies on abdominal POCUS, we chose this as the focus for our study [16]. Since the evidence-based PEx, founded on the Journal of the American Medical Association (JAMA) Rational Clinical Examination (RCE), devotes numerous complex maneuvers to the detection of hepatomegaly and ascites, we chose these pathologies to assess the role of learner-performed POCUS [17–19].

This randomized controlled study of first-year medical students compares the efficacy of brief, targeted instruction in either POCUS or the evidence-based PEx for the diagnosis of hepatomegaly and ascites. Outcomes assessed include interobserver reliability, sensitivity, specificity, accuracy, and positive and negative predictive values (PPV and NPV), as well as trainee confidence and preference.

Methods

Study design

This study was a prospective, single-center, randomized controlled trial performed at a tertiary teaching hospital in January 2015. Institutional approvals were obtained from the Research Ethics Boards of this hospital and the university medical education program.

Students were randomized to one of the two modalities using a random numbers table, and were blinded to patient clinical information. Two volunteer patients were used as models for all teaching sessions. The remaining eight patients were allocated to two tracks with equal distribution of abnormal clinical findings so that each track consisted of four patients. Each student examined four patients, with 7 min allotted for each exam. The reference standard for both the POCUS and PEx groups was the diagnoses and measurements obtained by an expert radiologist, who performed POCUS on these patients on the same day as the study, using the same ultrasound machine as the trainees (Fujifilm Sonosite Edge^®).

Participants

Students

Study subjects were recruited on a volunteer basis from the first-year medical school class at the University of Toronto. No monetary incentive was provided, and students did not receive academic credit for participating in this study. Students in subsequent years of training were excluded from the primary analysis given prior training on diagnostic abdominal PEx and POCUS. The medical students recruited for this study were in their second semester of first year, and had been introduced to ultrasound during the previous semester via online modules and ultrasound-enhanced anatomy sessions. The modules introduced students to relevant physics and knobology, and provided an overview of the ultrasound views of major organ systems. The ultrasound-enhanced anatomy labs consisted of three 30 min hands-on sessions, with a 1:4 instructor to learner ratio. Labs focused on the spatial relationships of abdominal, thoracic, and head and neck organs. With respect to prior PEx training, students had been introduced to cardiac, respiratory, and head and neck exams, but not the abdominal exam.

Patients

A total of ten consecutive, eligible patients were recruited from a tertiary hepatology clinic affiliated with the university. Informed written consent was obtained and a small monetary honorarium was provided for all participating patients.

Interventions

Half of students were randomized to receive 30 min of POCUS training, while the remainder received 30 min of PEx teaching [19]. Two investigators, one proficient in teaching POCUS, and another in teaching the JAMA RCE, taught each of these sessions. All sessions involved 10 min didactic instruction and demonstration on the volunteer patient, and 20 min student practice with instructor feedback. Both groups learned to assess (1) liver span, with hepatomegaly defined as liver span >14 cm at the mid-clavicular line (MCL) and (2) the presence or absence of ascites. The choice of 14 cm as the threshold for hepatomegaly for both POCUS and PEx was guided by the JAMA RCE, which states that a detected liver span >12–13 cm correlates with increased probability of hepatomegaly [18]. It was necessary to choose a consistent threshold for both groups; however, we acknowledge the wide variability in definitions of hepatomegaly, and the contribution of factors such as age, sex, body size, and accessory lobes [20–24].

Point-of-care ultrasound instruction included a review of probe orientation, image acquisition, and image optimization using depth and gain. Students were taught to identify the liver, spleen and hepatorenal recess in the supine position, identify fluid in the hepatorenal recess, and measure liver span in the MCL from dome-to-tip. MCL dome-to-tip was chosen given its high interrater and intrarater reliability [25]. Students demonstrated these maneuvers individually, with instructor feedback.

The PEx session was guided by teaching slides provided by JAMA. To detect ascites, students were taught to inspect for bulging flanks, and percuss for flank dullness, shifting dullness, and fluid wave. To detect hepatomegaly, students were taught to palpate for the lower border, percuss for the upper border, and measure liver span in the MCL. Likelihood ratios for each maneuver were discussed. Students demonstrated these maneuvers individually, with instructor feedback.

Data collection

Student data were collected after being anonymized and de-identified. Students recorded their results regarding the presence or absence of ascites and hepatomegaly, diagnostic confidence for each using a ten-point Likert scale (1 = not at all confident, 10 = very confident), and their measured liver span (Supplement 1). Following the study, students completed a questionnaire regarding their perceptions on the ease of learning, utility and applicability of their modality of training, each on a ten-point Likert scale (Supplement 2).

Outcome measures

The primary study outcome was to compare the interobserver variability between both POCUS and PEx to the reference standard. The secondary outcomes included: (1) participant confidence in results obtained either through POCUS or PEx, (2) ease of learning, (3) perceived utility, and (4) applicability (i.e., likelihood of future use).

Statistical analysis

Interobserver reliability between the reference standard and both POCUS and PEx was analyzed using Cohen’s Kappa for the diagnosis of ascites and hepatomegaly, and by the intraclass correlation coefficient (ICC) for liver span. ICC was also used to assess the correlation between all secondary outcomes and the examination modality used. Parameters were recorded as mean with standard deviation or 95% confidence intervals (95% CI). Data analyses were performed with IBM SPSS version 20.0 for Macintosh (SPSS Inc., USA) and R statistical software (version 3.1.0).

Results

Participant characteristics

A total of eight patients were enrolled in the study, the majority of whom were male (63%), with a mean age of 61.1 years (range 55–78) and mean BMI of 26.1 kg/m² (range 20–36). The eight patients were divided into two groups such that both normal and abnormal findings for ascites and hepatomegaly were represented in each group. Each student was randomly assigned to examine one group of patients using POCUS or PEx, such that each student examined four patients in total. Hepatomegaly, as defined in this study, was present in 38% of patients (mean 13.4 cm, range 10–17), and ascites in 50% of patients (mild-moderate 1, severe 3). A total of 44 first-year medical students were enrolled in the study, with 22 in the POCUS group and 22 in the PEx group.

Reliability, validity and performance for the diagnosis of ascites

A total of 88 POCUS and 88 PEx results were recorded for the diagnosis of ascites. The Cohen’s kappa for POCUS and PEx were 0.59 and 0.45 (p = 0.29), respectively, both indicating moderate agreement with the reference standard. There were no statistically significant differences in sensitivity, specificity, accuracy, PPV, or NPV between the PEx and POCUS groups (Table 1).

Table 1.

Comparison of interrater reliability, sensitivity, specificity, accuracy, and positive and negative predictive values of ultrasound compared with physical examination for ascites and hepatomegaly

	Ascites			Hepatomegaly
	POCUS (n = 88)	PEx (n = 88)	P value	POCUS (n = 84)	PEx (n = 88)	P value
Kappa	0.59	0.45	0.29	0.35	0.19	0.27
Sensitivity	77	84	0.59	67	48	0.21
Specificity	82	61	0.06	70	70	1.00
Accuracy	80	73	0.38	69	62	0.46
PPV	83	68	0.25	58	50	0.68
NPV	79	79	1.00	77	69	0.49

POCUS point-of-care ultrasound, PEx physical examination, PPV positive predictive value, NPV negative predictive value

Reliability, validity and performance for the diagnosis of hepatomegaly and measurement of liver span

There were a total of 86 POCUS and 87 PEx results for the diagnosis of hepatomegaly. The Cohen’s kappa was 0.35 for POCUS and 0.19 for PEx (p = 0.27). There were no statistically significant differences in sensitivity, specificity, accuracy, PPV, or NPV with POCUS as compared with PEx (Table 1). There were a total of 84 POCUS and 88 PEx measurements recorded for liver span. The ICC for correlation of measured liver span with reference standard was not significantly different between the POCUS and PEx group 0.25 (0.04–0.44) vs 0.21 (−0.001 to 0.40), p = 0.52.

Confidence, ease of learning, utility and future likelihood of use

The POCUS group was no more confident in their findings than the PEx group for either hepatomegaly or ascites (Table 2). Students rated POCUS and PEx similarly on ease of learning, but POCUS had higher perceived utility and likelihood of future use (Table 3). There were no significant ICCs between confidence level and accuracy for either the POCUS (hepatomegaly −0.65 (−0.76 to −0.51), p = 1.00; ascites −0.78 (−0.85 to −0.68), p = 1.00) or PEx (hepatomegaly −0.65 (−0.75 to −0.50), p = 1.00; ascites −0.65 (−0.75 to −0.50), p = 1.00) groups.

Table 2.

Differences in confidence levels between POCUS and physical examination

Exam Finding	Confidence, mean (range)
	POCUS	PEx	P value
Hepatomegaly	6.00 (1–10)a	5.83 (1–9)b	0.57
Ascites	7.33 (2–10)	7.34 (2–10)	0.91

POCUS point-of-care ultrasound, PEx physical examination

The number of missing confidence ratings are as follows: ^a 2, ^b 4

Table 3.

Post-study questionnaire to assess ease of learning, utility and likelihood of future use of either ultrasound or evidence-based physical examination

Question	POCUS (n = 25)	PEx (n = 25)	P value
Ease of learning to detect ascites	8.12 ± 1.09	7.76 ± 1.92	0.42
Ease of learning to detect hepatomegaly	7.28 ± 1.43	7.04 ± 2.35	0.67
Utility of learning to detect ascites*	8.38 ± 1.35	7.08 ± 1.86	0.008
Utility of learning to detect hepatomegaly	7.68 ± 1.52	5.36 ± 2.48	<0.001
Likelihood of future use to detect ascites*	8.67 ± 1.61	7.46 ± 1.79	0.02
Likelihood of future use to detect hepatomegaly*	8.12 ± 1.90	5.92 ± 2.32	0.001

All scores presented as mean ± SD

POCUS point-of-care ultrasound, PEx physical examination

* 1 Score missing from both ultrasound and physical examination groups

Discussion

This study compared the reliability, validity, and performance of medical trainees following brief instruction on either POCUS or PEx for the diagnosis of ascites and hepatomegaly. As many centers have begun introducing POCUS to first-year medical students alongside their clinical skills training [26–29], an understanding of how junior medical trainees are able to learn, perform, and interpret POCUS is crucial to both frontline educators and those involved in curriculum development.

We demonstrated that first-year medical students can be taught to use POCUS to accurately diagnose ascites and hepatomegaly after only brief instruction. POCUS assessments demonstrated comparable reliability, validity and performance to PEx. Trainees who learned POCUS rated it as more useful for this examination, and as more applicable to future practice, compared with their colleagues’ ratings of the evidence-based PEx. Students who performed POCUS demonstrated similar confidence in their findings to those who performed PEx, in keeping with their comparable levels of accuracy.

Numerous studies have demonstrated the efficacy of POCUS in the hands of medical students to enhance the cardiac exam, though there have been few studies evaluating the utility of trainee-performed abdominal POCUS [30]. The performance of PEx in our study was comparable to that in the literature [31–33]. However, our study did not demonstrate a superiority of POCUS over PEx in accuracy, reliability and performance, contrary to some studies assessing other pathologies [3]. There may be several reasons for this, including teaching time constraints and the limited number of patients in our study. Our study was designed to assess the efficacy of teaching interventions brief enough to be replicable as curricular additions or as part of bedside teaching moments. The fact that medical students in this study were unable to fully leverage the diagnostic power of abdominal ultrasound over PEx is of some concern, and may have significant implications in the design of future studies and the planning of curricula.

Related to study design, we included a limited number of patients to ensure that both normal and pathological findings could be assessed within a short time frame. However, the high representation of pathological findings overestimated PPV and underestimated NPV. Further, medical students not yet familiar with the full spectrum of disease may have been particularly impacted. For example, since the majority of patients had severe ascites (75%), a simultaneous view of the liver and the right kidney may not have been possible. Medical students relying excessively on viewing intervening fluid in the hepatorenal recess for the diagnosis of ascites would have made a false negative diagnosis. We suggest that future studies consider enrolling a larger number of patients with varying degrees of pathology. We also suggest that students are exposed to images and video clips displaying a broad range of disease severity, which may be done through online modules if time is limited.

Our result that medical students find POCUS useful and relevant to their future practice is similar to previous surveys of pre-clinical POCUS curricula [13, 34]. Our study also suggests that POCUS can be clinically useful and an enjoyable learning experience even when taught in a brief timeframe comparable to PEx. These findings are also consistent with existing literature [34, 35].

In conclusion, our study demonstrates that first-year medical students can use POCUS to diagnose ascites and hepatomegaly after brief instruction, with findings comparable to those using PEx. Students rated POCUS as more useful for this exam, and more likely to be applicable to their future practice. This study also highlights possible pitfalls requiring educator awareness and further research.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflicts of interest related to this study.

Ethical standard

Our study has been reviewed by the Research Ethics Board of Saint Michael’s Hospital and the University Health Network (Toronto, Canada).

Informed consent

All patient participants and medical trainees gave their informed consent prior to their inclusion in the study and had the opportunity to withdraw consent at any time during the study. All details that might disclose the identity of patient and trainee participants have been omitted.