What are the minimum requirements to establish proficiency in lung ultrasound training for quantifying B‐lines?

Abstract

Aims

The goal of this study was to determine the number of scans needed for novice learners to attain proficiency in B‐line quantification compared with expert interpretation.

Methods and results

This was a prospective, multicentre observational study of novice learners, physicians and non‐physicians from three academic institutions. Learners received a 2 h lung ultrasound (LUS) training session on B‐line assessment, including lecture, video review to practice counting and hands‐on patient scanning. Learners quantified B‐lines using an eight‐zone scanning protocol in patients with suspected acute heart failure. Ultrasound (US) machine settings were standardized to a depth of 18 cm and clip length of 6 s, and tissue harmonics and multibeam former were deactivated. For quantification, the intercostal space with the greatest number of B‐lines within each zone was used for scoring. Each zone was given a score of 0–20 based on the maximum number of B‐lines counted during one respiratory cycle. The B‐line score was determined by multiplying the percentage of the intercostal space filled with B‐lines by 20. We compared learner B‐line counts with a blinded expert reviewer (five US fellowship‐trained faculty with > 5 years of clinical experience) for each lung zone scanned; proficiency was defined as an intraclass correlation of > 0.7. Learning curves for each learner were constructed using cumulative sum method for statistical analysis. The Wilcoxon rank‐sum test was used to compare the number of scans required to reach proficiency between different learner types.

Twenty‐nine learners (21 research associates, 5 residents and 3 non‐US‐trained emergency medicine faculty) scanned 2629 lung zones with acute pulmonary oedema. After a mean of 10.8 (standard deviation 14.0) LUS zones scanned, learners reached the predefined proficiency standard. The number of scanned zones required to reach proficiency was not significantly different between physicians and non‐physicians (P = 0.26), learners with no prior US experience vs. > 25 prior patient scans (P = 0.64) and no prior vs. some prior LUS experience (P = 0.59). The overall intraclass correlation for agreement between learners and experts was 0.74 and 0.80 between experts.

Conclusions

Our results show that after a short, structured training, novice learners are able to achieve proficiency for quantifying B‐lines on LUS after scanning 11 zones. These findings support the use of LUS for B‐line quantification by non‐physicians in clinical and research applications.

Introduction

B‐line assessment on lung ultrasound (LUS) is an objective, quantitative measure of pulmonary congestion. ¹ , ² , ³ Incorporation of LUS B‐lines into the assessment of patients with acute heart failure (AHF) can improve diagnostic accuracy, guide acute management and may impact prognosis. ⁴ , ⁵ , ⁶ Recent consensus guidelines from the European Society of Cardiology support the use of LUS in the diagnosis and management of AHF. ⁷

Lung ultrasound has been used for over two decades in the intensive care unit and emergency department to evaluate patients with dyspnoea. ⁸ , ⁹ , ¹⁰ It is now being utilized by an increasing number of specialties. Despite its growing use, prospective data to support benchmarks for ultrasound competency are lacking. Thus, current guidelines are largely based on expert consensus. ¹¹ , ¹² Additionally, different benchmarks for LUS competency exist based on specialty. There have been a few studies published previously that focused on evaluating learning curves and ultrasound proficiency. ¹³ , ¹⁴ , ¹⁵ , ¹⁶ However, none of these studies have evaluated the learning curve of LUS for B‐line assessment.

The primary goal of this study was to determine the number of scans needed for novice learners to be proficient in B‐line quantification when compared with experts in patients with AHF.

Methods

Study design and setting

This was a multicentre, prospective study of learners, novice to LUS, from three academic institutions. This study had previously been approved by the institutional review board at all study sites.

Learners and scanning protocol

Learners included research associates, postgraduate year 1–3 emergency medicine and internal medicine residents and non‐ultrasound‐trained emergency medicine faculty. Leaners were considered novice and included in the study if they had performed <25 prior LUS exams. This cut‐off was chosen as 25–50 ultrasound exams are generally accepted as a benchmark for proficiency. ¹¹ All learners completed a 2 h structured LUS training course that included didactics, LUS image review to practice counting B‐lines and proctored hands‐on scanning. Learners then independently scanned adult patients with suspected pulmonary oedema from AHF.

All LUS exams were performed using Zonare ZS3, Z One Pro (Mindray, Mountain View, CA) or Sonosite MTurbo (FUJIFilm Sonosite, Bothell, WA) ultrasound machines with the curvilinear transducer. Ultrasound machine settings were standardized at a depth of 18 cm and clip length of 6 s, and tissue harmonics and multibeam former were turned off. The gain was adjusted to the individual patient so that the rib shadows appeared black, and the pleural line was distinct. Patients were scanned in a semi‐upright position, with the head of the bed at 45°. Our LUS scanning protocol was based on similar protocols previously published, ¹⁰ using an eight‐zone approach, with four zones in each hemithorax. We acquired videos with the probe in a transverse (horizontal) orientation, with the probe indicator facing the patient's right side and the probe face parallel to the adjacent ribs. Findings were recorded on a standardized data collection form.

Quantification of B‐lines

To quantify the number of B‐lines visualized, the intercostal space with the greatest number of B‐lines within each zone was used for scoring. Figure 1 shows normal lung (A‐lines) and lung with evidence of pulmonary oedema (B‐lines). Each lung zone was given a B‐line score of 0–20 based on the maximum number of B‐lines counted during one respiratory cycle. For B‐lines that were wide or fused together, the score was determined by multiplying the percentage of the intercostal space filled with confluent B‐lines by 20. For example, if 25% of the intercostal space was filled with confluent B‐lines, then a score of 5 would be given for that zone. If, within a single zone, no lung was visualized and only a pleural effusion or heart was seen, we considered this not applicable and did not use for data analysis. B‐lines were counted offline on a computer. This methodology was prespecified in the study protocol and emphasized during LUS training. Our method for B‐line quantification was based on previously published recommendations ¹⁰ ; however, our protocol differed in that we used 20 B‐lines as the maximum (instead of 10) and a horizontal probe orientation to maximize assessment of the pleural interface.

FIGURE 1.

Normal vs. abnormal lung. The lung image on the left shows A‐lines (arrows), an imaging artefact seen in patients with normal lung. The image on the right shows B‐lines (B), which are visualized in patients with pulmonary oedema.

Evaluation of proficiency

All images were reviewed, and B‐lines were counted by expert sonographers blinded to learner counts and patient information. We considered each lung zone as a separate data point. Therefore, a patient who had a complete eight‐zone LUS exam had eight unique LUS scans performed that were used for analysis. The five expert reviewers consisted of three ultrasound site principal investigators (PIs) and two independent physicians who were not associated with any of the study sites. A randomized subset of LUS videos were assessed by both ultrasound site PIs and the independent experts to assess for expert agreement. Independent experts were blinded to clinical data, chest zone and site PI B‐line quantification.

Statistical analysis

Cumulative sum (cusum) statistical methods were used to evaluate the number of scans required to reach an acceptable level of training. ¹⁷ , ¹⁸ Cusum analyses are a statistical analysis method used for evaluating sequential data to determine when a learner has reached proficiency with a technique. It is a probability‐based method that incorporates statistical testing of proficiency within the graphical representation of the results. The predefined acceptable and unacceptable failure rates determine a cut‐off value of the calculated statistic for proficiency. The calculated statistic is updated with each new observation added to the prior observation to determine when and if proficiency is reached. Because of the way the cusum statistic is calculated, the proficiency cut‐off still remains constant even as more results are added to the data set.

On individual scans, the learner was considered to be in agreement with the expert if counts were within two B‐lines. The acceptable failure rate was set at 30% (70% agreement), and the unacceptable failure rate (when retraining would likely need to occur) was set at 70%. Type 1 and Type 2 errors (alpha and beta, respectively) were both set at 0.1. The cusum analysis incorporated this information into a calculation to determine decision limits for whether or not the learner differed significantly from the acceptable failure rate.

Wilcoxon rank‐sum tests were used to compare the number of scans required to reach proficiency between physicians and non‐physicians, learners with no prior ultrasound experience and those with > 25 prior patient scans of any kind (i.e. cardiac and aorta) and learners with no prior and prior LUS experience. Intraclass correlation coefficients were used as an additional measure of agreement between learner and expert sonographer for the number of B‐lines and to determine the agreement between experts.

Results

Learners scanned 340 patients with acute pulmonary oedema, and 2629 (96.6%) lung zones were included in the analysis. We excluded videos where no lung was visualized (i.e. heart or pleural effusion). Of the lung zones included, 1116 videos (42%) had 0–2 B‐lines, 1015 (39%) had 3–10 B‐lines and 498 (19%) had 11–20 B‐lines. The mean number of B‐lines was 5.4 [standard deviation (SD) 5.8]. Learners included 21 research associates, 5 residents and 3 emergency medicine faculty. We found that after 10.8 (SD 14.0; range 3–51) LUS zones scanned, learners, including physicians and non‐physicians, reached proficiency for quantifying B‐lines when compared with an expert (Figures 2 and 3 ). Table 1 provides a demographic breakdown of learners and the number of scans required to meet proficiency. Table 2 provides a summary of statistics for the number of scans required to reach an acceptable level of training.

FIGURE 2.

Cumulative sum (cusum) for all learners. Cusum charts were used to evaluate the number of lung zones required for each learner to reach proficiency. This figure shows all learners on a single chart; y‐axis is the cusum, and x‐axis is the number of lung zones scanned. A negative trend of the cusum line indicates success, whereas a positive trend in the cusum line indicates failure. When the line crosses the lower decision limit (h0) from above, the true failure rate does not differ significantly from the acceptable failure rate. The acceptable failure rate was set at 30% (70% agreement), and the unacceptable failure rate was set at 70%.

FIGURE 3.

Cumulative sum (cusum) for non‐physician learners. This figure shows only non‐physician learners on a single chart; y‐axis is the cusum, and x‐axis is the number of lung zones scanned.

TABLE 1.

Summary statistics for the number of scans required to reach proficiency

Learners	N	Min	Max	Mean	SD
All	29	3	51	10.8	14.0
Non‐physician	21	3	51	11.5	14.1
Physician	8	3	45	9.0	14.7
No prior US	17	3	51	10.4	15.0
> 25 prior US	7	3	45	9.9	15.7
No prior LUS	20	3	51	10.6	13.8
Prior LUS	9	3	45	11.2	15.3

LUS, lung ultrasound; SD, standard deviation; US, ultrasound.

TABLE 2.

Number of scans required to reach proficiency for each learner

Learner	Scans required	Job title	US exams prior (0 = 0; 1 = 1–25; 2 = > 25)	LUS exams prior (0 = 0; 1 = 1–25)
1	47	RA	0	0
2	3	Resident	2	1
3	3	Resident	1	1
4	9	Resident	2	1
5	3	Faculty	2	1
6	45	Faculty	2	1
7	3	Resident	2	1
8	9	RA	1	0
9	15	RA	1	0
10	3	RA	0	0
11	3	RA	0	0
12	3	RA	0	0
13	5	RA	0	0
14	15	RA	0	0
15	29	RA	1	1
16	51	RA	0	0
17	9	RA	0	0
18	11	RA	1	0
19	3	Faculty	2	1
20	3	RA	0	0
21	3	RA	0	0
22	7	RA	0	0
23	13	RA	0	0
24	3	RA	0	0
25	3	RA	0	0
26	3	Resident	2	1
27	3	RA	0	0
28	3	RA	0	0
29	3	RA	0	0

LUS, lung ultrasound; RA, research associates; US, ultrasound.

The mean number of scans required to reach an acceptable level of training was not significantly different between physicians and non‐physicians: 9.00 (SD 14.7) compared with 11.5 (SD 14.1), P = 0.26. There was no significant difference in the number of scans required to reach an acceptable level of training between learners with no prior ultrasound experience and with > 25 prior exams of any kind: 10.4 (SD 15.0) compared with 9.90 (SD 15.7), P = 0.64. There was no significant difference in the number of scans required to reach an acceptable level of training between learners with no prior and prior LUS experience: 10.6 (SD 13.8) compared with 11.2 (SD 15.3), P = 0.59. The overall intraclass correlation for agreement between learners and experts was 0.74. The overall intraclass correlation for agreement among experts was 0.80.

Discussion

Lung ultrasound B‐line assessment is a useful tool for diagnosis, prognosis and monitoring response to treatment in patients with AHF. ⁴ , ⁵ , ⁶ B‐lines on LUS have been shown to outperform physical examination, chest radiography and B‐type natriuretic peptide for the diagnosis of AHF. ¹⁹ B‐lines are a dynamic marker of pulmonary congestion that clear in response to treatment. ²⁰ , ²¹ , ²² , ²³ Additionally, persistence of B‐lines after hospital discharge in patients with AHF is associated with a worse prognosis, including a greater risk of hospital readmission and mortality. ⁵ , ⁶ , ²⁴ , ²⁵ European Society of Cardiology expert consensus supports the use of LUS in the diagnosis and management of AHF. ⁷

The results of our study suggest that learners with limited to no prior ultrasound training, including non‐physicians, are able to accurately quantify B‐lines on LUS when compared with an expert. All learners in this study achieved an acceptable level of training, requiring on average 11 LUS zones scanned to reach proficiency.

These data are consistent with prior research suggesting that LUS for B‐line assessment is one of the easier ultrasound exam types to perform and interpret. ¹⁰ Previous studies have found high interrater reliability between learners and experts for the detection of B‐lines on LUS, ²⁶ , ²⁷ high intraclass correlation for quantifying B‐lines ²⁸ and high interobserver agreement for detecting B‐lines in patients with AHF. ¹⁶ , ²⁷ , ²⁹

Our data differ from prior studies in that this was the first study to assess the number of scans needed to reach proficiency, deemed as agreement > 70% with experts. To date, prospective data to support benchmarks for ultrasound competency have been lacking, and current guidelines are based primarily on expert consensus. ¹¹ , ¹² There have been multiple prior studies looking at ultrasound learning curves and proficiency. ¹³ , ¹⁴ , ¹⁵ However, none of these studies have investigated B‐line quantification. The one study assessing LUS focused solely on learning curves for pneumothorax. ¹⁴

When comparing physicians to non‐physicians, we found no significant difference in the number of scans needed to reach proficiency (P = 0.26). In a prior study by See et al., ³⁰ respiratory therapists were able to acquire and interpret LUS in intensive care patients with > 95% accuracy compared with an expert reviewer after 10 patient scans (120 images). Gustafsson et al. ²⁷ found that four heart failure nurses after 4 h of training had high agreement with a single cardiologist for detecting B‐lines. These data along with ours suggest that non‐physicians with little or no prior ultrasound experience can reach proficiency in quantifying B‐lines equivalent to an expert after limited training. Our study differed from these two prior studies of non‐physician sonographers in that we quantified the number of scans needed to obtain proficiency and also included research associates with limited medical background knowledge.

Clinical application

Pulmonary decongestion is an important treatment goal for patients with AHF. B‐line quantification on LUS is an objective measure of pulmonary congestion useful for monitoring response to therapy and for prognostic stratification. ⁵ B‐lines clear rapidly in response to therapy, and their lack of clearance at hospital discharge in patients with AHF is associated with a high risk of adverse events. ²⁴ Traditionally, B‐line quantification has been performed by trained clinicians; however, our data support that this skill could be expanded to non‐physicians, including a nurse, respiratory therapist or technician, prior to a physician seeing or re‐evaluating a patient. For example, a medical assistant could evaluate a patient for a decrease, increase or no change in LUS B‐lines after treatment. A clinician could then use this new information to tailor their management plan. This multidisciplinary strategy would allow for a data‐driven approach—where other methods, such as physical exam, have failed—to guide management and disposition decisions, potentially improving patient outcomes and reducing 30 day readmission rates.

Limitations

Although all learners reached proficiency in quantifying B‐lines, there was a wide range in the number of lung zones scanned required to reach proficiency, ranging from 3 to 51 (Table 2 ). Three learners required >30 scans to reach proficiency: two were research associates with no prior ultrasound experience, and one was a non‐ultrasound‐trained emergency medicine faculty who had performed <25 lung ultrasounds prior to the study. Because ultrasound is operator dependent, it is possible that image quality contributed to this wide range. Two potential factors directly related to image quality include skill in image acquisition and patient body mass index (BMI), neither of which were directly assessed in this study. Anecdotally, through review of images, at times, learners struggled with remaining parallel to the ribs in the lateral zones, adjusting gain to the individual patient and acquiring images in patients where the pleural line was relatively deep to the skin surface (patients with larger BMIs). Although these were addressed in the protocol and emphasized during training, future studies and training protocols for B‐line quantification should re‐emphasize remaining parallel to the ribs and adjusting gain for every image acquired. These studies should also assess impact of BMI on proficiency and assess technical quality of acquired images for every scan.

Conclusions

We found after a limited structured training and 10.8 independent scans that novice learners were able to achieve proficiency for quantifying B‐lines on LUS when compared with an expert.

Conflict of interest

F.M.R., R.R.E., V.N., K.L.F., and G.J.E. have no conflict of interest. R.F. is or has been in the last 1 year a consultant for and/or received honoraria from FUJIFilm SonoSite, 3rd Rock Ultrasound and InnovatED Ultrasound. P.S.P. is or has been in the last 1 year a consultant for and/or received honoraria from BMS, Novartis, Trevena and Roche Diagnostics. He has also received research support from Roche, Novartis, PCORI, NHLBI, AHA and AHRQ. L.G. received speaker honoraria from GE Healthcare and GlaxoSmithKline. She has also received research support from the Italian Ministry of Health. P.D.L. is or has been in the last 1 year a consultant for and/or received honoraria from Novartis, Trevena, Roche Diagnostics, Astra Zeneca, SCIEX and Siemens. He has also received research support from Roche, Novartis, Gilead, Edwards Lifesciences, Amgen, BCBSMF, GE/EMF, PCORI, NHLBI and AHRQ. S.P.C. received research support from NIH, AHRQ, AHA, PCORI and BMS. He also served on a steering committee and as a consultant for Novartis.

Funding

This work was supported by the National Institutes of Health (1R34HL136986‐01).

Russell, F. M. , Ferre, R. , Ehrman, R. R. , Noble, V. , Gargani, L. , Collins, S. P. , Levy, P. D. , Fabre, K. L. , Eckert, G. J. , and Pang, P. S. (2020) What are the minimum requirements to establish proficiency in lung ultrasound training for quantifying B‐lines?. ESC Heart Failure, 7: 2941–2947. 10.1002/ehf2.12907.