Optimal Endobronchial Ultrasound Strain Elastography Assessment Strategy: An Explorative Study

Abstract

Background

In lung cancer staging, mediastinal lymph nodes are currently aspirated using endobronchial ultrasound transbronchial needle aspiration (EBUS-TBNA) based on size and FDG-PET avidity. EBUS strain elastography (SE) is a new technique that may help predict the presence of malignancy. However, a standardized assessment strategy for EBUS-SE measurement is lacking.

Objectives

The aim of this study was to determine the optimal assessment strategy for investigating the predictive value of EBUS-SE in mediastinal lymph nodes.

Methods

Two qualitative visual analogue scale strain scores and two semiquantitative strain elastography measurements (a strain histogram and strain ratio) were acquired in 120 lymph nodes of 63 patients with (suspected) lung cancer. The dataset was randomized into an 80% training dataset to determine cut-off values. Performance was consecutively tested on the remaining 20% and the overall dataset.

Results

The semiquantitative mean histogram scoring strategy with a cut-off value of 78 (range 0–255) showed the best and most reproducible performance in prediction of malignancy with 93% overall sensitivity, 75% specificity, 69% positive predictive value, 95% negative predictive value, and 82% accuracy. Combining the EBUS-SE mean histogram scoring outcome with PET-CT information increased the post-test probability of disease in relevant clinical scenarios, having a positive test likelihood ratio of 4.16 (95% CI 2.98–8.13) and a negative test likelihood ratio of 0.14 (95% CI 0.04–2.81) in suspicious lymph nodes based on FDG-PET or CT imaging.

Conclusions

EBUS-SE can potentially help predict lymph node malignancy in patients with lung cancer. The best semiquantitative assessment method is the mean strain histogram technique.

Established Facts

• Accurate and systematic mediastinal lymph node staging through endobronchial ultrasound (EBUS) transbronchial needle aspiration is important for the diagnosis and decision of treatment strategy in lung cancer. In order to predict malignancy in lymph nodes, ultrasound characteristics can be used in adjunct to PET and CT characteristics. A new ultrasound technique termed strain elastography (SE) has become available in EBUS and measures tissue elasticity. However, the elastography assessment lacks standardization.

Novel Insights

• We systematically compared different EBUS-SE assessment strategies in patients with lung cancer and show that the mean strain histogram technique is the most robust, objective, and reproducible technique. Applied in daily clinical practice, adding the EBUS-SE histogram mean has a clear additional predictive value in both imaging suspicious and imaging normal lymph nodes, with stiffer lymph nodes being more likely malignant. These findings could be of potential value to better select patients for additional surgical staging procedures.

Introduction

Endobronchial ultrasound transbronchial needle aspiration (EBUS-TBNA) has evolved from a diagnostic procedure to a staging procedure, and is now recommended as the first-line staging procedure in lung cancer [1, 2]. For accurate and complete staging, it is imperative that the endoscopist performs a full and systematic evaluation of the entire mediastinum. The evaluation should start in the contralateral hilum and proceed to the ipsilateral hilum, and preferably combine EBUS evaluation with esophageal ultrasound evaluation using the EBUS scope (EUSb) [1]. The goal of EBUS has herein shifted from the confirmation of metastatic disease to proving the absence of metastatic disease. Besides the available CT and FDG-PET information, ultrasound characteristics are often used for selecting nodes to aspirate in daily clinical practice. A study by Fujiwara et al. [3] introduced the B-mode ultrasound characteristics of round shape, distinct lymph node margins, heterogeneous echogenicity, and presence of a central necrosis sign as independent predictive markers for malignancy. Later however, Evison et al. [4] showed that heterogeneous echogenicity was the only significant predictor of malignancy. Several other studies also found discording results in using these ultrasound features [5, 6, 7], resulting in limited widespread diagnostic application of these echo characteristics [8].

Additional tools like ultrasound strain elastography (SE) have become available that may help predict the probability of malignancy in mediastinal lymph nodes. Ultrasound SE visualizes relative tissue stiffness by measuring axial tissue deformation. In EBUS-TBNA specifically, the periodical motion of the heart and greater vessels cause tissue deformation. Monitoring this tissue deformation by means of ultrasound over time enables a derivation of relative tissue strain which is a marker of tissue stiffness. A region of tissue that is heavily deformed over time when compared to another region in the same image is relatively soft (and thus high in strain). A region that is barely deformed by this same deformation force is in contrast relatively stiff (and thus low in strain). In ultrasound SE, the received signal intensities over time are correlated to one another for quantifying these deformations. As current ultrasound technology only allows accurate cross-correlation of signal intensity along the propagation direction of ultrasound itself, it should be acknowledged that it is primarily an axial measurement of strain (for more information, also refer to the online supplementary material; for all online suppl. material, see www.karger.com/doi/10.1159/000494143). Ultrasound SE has already shown value in differentiating benign from malignant lesions in the thyroid and breast, where an external force can be used and standardized [9, 10, 11, 12]. Several pilot studies now report that endoscopic ultrasound SE can also be used for differentiating malignant and benign lymph nodes in the mediastinum with high accuracy, with lesions low in strain being more likely malignant [13, 14, 15, 16, 17, 18, 19, 20]. However, the optimal method for scoring the presentation of the SE values in EBUS applications remains unclear [21]. Several qualitative and semiquantitative techniques have been reported so far. For qualitative presentation of strain, visual analogue scale (VAS) scoring techniques can be used. This is done by converting calculated relative strain into a color gradient and superimposing it on the B-mode image. This strain color image is then used for scoring the lymph node region of interest upon its major color component [14, 17, 18, 22] or pattern-color combination [12]. Another frequently reported method is the semiquantitative strain ratio measurement [17, 19, 23]. This technique requires that the operator manually selects the lymph node region and a secondary surrounding tissue region. By doing so, a ratio of strain can be calculated [17, 19, 23, 24]. A third less frequently used method is the semiquantitative strain histogram scoring method. This method quantifies the relative strain found in a manually selected lymph node area, and presents this as a histogram of strain counts [25].

Ideally, every assessment is operator independent, swift, and safe. Yet in EBUS several factors will influence measurement results as a consequence of the technological principles used in SE (see online suppl. material). For example, the position of the target node in relation to the source of compression and the tip of the EBUS scope affects the measured axial strain in EBUS-SE.

In this explorative study, we aim to determine the optimal EBUS-SE scoring and assessment strategy in patients with suspected or proven lung cancer. This is done by comparing the performance of predicting mediastinal lymph node malignancy among different scoring methods. We hypothesize that the strain histogram method will give the best overall predictive accuracy for diagnosing the lymph node malignancy since this will be the most operator independent and thus objective technique. We furthermore hypothesize that EBUS-SE will increase the predictive value of mediastinal lymph node ultrasound imaging. To do so, we aim to explore what the clinical effect of adding EBUS-SE is on the probability of malignancy in combination with FDG-PET and CT imaging.

Materials and Methods

All patients at the Radboud University Medical Center (Nijmegen, The Netherlands) who had an indication for a diagnostic or staging EBUS-TBNA procedure for suspicion of lung cancer were eligible and included. This study was conducted with approval of independent ethics committee as part of the E-predict multicenter study (NCT02488928) and all included patients gave informed consent. The inclusion period ranged from July 2015 to April 2016. All subjects had no history of prior radio- or chemotherapy and no general EBUS-TBNA contraindications. The EBUS-TBNA procedure was performed according to local and international guidelines [1, 26]. Measurements were acquired using the Hitachi Preirus Hi Vision processor (Hitachi Corp., Tokyo, Japan) with Hitachi Real-Time Elastography software in combination with Pentax EB-1970UK echo-endoscopes (Pentax Medical, Tokyo, Japan). The recommended diagnostic workup of our hospital was used; all nodes with a short-axis size > 10 mm on CT, ultrasound-based short axis size > 5 mm, or FDG-PET avid nodes in at least regions 4L, 4R, and 7 were routinely aspirated. FDG-PET scans were obtained and evaluated by the radiologist following the European Association of Nuclear Medicine guidelines [27]. While standard uptake values of individual lymph nodes were not routinely reported, a cut-off of 2.5 was typically used. EBUS-SE measurement collection and assessment was performed using a standardized operating protocol (online suppl. Fig. E.1.). In brief, real-time dual ultrasound B-mode and elastography imaging were used for data collection of both B-mode lymph node ultrasound characteristics (size, shape, margin, echogenicity, central hilar structure presence, and coagulation necrosis sign) and quantitative and qualitative EBUS-SE measurements. After these measurements, routine EBUS-TBNA sampling was performed under guidance of B-mode ultrasound alone. Cytology and consecutive histology (if available) of the EBUS-TBNA specimen combined with clinical follow-up were used as the gold standard for determining individual lymph node diagnosis.

EBUS-SE Evaluation Techniques

Qualitative Evaluation

For visual scoring of the strain, two VAS scoring methods based upon earlier reports were used [14, 17, 22]. The first VAS system requires the operator to classify the image into one of five groups based on the major color component. From low to high strain, respectively, these scores correspond to: 1, predominantly blue; 2, blue and green; 3, predominantly green; 4, green and red; 5, predominantly red (Fig. 1).

Fig. 1.

Typical examples of EBUS-SE imaging of 4L lymph nodes as captured by Hitachi Real-Time Strain Elastography software on the Preirus Hi Vision processor (Hitachi Corp. Tokyo, Japan) in combination with the Pentax UB-1970UK endoscope (Pentax Medical, Tokyo, Japan). In each panel on the left a conventional B-mode image with strain map overlay in color is seen; the grey box is the manually set region of interest (ROI) selection for calculation of the strain histogram mean. The different colors correspond to measured relative strain. Red corresponds to high deformation, green to intermediate measured deformation, and blue to relatively low deformation. On the right side of each panel, the B-mode image can be seen and used as a reference. In the bottom left of each panel a strain rate graph is shown, to be used for quality control and frame selection of elastography measurements. In the bottom right of each panel the strain histogram as calculated based on the selected ROI is also shown. Finally, the strain histogram parameters are presented in the table at the bottom of each panel. All panels depict 4L lymph nodes: EUSb approach (a), EBUS approach (b–d). Malignant nodes (a, b), benign nodes (c, d).

The second qualitative VAS scoring method used a modified version of the Tsukuba score, a scoring system based on a combination of the visualized color and the pattern of the strain image. This scoring method has shown to be promising in the classification of breast lesions [12]. However, the scoring was slightly altered by the authors to accommodate for the differences in breast US and EBUS imaging. The operator grades the lymph node EBUS-SE image following a six-category classification system (Fig. 2).

Fig. 2.

Modified Tsukuba score classification system used for VAS scoring of lymph node EBUS-SE, adapted from Itoh et al. [12]. The depicted typical images are all EBUS-SE lymph node images obtained following the standard acquisition protocol described in the Methods section and online supplementary material. The original Tsukuba scoring system as introduced for breast imaging was adjusted by the authors to accommodate for differences in endobronchial and breast imaging [12].

Semiquantitative EBUS-SE Evaluation

For semiquantitative scoring of EBUS-SE, two measurement methods are evaluated (see online suppl. material). The first measurement method is the mean SE histogram. This requires a (rectangular) hand selection of a lymph node region of interest. After selection, strain values are computed from the selected area and relayed back to the operator through a histogram of strain counts (Fig. 1). The second semiquantitative scoring method is the strain ratio. Herein, the lymph node region of interest as selected during the strain histogram method are compared against a hand-selected surrounding reference tissue area using the Matlab Image Processing Toolbox (version 9.1.0, MathWorks Inc., Natick, MA, USA; online suppl. Fig. E.2). As opposed to the other methods, the strain ratio was measured after completion of the EBUS procedure by one operator (R.L.J.V.) blinded for patient details and pathology outcome.

Statistics

The performance of the different EBUS-SE scoring techniques was evaluated by a priori randomizing the final dataset into an 80% training dataset for determining scoring method cut-off values and using the remaining 20% of data to test the scoring method performance on unseen data. To evaluate the performance of the models, sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), accuracy, negative and positive likelihood ratios, and post-test probability of malignancies (using the likelihood ratios) with their respective 95% confidence intervals (CI) are calculated among the separate and combined datasets. Area under the curve (AUC) is calculated for continuous variables.

To exploratively correlate SE imaging to outcome, pathology results were classified as malignant or benign. Patients diagnosed with granulomatous disease or patients for whom measurement data were incomplete were excluded from the final dataset before randomization and analysis (Fig. 3). The statistical computing language R and interface R-studio (version 1.1.414) were used for statistical analysis [28].

Fig. 3.

CONSORT flow diagram of the study design.

Results

In total, 150 lymph node SE measurements were obtained in 81 patients. After the exclusion of other pathologies and incomplete measurements, 120 lymph nodes in 63 unique patients remained (Fig. 3). A total of 45 lymph nodes herein proved to be malignant based on EBUS- TBNA alone (43 lymph nodes, 28 patients) or by surgical follow-up of false negative benign EBUS-TBNA cytology findings (2 lymph nodes, 2 patients). For the 75 lymph nodes (42 unique patients) eventually classified as benign, follow-up data were available in 61 nodes (81.3%). Surgical proof of benign EBUS-TBNA findings was available in 41 lymph nodes (55%) by either resection (15 nodes, 9 patients) or mediastinoscopy (26 nodes, 13 patients). Long-term clinical follow-up was available in 20 lymph nodes (27%, 12 patients) with a median follow-up time of 23 months (min.–max. 3–34). No follow-up of benign EBUS-TBNA findings was available in 14 lymph nodes (19%, 9 patients) due to rapid disease progression (6 nodes, 5 patients) and loss to follow-up (8 nodes, 4 patients).

The general patient demographics and lymph node details are summarized in Table 1. The lymph nodes were randomized into the training and test dataset (Fig. 3). The a priori-defined training dataset included 100 randomly selected lymph nodes. The remaining testing dataset consisted of 20 lymph nodes. The prevalence of malignancy or pre-test probability of malignant disease in both datasets was approximately equal at 0.38 and 0.35, respectively. The results of general B-mode ultrasound feature classification are presented in online supplementary Table E.1. In the final dataset, 79 of 120 lymph nodes (66%) were classified as FDG-PET positive. No PET scan was available in 3 patients (three lymph nodes), and no contrast CT was available in 1 patient (one lymph node).

Table 1.

Overall patient demographics and lymph node characterization

Demographics and characterization			Measured lymph nodes
				region	n (malignant/benign)	% total
Study subjects, n	63		2L	1 (1/0)	0.8
Mean age, years	64.3 (41–83)		2R	2 (2/0)	1.6
Sex (male/female)	39 (61)/24 (39)		4L (EB)	23 (11/12)	19.2
			4L (EUSb)	7 (4/3)	5.8
Aspirated lymph node characteristics			4R	25 (7/18)	20.8
CT size(SA),mm	11.95 (2–41)		7 (LMB)	3 (0/3)	2.5
FDG-PET avidity	79 FDG+/38 FDG–/3 NA		7 (RMB)	23 (4/19)	19.2
US size(SA),mm	9.95 (4–26)		7 (EUSb)	10 (5/5)	8.3
			10L	0 (0/0)	0
Lymph node pathology outcome			10R	2 (1/1)	1.6
Benign		75 (62.5)	11L	10 (3/7)	8.3
Malignant		45 (37.5)	11R	14 (7/7)	11.7
NSCLCadenocarcinoma		19 (15.8)
NSCLCSqCC		15 (12.5)
NSCLCNOS		1 (0.8)
Small cell lung carcinoma		5 (4.2)
LCNEC		3 (2.5)
Invasive ductal carcinoma		2 (1.6)

Values are presented as the mean (range) or n (%).SA, short axis; NA, not available;NSCLC, non-small cell lung cancer; SqCC, squamous cell carcinoma; NOS, not otherwise specified; LCNEC, large cell neuroendocrine carcinoma; EB, endobronchial; EUSb, esophageal using EBUS scope; LMB, left main bronchus; RMB, right main bronchus.

EBUS-SE Scoring Method Assessment Cut-Off Values

Cut-off values of all four SE scoring methods were first optimized by analyzing performance on the training dataset using accuracy, AUC (in the cases of strain histogram and strain ratio scoring), and individual predictive values of sensitivity, specificity, NPV, and PPV.

For the color VAS scoring system, optimal predictive performance was found if lymph nodes categorized as “predominantly blue” or “blue and green” were grouped together (predicting malignancy) versus all other categories grouped (predicting benign outcome, online suppl. Fig. E.3).

The modified Tsukuba scoring method performed best when placing the cut-off at the two lowest of five scores (online suppl. Fig. E.4), grouping scores 4–5 as malignant and scores 1–3 as benign. For the semiquantitative measurements, the ROC curve analysis of the strain histogram method (AUC 0.846, online suppl. Fig. E.5) showed a relative mean strain value of < 78 (range 0–255, with a lower value representing less strain) to be most predictive of malignancy using our software settings. The strain ratio was most predictive of malignancy when the mean of the lymph node strain divided by the reference tissue strain was more than 1.67 (AUC 0.637, online suppl. Fig. E.5).

EBUS-SE Predictive Value

With the above-described cut-off values, the different predictive values as introduced in the Methods section were calculated on both datasets. The results are summarized in Table 2, and the calculation of likelihood ratios are also presented in online supplementary Table E.2. The prospectively collected color VAS scoring, modified Tsukuba scoring, and strain histogram scoring had similar accuracies in the training dataset of 76, 82, and 82%, respectively. The most notable differences in individual performance factors among these methods were found in sensitivity and NPV. Sensitivity and NPV are upwards of 76% in all three methods. However, fewer false negatives were found in the strain histogram method, having a sensitivity of 92% and NPV of 94% (being 3–16% higher than the other two methods). Consequently, it also had the lowest negative post-test probability of malignancy in our dataset (0.06). Following the specificity and PPV, positive post-test probability of malignancy in our dataset seems to be similar in the three methods at around 0.70. While performance among the three prospective methods seems comparable, the strain ratio most markedly has lower sensitivity (50%) and NPV (73%) in the training dataset, which is further reflected by the lower accuracy (71%).

Table 2.

Summary of main EBUS-SE scoring method performance results

EBUS-SE scoring method	Sens, %	Spec, %	PPV, %	NPV, %	Accuracy, % (95% CI)	Positive post-test probability of malignancy (95% CI)	Negative post-test probability of malignancy (95% CI)
Training dataset (100 lymph nodes, 38 malignancies – pre-test probability of malignancy 0.38)
Color VAS score	76	77	67	84	77 (68–85)	0.67 (0.56–0.78)	0.16 (0.08–0.23)
Modified Tsukuba score	87	79	72	91	82 (73–89)	0.72 (0.61–0.82)	0.09 (0.03–0.16)
Strain histogram mean	92	76	70	94	82 (73–89)	0.70 (0.61–0.80)	0.06 (0.00–0.13)
Strain ratio	50	84	65	73	71 (61–80)	0.66 (0.50–0.80)	0.27 (0.20–0.33)
Test dataset (20 lymph nodes, 7 malignancies – pre-test probability of malignancy 0.35)
Color VAS score	100	69	64%	100	80	0.64	0.00*
Modified Tsukuba score	100	62	58	100	75	0.58*	0.00*
Strain histogram mean	100	69	64	100	80	0.64	0.00
Strain ratio	29	85	50	69	65	0.5	0.31
Overall dataset (120 lymph nodes, 45 malignancies – pre-test probability of malignancy 0.375)
Color VAS score	80	76	67	86	78	0.67	0.14
Modified Tsukuba score	89	76	69	92	81	0.69	0.08
Strain histogram mean	93	75	69	95	82	0.69	0.05
Strain ratio	47	84	64	72	70	0.64	0.28

Scoring method performance was obtained by randomly partitioning the total dataset into a training (80%) and test dataset (20%). The cut-off scores of the scoring methods are first determined by optimization of performance on the training dataset (online suppl. Fig. E.3–E.5). Transferability of performance using these scores is subsequently assessed in the test dataset. The post-test probabilities of malignancy are determined by the pre-test probability of malignancy and the likelihood ratios as defined in online supplementary Table E.2.

^*Denotes significant differences between training and test dataset, being outside the 95% CI as determined in the training dataset. Sens, sensitivity; Spec, specificity; PPV, positive predictive value; NPV, negative predictive value; VAS, visual analog scale.

When the cut-offs of the four scoring methods are consecutively used to analyze performance on the testing dataset, the accuracies remain within the earlier determined 95% CI of the training dataset (Table 2). However, the positive and negative post-test probability of malignancy as based on the likelihood ratios (online suppl. Table E.2) do show differences. All collected methods except the strain ratio showed no false negatives in the test dataset. Consequently, both the color VAS score and modified Tsukuba score performed better than the 95% CI as determined from the training dataset. Furthermore, we would expect a higher than 0.58 positive post-test probability of malignancy of the modified Tsukuba scoring method based on the training dataset results (95% CI 0.61–0.82).

When combining performances of both the training and test dataset, the mean of the strain histogram is shown to best predict lymph node malignancy. It has the lowest negative post-test probability of malignancy while accuracy is similar to the best of the methods. The mean of the strain histogram furthermore shows similar performance when testing on new and unseen data.

Effect of EBUS-SE on Probability of Malignancy Assessment

Based on the analysis of the different EBUS-SE scoring methods, the SE histogram mean scoring method was used to further investigate if EBUS-SE could be of potential added value in clinical workup. This was investigated by calculating the effect on the probability of malignancy when combined with PET-CT imaging information of both the abnormal and normal mediastinal lymph nodes in the overall dataset (Fig. 4; online suppl. Table E.2, E3). Based on available combined PET avidity and/or CT size > 10 mm, a group of normal (n = 26) and suspicious (n = 90) mediastinal lymph nodes were identified in our complete dataset. In these groups, the probability of malignancy was 8 and 46%, respectively. Explorative analyses showed that the addition of EBUS-SE histogram scoring in the subgroup of suspicious lymph nodes increases the post-test probability of malignancy from 46 to 73% (positive likelihood ratio 4.61, 95% CI 2.98–8.13). Oppositely, a negative EBUS-SE histogram analysis in this group of suspicious nodes reduces the post-test probability of malignancy from 46 to 8% (negative likelihood ratio 0.143, 95% CI 0.04–2.81; Fig. 4; online suppl. Table E.3). In the subgroup of imaging normal lymph nodes, a mean EBUS-SE histogram < 78 increased the probability of malignancy from 8 to 45%. If EBUS-SE outcome indicated non-malignancy in the PET/CT negative lymph nodes, the probability of malignancy dropped from 8 to 0%, negating 2 cases where PET-CT imaging was found to be false negative (Fig. 4; online suppl. Table E.3).

Fig. 4.

Graphical representation of added value of performing cumulative diagnostic tests in our complete dataset. The first point on the left at 0.375 corresponds to the prevalence of malignancy in our complete dataset (116 lymph nodes). FDG-PET and/or CT imaging is able to increase the probability of malignancy to 46% if found positive. Negative PET and CT imaging is able to decrease the probability of malignancy from 37.5 to 7.8%. After having performed PET-CT imaging, EBUS-SE imaging is performed. In PET-CT-suspected nodes (n = 90), the probability of malignancy increases from 46 to 73% if EBUS-SE imaging is positive. If EBUS-SE imaging is negative in these suspected nodes, the probability of malignancy decreases to 7.9%. In normal lymph node findings based on PET-CT imaging (n = 26), the probability of malignancy decreases from 7.8 to 0% if negative on EBUS-SE imaging (> 78 strain histogram mean). However, if these lymph nodes are EBUS-SE imaging positive (< 78 strain histogram mean), the probability of malignancy increases from 7.9 to 45% (online suppl. Table E.3).

Discussion

EBUS-SE could potentially be a sensitive technique to help predict lymph node malignancy in patients with lung cancer. We found that the semiquantitative mean histogram scoring method with a cut-off value of 78 (range 0–255) showed the best, most objective and most reproducible performance in the prediction of malignancy with 93% overall sensitivity, 75% specificity, 69% PPV, 95% NPV, and 82% accuracy. Exploratively combining relative lymph node stiffness by using the EBUS-SE histogram mean scoring outcome with CT and/or PET-CT information further increased confidence in post-test probability of disease, having a positive test likelihood ratio of 4.16 (95% CI 2.98–8.13) and a negative test likelihood ratio of 0.14 (95% CI 0.04–2.81) in imaging suspicious lymph nodes. EBUS-SE scoring may thus enable a better prediction and separation of lymph nodes with a high and low probability of disease but will not replace a tissue diagnosis. It can, however, be a helpful tool for guiding which node to aspirate in a distinct region, or ultimately deciding whether to proceed with, or omit, additional diagnostic procedures if ROSE or conclusive cytology results are unavailable (especially when FDG-PET information is lacking). Yet, care needs to be taken in interpreting these results. This explorative study is of limited size and further research to corroborate our findings using a standardized assessment procedure based on clinical findings and technological considerations is recommended.

Differences in EBUS-SE Scoring Methods

In the qualitative scoring, both the scoring of a major color component (color VAS method) and a combined scoring of pattern and color (modified Tsukuba score) perform similarly in terms of overall accuracy. Our proposed use of the modified Tsukuba scoring system has not previously been introduced in EBUS applications. Even though we expected that taking into account both pattern and color above color alone would provide more information, it does not seem to be of clear additional value over the color VAS system in this dataset.

Three studies used a scoring system like the color VAS scoring system, with comparable results [14, 18, 22]. He et al. [14] reported best accuracy if the visualized color was over 50% blue, then indicative of malignancy with an accuracy of 73.7%. Huang et al. [22] and Izumo et al. [18] introduced an additional cut-off; only predominantly blue lesions were indicative of malignancy, while predominantly non-blue lesions were indicative of benign lymph nodes. By leaving out an intermediary group (part blue, part non-blue) they were able to report an overall accuracy in these categories of 91.4 and 96.7%, respectively. Leaving out uncertain cases (part blue, part non-blue) was not done in our study for accurate comparison of scoring methods (but see online suppl. Fig. E.3, E4). Possibly, such a method could increase diagnostic success.

In semiquantitative scoring, the majority of previous pilot studies report using the strain ratio for scoring [17, 19, 23]. Our study, however, clearly shows that the mean value of the strain as obtained from the histogram has best performance. As reflected by the comparatively low performance of the strain ratio scoring method, we found that the strain ratio outcome relied heavily on the selected secondary reference tissue region in EBUS. Adequate selection of a uniform secondary reference tissue while taking into account the technical and clinical boundary conditions is difficult not only because of limited tissue uniformity, but also heterogeneous sources of strain (online suppl. material). This is likely one of the reasons why other pilot studies investigating EBUS-SE with strain ratios reported different cut-off values [29] and collected multiple measurements for combining them into one average reference value [17, 19, 23]. A potential limitation of our study is that we collected strain ratio measurements retrospectively rather than prospectively, which could have affected the strain ratio scoring outcome.

Combining EBUS-SE with Other Predictive Variables

FDG-PET and/or CT imaging of lymph nodes in this study had sensitivities comparable to the reported literature [2], but a lower combined specificity of 33% (online suppl. Table E.3). This could in part be explained by the systematic staging of all lymph nodes by EBUS-TBNA as applied in our clinical practice. Combining EBUS-SE with pre-test contrast CT and FDG-PET avidity, however, was able to improve the predictive accuracy of lymph node pathology in both PET-CT negative and positive imaging cases. The relative stiffness as measured by EBUS-SE thus seems to show additional value to the metabolic and size information as given by FDG-PET and CT. Yet again, care needs to be taken in interpreting the results of adding EBUS-SE information to PET and CT negative nodes, as only 26 lymph nodes, of which 2 were false negative, were available for an exploratory analysis. While adding EBUS-SE correctly changed the diagnosis to malignant in both cases and could be promising, additional studies are warranted.

Besides PET-CT information, ultrasound B-mode characteristics as described by Fujiwara et al. [3] were also collected in this explorative study (online suppl. Table E.1). Since different studies have shown conflicting results in the value of these features, we did not integrate these into a combined predictive value or integrate them with EBUS-SE findings [3, 4, 5, 6, 7]. Analysis of these features in our dataset showed the presence of heterogeneous echogenicity and a central necrosis sign to be most predictive of malignancy, with positive likelihood ratios of 5.83 and 3.65. An AUC of 0.73 of the B-mode-based size of lymph nodes revealed a correlation between size and malignancy.

General Remarks

The mean lymph node short axis diameter in this study was 9.9 mm (range 4–26). This indicates it was not a population with bulky mediastinal disease, but rather a group with a low disease burden. Malignant lymph node involvement can be heterogeneous and microscopic. This is cause for concern in both TBNA and SE of individual lymph nodes. TBNA provides only limited information about the complete node, as it samples limited amounts of tissue within the node. SE can provide information about the entire node, but is based only on the strain of tissue upon deformation. Whether elastography is able to differentiate between micrometastatic disease accurately and if it may guide the site of aspiration within a lymph node to detect intranodal fields at risk of micrometastatic disease is unclear. The current dataset does not allow this level of analysis.

Observer Scoring Variability

A general point of concern for all EBUS-SE measurement methods is the inter- and intraobserver variability in lymph node scoring and frame selection through different platforms. System settings and operator protocol can affect the EBUS-SE measurement outcome (online suppl. material). A standardized operating procedure is essential. In this current study, the prospective elastography measurements and scoring were done by one pulmonologist using a standardized operating protocol (E.v.d.H.). Interobserver variability was thus minimal in order to adequately study the differences in scoring method performance. Reproducibility of this technique based upon operator expertise, operator color acuity, and anatomical site and region needs to be further investigated. Widespread implementation of this technique can only be realized when system settings and operator methods are unified. Transferability of these findings to other diseases, such as sarcoidosis for example, are unknown and also need to be studied. In doing so, we recommend using the mean of the strain histogram for scoring as it showed the best performance in this study, requires the least operator input, and will have highest interobserver agreement based upon technique background.

Conclusion

EBUS-SE is a sensitive technique which may help predict lymph node malignancy in patients with lung cancer. In this study we found that the semiquantitative mean histogram scoring method with a cut-off value of 78 (range 0–255) showed the best and most reproducible performance in the prediction of malignancy, with 93% overall sensitivity, 75% specificity, 69% PPV, 95% NPV, and 82% accuracy. Combining the EBUS-SE mean histogram scoring outcome with CT and/or PET-CT information increased confidence in post-test probability of disease, with a positive test likelihood ratio of 4.16 (95% CI 2.98–8.13) and a negative test likelihood ratio of 0.14 (0.04–2.81 95% CI) in suspicious lymph nodes based on PET or CT imaging. We further present a standard operating procedure for EBUS-SE assessment and future research.

Statement of Ethics

The research was conducted ethically in accordance with the latest World Medical Association Declaration of Helsinki. The study protocol was approved by the independent local medical ethical committee and institutional review body before subject inclusion commenced. Informed consent was obtained. The study is registered and can be found on ClinicalTrials.gov (identifier: NCT02488928).

Disclosure Statement

The authors have no conflicts of interest to declare in relation to this study.

Funding Sources

Unrestricted research grants for this study were obtained from the Ankie Hak Fund, Astra Zeneca Oncology, and Pentax Medical.

Author Contributions

R.L.J.V. and E.v.d.H. contributed to the design, acquisition, analysis and interpretation of data for the work, drafted the work, revised it critically, and gave final approval of the version to be published. C.L.d.K. contributed to the design and analysis and interpretation of data for the study, and provided feedback for important intellectual content revisions.

All authors gave final approval of the version to be published and agree to be accountable for all aspects of the work in ensuring that questions relating to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Supplementary Material

Supplementary data

Supplementary data

Supplementary data

Supplementary data

Supplementary data

Supplementary data

Supplementary data

Supplementary data

Supplementary data