Mediastinal lymph node detection and station mapping on chest CT using spatial priors and random forest

Abstract

Purpose:

To develop an automated system for mediastinal lymph node detection and station mapping for chest CT.

Methods:

The contextual organs, trachea, lungs, and spine are first automatically identified to locate the region of interest (ROI) (mediastinum). The authors employ shape features derived from Hessian analysis, local object scale, and circular transformation that are computed per voxel in the ROI. Eight more anatomical structures are simultaneously segmented by multiatlas label fusion. Spatial priors are defined as the relative multidimensional distance vectors corresponding to each structure. Intensity, shape, and spatial prior features are integrated and parsed by a random forest classifier for lymph node detection. The detected candidates are then segmented by the following curve evolution process. Texture features are computed on the segmented lymph nodes and a support vector machine committee is used for final classification. For lymph node station labeling, based on the segmentation results of the above anatomical structures, the textual definitions of mediastinal lymph node map according to the International Association for the Study of Lung Cancer are converted into patient-specific color-coded CT image, where the lymph node station can be automatically assigned for each detected node.

Results:

The chest CT volumes from 70 patients with 316 enlarged mediastinal lymph nodes are used for validation. For lymph node detection, their system achieves 88% sensitivity at eight false positives per patient. For lymph node station labeling, 84.5% of lymph nodes are correctly assigned to their stations.

Conclusions:

Multiple-channel shape, intensity, and spatial prior features aggregated by a random forest classifier improve mediastinal lymph node detection on chest CT. Using the location information of segmented anatomic structures from the multiatlas formulation enables accurate identification of lymph node stations.

1. INTRODUCTION

Identification of enlarged lymph nodes plays a very important role in the diagnosis, staging, and treatment planning of lung cancer.^1,2 With regard to lymph node size, according to the response evaluation criteria in solid tumors (RECIST) 1.1 criteria,^3–5 lymph nodes measuring more than 15 mm in the short axis are designated as pathologically enlarged target lesions to be tracked for treatment response for follow-up imaging examinations, whereas lymph nodes measuring more than 10 mm but less than 15 mm are designated as pathologically enlarged but not to be tracked for treatment response.

With regard to reporting pathologically enlarged mediastinal lymph node locations in the context of lung cancer, the International Association for the Study of Lung Cancer (IASLC) Staging Project⁶ has recently formulated a lymph node map to standardize reporting lymph node locations. This reference is intended to reconcile the discrepancies in previous reporting systems including the Mountain–Dresler map⁷ and the Naruke map.⁸

The purpose of our work is to automatically detect pathologically enlarged mediastinal lymph nodes and to automate the standardized reporting of these lymph nodes according to the IASLC system.

Automated detection of mediastinal lymph nodes is inherently challenging for two primary reasons. First, given the similar attenuation of lymph nodes relative to adjacent anatomic structures such as vessels, esophagus, and heart, the relative contrast between lymph nodes and mediastinal structures is very low. Second, the shape and size of lymph nodes vary considerably (Fig. 1). Proper lymph node station labeling amplifies the challenge of mediastinal lymph node detection because it depends not only on accurate detection but also on proper localization of the lymph node relative to surrounding anatomic structures. Figure 2 shows the illustration of IASLC mediastinal lymph node map.

FIG. 1.

Examples of enlarged lymph nodes.

FIG. 2.

IASLC lymph node station map. (Partially reproduced with permission from Hoffman et al., “Automatic identification of IASLC-defined mediastinal lymph node stations on CT scans using multiatlas organ segmentation,” Proc. SPIE 9414, 94141R (2015).)

Several previous works have investigated the automated detection of lymph nodes in CT imaging. Feuerstein et al.¹⁰ built a statistical station atlas for lymph node detection on chest CT. Feulner et al.^11,12 employed a spatial prior from segmentation of the heart and esophagus and discriminative learning for lymph node detection on chest CT. In addition to mediastinal lymph node detection, there have been several preceding works which have investigated lymph node detection in the abdomen, pelvic, and axillary.^13–18 For example, Kitasaka et al.¹⁶ presented a 3D minimum directional difference filter for abdominal lymph nodes detection and applied Hessian-based vesselness to reduce the false positives. Barbu et al.^13–18 used a robust learning-based method with marginal space learning to detect and segment axillary and pelvic lymph nodes in CT imaging.

There have also been preceding works which have investigated the labeling of mediastinal lymph node stations. Feuerstein et al.¹⁰ built a station atlas by deformable registration. For a new CT image, lymph node stations were determined by registering the atlas to it. Lu et al.¹⁹ described each lymph node station as one or multiple cuboids according to their geometric constraints to adjacent structures. Matsumoto²⁰ trained a fuzzy model to classify different lymph node stations. Ei-Sherief²¹ and Lynch²² manually traced the lymph node stations and demonstrated the IASLC nodal map on CT images. However, this technique was prone to poor interobserver variability.

Compared to the aforementioned methods, the major contribution of this work includes the following: (i) Rather than using different methods for different structures, multiple anatomic structures are simultaneously segmented by a multiatlas segmentation method. (ii) Spatial prior features from the segmented structures are integrated by random forest for lymph node candidate generation. (iii) Lymph node stations are determined based on their surrounding anatomic structures. (iv) The text definitions of the IASLC nodal map are translated into patient-specific color-coded CT images to address the ambiguities in the text definitions of IASLC.

In this work, several surrounding or contextual anatomic structures are first segmented via multiatlas label fusion. Then, the spatial prior from the segmentation of the anatomic structures is concatenated with other local features to be processed by random forest classification²³ for lymph node detection. Finally, for each detected lymph node, with the segmentation of anatomic structures, its station is automatically determined according to the IASLC reference.

Our overall system has six steps: (1) region of interest (ROI) identification, (2) key structures segmentation by multiatlas label fusion, (3) candidate generation by random forest classification, (4) false positive rejection by lymph node segmentations, (5) support vector machine (SVM) classification, and (6) lymph node station mapping. This work flow is illustrated in Fig. 3.

FIG. 3.

System work flow. (ROI: Region of interest. SVM: Support vector machine.)

2. LYMPH NODE DETECTION

2.A. Region of interest

The mediastinum is automatically located as a ROI since this work focuses on detecting mediastinal lymph nodes. We first segment the lungs and the trachea by thresholding. The spine is segmented by watershed and directed graph search.²⁴ A convex hull is then computed and used to cover the lungs and spine. By taking the convex hull, the muscle and fat close to the body wall are excluded from further analysis. The ROI is defined as the convex hull excluding the segmentations of trachea, lungs, and spine. Figure 4 shows an example of a mediastinal ROI.

FIG. 4.

An example of a mediastinal ROI (blue). Segmentations of lungs (brown) and spine (yellow) are shown. (See color online version.)

2.B. Anatomic structure segmentation

In lymph node analysis, identification of anatomic structures plays an important role for two reasons. First, lymph nodes reside outside anatomic organs. Hence, identification of anatomic structures decreases the volume that needs to be searched. Second, with the identification of anatomic structures, the lymph node stations can be automatically determined according to the IASLC reference (Fig. 2). Some structures, such as the trachea, lungs, and spine, are easy to be segmented in CT images with intensity-based segmentation methods. Other structures such as esophagus and heart are challenging to be segmented in CT images. Here, eight more structures, esophagus, pulmonary trunk, aortic arch (AA), ascending aorta, descending aorta, azygos arch, heart, and super vena cava, are simultaneously segmented by multiatlas with joint label fusion method.²⁵ In total, 11 anatomic structures are segmented in this work.

An atlas A is defined as a triple of images A = (I, L¹, L²) where I is the CT image and the L’s are its corresponding labeled structures. We further divide L into two subgroups. L¹ denotes the structures (trachea, lungs, and spine) that can be easily segmented automatically by intensity-based methods in I. L² represents the eight structures (esophagus, pulmonary trunk, etc.) manually labeled in I. Let $U=(I_T,L_T^1,L_T^2)$ be an image to be segmented ($L_T^1$ can be easily segmented automatically and $L_T^2$ denotes the target structures that must be segmented by atlas method) and $A_1=(I_1,L_1^1,L_1^2)$, …, $A_n=(I_n,L_n^1,L_n^2)$ be n atlases. A registration between A_i and U is required to propagate the label $L_i^2$ to U. Each atlas A_i is registered to U by the label-guided diffeomorphic registration method in advanced normalization tools (ANTs),²⁶ which results in transformations u_i. The atlas labels $L_i^2$ are then warped to the target U by u_i. Finally, the label of voxel x in $L_T^2$ is determined by

$${\displaystyle L_T^2\left(x\right)=\sum _{i=1}^n{\omega }_i\left(x\right)u_i\left[L_i^2\left(x\right)\right],}$$ (1)

where $u_i\left[L_i^2\left(x\right)\right]$ is the warped ith atlas label for x and ω_i(x) is its weight with $\sum _{i=1}^n{\omega }_i\left(x\right)=1$. Similarity-weighted voting is usually used to estimate ω_i(x). That is, atlas that is more similar to the target has higher weight. One potential limitation of the similarity-weighted method is the voting weights ignore the correlations between atlases and are computed independently. In this work, we use an advanced fusion algorithm, the joint label fusion²⁵ that estimates optimal weights ω_i(x) by considering the similarity and correlation between the atlases,

$${\displaystyle {\omega }_i\left(x\right)=\frac{M_x^{-1}{1}_n}{1_n^t{M}_x^{-1}{1}_n},}$$ (2)

where 1_n is a constant vector of size n and M_x is a dependency matrix describing the correlation between ith and jth atlases. More detail about M_x can be found in Ref. 25. In Ref. 25, joint label fusion outperformed the similarity-based fusion methods.

We use five atlases (n = 5). According to Ref. 27, n = 5 is appropriate for aorta and heart segmentation in chest CT images for multiatlas segmentation method. Increasing the number of atlases may not significantly improve the accuracy of segmentation. Figure 5 shows the example of one of our atlases.

FIG. 5.

Example of one atlas. Heart (blue), mainstem bronchi (light gray), pulmonary trunk (light green), esophagus (teal), descending aorta (dark purple), superior vena cava (dark gray), ascending aorta (yellow), azygos arch (pink), and aortic arch (brown) are manually segmented. (See color online version.)

2.C. Lymph node candidate generation

Typical lymph nodes are round in 2D or ellipsoidal in 3D (i.e., blob-like) with CT attenuations in the soft tissue range (approximately 25–60 HU). They tend to have relatively sharp borders unless they become confluent with adjacent nodes or abut other anatomic structures. Hence, all voxels between −10 and 130 HU in the ROI are included as target voxels. For each target voxel, a set of features are computed.

2.C.1. Shape features

2.C.1.a. Object scale-based blobness.

As a blob-like structure, blobness is an important feature of lymph nodes. An object scale-based blobness B_σ(x) at voxel x²⁸ is defined as

$${\displaystyle B_{\sigma }\left(x\right)=\left\{\begin{array}{cl}\hfill (1-e^{\frac{-R_A^2}{2}})(1-e^{\frac{-S^2}{2}})e^{\frac{-R_c^2}{2}}\hfill & \text{if}{\lambda }_i<0\text{for}i=1,\hspace{0.17em}2,\hspace{0.17em}3\hfill \\ \hfill 0\hfill & \text{otherwise}\hfill \end{array}\right.,}$$ (3)

where $R_A=\left|{\lambda }_2\right|/\left|{\lambda }_3\right|$, $S=\sqrt{{\lambda }_1^2+{\lambda }_2^2+{\lambda }_3^2}$, and R_c = k − 2σ. λ₁, λ₂, λ_3, and (|λ₁| ≤|λ₂| ≤|λ₃|) are eigenvalues of the Hessian matrix. k is the local object scale at x. Object scale k is defined as the largest radius of a ball centered at x. All voxels within the ball should satisfy some homogeneity criterion.²⁹ In this way, object scale contains the size information for local structures. Multiple image scales (σ) are used to capture the size variation of lymph nodes. Blobness B_σ(x) is intended to suppress the line-like and plane-like structures and enhance the blob-like structures. It will be maximal when object scale (k), the radius of the blob, and image scale (2σ) approximately match. Ideally, the blob center has the maximal blobness B_σ(x). In this work, B_σ(x) is computed at multiple image scales σ = 2.5, 3.5, and 4.5 mm (to capture lymph nodes measuring in 10, 14, and 18 mm respectively) and the maximal is selected.

2.C.1.b. Circular transformation.

The rationale of circular transformation Ct(x) is based on the fact that the center of a circular object has the minimal variation of the distance to object boundary.³⁰ For each voxel x, m evenly oriented radial lines from x with length l are built and its circular transform is defined as

$${\displaystyle Ct\left(x\right)=\frac{{\displaystyle \sum _{i=1}^m{g}_i}}{{\displaystyle \frac{1}{\overline{D}}\sqrt{\sum _{i=1}^m{(D_i-\overline{D})}^2}}}.}$$ (4)

For each radial line ith, g_i is its closest local maximal gradient to x. D_i is the corresponding distance of the voxel with g_i to x. $\overline{D}$ is the average of all D_i. Like blobness B_σ(x), the center of a circular object will have maximal Ct(x) since its variation of D_i is minimal. In this work, we build radial lines at every 20° on three orthogonal views (m = 54 andl = 20 voxels) because 18 (360°/20°) directions and 20 voxels in each direction are enough for circular transformation computation of big lymph nodes, such as lymph nodes greater than 20 mm in size.

2.C.2. Spatial prior feature

Lymph nodes always lie between two or more adjacent anatomic organs. For example, the prevascular lymph nodes lie between the superior vena cava (SVC) and aortic arch (Fig. 11). That is, the segmentations of superior vena cava and aortic arch are more important than other organ segmentations for prevascular lymph node detection. Therefore, in our previous work,^31,32 the spatial prior of having a lymph node at x was defined as the minimal distance of segmented structures (closest organ) to x,

$${\displaystyle S_m\left(x\right)=min\left(d_j\left(x\right)\right)1\le j\le O,}$$ (5)

where d_j(x) is the smallest distance from x to the jth structure. S_m(x) indicates that larger gaps between adjacent structures have higher possibility to have enlarged lymph nodes. O is the number of segmented structures.

FIG. 11.

Demonstration of stations 3A and 3P.

Since lymph nodes reside outside the organs, we define a binary mask G to reduce the search region for lymph nodes detection,^31,32

$${\displaystyle G\left(x\right)=\left\{\begin{array}{lc}0\hfill & \hfill \text{if}x\text{is}\text{inside}\text{any}\text{organ}\hfill \\ 1\hfill & \hfill \text{else}\hfill \end{array}\right..}$$ (6)

G is a binary mask (0 for regions which cannot contain lymph nodes and 1 for other regions). In this way, all structures segmented in the previous steps are excluded by the mask G.

Finally, the local object scale-based blobness B_σ, the circular transformation Ct, and the spatial prior S are hard integrated by the binary mask G as a lymph node detection response,^31,32

$${\displaystyle R\left(x\right)=G\left(x\right)max\left(B_{\sigma }\left(x\right)\right)Ct\left(x\right)S_m\left(x\right).}$$ (7)

That is, voxels with high blobness, circular transformation, and spatial prior are enhanced. Voxels with higher R(x) are selected and clustered as initial lymph node candidates.

However, one limitation of this method^31,32 is that if the segmentation of some structure incorrectly covers some lymph nodes (G(x) = 0), the system will not be able to detect them since R(x) = 0 for these lymph nodes. Therefore, in this work, features are not hard integrated by the binary mask G [Eq. (6)]. The shape and spatial prior features are soft integrated by a random forest classifier²³ for lymph node candidate generation.

Spatial prior is redefined as the relative multidimensional distance vector corresponding to each structure instead of minimal distance in Refs. 31 and 32,

$${\displaystyle S_v\left(x\right)=[d_1\left(x\right),\dots ,d_O\left(x\right)],}$$ (8)

where d_j(x), 1 ≤ j ≤ O is the signed distance from x to jth segmented structure. There are 11 structures (O = 11) in this work. Fourteen features (intensity, blobness B_σ, the circular transformation Ct, and the spatial prior vector S_v(x)) are then parsed by the random forest classifier.

During random forest training, positive samples are all voxels within the lymph nodes demarcated by a radiologist. An equal number of negative samples are obtained within a shell (beyond 3 but within 15 pixels) from the true lymph nodes. Negative samples near the lymph node boundary (less than 3 pixels) are ignored to avoid uncertainty. Figure 6 shows the positive and negative sample regions. Twenty patients with 62 lymph nodes are used for random forest training.

FIG. 6.

Positive sample (green “+”): all voxels of a manually labeled lymph node (green curve). Negative samples (blue “−”): randomly selected from a surrounding shell (blue curves). (See color online version.)

For every target voxel in the ROI, intensity, blobness B_σ, circular transformation Ct, and spatial prior S_v(x) are pushed through the final trained ensemble of T decision trees, and a lymph node probability P_ln is computed. Voxels with P_ln ≥ th are selected as lymph node candidates. We use a small validation set (5 patients with 18 lymph nodes) and lymph node detection recall and precision are computed to estimate the T and th. Different configuration of the number of decision trees (T = {25,  50,  100}) and the lymph node probability threshold (th = {0.4,  0.5,  0.6,  0.7,  0.8}) are tested. Figure 7 shows the detection recall and precision curves on different combination of T and th. We set T = 50 and th = 0.7 in our experiment since it has the best performance (100% recall and 7% precision on five patients).

FIG. 7.

Lymph node detection recall and precision curves on different T and th. T = 50 and th = 0.7 have the largest recall and the second largest precision.

2.D. False positive rejection

Since only the enlarged lymph nodes (short axis diameter ≥10 mm) are clinically meaningful, small lymph nodes are eliminated from the lymph node candidates. Therefore, the detected lymph node candidates are segmented and their short axis is estimated for early false positive rejection. Many works about lymph node segmentation in CT images have been developed.^33–41 In this work, using the centroids of lymph node candidates as seed points [Fig. 8(a)], a level-set-based curve evolution is applied [Fig. 8(b)]⁴² to segment the lymph nodes. An ellipsoid [Fig. 8(c)] is fitted and its short axis is estimated. In our implementation, the lymph node candidates which have short axis less than 8 mm are removed from the detections since they do not meet the clinical criteria for abnormality and our system is designed to detect enlarged lymph nodes (lymph nodes measuring 10 mm or greater).

FIG. 8.

Graphical depictions of (a) centroids (yellow dots) of detected lymph node candidates, (b) segmentations (yellow curves) of lymph node candidates, and (c) fitted ellipse (green ellipse) and short axis (green line) of segmented lymph node. (See color online version.)

2.E. Texture feature for SVM classification

In Sec. 2.C, shape and spatial prior features are integrated by random forest for lymph node candidate generation. Texture features are also important to classify lymph nodes with other soft tissues. Texture features computed by Haralick gray-level co-occurrence matrix (GLCM)⁴³ have been applied to many medical image applications^44–47 since it measures not only the intensity distribution but also the spatial correlation to neighbor voxels. In this work, GLCM features are computed from the segmented lymph node candidates. For each lymph node candidate, 52 matrices (13 directions and 4 offset distances) and 12 features (energy, correlation, entropy, contrast, sum of mean, variance, inertia, cluster shade, homogeneity, cluster tendency, inverse variance, and maximal probability) from the matrix are computed. In total, 624 GLCM features are generated for each detected candidate.

An SVM committee⁴⁸ is used for final classification. In the training phase, to optimize the SVM committee configuration, two-way ANOVA is performed on the training set to analyze the effect of committee number and feature length. The area under the receiver operating characteristic (ROC) curve in a leave-one-patient-out validation is used as the performance metric. The committee optimization process forms a committee of seven SVMs with three features per SVM. More details about feature selection can be found in Ref. 48. In the test phase, the detections and features are fed into the trained SVM committee to determine whether they are true lymph nodes or false detections.

3. LYMPH NODE STATION LABELING

With the segmentation of anatomic structures, we successfully assign the manually labeled lymph nodes to their stations in our previous work.⁹ In this work, we assign the detected lymph nodes to their stations to automate the standardized lymph node reporting according to the IASLC system.

The IASLC lymph node station map is comprised of 14 lymph node stations which are in turn grouped into five zones (Fig. 2).⁶ Because chest CT examinations commonly do not include the full extent of the supraclavicular zone, we exclude station 1 from our analysis. The geographic range of our detections spans from stations 2 to 14.

3.A. Superior zone: Stations: 2–4

The superior zone has six stations (2R, 2L, 3A, 3P, 4R, 4L) with the clavicles defining the cranial border and the pulmonary trunk defining the caudal border.

The right and left lower paratracheal lymph nodes (stations 4R and 4L, respectively) lie directly adjacent to the trachea, superior to the level of the aortic arch, and inferior to the bifurcation of the superior vena cava in the axial plane (Fig. 9). Thus, the border of station 4R is formed by the SVC and trachea [Fig. 9(a)]. The border of station 4L is formed by the aortic arch [Fig. 9(b)].

FIG. 9.

Demonstration of stations 4R (a) and 4L (b).

Paratracheal stations 2R and 2L span between the level of the clavicles and carina in the axial plane (Fig. 10). Thus, with the border of stations 4R and 4L from the previous step, 2R and 2L are determined. Left (L) or right (R) is in relation to the trachea.

FIG. 10.

Demonstration of stations 2R and 2L. (Arrows: upper borders of 4R and 4L.)

The prevascular station 3A is comprised of lymph nodes which lie between the superior vena cava and aortic arch. The prevertebral station 3P is comprised of lymph nodes lying between the esophagus and spine (Fig. 11).

3.B. Aortic zone: Stations 5–6

The aortic zone is comprised of para-aortic (station 5) and subaortic (station 6) lymph nodes between the aorta and pulmonary trunk in the axial plane [Fig. 9(b)].

3.C. Inferior zone: Stations 7–9

The inferior zone is comprised of subcarinal (station 7), paraesophageal (station 8), and pulmonary ligament (station 9) lymph nodes, all of which lie caudal to the axial level of the carina. The name of a particular station is derived from the closest anatomic structure lying adjacent to the lymph node of interest (Fig. 12).

FIG. 12.

Demonstration of stations 7–9 and 10–14.

3.D. Hilar, lobar, and (sub)segmental zone: Stations 10–14

The hilar, lobar, and (sub)segmental zones lie distal to the mediastinal pleural reflection. The name of a particular station is derived from the closest airway structure lying adjacent to the lymph node of interest (Fig. 12).

For each detected lymph node, it is assigned a station/zone if its centroid falls into the station/zone.

The lymph node detection and station labelling are implemented with matlab 2013. The multiatlas label fusion is implemented with c ++ for anatomical structures segmentation.⁴⁹ jafroc (jackknife alternative free-response receiver operating characteristic) version 4.2⁵⁰ is used for computing statistical differences between the performances of two detection methods (with and without random forest).

4. RESULTS

4.A. Dataset

Our patient population consisted of 95 patients with 378 enlarged lymph nodes on contrast-enhanced abdominal CT scan documented in the radiologist’s report. Each patient has one or more lymph nodes measuring greater than 10 mm in diameter. A radiologist manually traced these lymph nodes slice-by-slice on the CT images. For each lymph node, its station was also recorded. These tracings and station labels served as the ground truth. The average lymph node size was 15.6 ± 5 mm. The CT voxel spacing within an axial slice was 0.7–0.9 mm and slice thickness was 1 mm. This dataset is online available.⁵¹

Five patients were randomly selected as the atlases to segment anatomic structures. For each atlas, eight structures (esophagus, pulmonary trunk, aortic arch, descending aorta, super vena cava, ascending aorta, and heart) were manually segmented using 3D slicer. These structures were also manually segmented in another 20 randomly selected patients for evaluation of anatomic structure segmentation. Sixty two enlarged lymph nodes in these 20 patients were used for random forest training. In the remaining 70 patients, 316 enlarged lymph nodes were used for lymph node detection and station labelling.

4.B. Performance of anatomic structure segmentation

The segmentation of anatomic structures not only provides important spatial prior information for lymph node detection but also provides references to define the station map of lymph nodes. The performance of multiatlas label fusion for structure segmentation is reported in Table I. Figure 13 shows some examples of segmented organs.

TABLE I.

Performance of structure segmentation.

Organs	Dice coefficient
Heart	0.90 ± 0.03
Descending aorta	0.82 ± 0.05
Ascending aorta	0.81 ± 0.05
Aortic arch	0.79 ± 0.08
SVC	0.77 ± 0.09
Pulmonary trunk	0.70 ± 0.10
Azygos arch	0.65 ± 0.11
Esophagus	0.57 ± 0.18

FIG. 13.

Three examples of organ segmentation, one per row. First column: Original CT scan. Second column: Segmentation by using multiatlas label fusion. Third column: Ground truth. Heart (blue), esophagus (teal), mainstem bronchi (light gray), ascending aorta (yellow), descending aorta (dark purple), pulmonary trunk (light green), azygos arch (pink), and superior vena cava (dark gray). (See color online version.)

4.C. Performance of lymph node detection

Lymph node detection performance is evaluated using tenfold cross-validation. For each fold, SVM committee is optimized. Figure 14(a) shows the free response receiver operating characteristic (FROC) curves of the detection performance for the 316 lymph nodes. With random forest, the system achieves a sensitivity of 88% at 8 false positives per patient. This is significantly (p = 0.0013 from jafroc) better than the performance on the same dataset of our previous method without random forest (68% sensitivity at 8 false positives per patient).³¹

FIG. 14.

(a) FROC comparisons of methods with and without random forest. (b) FROC comparisons of the method with random forest on lymph nodes of different sizes.

The FROC curves on different sizes of lymph nodes are shown in Fig. 14(b). The lymph node size is represented by its short axis. With random forest, the system achieves a sensitivity of 85% at 4 false positives per patient for large lymph nodes (short axis is greater than 20 mm) and a sensitivity of 66% at 4 false positives per patient for small lymph nodes (short axis is between 10 and 20 mm). Examples of detected lymph nodes are shown in Fig. 15. Examples of missed and false lymph node detections are shown in Fig. 16. One lymph node [Fig. 16(a)] is missed because it is misidentified as esophagus and another lymph node [Fig. 16(b)] is missed because it is not covered by the ROI (the patient’s left lung has volume loss and the convex hull of lungs does not cover the lymph node). Pericardial recess [Figs. 16(d) and 16(e)], azygous vein [Fig. 16(c)], and sternohyoid and sternothyroid muscles [Fig. 16(f)] are the main sources of false positive detections (47.6%, 19.1%, and 28.6%, respectively).

FIG. 15.

Examples of detected lymph nodes. The anatomic locations are (a) aorticopulmonary window, (b) precarinal, (c) left hilar, and (d) subcarinal.

FIG. 16.

False negative (red) and false positive (yellow) examples. (See color online version.)

4.D. Performance of lymph node station labeling

In clinical practice, assessing lymph nodes by nodal zone rather than individual station is useful because multiple adjoining stations may involve in the mediastinal adenopathy. Zonal labeling performance is shown in Tables II and III. Figure 17 shows the comparison between the stations defined in the IASLC guideline and the stations defined in our lymph node station map. Figure 18 shows two examples of detected lymph nodes and their correctly labeled stations.

TABLE II.

Performance of lymph node station mapping.

Zone	Station numbers	Number of nodes	Detected	Correctly labeled	Accuracy (%)
Superior	2–4	123	114	106	86.2
Aortic	5–6	31	27	22	71.0
Inferior	7–9	77	73	66	85.7
Hilar and Lobar	10–14	85	79	73	85.9
Overall	2–14	316	293	267	84.5

TABLE III.

Confusion matrix of lymph node station mapping.

		Zone (labeled)
		Superior	Aortic	Inferior	Hilar and Lobar
Zone (reference)	Superior	106	2	5	1
	Aortic	2	22	2	1
	Inferior	3	3	66	1
	Hilar and Lobar	0	2	4	73

FIG. 17.

Comparison of IASLC stations [(a), (c), and (e)] and our lymph node station map [(b), (d), and (f)]. The labels have lymph node stations defined in Fig. 2.

FIG. 18.

(a) Example of detected lymph nodes and (b) their station (4R). (c) Example of detected lymph node and (d) its station (10). Partially reproduced with permission from Hoffman et al., “Automatic identification of IASLC-defined mediastinal lymph node stations on CT scans using multiatlas organ segmentation,” Proc. SPIE 9414, 94141R (2015).

4.E. Computation time

Random forest training on 20 patients with 62 lymph nodes took around 45 min. Lymph node detection is evaluated with tenfold cross-validation on 70 patients with 316 lymph nodes. For each fold, SVM committee optimization took around 20 min. The computation time for each step is summarized in Table IV.

TABLE IV.

Computation time.

Computation	Time (min)
ROI by lung and spine segmentation	1
8 key structures segmentation by multiatlas label fusion	30
Candidates generation by random forest	<1
False positive rejection by lymph node segmentation	<1
Lymph node classification by SVM	1
Lymph node station labeling	2

5. DISCUSSION AND CONCLUSION

In this work, we present an automatic method for mediastinal lymph nodes detection and station labeling on chest CT images, which is challenging due to the low contrast between surrounding structures and lymph nodes. In addition, lymph node stations are closely located together with ambiguous borders.

First, several anatomic structures are segmented via multiatlas label fusion. Multiatlas label fusion uses multiple atlases to compensate for potential error from using a single atlas. The labels from all atlases are fused for the final segmentation. Multiatlas segmentation methods have been used in many medical image analyses and outperform the methods using single-atlas.^25,52–55

The organ segmentation in this work is similar to the work of del Toro and Muller,⁵⁵ in which a multiatlas with anatomical hierarchy is proposed for multiple organs segmentation in the thorax and abdomen. The differences include the following: (1) In Ref. 55, small organs (such as gallbladder and trachea) are guided by large organs (such as liver, lungs) during the atlas to target registration. In this work, presegmented spine and lungs are used for label-guided registration. (2) Majority voting is used for label fusion in Ref. 55. We use joint label fusion to consider the similarity and correlation between the atlases.

Table I shows that the segmentations of heart and esophagus have the best and worst performance, respectively. Therefore,⁵⁵ if we use the segmentation of the heart to guide the registration of small organs such as esophagus, the organs’ segmentation accuracy may be further improved.

Then, intensity, shape, and the spatial prior from the segmented structures are aggregated by a random forest classifier²³ for lymph node detection. A random forest classifier is an ensemble method and consists of many trees that generate a number of classifiers. The results from those classifiers are aggregated to make classifications. The benefits of using a random forest classifier include: (i) efficient computation in training and classification, (ii) probabilistic output, (iii) robust tolerance of variability within input training data and, and (iv) built-in inherent feature selection. It has been successfully applied to various organ localization and segmentation tasks in different imaging modalities.^56–59

Finally, for each detected lymph node, with the segmentation of anatomic structures, its station is automatically determined according to the IASLC reference.

Evaluation of 70 patients shows the advantage of incorporating various heterogeneous features into the random forest classification. Using the location of anatomic structure from the multiatlas enabling accurate station, identification is also evident.

As a comparison, the method without using random forest demonstrates lower performance on lymph node detection. The main reason is because the detection is limited within the binary mask G [Eq. (6)], defined by the segmentation of the anatomic structures. If the segmentation of some structure incorrectly covers lymph nodes because of their low contrast and similar appearance, the system will not be able to detect them. However, with the random forest, the spatial priors from anatomic structures are defined as a soft constraint of thresholding on distance vectors S_v within the random forest weak hypothesis optimization [Eq. (8)]. By including all the distance vectors during training, random forest optimally finds thresholds and more lymph nodes could be detected at a given false positive rate even with negative distance measurements (lymph nodes are inside inaccurate structure segmentations). In this way, the structure segmentation inaccuracy in the preprocessing can be largely compensated.

The random forest classifier is used for lymph node candidate generation. The subsequent SVM committee is utilized for final classification. The reason is that unlike the SVM committee, the random forest is highly computationally efficient and used to winnow the search space before final classification. For example, the search region for lymph nodes has around 3.2 × 10⁶ voxels for each patient. Intensity, shape, and prior features are computed on all voxels. The random forest classifier takes less than 1 min to generate about 50 lymph node candidates for each patient. For those 50 candidates, texture features are fed into the SVM committee for final classification.

Our mediastinal lymph node detection performance is slightly better than the state-of-the-art performance¹¹ (72% sensitivity at 8 false positives per patient). Based on the detections outputted from our system,^60,61 further improve the detection performance to a sensitivity of 80% at three false positives per patient.

Compared to previously published work,^10,19 our lymph node station labeling improves in three aspects. (i) We build the atlases as a number of unambiguous anatomical structures rather than ambiguous nodal stations in Ref. 10. (ii) Apart from the cubic lymph node stations defined in Ref. 19, our stations are bordered by the segmentations of surrounding structures. (iii) We simultaneously segment the key structures by a coherently optimized multiatlas method rather than using different methods for different structures in Ref. 19.

A potential limitation of our method is the use of classic anatomy atlases to identify anatomical structures for lymph node detection and station labelling. Any extreme variation from classic anatomy, for example, severe scoliosis or postsurgical resection of large organs, may confound our system as it relies on atlas-to-patient registration.

In summary, we showed that a random forest classifier with spatial prior enabled state-of-the-art mediastinal lymph node detection and labeling. This system can be adapted for abdominal, pelvic, and axillary lymph node detection. Furthermore, to augment the clinical utility of our system, we have automated the translation of a mediastinal lymph node location to a precise IASLC station label with the greater aim of standardizing the radiologic reporting of metastatic mediastinal lymph nodes in the context of lung cancer.

CONFLICT OF INTEREST DISCLOSURE

Authors Summers and Yao have pending and/or awarded patents for related automated analyses and receive royalty income for a patent license from iCAD. All other authors have no conflicts of interest.