Computerized tertiary lymphoid structures density on H&E-images is a prognostic biomarker in resectable lung adenocarcinoma

Summary

The increased amount of tertiary lymphoid structures (TLSs) is associated with a favorable prognosis in patients with lung adenocarcinoma (LUAD). However, evaluating TLSs manually is an experience-dependent and time-consuming process, which limits its clinical application. In this multi-center study, we developed an automated computational workflow for quantifying the TLS density in the tumor region of routine hematoxylin and eosin (H&E)-stained whole-slide images (WSIs). The association between the computerized TLS density and disease-free survival (DFS) was further explored in 802 patients with resectable LUAD of three cohorts. Additionally, a Cox proportional hazard regression model, incorporating clinicopathological variables and the TLS density, was established to assess its prognostic ability. The computerized TLS density was an independent prognostic biomarker in patients with resectable LUAD. The integration of the TLS density with clinicopathological variables could support individualized clinical decision-making by improving prognostic stratification.

Graphical abstract

Highlights

• •Automated workflow was developed for quantifying TLS density on H&E-stained WSIs
• •The TLS density was an independent prognostic factor in resectable LUAD patients
• •Prediction model integrated with the TLS density had stronger predictive power

Health informatics; Computational bioinformatics; Cancer

Introduction

Lung adenocarcinoma (LUAD) is the most common subtype of lung cancer,¹ which has high incidence and mortality. Accurate stratification for LUAD patients could aid individualized clinical decision-making. The tumor-node-metastasis (TNM) staging² system plays a key role in the risk stratification and management of LUAD patients, but LUAD patients at the same TNM stage may exhibit significantly different prognosis due to tumor heterogeneity.^1,3 Recent reports have found that tertiary lymphoid structures (TLSs)^4,5,6 play an important role in antitumor immune response.^7,8 The increased amount of TLSs is correlated with better prognosis in various cancer types including melanoma,⁹ lung cancer,¹⁰ and gastrointestinal cancer,¹¹ etc. Thus, TLSs have the potential to be a complementary prognostic biomarker in LUAD.

Most previous studies have quantified and analyzed the TLS-associated indicators typically using multiplex immunohistochemistry (mIHC) or immunofluorescence (mIF) approach.^12,13,14 However, mIHC and mIF are not routine examinations with high costs, which limited their clinical application. In comparison, hematoxylin and eosin (H&E) staining is the most widely used staining technique in histopathology. H&E-stained histopathology slides are inexpensive and easily accessible. Representative images of intra-tumoral TLSs on H&E-stained slides are shown in Figure 1. Therefore, the assessment of TLSs in H&E-stained slides might be more easily promoted to clinical practice.

Figure 1.

Example images of TLSs in the tumor region

(A) Immature TLSs present dense, round, or oval-shaped lymphoid aggregates (blue arrow, 10× magnification; scale bar, 200 μm).

(B) Higher magnification image of an immature TLS (red arrow, 40× magnification, corresponding to the bottom left one in subfigure A; scale bar, 50 μm).

(C) Mature TLSs with a germinal center (blue arrow, 10× magnification; scale bar, 200 μm).

(D) Higher magnification image of a mature TLS, and germinal center is shown as a pale area (red arrow, 40× magnification, corresponding to the center one of subfigure C; scale bar, 50 μm). TLSs, tertiary lymphoid structures.

Several approaches^15,16 have been developed for quantifying the TLSs based on routine H&E-stained whole-slide images (WSIs). Nevertheless, these methods for identifying TLSs are manually assessed by pathologists, which are time-consuming and experience-dependent with poor reproducibility.¹⁷ With the rapid development of computer technology,¹⁸ an automated and standardized workflow has hopefully been developed for the efficient and accurate assessment of TLSs in routine H&E-stained WSIs of LUAD.

In this study, we developed an automated workflow for quantifying the density of TLSs in the tumor regions on H&E-stained WSIs and then evaluated the prognostic value of TLS density using three independent datasets of patients with resectable LUAD. We hypothesized that the TLS density was an independent prognostic factor for disease-free survival (DFS), and the integration of the TLS density with clinicopathological variables would improve prognostic stratification in patients with resectable LUAD.

Results

Patients

Totally 1196 patients with resectable LUAD were collected in this study, and ultimately 802 patients met the inclusion and exclusion criteria (Additional file: Figure S1). Among them, 322 patients from Guangdong Provincial People’s Hospital (GDPH) formed the discovery set, 226 patients from The Second Xiangya Hospital of Central South University (XYSH) formed the external validation set V₁, and 134 patients from Shanxi Provincial Cancer Hospital (SXCH) and 120 patients from Yunnan Cancer Hospital (YNCH) formed the external validation set V₂. The median follow-up was 85.3(83.1–89.8) months for the discovery set, 60.2(58.5–62.7) months for the external validation set V₁, and 37.1(35.5–48.3) months for the external validation set V₂. The baseline clinicopathological variables of the three datasets are shown in Table 1. Significant differences were found in all included clinicopathological variables among the three datasets, except for sex (p = 0.158) and tumor site (p = 0.055).

Table 1.

Distributions of demographic and clinicopathological variables of patients with resectable LUAD in the discovery set and the two external validation sets

Variable	Discovery set	Validation set V1	Validation set V2	p
	(n = 322)	(n = 226)	(n = 254)
Age				<0.001
≤65 years	206(64.0%)	186(82.3%)	200(78.7%)
>65 years	116(36.0%)	40(17.7%)	54(21.3%)
Sex				0.158
Male	168(52.2%)	105(46.5%)	140(55.1%)
Female	154(47.8%)	121(53.5%)	114(44.9%)
Smoking status				< 0.001
Ever	80(24.8%)	66(29.2%)	101(39.8%)
Never	242(75.2%)	160(70.8%)	153(60.2%)
Tumor site				0.055
Upper/Middle	223(69.3%)	151(66.8%)	152(59.8%)
Lower	99(30.7%)	75(33.2%)	102(40.2%)
Family history				0.005
Yes	33(10.2%)	13(5.8%)	9(3.5%)
No	289(89.8%)	213(94.2%)	245(96.5%)
Differentiation				< 0.001
G1/G2	252(78.3%)	159(70.4%)	158(62.2%)
G3/G4	70(21.7%)	67(29.6%)	96(37.8%)
TNM stage				< 0.001
I	235(73.0%)	158(69.9%)	142(55.9%)
II	40(12.4%)	36(15.9%)	49(19.3%)
III	47(14.6%)	32(14.2%)	63(24.8%)
VPI				< 0.001
Yes	149(46.3%)	132(58.4%)	35(13.8%)
No	173(53.7%)	94(41.6%)	219(86.2%)
LVI				0.004
Yes	19(5.9%)	1(0.4%)	13(5.1%)
No	303(94.1%)	225(99.6%)	241(94.9%)
Type of surgery				0.003
Segmentectomy/Wedge	31(9.6%)	9(4.0%)	9(3.5%)
Lobectomy/Pneumonectomy	291(90.4%)	217(96.0%)	245(96.5%)
EGFR				< 0.001
Positive	114(35.4%)	0(0.0%)	25(9.8%)
Negative	64(19.9%)	0(0.0%)	34(13.4%)
Unknown	144(44.7%)	226(100%)	195(76.8%)
ALK				< 0.001
Positive	17(5.3%)	0(0.0%)	3(1.2%)
Negative	243(75.5%)	0(0.0%)	51(20.1%)
Unknown	62(19.2%)	226(100%)	200(78.7%)
Adjuvant chemotherapy				< 0.001
Yes	71(22.0%)	56(24.8%)	116(45.7%)
No	251(78.0%)	147(65.0%)	138(54.3%)
Unknown	0(0.0%)	23(10.2%)	0(0.0%)
Follow-up (month, median [95%CI])	85.3(83.1–89.8)	60.2(58.5–62.7)	37.1(35.5–48.3)
No. of events	134(41.6%)	100(44.2%)	95(37.4%)

The p-value is obtained using Pearson’s chi-squared test. The median follow-up time was estimated by the reverse Kaplan-Meier method.

G1/G2, well-differentiated or moderately differentiated; G3/G4, poorly differentiated or undifferentiated; VPI, visceral pleural invasion; LVI, lymphovascular invasion; EGFR, epidermal growth factor receptor; ALK, anaplastic lymphoma kinase; CI, confidence interval.

Segmentation results and consistency analysis

The Bland-Altman plot exhibited favorable consistency between our automatic TLSs counting and the pathologist’s manual TLSs counting based on H&E-stained WSIs (intra-class correlation coefficient [ICC] = 0.894, 95%CI: 0.821–0.938, p < 0.001; Figure 2A), or based on IHC-stained WSIs (ICC = 0.903, 95%CI: 0.836–0.944, p < 0.001; Figure 2B). The example images of manual TLSs annotation on H&E-stained WSIs and automatic TLSs segmentation are shown in the Additional file: Figure S2. A good qualitative consistency was observed in the TLSs identification.

Figure 2.

Bland-Altman plot for agreement between manual TLSs counting and automated TLSs counting

(A) Manual TLSs counting based on H&E-stained WSIs.

(B) Manual TLSs counting based on IHC-stained WSIs. The red solid line shows the mean of the difference between manual TLSs counting and automated TLSs counting. The blue dashed lines show the upper/lower bounds of 95% limits of agreement (95% LoA). Abbreviations: ICC, intraclass correlation coefficient; CI, confidence intervals.

Prognostic value of TLS density

The cutoff value of TLS density was determined to be 0.489 TLSs per mm² using the maximally selected rank statistics method in the discovery set. Kaplan-Meier curves showed that the TLS-low group had a significantly worse DFS compared with the TLS-high group in the discovery set (HR 0.66, 95%CI 0.44–0.98, p = 0.037; Figure 3A) and the two external validation sets (V₁: 0.60, 0.39–0.93, p = 0.022; V₂: 0.29, 0.16–0.55, p < 0.0001; Figures 3B and 3C). After combining the TLS density and TNM stage, the risk stratification became more refined for resectable LUAD patients in the three datasets (p < 0.0001; Figures 3D–3F). In Figure 4A, violin plots presented that the lymphocyte density within the tumor region was significantly higher in the TLS-high group compared with the TLS-low group in the three datasets (p < 0.001). Furthermore, we observed that the low TLS density was significantly associated with the immune-desert phenotype in the three datasets (p < 0.001; Figure 4B).

Figure 3.

Prognostic value of the TLS density

Kaplan-Meier curves of patients stratified by the TLS density in the discovery set (A) and the two validation sets (B and C). Kaplan-Meier curves of patients stratified by combining the TLS density with the TNM stage in the three datasets in the discovery set (D), and the two validation sets (E and F).

Figure 4.

The association of TLS density with lymphocyte density and immune phenotypes, and the performance of predictive models

(A) The lymphocyte density within the tumor region is significantly higher in the TLS-high group compared with the TLS-low group in the three datasets (p < 0.001). The p value is analyzed using the Wilcoxon rank-sum tests.

(B) The TLS density distribution across immune phenotypes in the three datasets. The lower TLS density is associated with the immune-desert phenotype in the three datasets (p < 0.001). The p value is analyzed by the Cochran-Armitage trend test.

(C) The iAUC with 1000 times bootstrap for each model is shown as a boxplot in the three datasets. Abbreviations: iAUC, the integrated area under the curve. Clinical model: smoking status + differentiation + TNM stage + LVI; Full model: smoking status + differentiation + TNM stage + LVI + TLS density.

Based on the results of the univariable analysis, sex, smoking status, TNM stage, differentiation grade, visceral pleural invasion (VPI), lymphovascular invasion (LVI), adjuvant chemotherapy, immune phenotypes, and TLS density were used as candidate factors in subsequent multivariable analysis. In the multivariable stepwise Cox regression analysis, TLS density was demonstrated as an independent prognostic factor for DFS in the discovery set (HR 0.61, 95%CI 0.41–0.92, p = 0.017) and the two external validation sets (V₁: 0.60, 0.37–0.95, p = 0.031; V₂: 0.32, 0.17–0.60, p < 0.001; Table 2). Furthermore, after adjustment of the multivariable analysis without stepwise regression for the factors with a p value less than 0.05 in the univariable analysis (Table 2), the p values of TLS density remained below 0.05 in the three datasets, further indicating that TLS density was an independent prognostic factor for DFS (Additional file: Table S2).

Table 2.

Univariable and multivariable analysis for DFS in the discovery set and the two external validation sets

	Discovery set		Validation set V1		Validation set V2
	HR (95%CI)	p	HR (95%CI)	p	HR (95%CI)		p
Univariable analysis
Age
> 65 years vs. ≤ 65 years	1.32(0.93–1.86)	0.115	1.02(0.60–1.71)	0.953	0.80(0.47–1.37)		0.410
Sex
Female vs. Male	0.64(0.45–0.91)	0.012	0.69(0.46–1.02)	0.061	0.62(0.41–0.94)		0.025
Smoking status
Ever vs. Never	1.72(1.20–2.47)	0.003	1.25(0.81–1.93)	0.316	1.32(0.88–1.97)		0.181
Tumor site
Lower vs. Upper/Middle	0.87(0.60–1.27)	0.482	1.33(0.88–2.01)	0.182	0.95(0.63–1.44)		0.820
Family history
Yes vs. No	1.20 (0.71–2.02)	0.503	1.25(0.58–2.70)	0.574	0.23(0.03–1.64)		0.142
Differentiation
G3/G4 vs. G1/G2	3.03(2.12–4.33)	<0.001	2.07(1.38–3.12)	<0.001	1.76(1.18–2.64)		0.006
TNM stage
II vs. I	2.58(1.61–4.14)	<0.001	2.00(1.18–3.39)	0.010	2.43(1.41–4.20)		0.001
III vs. I	5.51(3.69–8.22)	<0.001	5.95(3.71–9.54)	<0.001	4.25(2.66–6.77)		< 0.001
VPI
Yes vs. No	1.42(1.01–2.00)	0.042	1.06(0.71–1.59)	0.759	1.72(1.00–2.96)		0.051
LVI
Yes vs. No	4.37(2.54–7.53)	<0.001	2.96(0.41–21.36)	0.281	2.05(0.94–4.43)		0.070
Type of surgery
Segmentectomy/Wedge vs.	0.87(0.48–1.57)	0.644	1.14(0.46–2.81)	0.775	1.61(0.65–3.97)		0.300
Lobectomy/Pneumonectomy
Adjuvant chemotherapy
Yes vs. No	3.85(2.71–5.47)	<0.001	2.31(1.53–3.50)	<0.001	3.06(1.97–4.74)		< 0.001
Immune phenotypes
IP2 vs. IP1	0.57(0.38–0.86)	0.007	0.52(0.30–0.90)	0.019	0.37(0.21–0.64)		< 0.001
IP3 vs. IP1	0.50(0.33–0.76)	0.001	0.43(0.21–0.86)	0.016	0.37(0.18–0.77)		0.008
TLS density
High vs. Low	0.66(0.44–0.98)	0.039	0.60(0.39–0.93)	0.024	0.29(0.16–0.55)		< 0.001
Multivariable analysis
Smoking status
Ever vs. Never	1.69(1.17–2.45)	0.005	1.42(0.89–2.25)	0.137
Differentiation
G3/G4 vs. G1/G2	1.66(1.10–2.51)	0.016	1.39(0.89–2.15)	0.146	1.64(1.09–2.48)		0.019
TNM stage
II vs. I	1.95(1.18–3.23)	0.010	2.07(1.20–3.59)	0.009	2.20(1.27–3.81)		0.005
III vs. I	3.81(2.40–6.04)	<0.001	5.16(3.11–8.56)	<0.001	3.46(2.16–5.56)		< 0.001
LVI
Yes vs. No	1.78(0.97–3.25)	0.062
TLS density
High vs. Low	0.61(0.41–0.92)	0.017	0.60(0.37–0.95)	0.031	0.32(0.17–0.60)		< 0.001

HR, hazard ratio; CI, confidence interval; G1/G2, well-differentiated or moderately differentiated; G3/G4, poorly differentiated or undifferentiated; IP1, immune-desert phenotype; IP2, immune-excluded phenotype; IP3, inflamed phenotype; VPI, visceral pleural invasion; LVI, lymphovascular invasion.

In subgroup analysis, all the patients in the three datasets were merged to improve discovery power. For early stage (stage I–II) or TNM stage I LUAD patients, the DFS in the TLS-high group were significantly better than those in the TLS-low group (p < 0.001; Additional file: Figures S3A and S3B). For TNM stage I and II LUAD patients, there was still a trend that the TLS-low groups had a poor prognosis (Additional file: Figures S3C and S3D), although no statistically significant difference was found. In addition, similar trends were also observed in most subgroups stratified the patients by clinicopathological variables including age, sex, smoking status, tumor site, differentiation grade, VPI, LVI, and adjuvant chemotherapy (Additional file: Figures S4 and S5), except in the subgroup stratified by family history, type of surgery, epidermal growth factor receptor (EGFR), and anaplastic lymphoma kinase (ALK).

Development and validation of the prediction model

Based on the independent prognostic factors (i.e., smoking status, differentiation grade, TNM stage, LVI, and TLS density; Table 2) determined by the stepwise selection method in the discovery set, a Cox model (full model) for predicting DFS was established. Furthermore, four other reference models were built for performance comparison, including TNM stage model, TLS density model, TNM stage and TLS density model, and clinical model (smoking status and differentiation grade and TNM stage and LVI). The coding, regression coefficients, and estimated 5-year baseline hazard for each prediction model were summarized in the Additional file: Table S1.

The performance of the five models was evaluated in the three datasets (Table 3). The full model had better discrimination and calibration ability than the clinical model in the discovery set (concordance index [C-index] 0.716 vs. 0.702, the integrated area under the curve [iAUC] 0.702 vs. 0.690, Akaike information criterion [AIC] 1373.345 vs. 1377.481, likelihood ratio test [LRT] p = 0.013; Figure 4C) and the two external validation sets (V₁: C-index 0.710 vs. 0.694, iAUC 0.691 vs. 0.674, AIC 929.065 vs. 931.250; V₂: C-index 0.697 vs. 0.681, iAUC 0.694 vs. 0.678, AIC 923.722 vs. 937.909). Similarly, the TNM stage and TLS density model achieved better performance compared with the TNM stage model in the three datasets. Additionally, the time-dependent receiver operating characteristic (ROC) curves at 5-year and time-dependent area under the curve (AUC) curves at different times were shown in the Additional file: Figure S6. The full model exhibited better predictive performance at most time points compared with the clinical model in the three datasets.

Table 3.

Performance of models in the discovery set and the two external validation sets

Model	Discovery set			Validation set V1			Validation set V2
	iAUC	C-index(95%CI)	AIC	iAUC	C-index(95%CI)	AIC	iAUC	C-index(95%CI)	AIC
TLS density model	0.543	0.548(0.510–0.586)	1442.527	0.555	0.564(0.517–0.612)	970.278	0.585	0.605(0.568–0.642)	954.701
TNM stage model	0.643	0.658(0.617–0.699)	1387.206	0.656	0.673(0.624–0.721)	931.919	0.668	0.669(0.616–0.722)	938.821
TLS density & TNM stage model	0.664	0.691(0.651–0.731)	1385.670	0.679	0.707(0.661–0.752)	929.558	0.687	0.696(0.646–0.747)	927.186
Clinical model	0.690	0.702(0.659–0.745)	1377.481	0.674	0.694(0.643–0.745)	931.250	0.678	0.681(0.625–0.736)	937.909
Full model	0.702	0.716(0.673–0.758)	1373.345	0.691	0.710(0.660–0.759)	929.065	0.694	0.697(0.643–0.751)	923.722

Clinical model: Smoking status + Differentiation + TNM stage + LVI; Full model: Smoking status + Differentiation + TNM stage + LVI + TLS density.

AUC, the integrated area under the curve; CI, confidence interval; AIC, Akaike information criterion; LVI, lymphovascular invasion.

Discussion

Accurate prognosis prediction for resectable LUAD could guide clinical decision-making and improve risk stratification. The TLSs had shown important prognostic value. However, the existing approaches for TLSs evaluation in resectable LUAD are limited by lacking an objective, efficient, and accurate TLSs quantification workflow. Consequently, in this multi-center study, we developed an automated workflow to access the density of TLSs in the tumor region based on routine H&E-stained WSIs and evaluated the prognostic value of TLS density for predicting DFS in three independent datasets of patients with resectable LUAD. We found that the TLS density was an independent prognostic factor in patients with resectable LUAD, and integrating the TLS density with clinicopathological variables could improve the prognostic stratification in resectable LUAD.

TLSs play a crucial role in effective antitumor immunity.⁷ Based on available evidence, the presence and higher density of TLSs are associated with better outcomes.^9,14,19 The prognostic value of TLSs has been evaluated in several solid tumor types. For instance, Horeweg et al.¹⁹ found that intratumoral TLSs had a strong favorable prognostic impact beyond clinicopathological and molecular variables for endometrial cancer patients. Silina et al.¹⁴ analyzed TLS formation in peritumoral and intra-tumoral regions for human lung squamous cell carcinoma, and determined that TLS density was an independent prognostic marker in untreated patients. Recent studies^15,16 determined the prognosis value of TLSs in patients with liver cancer, accounting for the TLSs at varying anatomic subregions (intra-tumor and peri-tumor region) and between different immune classes. However, most of the previous studies evaluated TLSs using manual and qualitative ways. The manual assessment-based approach for quantifying TLSs is highly labor-intensive and susceptible to inter-observer variability¹⁷ due to the heterogeneous morphology of TLSs.

In this study, we referred to current studies comprehensively and quantified the TLS density in the tumor region. Unlike the aforementioned studies, our study developed an automated pipeline to evaluate the TLS density in a standardized way. In the tumor segmentation phase, the transfer learning approach was adopted to pre-train the segmentation model using an annotated publicly available dataset. The pre-trained model was fine-tuned using only a small number of histopathological images of LUAD with annotations. In the tissue segmentation and classification phase, we employed a weakly supervised method based on patch-level classification labels to perform tissue semantic segmentation on histopathological images. Moreover, there was a good agreement between manual TLSs counting based on H&E-stained WSIs (ICC = 0.894; p < 0.001) or based on IHC-stained WSI (ICC = 0.903, p < 0.001) and automated TLSs counting in this study, as shown in Figure 2 and Additional file: Figure S2. Thus, compared with the method based on manual evaluation, the automated workflow could not only reduce the workload of annotation for pathologists but also increase reproducibility and reliability.

TLSs were assessed based on mIHC or mIF images in most studies¹³ for lung cancer, which were not routinely used in the clinical setting. In our study, a workflow based on routine H&E stained WSIs of LUAD was designed for evaluating TLSs, which might be easier to promote the clinical application. In the context of lung cancer, only a limited number of studies have made progress in developing automated methods for TLSs detection and analysis at present. Barmpoutis et al.²⁰ proposed a methodology for the automated identification and quantification of TLS, and performed segmentation of lymphocytes through an ellipsoidal model. This pipeline showed great potential for clinical application, but the study did not analyze its value for risk stratification. A pilot study²¹ conducted by Kushnarev et al. designed a digital imaging analysis platform for identifying TLSs on H&E stained WSIs of LUAD, and performed overall survival analysis by patient stratification in an independent cohort. In contrast, we validated the prognostic ability of TLS density quantified in the automated workflow at three independent datasets from different regions in China. We found that the TLS density was an independent prognostic factor in the three datasets, even though there were statistically significant differences in baseline characteristics among these datasets (Table 1). Moreover, the association of the quantified TLS density with DFS was further validated in most of the subgroups stratified by age, sex, smoking status, tumor site, differentiation, and TNM stage (Additional file: Figures S3 and S4). While integrating the TLS density into the TNM stage model and clinical model, respectively, the models with the TLS density showed better discrimination and calibration than those without the TLS density in the three datasets (Table 3 and Figure 4C). These indicated that the quantified TLS density is a robust biomarker for predicting DFS in resectable LUAD, which was expected to achieve further clinical generalization.

Based on the spatial distribution of tumor-infiltrating lymphocytes, the tumor microenvironment can be classified into three immune phenotypes, including inflamed, immune excluded, and immune-desert.²² In this study, we observed that the patients with inflamed immune phenotype tend to have favorable prognosis, while those with immune-desert phenotype tend to have worse outcomes (Table 2; Additional file; Figure S7). Recent studies^23,24 showed that different immune phenotypes might correspond to distinct anti-tumor immune responses. Among them, the inflamed phenotype is usually accompanied by an intense inflammatory response, and the “inflamed” tumors demonstrate infiltration more densely with immune cells.^25,26 The formation of TLSs in the tumor region may promote the generation of immune responses.⁸ Therefore, the presence or increase of TLSs is more common in the inflamed phenotype of LUAD, whereas the presence of TLS may be relatively less in the immune-excluded and immune-desert phenotypes. The association was consistent with our findings (p < 0.001; Figure 4B). Moreover, the computerized TLS density was identified as an independent prognostic factor for DFS in the multivariable analysis (Table 2). Instead, immune phenotypes were not independently associated with DFS in resectable LUAD, which might indicate that the quantified TLS density had the potential to provide stronger predictive power.

In conclusion, we developed an automated computational workflow for quantifying the TLSs in the tumor region of routine H&E-stained WSIs, and analyzed the association between the TLS density and DFS in patients with resectable LUAD. This study demonstrated that the quantified TLS density was an independent prognostic biomarker with good reliability, reproducibility, and generalizability, and integration of the TLS density with clinicopathological variables could guide individualized clinical decision-making by improving patient stratification.

Limitations of the study

This study also had several limitations. First, as a retrospective study, it had an inherent limitation. We will further validate the prognostic value of the TLS density on larger prospective cohorts in the future. Second, this study indirectly quantified TLSs based on tissue segmentation results. Due to the variations in the morphology of TLSs, there was a certain level of bias for the TLSs counting. Nevertheless, there was good agreement between automated TLSs counting and manual TLSs counting methods. Meanwhile, we are currently developing a fully automated workflow to directly identify TLSs on H&E-stained WSIs further improving the accuracy of TLSs counts. Besides, we will also classify TLSs into lymphoid aggregates, primary follicles, and secondary follicles with a germinal center according to their maturation degree, and assess their differences in prognostic effect in patients with resectable LUAD.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Software and algorithms
R code for statistical analysis	This paper	https://github.com/YuMeng-W/TLS_DENS-HE-WSI
Hover-Net	Graham et al.27	https://github.com/vqdang/hover_net
Tissue segmentation toolbox	Han et al.28	https://github.com/ChuHan89/WSSS-Tissue
ResNet-50	He et al.29	https://arxiv.org/abs/1512.03385
Python	Open resource	https://www.python.org/
RStudio	Open resource	https://www.rstudio.com/
R software	R CRAN	https://cran.r-project.org/
survival R package (v3.5-5)	R CRAN	https://cran.r-project.org/web/packages/survival/index.html
survminer R package (v0.4.9)	R CRAN	https://cran.r-project.org/web/packages/survminer/index.html
timeROC R package (v0.4)	R CRAN	https://cran.r-project.org/web/packages/timeROC/index.html
lmtest R package (v0.9-40)	R CRAN	https://cran.r-project.org/web/packages/lmtest/index.html
lpSolve R package (v5.6.18)	R CRAN	https://cran.r-project.org/web/packages/lpSolve/index.html
irr R package (v0.84.1)	R CRAN	https://cran.r-project.org/web/packages/irr/index.html
ggplot2 R package (v3.4.2)	R CRAN	https://cran.r-project.org/web/packages/ggplot2/index.html
tidyr R package (v1.3.0)	R CRAN	https://cran.r-project.org/web/packages/tidyr/index.html
RColorBrewer R package (v1.1-3)	R CRAN	https://cran.r-project.org/web/packages/RColorBrewer/index.html
gbm R package (v2.1.8.1)	R CRAN	https://cran.r-project.org/web/packages/gbm/index.html
blandr R package (v0.5.1)	R CRAN	https://cran.r-project.org/web/packages/blandr/index.html
openxlsx R package (v4.2.5.2)	R CRAN	https://cran.r-project.org/web/packages/openxlsx/index.html
boot R package (v1.3-28)	R CRAN	https://cran.r-project.org/web/packages/boot/index.html
cutoff R package (v1.3)	R CRAN	https://cran.r-project.org/web/packages/cutoff/index.html
Other
Camelyon Challenge dataset	Litjens et al.30	https://camelyon17.grand-challenge.org

Resource availability

Lead contact

Further information and requests for resources and reagents should be directed to and will be fulfilled by the lead contact, Zaiyi Liu (liuzaiyi@gdph.org.cn).

Materials availability

This study did not generate new unique reagents.

Experimental model and study participant details

Patients

This retrospective study was performed using resectable LUAD patients from four institutions across different regions in China: GDPH (2007-11 to 2014-12), XYSH (2013-08 to 2017-12), SXCH (2014-07 to 2020-11) and YNCH (2012-09 to 2014-07). The multi-center study was approved by the institutional ethics committees of the above four hospitals (approval number: KY-Q-2022-338-01). The informed consent of patients was waived because this was a retrospective image analysis study.

DFS, defined as the time interval from surgery to the recurrence or death, was identified as the endpoint of interest. We collected the baseline and clinicopathological variables from the medical information system, including age at surgery, sex, smoking status, family history, tumor site, differentiation grade, TNM stage, VPI, LVI, type of surgery, EGFR, ALK, and adjuvant chemotherapy. The cases with missing clinicopathological variables, receiving neoadjuvant therapy, remaining residual tumors (R1/R2 resection), and having a history of other malignant tumors were excluded.

Tissue slides digitalization

We acquired digital WSIs from the H&E-stained diagnostic slides of the primary tumor. The H&E-stained diagnostic slides were scanned under 40× magnification using a Leica Aperio-AT2 scanner (USA, 0.252μm/pixel) in GDPH and YNCH, a Leica Aperio-GT450 scanner in XYSH (USA, 0.264μm/pixel), and a Hamamatsu scanner in SXCH (Japan, 0.220μm/pixel), respectively. In addition, we collected the available slides stained with CD3 in GDPH which were adjacent to the routine H&E-stained slides used for this study. These IHC-stained slides were digitalized by a Leica Aperio-AT2 scanner (USA, 0.252 μm/pixel) at 40× magnification. The image quality was reviewed by an experienced pathologist (B.B.L.) without knowledge of patients’ clinical information, and excluded WSIs that were blurry, contained artifacts, exhibited poor staining, or lacked sufficient tumor tissues.

Method details

Tissue and cell segmentation and classification

The overview of this study is shown in below figure. The semi-automatic segmentation of the tumor region was performed using the same strategy as in our previous study.³¹ To reduce the amount of annotation, we used transfer learning to train a ResNet50²⁹ model for the tumor region segmentation. We extracted millions of small positive and negative image patches from the Camelyon16³⁰ dataset to pre-train the model. We fine-tuned the pre-trained model using 67 annotated WSIs of LUAD from GDPH. The fine-tuning model was adopted in the tumor region segmentation for all WSIs from the four hospitals. The segmentation masks of the tumor region were visually checked by a pathologist (B.B.L.), and the imperfect masks were manually corrected (see below figure).

Proposed workflow of our study

An automated pipeline is employed to identify and analyze the TLSs on routine H&E-stained WSIs.

(A) Semi-automated tumor region segmentation.

(B) Image tile extraction from tumor region and fully automated tissue segmentation at 10× magnification, including tumor epithelium (red), tumor stroma (orange), and lymphocyte aggregate (green).

(C) TLSs identification in the tumor region. Based on the tissue segmentation mask, the lymphoid aggregates greater than or equal to 150μm in diameter (or major axis in oval-shaped TLSs) are identified as TLSs.

(D) Biomarker construction. TLS density is calculated as “the number of TLSs in the tumor region/the area of tumor region.” The cutoff value is determined by the maximally selected rank statistics method in the discovery set.

(E) Survival analysis and external validation. Abbreviations: TLSs, tertiary lymphoid structures; hematoxylin and eosin, H&E; WSIs, whole-slide images. Scale bar for WSIs in subfigures (A–C) indicates 3 mm; scale bar for patches in subfigure (B) indicates 200 μm; scale bar for a patch in subfigure (C) indicates 50 μm.

Based on the results of tumor region segmentation, we employed the weakly-supervised semantic segmentation method²⁸ based on patch-level classification labels to perform tissue segmentation in the tumor region, finally reducing the effort of annotation. The method classified the tumor region into tumor epithelium, tumor stroma, and lymphocyte aggregate at 10× magnification. The framework is shown in the above figure.

Furthermore, we deployed Hover-Net²⁷ to segment and classify the cell in the tumor region at 40× magnification. These cells were categorized into tumor cells, stromal cells, lymphocytes, and other cells. We identified the tissue region where the lymphocytes are located and obtained the lymphocyte density within the cancer epithelium, stroma, and whole tumor region.^23,32 Then, we quantified three immune phenotypes (i.e., inflamed, immune-excluded, and immune-desert) using the lymphocyte density data.²⁴

TLS density quantification

The TLS density was quantified using the following procedure (see figure in tissue and cell segmentation and classification). First, we calculated the area of the tumor region for each WSI. Based on the pre-segmented tissue segmentation mask, we then excluded the lymphoid aggregates with a diameter (or the major axis of the ellipse) less than 150μm¹³ and obtained final segmentation masks of TLSs. Finally, we defined the TLS density as the ratio between the number of TLSs in the tumor region and the whole tumor area as follows:

$$TLSdensity=\frac{NumberofTLSsinthetumorregion}{Areaoftumorregion}$$

For stratifying these patients, the maximally selected rank statistics method³³ was used to determine the cutoff value within the discovery set. Any patient with TLS density less than the cutoff value belonged to the TLS-low group, whereas any patient greater than or equal to the cutoff value belonged to the TLS-high (see figure in tissue and cell segmentation and classification).

To analyze the consistency between manual counting and automated counting of TLSs, we randomly selected 50 H&E-stained WSIs and 50 IHC-stained WSIs, respectively, from all available histopathological images. An experienced pathologist (B.B.L.), who was blind to the result of automatic TLSs segmentation, manually identified TLSs on these selected WSIs and determined the TLSs count in the tumor region for each WSI. The manual counting was conducted using Aperio ImageScope (version 12.3.2) in this study. The ICC and Bland-Altman plot were used to assess the agreement between manual TLSs counting based on H&E-stained WSIs or based on IHC-stained WSIs and automated TLSs counting using our quantitative method.

Quantification and statistical analysis

Categorical data were reported as count (percentage). Differences between the three datasets were compared via Pearson’s chi-squared test. The median follow-up was estimated by the reverse Kaplan–Meier method. The log-rank test was used to estimate differences in DFS between the TLS-low and TLS-high groups. The distribution of the lymphocyte density within the tumor region between TLS-high and TLS-low groups was compared by the Wilcoxon rank-sum tests. The Cochran-Armitage trend test was used to analyze the association between TLS density and immune phenotypes.

The prognostic ability of TLS density and other clinical variables was assessed using the Wald test in univariable Cox regression analysis. The variables that achieved statistical significance (P-value less than 0.05) in the univariable analysis were retained as the candidates for the subsequent multivariable analysis. Multivariable stepwise Cox regression analysis based on the AIC was used to determine an optimal prediction model for DFS. The discrimination ability of predicted models was measured using Harrell’s C-index³⁴ and the iAUC (resampling with 1000 times bootstrapping).³⁵ The model calibration was evaluated by the AIC. The predicted accuracy at 5-year DFS was measured by the time-dependent ROC and AUC curves. The LRT was employed to determine whether the differences between the models were significant. Furthermore, to further demonstrate that the TLS density was an independent prognostic factor, multivariable analysis without stepwise regression was also used to analyze all variables that reached P<0.05 in the univariable analysis.

All statistical analyses were conducted using R³⁶ software (version 4.1.2, www.R-project.org) with packages survival, survminer, lmtest, risksetROC, and vioplot. P<0.05 (two-tailed) was considered statistically significant.

Published: August 16, 2023

Data and code availability

• •The patients’ data reported in this study cannot be deposited in a public repository due to the privacy of patients. To request access, please contact the lead contact (Zaiyi Liu, liuzaiyi@gdph.org.cn) upon reasonable request approved by the institutional review board of all enrolled centers.
• •All original code has been deposited at the repository (https://github.com/YuMeng-W/TLS_DENS-HE-WSI) and is publicly available as of the date of publication. DOIs are listed in the key resources table.
• •Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.