Clinical utility of CT-based deep learning for differentiating pulmonary aspergillosis from bacterial pneumonia: a multicenter retrospective study

Abstract

Background

Early differentiation of pulmonary aspergillosis (PA) from bacterial pneumonia (BP) remains challenging in respiratory practice because of overlapping clinical and computed tomography (CT) features. Conventional microbiological and serological tests may be invasive, delayed, or insufficiently sensitive in selected patients. This study aimed to develop and externally validate CT-based deep learning (DL) models as adjunctive tools for differentiating PA from BP.

Methods

This multicenter retrospective study included patients diagnosed with PA or BP at The First Affiliated Hospital of Guangzhou Medical University and The Second Affiliated Hospital of Guangzhou Medical University. CT images from The First Affiliated Hospital of Guangzhou Medical University were used for model development and internal validation, while data from The Second Affiliated Hospital of Guangzhou Medical University were used for external validation. Two CT-based DL models were developed: an unannotated supervised training (UST) model using CT images without manual lesion annotation and an annotated supervised training (AST) model incorporating physician-annotated lesions. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV). A standardized image-only questionnaire was also used to explore clinician performance in differentiating PA from BP.

Results

A total of 387 PA cases and 311 BP cases were included. In internal validation, the AST model achieved an AUC of 0.83 [95% confidence interval (CI): 0.76–0.90], with 81.7% sensitivity, 73.2% specificity, and 78.0% accuracy, whereas the UST model achieved an AUC of 0.78 (95% CI: 0.69–0.86). In external validation, the AST model achieved an AUC of 0.81 (95% CI: 0.71–0.92), compared with 0.56 (95% CI: 0.41–0.70) for the UST model. In the standardized image-only clinician assessment, 158 valid responses were analyzed, with an average diagnostic accuracy of 68.2%. Exploratory analysis suggested that model performance varied across disease stages, highlighting the potential influence of temporal radiological changes.

Conclusions

The annotated CT-based DL model showed potential as an adjunctive imaging tool for differentiating PA from BP. However, the external sensitivity of 60.6% indicates that the model should not be used as a stand-alone diagnostic or rule-out tool. Prospective multicenter validation and integration of clinical and microbiological data are required before clinical implementation.

Highlight box.

Key findings

• The annotated supervised training (AST) model achieved an area under the receiver operating characteristic curve (AUC) of 0.83 and an accuracy of 78.0% in internal validation. In external validation, the AUC was 0.81 and specificity was 86.7%, but sensitivity decreased to 60.6%, with 13 of 33 pulmonary aspergillosis (PA) cases misclassified as bacterial pneumonia (BP). Thus, the AST model may support image-based differentiation but should not be used as a stand-alone diagnostic or rule-out tool. The image-only clinician assessment was exploratory and did not reproduce the complete clinical workflow.

What is known and what is new?

• PA and BP share overlapping clinical and computed tomography (CT) features, making early differentiation challenging. Previous CT-based artificial intelligence studies of pulmonary fungal infections have been limited by sample size, disease scope, or external validation.

• This multicenter study developed and externally validated two CT-based deep learning models, with and without physician-guided lesion annotation, in 698 patients and compared their performance with an image-only clinician benchmark.

What is the implication, and what should change now?

• The AST model should be used only as an adjunct to comprehensive clinical assessment. A BP prediction should not exclude PA or stop mycological investigation when clinical suspicion remains high. Model outputs should be interpreted with host factors, symptoms, laboratory findings, microbiological evidence, and complete radiological assessment. Prospective multicenter validation is required before routine clinical implementation.

Introduction

Pulmonary aspergillosis (PA) comprises a spectrum of acute and chronic fungal diseases caused by Aspergillus species, and its incidence has increased in recent years (1), particularly among immunocompromised individuals, including those with neutropenia, human immunodeficiency virus (HIV) infection, or post-transplant immunosuppression (2). Invasive pulmonary aspergillosis (IPA), the most aggressive form, is especially perilous because of its insidious onset and rapid progression and is associated with high mortality rates if left undiagnosed (3,4). Timely differentiation of PA from other pulmonary infections is therefore critical to guide potentially life-saving interventions.

Achieving an early diagnosis remains challenging. Although histopathology and sterile-site cultures are diagnostic reference standards, their utility is limited by invasiveness, contamination risks, and low microbial yields. Noninvasive tests such as serum galactomannan (GM) have limited sensitivity in nonneutropenic patients (5-7), and metagenomic next-generation sequencing (mNGS) lacks standardized interpretation. PA frequently mimics bacterial pneumonia (BP), including community-acquired pneumonia (CAP) and hospital-acquired pneumonia (HAP), because of overlapping clinical and radiological features (8-12). This diagnostic ambiguity underscores the need for tools that can help distinguish PA from BP.

Artificial intelligence (AI), particularly deep learning (DL) applied to radiological imaging, has emerged as a potential approach to the diagnosis of pulmonary infections. Compared with more extensively studied pulmonary conditions, applications to fungal infections remain relatively limited. Prior studies have shown potential: one computed tomography (CT)-based AI model distinguished IPA from non-fungal pneumonia (13), while another integrated clinical data to differentiate PA from Staphylococcus aureus pneumonia (78% versus radiologists’ accuracy of 40–75%) (14). More recent studies have extended CT-based AI to the diagnosis and subtype classification of chronic pulmonary aspergillosis (CPA) (15) and to multicentre segmentation of IPA, pulmonary mucormycosis, BP, and pulmonary tuberculosis lesions (16). Nevertheless, existing studies remain limited by sample size, disease scope, subtype heterogeneity, or limited external validation.

Building on the clinical resources available at a national respiratory medical center in China, we previously developed DL models for differentiating pulmonary cryptococcosis from lung adenocarcinoma (17) and distinguishing among pulmonary mucormycosis, IPA, and pulmonary tuberculosis (18). In this study, we aimed to develop and externally validate CT-based DL models as adjunctive tools for the initial differentiation of PA from BP in respiratory practice. Specifically, we compared two model development strategies, with and without physician-guided lesion annotation, and explored clinician performance using a standardized image-only questionnaire. We also performed an exploratory analysis of serial CT examinations to assess whether temporal radiological changes may influence model performance. By addressing this common diagnostic challenge, we sought to evaluate the performance of CT-based DL as a potential adjunctive imaging approach for differentiating PA from BP. The model is intended as adjunctive imaging support for respiratory physicians and other clinicians evaluating patients with suspected pulmonary infection. We present this article in accordance with the TRIPOD reporting checklist (19) (available at https://jtd.amegroups.com/article/view/10.21037/jtd-2026-1253/rc).

Methods

Study design

This retrospective study enrolled patients diagnosed with PA or BP from The First Affiliated Hospital of Guangzhou Medical University and The Second Affiliated Hospital of Guangzhou Medical University, both located in Guangzhou, China. The First Affiliated Hospital of Guangzhou Medical University provided data for model training and internal validation, with patient inclusion from January 2013 to December 2022. The Second Affiliated Hospital of Guangzhou Medical University provided external validation data, with patient inclusion from 1 January 2012 to 30 June 2021. The primary-center cohort was randomly divided at the patient level into training and internal validation sets at an approximate 8:2 ratio. The training set included 283 PA cases and 225 BP cases, while the internal validation set included 71 PA cases and 56 BP cases.

The primary-center cohort was randomly divided at the patient level into the training and internal-validation sets at an approximate ratio of 8:2 before image preprocessing. All CT examinations belonging to the same patient were assigned exclusively to one dataset, and no patient appeared in both the training and internal-validation sets. For patients with multiple CT examinations, only the initial diagnostic CT examination acquired at the time of PA or BP diagnosis was used for model development and primary performance evaluation. Subsequent examinations were excluded from the primary analysis and were evaluated separately in the exploratory longitudinal assessment. Treatment variables were not used as model predictors.

The study size was determined by the availability of eligible cases during the predefined retrospective study periods at the participating centers. All patients who met the inclusion and exclusion criteria were included for model development, internal validation, or external validation, as appropriate. The same diagnostic criteria, eligibility criteria, CT preprocessing procedures, and outcome definitions were applied to the development/internal validation and external validation cohorts. To evaluate the DL models’ performance across disease progression stages, we additionally selected a subgroup of patients from the primary center who had undergone at least three chest CT examinations during treatment. This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Ethics Committee of The First Affiliated Hospital of Guangzhou Medical University (No. ES-2023-017-01). The Second Affiliated Hospital of Guangzhou Medical University was informed of the study and agreed to its conduct. The requirement for informed consent was waived owing to the retrospective nature of the study.

Study population

PA diagnosis adhered to the 2019 European Organisation for Research and Treatment of Cancer and Mycoses Study Group Education and Research Consortium (EORTC/MSGERC) consensus guidelines for IPA (20), with CPA and allergic bronchopulmonary aspergillosis (ABPA) diagnosed according to Infectious Diseases Society of America guidelines (2). The exclusion criteria for PA were (I) prior surgical treatment for PA; (II) concurrent fungal infections; (III) coexisting pulmonary conditions (active tuberculosis, lung tumors, interstitial lung disease, pneumoconiosis, significant pleural effusion, or pneumothorax); and (IV) suboptimal CT image quality with substantial artifacts.

Patients with BP met three inclusion criteria: (I) documented diagnosis of pneumonia; (II) confirmed etiological evidence; (III) antibiotic treatment during hospitalization. The exclusion criteria were (I) fungal or viral pneumonia; (II) the aforementioned pulmonary comorbidities; (III) aspiration or postoperative pneumonia; and (IV) inadequate CT image quality for analysis. The outcome predicted by the models was the reference diagnosis of PA versus BP. Reference diagnoses were retrospectively determined from the available clinical records according to prespecified diagnostic criteria before model development and validation; information on assessor demographics was not collected. Based on the available medical records, clearly documented major immunocompromising conditions were defined as confirmed neutropenia, hematologic malignancy, hematopoietic stem-cell transplantation, solid-organ transplantation, or explicitly documented severe immunodeficiency. Suspected or unconfirmed diagnoses were not included in this category.

CT image processing

Two qualified specialists, including one clinician and one radiologist, independently evaluated CT image quality using the hospital's Picture Archiving and Communication System (PACS). Examinations meeting the predefined quality criteria were downloaded and anonymized to ensure patient confidentiality. For patients with multiple CT examinations, the initial examination acquired at the time of PA or BP diagnosis was selected for the primary analysis. Both non-contrast and contrast-enhanced CT series were eligible for inclusion if they met the predefined image-quality criteria; contrast enhancement was not an exclusion criterion. The same image preprocessing pipeline was applied to all selected CT series, irrespective of contrast-enhancement status. Slice thickness for the selected CT series was retrieved from the imaging metadata and recorded in millimeters. The distribution of slice thickness was summarized separately for the primary-center and external-validation cohorts. The Reconstruction Kernel attribute (0018,1210) and Reconstruction Algorithm attribute (0018,9315) were also queried from the available Digital Imaging and Communications in Medicine (DICOM) headers. Reconstruction-kernel metadata were available and contained vendor-specific labels, whereas the Reconstruction Algorithm attribute was absent or empty in the available CT series. Because reconstruction-kernel nomenclature is manufacturer-specific and reconstruction-algorithm metadata were unavailable, these parameters could not be harmonized across vendors or included in formal stratified analyses. Lesion annotation was performed using the Neusoft Darts platform (https://carevault.neusoft.com/darts) by eight board-certified respiratory physicians who received standardized training before annotation. The CT examinations were divided among the eight physicians, and each examination was assigned to one physician for initial annotation according to a predefined annotation protocol. The protocol required precise delineation of the involved lung lobes or segments and their associated vascular and bronchial structures.

All initial annotations were subsequently reviewed and refined by a three-member expert panel comprising a senior respiratory clinician with more than 10 years of clinical experience, a professor of respiratory medicine, and a chief radiologist. The panel jointly reviewed all annotated CT slices, and the panel-approved annotations were used as the final lesion masks for AST model development and validation. Because the annotation workflow consisted of one initial annotation followed by joint expert-panel review, rather than independent duplicate annotation by two physicians, formal pairwise interobserver agreement metrics, such as the Dice similarity coefficient or kappa, were not calculated. Annotation time was not prospectively recorded. In a sample of 10 consecutive annotations from a continuous work session, the mean annotation time, estimated from consecutive case-completion timestamps, was approximately 9 minutes per case.

All CT volumes and the corresponding lung or lesion masks were loaded with an additional channel dimension, reoriented to the right-anterior-superior (RAS) coordinate system, and resampled to a uniform voxel spacing of 0.8 mm × 0.8 mm × 1.0 mm. Image intensities were clipped to −1,000 to 500 Hounsfield units and normalized. Foreground cropping was performed using the automatically generated lung masks for UST inputs and the expert-reviewed lesion masks for AST inputs.

Lung segmentation was performed using a pretrained three-dimensional U-Net developed from 95 manually annotated chest CT examinations. The segmentation model was applied to the CT volumes in the present study to generate lung masks, which defined the pulmonary foreground for preprocessing of UST inputs. For AST inputs, the lesion-containing bounding box derived from the expert-reviewed lesion mask defined the foreground region, and a 10-voxel margin was retained around the lesion to preserve surrounding anatomical context.

Supervised model development

We developed two CT-based DL models with the same classification architecture but different input construction: (I) the unannotated supervised training (UST) model, developed without manual lesion delineation and using the automatically generated lung mask to define the pulmonary foreground; and (II) the annotated supervised training (AST) model, which used expert-reviewed lesion masks to define the lesion-containing foreground. For the AST model, only the final lesion masks reviewed and approved by the expert panel were incorporated into model development and validation. As illustrated in Figure 1, both models followed the same overall sequence of data acquisition, image preprocessing, data augmentation, three-dimensional feature extraction, model training, and performance evaluation, while their foreground definitions differed. The model architecture was adapted from our previously published framework (17); all preprocessing, input-construction, training, and implementation procedures used in the present study are described below.

Figure 1.

Workflow and network architecture of the deep learning models. The workflow includes data collection, lung segmentation, image preprocessing, data augmentation, feature integration, model training, and performance evaluation. Two model-development strategies were used: UST with CT images processed without manual lesion annotation and the automatically generated lung mask used to define the pulmonary foreground, and AST with expert-reviewed lesion masks used to define the lesion-containing foreground. 3D, three-dimensional; AST, annotated supervised training; CT, computed tomography; FC, fully connected; UST, unannotated supervised training.

Both models used the same three-dimensional residual convolutional neural network architecture. The network comprised an initial convolutional stem, two residual blocks (ResBlock1 and ResBlock2), subsequent three-dimensional convolutional and pooling layers, fully connected layers, and a two-node Softmax output for PA and BP. The UST and AST models differed in input construction rather than in the underlying classification architecture.

Model-specific batch-processing dimensions were set to 224×224×128 voxels for the UST model and 160×160×80 voxels for the AST model. Data augmentation was applied only to the training data and included random flipping, rotation, affine transformation, mild contrast adjustment, and Gaussian noise. The training and validation batch sizes were 4 and 1, respectively.

Model training used the AdamW optimizer with an initial learning rate of 1×10⁻⁴ and a weight decay of 1×10⁻⁵. A cosine annealing learning-rate scheduler was applied, with a minimum learning rate of 1×10⁻⁶. Class imbalance was addressed using weighted cross-entropy loss. Class weights were calculated from the training-set class frequencies using the inverse-frequency formula w_c = N/(K × n_c), where N is the total number of training cases, K is the number of classes, and n_c is the number of training cases in class c. The models were trained for a maximum of 200 epochs, with early stopping if the mean validation loss did not improve for 15 consecutive epochs. The checkpoint with the highest mean validation accuracy was retained for final inference. Predicted class labels were determined according to the larger of the two Softmax output probabilities. The principal model-development and training parameters are summarized in Table S1.

No formal grid, random, or Bayesian hyperparameter search was performed. Hyperparameters were selected empirically based on prior experiments and training stability. The internal validation set was used for early stopping and checkpoint selection.

Reproducibility was supported by setting the random seed to 42 using MONAI’s set_determinism function, with deterministic operations enabled where supported. Model development was implemented in Python 3.9.16 using PyTorch 1.12.1, MONAI 1.0.1, and SimpleITK 2.1.1.1. Performance visualization used scikit-learn 1.1.2 and matplotlib 3.5.2. Model training was performed under CUDA 11.7 on a workstation equipped with a single NVIDIA Tesla V100 GPU.

During internal and external validation, the finalized models were applied to the preprocessed CT images without further training or parameter adjustment. Model-generated probability scores and predicted labels were compared with the reference diagnoses to calculate diagnostic performance metrics.

Model explainability analysis

Gradient-weighted class activation mapping (Grad-CAM) was applied to the AST model to visualize image regions contributing to its classification output. Eight representative cases from the validation cohort were included in a balanced set comprising two correctly classified PA cases, two correctly classified BP cases, two false-negative PA cases, and two false-positive PA cases. For each case, the original axial CT image, expert-reviewed lesion annotation, and corresponding AST Grad-CAM heatmap from the same imaging level were displayed side by side. The heatmaps were used solely as qualitative visualizations of model attention and were not interpreted as quantitative lesion-segmentation results or evidence of causal model reasoning.

The explainability analysis focused on the AST model because its expert-reviewed lesion annotations enabled direct visual comparison between model attention maps and manually defined lesion regions.

Clinician assessment

We developed a standardized image-only online questionnaire to evaluate clinicians’ ability to differentiate PA from BP using CT images alone. The questionnaire comprised two sections: 10 demographic questions (Appendix 1) and 20 diagnostic cases, including 11 PA and 9 BP cases randomly selected from the internal validation set (Appendix 2). Each case presented 5–7 representative CT images in a binary PA/BP format, with 5 points awarded for each correct answer and a maximum score of 100. No clinical history, symptoms, laboratory findings, microbiological results, prior imaging, treatment information, or radiological reports were provided. The assessment was designed to provide a standardized image-only benchmark rather than to reproduce the complete clinical diagnostic workflow. The survey was distributed through the National Respiratory Medicine Center’s website to clinicians nationwide.

Statistical analysis

Statistical analyses were performed using SPSS 23.0 (IBM) and Python 3.9.16, with visualization performed using scikit-learn 1.1.2, matplotlib 3.5.2, and GraphPad Prism 9. Continuous variables are reported as mean ± standard deviation (SD) or median [interquartile range (IQR)], and categorical variables are reported as frequencies (%). Group comparisons were performed using t tests or Wilcoxon rank-sum tests, as appropriate, for continuous variables and χ² tests or Fisher’s exact tests, as appropriate, for categorical variables. The same methods were used to compare baseline demographic and disease characteristics between the primary-center and external-validation cohorts. All statistical tests were two-sided, and P<0.05 was considered statistically significant.

The DL implementation followed standard Python protocols. Ninety-five percent confidence intervals (CIs) for accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated using the Wilson score method based on the corresponding patient-level confusion matrices. Patient-level confusion matrices were constructed separately for the UST and AST models in the internal and external validation cohorts, with PA designated as the positive class and BP as the negative class.

F1 scores and Matthews correlation coefficients (MCCs) were calculated from the corresponding patient-level confusion matrices, with PA designated as the positive class and BP as the negative class.

Receiver operating characteristic (ROC) curves were generated using patient-level continuous PA prediction probabilities. Ninety-five percent confidence intervals for individual area under the receiver operating characteristic curve (AUC) estimates were calculated using the DeLong method. The AUCs of the UST and AST models were compared separately in the internal and external validation cohorts using two-sided paired DeLong tests for correlated ROC curves. The AUC difference was defined as the AUC of the AST model minus that of the UST model, and the corresponding 95% confidence interval (CI) was calculated using the DeLong covariance estimates. PA was designated as the positive class.

Model calibration was evaluated separately in the internal and external validation cohorts. For each model, patients were ordered according to their continuous predicted PA probabilities and divided into five approximately equal-sized groups. The observed PA proportion in each group was plotted against the corresponding mean predicted probability, together with the 45-degree ideal calibration line and a logistic recalibration curve. Calibration intercepts and slopes were estimated using logistic recalibration of the observed binary outcome on the logit-transformed predicted PA probability. Overall probabilistic accuracy was quantified using the Brier score, with lower values indicating better performance. Precision-recall curves were generated using continuous PA prediction probabilities, and average precision was calculated with PA designated as the positive class. The PA prevalence in each validation cohort was displayed as the reference level in the precision-recall plots.

Decision curve analysis was performed separately in the internal and external validation cohorts over threshold probabilities from 0.05 to 0.80 in increments of 0.01 (21). A model-positive result was conceptualized as prompting additional diagnostic evaluation for PA rather than initiating antifungal treatment. The net benefit of the UST and AST models was compared with strategies of evaluating all patients and evaluating no patients for PA. The threshold probabilities used in decision curve analysis represented clinical trade-offs and were distinct from the probability threshold of 0.5 used to generate the confusion-matrix-based classification metrics.

Patient-level stratified bootstrap resampling with 2,000 iterations was used to estimate percentile 95% confidence intervals for the Brier score and average precision, with PA and BP patients resampled separately within each validation cohort.

Exploratory subtype analyses were performed separately for IPA, CPA, and ABPA. For each analysis, patients with the corresponding PA subtype were treated as the positive group, while all patients with BP in the same validation cohort were retained as the negative group. The UST and AST models were not retrained for individual subtypes. Subtype-specific sensitivity was calculated using the prespecified probability threshold of 0.5, and subtype-versus-BP AUCs were calculated using continuous PA prediction probabilities. Ninety-five percent confidence intervals for sensitivity and specificity were calculated using the Wilson score method, and confidence intervals for AUCs were calculated using the DeLong method. No formal comparisons among PA subtypes were performed because of the limited subgroup sample sizes. Performance for the single external ABPA case was reported descriptively without estimating an AUC.

Missing data were handled by complete-case analysis. Cases with missing essential clinical, microbiological, or CT imaging data required for eligibility assessment, model development, or model validation were excluded from the corresponding analyses. No imputation was performed. For between-center baseline comparisons, available-case denominators were reported for variables with missing data.

Declaration of generative AI in scientific writing

Generative AI tools were used during the preparation of this manuscript solely for language editing and grammatical refinement.

Results

Study population and data characteristics

The study flowchart is presented in Figure 2. The primary-center cohort included 354 patients with PA and 281 patients with BP for model training and internal validation, and the external-validation cohort included 33 patients with PA and 30 patients with BP. Overall, 387 patients with PA and 311 patients with BP were included across the primary-center and external-validation cohorts. The exploratory longitudinal assessment included 39 CT examinations from 8 patients with PA and 43 CT examinations from 9 patients with BP.

Figure 2.

Flowchart of the study population and dataset allocation. Patients from the primary center were used for model training and internal validation, while patients from another tertiary hospital were used for external validation. A longitudinal subset with serial CT examinations was used for exploratory assessment of model performance across disease stages. Hospital A: The First Affiliated Hospital of Guangzhou Medical University; Hospital B: The Second Affiliated Hospital of Guangzhou Medical University. BP, bacterial pneumonia; CT, computed tomography; PA, pulmonary aspergillosis.

Baseline demographic and disease characteristics of the primary-center and external-validation cohorts are summarized in Table S2. No statistically significant between-center differences were observed in age, body weight among patients with available data, sex distribution, smoking status, PA/BP composition, PA subtype distribution, or the prevalence of clearly documented major immunocompromising conditions (all P>0.05). Clearly documented major immunocompromising conditions were present in 22 of 635 patients (3.5%) in the primary-center cohort and 4 of 63 patients (6.3%) in the external-validation cohort (P=0.282).

Both non-contrast and contrast-enhanced CT series were included. In the primary-center cohort, 569 of 635 selected CT series (89.6%) were non-contrast and 66 (10.4%) were contrast-enhanced. In the external-validation cohort, 60 of 63 selected CT series (95.2%) were non-contrast and 3 (4.8%) were contrast-enhanced.

CT slice thickness differed markedly between centers. In the primary-center cohort, 630 of 635 examinations (99.2%) had a slice thickness of 1–2 mm. The remaining five examinations had slice thicknesses of 0.5, 0.8, 2.5, 3.8, and 5 mm, respectively. In the external-validation cohort, 61 of 63 examinations (96.8%) had a slice thickness of 5 mm, while the remaining two examinations had a slice thickness of 1 mm.

Inspection of the available DICOM headers identified vendor-specific reconstruction-kernel labels of FC10 for Toshiba, 150f\3 for Siemens, and LUNG for GE. Because these labels are manufacturer-defined, direct equivalence across vendors could not be established. The Reconstruction Algorithm attribute was absent or empty in the available CT series, precluding a systematic comparison of reconstruction algorithms between centers.

Clinical and radiological characteristics

As shown in Table 1, patients with PA were significantly younger and had lower levels of inflammatory markers [white blood cell count (WBC), neutrophils, erythrocyte sedimentation rate (ESR), high-sensitivity C-reactive protein (hs-CRP), and procalcitonin] than patients with BP (all P<0.05). However, patients with PA had higher positivity rates for fungal biomarkers [bronchoalveolar lavage fluid (BALF) [1→3]-β-D-glucan, serum GM, and BALF GM]. Radiologically, patients with PA more frequently exhibited cavitary lesions (single or multiple), air crescent signs, and tree-in-bud patterns, whereas patients with BP predominantly showed multifocal infiltrates, consolidation, bronchial signs, and pleural effusion (all P<0.05).

Table 1. Clinical and radiological characteristics of patients with pulmonary aspergillosis and bacterial pneumonia.

Variables	PA cases (n=354)	BP cases (n=281)	Statistics	P value
Age (years)	53 (42.2, 62.8)	64 (53.0, 74.0)	−8.028	<0.001
Male	208 (58.8)	185 (65.8)	3.328	0.07
BMI (kg/m2)	20.9±3.4	21.6±3.7	−1.921	0.06
Chronic underlying disease	248 (70.1)	169 (60.1)	6.830	0.009
Length of stay (days)	10.0 (7.0, 14.7)	8.2 (6.0, 11.8)	−4.143	<0.001
Hospitalization cost (yuan)	28,607.1 (19,070.4, 44,646.0)	17,335.2 (11,898.3, 26,402.1)	−9.324	<0.001
Laboratory tests
WBC count (×109/L)	7.2 (5.7, 9.7)	8.4 (6.2, 11.5)	−3.168	0.002
Neutrophil count (×109/L)	4.4 (3.3, 6.8)	6.0 (3.9, 9.2)	−4.515	<0.001
Lymphocyte count (×109/L)	1.6 (1.2, 2.1)	1.3 (0.9, 1.8)	−4.676	<0.001
ESR (mm/h)	34.0 (15.0, 78.5)	60.0 (30.0, 90.0)	−4.332	<0.001
Hs-CRP (mg/L)	4.0 (0.9, 31.4)	38.5 (5.2, 103.0)	−4.846	<0.001
Elevated procalcitonin	91 (37.0)	134 (57.0)	19.367	<0.001
Elevated serum(1→3)-β-D-glucan	42 (13.6)	26 (11.3)	0.649	0.42
Elevated serum GM, n (%)	23 (8.2)	5 (2.9)	5.329	0.02
Elevated BALF (1→3)-β-D-glucan	11 (84.6)	5 (2.9)	102.718	<0.001
Elevated BALF GM	41 (31.1)	3 (9.7)	5.825	0.02
Elevated IgM antibodies against M. pneumoniae	58 (40.0)	63 (36.8)	0.331	0.57
Location of the pulmonary lesion
Left upper lung	179 (50.6)	149 (53.0)	0.380	0.54
Left lower lung	149 (42.1)	192 (68.3)	43.372	<0.001
Right upper lung	191 (54.0)	150 (53.4)	0.021	0.89
Right middle lung	113 (31.9)	120 (42.7)	7.842	0.005
Right lower lobe	146 (41.2)	199 (70.8)	55.225	<0.001
Radiological signs
Halo	32 (9.0)	21 (7.5)	0.502	0.48
Single cavity	103 (29.1)	9 (3.2)	72.992	<0.001
Multiple cavities	115 (32.5)	13 (4.6)	75.546	<0.001
Air crescent sign	105 (29.7)	0	99.860	<0.001
Tree-in-bud	18 (5.1)	4 (1.4)	6.279	0.01
Multiple punctate or patchy infiltrates	144 (40.7)	225 (80.1)	99.866	<0.001
Consolidation	72 (20.3)	133 (47.3)	52.207	<0.001
Bronchus sign	33 (9.3)	88 (31.3)	49.133	<0.001
Subpleural nodule or mass	20 (5.6)	9 (3.2)	2.152	0.14
Nodules or masses in other locations	100 (28.2)	32 (11.4)	27.045	<0.001
Pleural effusion	10 (2.8)	38 (13.5)	25.659	<0.001

Data are presented as median (IQR), mean ± standard deviation or n (%). BALF, bronchoalveolar lavage fluid; BMI, body mass index; BP, bacterial pneumonia; ESR, erythrocyte sedimentation rate; GM, galactomannan; Hs-CRP, hypersensitive C-reactive protein; IQR, interquartile range; PA, pulmonary aspergillosis; WBC, white blood cell count.

Model performance in validation sets

In internal validation (71 PA and 56 BP cases), the AST model correctly identified 58 PA cases, with significantly higher detection rates for cavitary lesions and air crescent signs among correctly identified cases (both P<0.05). As detailed in Tables 2,3 and Figure 3, the UST and AST models achieved AUCs of 0.78 (95% CI: 0.69–0.86) and 0.83 (95% CI: 0.76–0.90), respectively. The UST model had higher sensitivity than the AST model (91.5% versus 81.7%) but lower specificity (44.6% versus 73.2%).

Table 2. Performance of the unannotated and annotated supervised training models in internal and external validation sets.

Performance metric	The UST model		The AST model
	Internal validation	External validation	Internal validation	External validation
AUC (95% CI)	0.78 (0.69–0.86)	0.56 (0.41–0.70)	0.83 (0.76–0.90)	0.81 (0.71–0.92)
Accuracy, % (95% CI)	70.9 (62.4–78.1)	60.3 (48.0–71.5)	78.0 (70.0–84.3)	73.0 (61.0–82.4)
Sensitivity, % (95% CI)	91.5 (82.8–96.1)	75.8 (59.0–87.2)	81.7 (71.2–89.0)	60.6 (43.7–75.3)
Specificity, % (95% CI)	44.6 (32.4–57.6)	43.3 (27.4–60.8)	73.2 (60.4–83.0)	86.7 (70.3–94.7)
PPV, % (95% CI)	67.7 (57.8–76.2)	59.5 (44.5–73.0)	79.5 (68.8–87.1)	83.3 (64.1–93.3)
NPV, % (95% CI)	80.6 (63.7–90.8)	61.9 (40.9–79.2)	75.9 (63.1–85.4)	66.7 (51.0–79.4)
F1 score	0.778	0.667	0.806	0.702
Matthews correlation coefficient	0.418	0.202	0.551	0.486

PA was designated as the positive class and BP as the negative class for confusion-matrix-based metrics. Ninety-five percent confidence intervals for individual AUCs were calculated using the DeLong method. Confidence intervals for accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were calculated using the Wilson score method. AST, annotated supervised training; AUC, area under the receiver operating characteristic curve; BP, bacterial pneumonia; CI, confidence interval; NPV, negative predictive value; PA, pulmonary aspergillosis; PPV, positive predictive value; UST, unannotated supervised training.

Table 3. Pairwise comparison of AUCs between the unannotated and annotated supervised training models in internal and external validation sets.

Cohort	ΔAUC (AST − UST)	95% CI	P value
Internal validation	0.055	−0.019 to 0.129	0.14
External validation	0.258	0.085 to 0.430	0.003

PA was designated as the positive class and BP as the negative class. AUC differences were calculated as AST minus UST. AUCs were compared within each cohort using two-sided paired DeLong tests for correlated ROC curves. AST, annotated supervised training; AUC, area under the receiver operating characteristic curve; BP, bacterial pneumonia; CI, confidence interval; PA, pulmonary aspergillosis; ROC, receiver operating characteristic; UST, unannotated supervised training.

Figure 3.

Receiver operating characteristic curves of the two deep learning models. ROC curves are shown for the unannotated supervised training model in internal validation (A) and external validation (C), and for the annotated supervised training model in internal validation (B) and external validation (D). AUC, area under the receiver operating characteristic curve; ROC, receiver operating characteristic.

In internal validation, the UST model yielded 6 false-negative PA cases (all serum GM negative; 1 of 5 BALF GM positive) and 31 false-positive BP cases (all serum and BALF GM negative), whereas the AST model yielded 13 false-negative PA cases (all serum GM negative; 1 of 9 BALF GM positive) and 15 false-positive BP cases (all serum and BALF GM negative).

Grad-CAM visualizations were generated for eight representative cases using the AST model. Four cases were correctly classified, comprising two PA cases correctly predicted as PA and two BP cases correctly predicted as BP. Four cases were misclassified, comprising two PA cases incorrectly predicted as BP, representing false-negative PA results, and two BP cases incorrectly predicted as PA, representing false-positive PA results. Representative PA and BP cases misclassified by the UST and AST models are shown in Figure 4. For each case, the original axial CT image, expert-reviewed lesion annotation, and corresponding attention map are presented in Figure S1. These examples provide a qualitative illustration of image regions contributing to both correct and incorrect AST model outputs.

Figure 4.

Representative cases misclassified by the two deep learning models. Panels (A-D) show cases of pulmonary aspergillosis misclassified as bacterial pneumonia, and panels (E-H) show cases of bacterial pneumonia misclassified as pulmonary aspergillosis.

In external validation, compared with the AST model, the UST model showed higher sensitivity but lower specificity (75.8% versus 60.6% and 43.3% versus 86.7%, respectively). The UST and AST models achieved AUCs of 0.56 (95% CI: 0.41–0.70) and 0.81 (95% CI: 0.71–0.92), respectively; however, the sensitivity of the AST model decreased from 81.7% in internal validation to 60.6% in external validation. The AST model correctly identified 20 of 33 PA cases, while 13 PA cases were misclassified as BP (Tables 2,3 and Figure 3C,3D). Complete patient-level confusion matrices for the UST and AST models in the internal and external validation cohorts are provided in Table S3.

The F1 scores of the UST and AST models were 0.778 and 0.806 in internal validation and 0.667 and 0.702 in external validation, respectively. The corresponding MCC values were 0.418 and 0.551 internally and 0.202 and 0.486 externally.

In internal validation, the AST model had a numerically higher AUC than the UST model [0.830 versus 0.775; ΔAUC, 0.055; 95% CI: −0.019 to 0.129], but the difference was not statistically significant (P=0.14). In external validation, the AST model had a significantly higher AUC than the UST model (0.813 versus 0.556; ΔAUC, 0.258; 95% CI: 0.085 to 0.430; P=0.003).

Calibration and probability-based performance

The AST model had lower Brier scores than the UST model in both internal validation [0.180 (95% CI: 0.132–0.230) versus 0.198 (95% CI: 0.174–0.224)] and external validation [0.195 (95% CI: 0.133–0.265) versus 0.257 (95% CI: 0.228–0.291)]. Average precision was also higher for AST internally [0.839 (95% CI: 0.757–0.916) versus 0.804 (95% CI: 0.730–0.879)] and externally [0.827 (95% CI: 0.716–0.928) versus 0.575 (95% CI: 0.483–0.723)]. Calibration intercepts and slopes deviated from their ideal values for both models, with a particularly limited calibration slope for the UST model in external validation. Detailed probability-based performance measures are provided in Table S4, and the calibration plots are shown in Figure S2.

Decision curve analysis showed that the relative net benefit of the models varied across threshold probabilities in internal validation, although AST generally provided higher net benefit over much of the evaluated range. In external validation, AST provided greater net benefit than UST and the evaluate-all and evaluate-none strategies over threshold probabilities of approximately 0.19–0.80. These findings were exploratory because no clinical decision threshold had been prespecified and the external cohort was limited in size. The decision curves are shown in Figure S2, and the precision-recall curves are presented in Figure S3.

Exploratory performance across PA subtypes

The primary-center PA cohort included 115 patients with IPA, 214 with CPA, and 25 with ABPA, including 27, 40, and 4 patients, respectively, in the internal validation cohort. The external validation cohort included 16 patients with IPA, 16 with CPA, and one with ABPA.

In internal validation, the AST model achieved subtype-versus-BP AUCs of 0.80 (95% CI: 0.70–0.90) for IPA, 0.86 (95% CI: 0.79–0.93) for CPA, and 0.78 (95% CI: 0.58–0.98) for ABPA. The corresponding sensitivities were 77.8%, 85.0%, and 75.0%, respectively. In external validation, the AST AUCs were 0.76 (95% CI: 0.61–0.92) for IPA and 0.85 (95% CI: 0.74–0.96) for CPA, with sensitivities of 56.3% and 62.5%, respectively. The single external ABPA case was correctly classified by both models, but stable subtype-specific performance could not be estimated. Full UST and AST results are presented in Table S5.

Disease stage identification

Both models showed comparable AUCs (UST, 0.78; AST, 0.79) in the stage-specific analysis (Figure 5A,5B). The AST model had higher specificity (72.1% versus 37.2%) and accuracy (73.2% versus 62.2%) than the UST model. In the longitudinal analysis, all serial CT examinations were correctly classified by both models in 2 of 8 patients with PA and 3 of 9 patients with BP (Figures S4,S5).

Figure 5.

Receiver operating characteristic curves for model performance across disease stages. ROC curves are shown for the unannotated supervised training model (A) and annotated supervised training model (B) in the exploratory analysis of serial CT examinations. AUC, area under the receiver operating characteristic curve; CT, computed tomography; ROC, receiver operating characteristic.

Clinician performance assessment

Among 158 valid responses from clinicians at 61 hospitals, the mean diagnostic score was 68.2±11.5 under standardized image-only conditions (Table 4 and Figure 6). The corresponding sensitivity, specificity, and accuracy for PA identification were 73.2%, 62.0%, and 68.2%, respectively. No statistically significant differences in questionnaire scores were observed according to hospital grade, department, or clinical experience (all P>0.05). These results reflect performance in the standardized CT image assessment and should not be interpreted as estimates of clinicians’ diagnostic performance in routine clinical practice. The 11 PA cases included in the questionnaire were divided into Groups A and B according to diagnostic accuracy. High-accuracy PA cases (Group A, 4 cases) consistently showed cavitary lesions and air crescent signs. Low-accuracy cases (Group B, 7 cases) predominantly exhibited non-specific infiltrates (85.7% versus 50% in Group A).

Table 4. Results of the standardized clinician questionnaire assessment.

Variables	Classification	N (%)	Score	Statistics	P value
Hospital grade	Tertiary first-class hospital	142 (89.9)	67.9±11.6	−1.015	0.31
	Other hospitals	16 (10.1)	70.9±10.4
Department	Respiratory Department	125 (79.1)	69.0±11.1	1.785	0.08
	Others	33 (20.9)	65.0±12.7
Years of clinical experience	≤10	89 (56.3)	66.7±11.7	−1.846	0.07
	>10	69 (43.7)	70.1±11.1
Number of pathologically confirmed PA cases diagnosed annually	≤50	108 (68.4)	67.9±11.7	−0.470	0.64
	>50	50 (31.6)	68.8±11.3
Independent development of fungal antigen detection	Yes	114 (72.2)	67.8±12.1	−0.703	0.48
	No	44 (27.8)	69.2±9.7
Routine use of histopathological special staining	Yes	89 (56.3)	69.4±11.5	1.585	0.12
	No	69 (43.7)	66.5±11.4
What do you think is the most difficult type of PA to diagnose and treat?	Invasive pulmonary aspergillosis	27 (17.1)	NA	NA	NA
	Invasive tracheobronchial aspergillosis	31 (19.6)
	Chronic necrotizing pulmonary aspergillosis	38 (24.1)
	Aspergilloma	3 (1.9)
	Allergic bronchopulmonary aspergillosis	59 (37.3)
What do you think about the prospect of imaging AI in the differential diagnosis of PA?	Undetermined	19 (12.0)	NA	NA	NA
	Bad	6 (3.8)
	So-so	49 (31.0)
	Promising	84 (53.2)

AI, artificial intelligence; NA, not available; PA, pulmonary aspergillosis.

Figure 6.

Results of the clinician questionnaire assessment. Panel (A) shows the frequency distribution of diagnostic scores. Panels (B-D) show the distribution of participating clinicians by hospital grade, department, and years of clinical experience, respectively.

Discussion

This study represents one of the first comprehensive evaluations of CT-based DL models for distinguishing PA from BP using a large, well-characterized dataset, with an exploratory comparison to clinician performance in a standardized image-only assessment. The AST model achieved AUCs of 0.83 and 0.81 in the internal and external validation cohorts, respectively, and showed higher specificity than the UST model. However, its sensitivity decreased from 81.7% in internal validation to 60.6% in external validation, indicating reduced detection of PA cases across centers. These findings support the potential adjunctive value of the model but also highlight the need for cautious interpretation of its external performance. In the external cohort, 13 of 33 PA cases were misclassified as BP, and the findings do not establish readiness for independent diagnostic use or routine clinical implementation. However, this exploratory comparison was not intended to establish superiority over clinicians in routine clinical practice, where diagnostic decisions incorporate clinical history, laboratory findings, microbiological evidence, prior imaging, and multidisciplinary interpretation. Below, we address key aspects of our study in the context of prior research and clinical relevance.

Rationale for including PA subtypes as a collective entity

The primary objective of this study was to distinguish PA from BP at initial presentation, when the specific form of aspergillosis may not yet have been established. We therefore developed a unified PA-versus-BP classifier rather than separate subtype-specific models. This approach increased the number of PA cases available for model development and reflected the broad initial diagnostic question addressed in the study.

Nevertheless, IPA, CPA, and ABPA have distinct clinical contexts, pathophysiological mechanisms, and imaging manifestations. For ABPA specifically, the revised 2024 International Society for Human and Animal Mycology (ISHAM) ABPA working group guideline provides updated diagnostic and radiological classification criteria, further underscoring that ABPA is a clinically distinct entity within the broader PA spectrum (22). The exploratory subtype analysis confirmed that pooled overall performance did not translate into identical performance across PA subtypes. In external validation, the AST model showed stronger discrimination for CPA than for IPA, with subtype-versus-BP AUCs of 0.85 and 0.76, respectively. However, sensitivities remained modest for both CPA and IPA, and the external IPA sensitivity was only 56.3%. This finding is particularly important because missed IPA may have serious clinical consequences.

The ABPA subgroup was too small for reliable performance estimation, particularly in the external cohort, which contained only one case. Accordingly, the subtype analyses should be regarded as exploratory and hypothesis-generating. The unified model may serve as an initial adjunctive PA-versus-BP classification approach, but larger subtype-enriched cohorts and separately developed subtype-specific models are required before stable performance for IPA, CPA, or ABPA can be established. Recent multicentre CT-based work has also demonstrated the feasibility of diagnosing CPA and classifying CPA subtypes with DL, supporting further development of subtype-specific models (15).

Performance analysis and innovations

DL has been widely applied to disease classification and prognostic evaluation (23). In internal validation, the AST model showed a more balanced sensitivity and specificity profile than the UST model. However, this pattern was not maintained in external validation, where AST sensitivity decreased to 60.6%. Previous CT-based AI studies of pulmonary infections have used heterogeneous model-development and input strategies, including weakly supervised classification and lesion localization (24), manually delineated or unannotated image inputs (17,18), and local-global modeling (25). Recent multicentre work has also evaluated automated lesion segmentation (16). In the present study, AST and UST shared the same classification architecture but differed in foreground definition, permitting a more direct assessment of the contribution of physician-guided lesion localization. The AST model showed higher specificity than the UST model in external validation, suggesting that lesion annotation may help improve the recognition of relevant imaging features. Nevertheless, the reduced external sensitivity indicates that lesion annotation did not eliminate the risk of missing PA cases. Conversely, the UST model’s lower specificity (43.3%) suggests that unsupervised approaches may struggle with BP mimics, such as non-specific infiltrates. However, it is important to acknowledge that the UST model offers the advantages of convenience and time efficiency, making it particularly suitable for certain settings with limited human resources.

Formal paired comparison showed that the AUC difference between AST and UST was not statistically significant in internal validation, despite the numerically higher AST AUC. In contrast, AST demonstrated significantly better discrimination than UST in external validation. This external AUC improvement was accompanied by substantially higher specificity but lower sensitivity, indicating that lesion annotation altered the balance between false-positive and false-negative classifications rather than improving every aspect of diagnostic performance. Therefore, AST should not be interpreted as uniformly superior to UST, particularly when the clinical priority is to avoid missed PA cases.

Evaluation beyond discrimination provided a more nuanced assessment of model performance. AST showed lower Brier scores and higher average precision than UST in both validation cohorts, with the largest differences observed externally. However, calibration remained imperfect. In particular, the calibration slopes deviated from the ideal value of 1 for both models, indicating that the predicted probabilities should not be interpreted as fully calibrated estimates of the probability of PA.

The exploratory decision curve analysis suggested that AST could provide greater net benefit than UST across a range of threshold probabilities, particularly in external validation. In this analysis, a positive model result was conceptualized as prompting further fungal evaluation rather than directly initiating treatment. Because no clinical threshold was prespecified and the analysis was based on a relatively small external cohort, the decision curves should not be considered evidence of established clinical effectiveness. Prospective impact studies are required to determine whether model-assisted evaluation improves clinician decisions or patient outcomes.

The qualitative Grad-CAM examples enabled direct visual comparison between AST model activation and expert-reviewed pulmonary lesions. The balanced inclusion of correctly classified and misclassified cases improved the transparency of the lesion-annotated model and illustrated that the presence of model attention within or around a pulmonary lesion does not, by itself, ensure correct diagnostic classification. Grad-CAM attention maps do not demonstrate causal model reasoning, do not confirm that the model used the same diagnostic process as clinicians, and should not be regarded as evidence of clinical validity.

In contrast to previous studies that compared AI systems with radiologists or clinicians (14,17,26), the present study focused on the image interpretation performance of respiratory physicians because of their central role in the diagnosis and management of pulmonary infections. Under standardized image-only conditions, clinicians achieved an average accuracy of 68.2%, with higher scores for cases exhibiting classic PA features, such as cavitary lesions and air crescent signs. Questionnaire scores did not differ significantly according to hospital grade, department, or clinical experience. However, these findings should be interpreted cautiously because the participants assessed only selected static CT images and did not have access to clinical history, symptoms, laboratory findings, microbiological results, prior imaging, treatment response, or radiological reports. These sources of information substantially influence diagnostic decision-making in routine practice. Therefore, the clinician assessment should be regarded as a standardized image-only benchmark rather than a direct comparison between model performance and real-world clinician performance.

In potential clinical use, CT-based model output should be regarded only as adjunctive imaging information rather than as an independent diagnostic conclusion. A high model-predicted probability of PA may prompt consideration of further fungal evaluation, such as bronchoscopy, bronchoalveolar lavage GM testing, fungal culture, or mNGS. However, the model should not independently determine whether these investigations are performed or whether antifungal treatment is initiated. Given the external sensitivity of 60.6%, careful interpretation is required. However, given the reduced sensitivity observed in external validation, a model prediction of BP should not be used to rule out PA or to discontinue further mycological investigation when clinical suspicion remains high. Model outputs should be interpreted together with host factors, clinical manifestations, laboratory findings, microbiological evidence, and comprehensive radiological assessment. Prospective clinical-impact studies are required before the model can be incorporated into routine diagnostic workflows.

Despite an external validation AUC of 0.81 and a specificity of 86.7%, the AST model sensitivity decreased from 81.7% in internal validation to 60.6% in external validation. In the external cohort, 13 of 33 PA cases were misclassified as BP, corresponding to a false-negative proportion of 39.4%. Importantly, the external validation cohort included only 33 PA cases, and the sensitivity estimate had a relatively wide 95% CI of 43.7–75.3%. Therefore, the limited external sample size was likely an important contributor to the imprecision and apparent magnitude of the sensitivity difference.

In addition, the primary-center and external-validation cohorts differed markedly in CT acquisition characteristics, whereas PA subtype distribution showed numerical but not statistically significant variation. CT slice thickness differed markedly between centers: 630 of 635 primary-center examinations (99.2%) had a slice thickness of 1–2 mm, whereas 61 of 63 external-validation examinations (96.8%) had a slice thickness of 5 mm. Thicker CT sections may reduce the visibility of subtle radiological findings, such as small cavities, tree-in-bud opacities, and small nodules, and this difference may have contributed to inter-center variation in model transportability. Vendor-specific reconstruction kernels may represent an additional source of inter-center image heterogeneity because kernel selection can affect image sharpness, noise texture, and edge appearance. However, the manufacturer-specific kernel nomenclature and absence of reconstruction-algorithm metadata prevented direct harmonization or formal assessment of their independent effects. Although all CT volumes were resampled to a uniform voxel spacing, resampling may not fully recover information lost from thicker source sections or eliminate texture differences introduced during image reconstruction. These acquisition and reconstruction differences may therefore have contributed to domain shift and reduced model transportability. Measured demographic and disease characteristics, including age, body weight, sex distribution, smoking status, PA/BP composition, PA subtype distribution, and the prevalence of clearly documented major immunocompromising conditions, did not differ significantly between centers. Nevertheless, the external cohort included a numerically higher proportion of IPA than the primary-center cohort (48.5% versus 32.5%), and residual patient-spectrum differences cannot be excluded. Standardized disease-severity measures and patient-level referral-source information were not systematically captured in the retrospective dataset; therefore, their potential contributions to inter-center performance differences could not be formally evaluated.

The proportion of PA cases was broadly comparable between the primary-center and external-validation cohorts (55.7% and 52.4%, respectively), suggesting that prevalence differences alone were unlikely to explain the reduced sensitivity. Overall, these findings indicate that a model prediction of BP should not be used to exclude PA when clinical suspicion remains high. Model outputs should be interpreted together with host factors, clinical manifestations, mycological testing, and comprehensive radiological assessment.

Disease progression and model adaptability

Longitudinal analysis revealed that both models faced challenges in tracking PA evolution, often misclassifying treated cases as BP due to resolving lesions. This finding reflects clinical settings in which antifungal therapy may alter radiological appearances. In contrast, BP cases were more consistently identified, likely owing to their less dynamic imaging trajectories. These findings emphasize the need for dynamic AI systems capable of incorporating temporal data—a direction for future research.

Strengths and limitations

Our study’s strengths include rigorous case adjudication (using EORTC/MSGERC criteria), a multicenter design, and the inclusion of clinician assessments. The sample size of PA cases (n=387) exceeds that of previous CT-based pulmonary infection AI studies (13,27). However, limitations must be acknowledged. Because both cohorts were derived from tertiary hospitals in Guangzhou and no formal fairness analyses across sociodemographic subgroups were performed, generalizability to other settings and populations remains uncertain. First, excluding comorbid conditions (e.g., co-infections) may limit generalizability. Future models should integrate such complexities. Second, lesion annotations were generated through initial delineation by trained respiratory physicians followed by joint review and refinement by a three-member expert panel. Although this workflow provided quality control for the final lesion masks, independent duplicate annotation and formal interobserver agreement assessment were not performed. Annotation time was not prospectively recorded, preventing formal quantification of the workload associated with manual lesion delineation. Future studies should evaluate annotation reproducibility and workload using independent multi-reader designs. Third, the models currently rely solely on CT images. Recent work integrating meta-analytic evidence with clinical data developed and externally validated a machine-learning model for early IPA risk stratification in non-neutropenic patients (28). These findings support future multimodal models combining CT features with host factors, laboratory findings, and microbiological data. Our study revealed a notably higher positive rate of GM testing in BALF than in serum, aligning with prior studies (29-31). Incorporating these biomarkers may improve model performance. Fourth, the external validation cohort included only 33 PA cases, resulting in a relatively wide confidence interval for sensitivity. The limited sample size may have contributed to the imprecision and apparent magnitude of the difference between internal and external sensitivity. In addition, differences in CT slice thickness, vendor-specific reconstruction kernels, and incompletely documented reconstruction algorithms may have affected model transportability. Because reconstruction-algorithm metadata were unavailable and kernel nomenclature was manufacturer-specific, their independent effects could not be formally evaluated. Numerical variation in PA subtype distribution between centers may also have affected model transportability. Larger prospective multicenter validation cohorts are required to obtain more precise performance estimates and further evaluate model generalizability. Accordingly, the current model is not suitable as a stand-alone screening, diagnostic, or rule-out tool, and its effect on clinician decision-making and patient outcomes remains unknown. Fifth, the clinician assessment was based on a limited number of selected static CT images without accompanying clinical history, symptoms, laboratory findings, microbiological results, prior imaging, treatment response, or radiological reports. Consequently, the questionnaire did not fully reproduce routine clinical diagnostic workflows, and the comparison between model and clinician performance should be considered exploratory. Future studies should evaluate clinician performance with and without AI assistance using complete clinical information in prospective, reader-based designs.

Although calibration, precision-recall, and decision curve analyses were added, these evaluations were based on retrospective validation cohorts, and the external cohort was relatively small. No clinically established threshold for triggering additional fungal evaluation was available; therefore, the decision curve analysis used an exploratory range of threshold probabilities. The predicted probabilities were not recalibrated for clinical use, and the clinical consequences of model-assisted decisions were not evaluated prospectively.

The exploratory subtype analyses were limited by small and imbalanced subgroup sample sizes. In particular, only four patients with ABPA were included in internal validation and one in external validation. The models were trained for overall PA-versus-BP classification rather than for individual PA subtypes, and the same BP comparator group was reused across subtype analyses. No formal comparisons among subtypes were performed. Consequently, the subtype-specific estimates, especially those for ABPA, should be interpreted as descriptive and hypothesis-generating rather than as evidence of stable subtype-specific performance.

Explainability analysis was limited to qualitative Grad-CAM visualization of eight deliberately selected AST cases. Although the displayed set was balanced across true class and classification outcome, it was not a systematic cohort-wide analysis. We did not perform quantitative assessment of attention-map localization or attention–lesion overlap, and comparable explainability analysis was not conducted for the UST model. The small and selectively sampled set of visualized cases limits the generalizability of the interpretability findings.

Conclusions

In summary, the AST framework showed potential for adjunctive CT-based differentiation of PA from BP. However, its external sensitivity of 60.6% indicates that the model should not be used as a stand-alone diagnostic, screening, or rule-out tool. In particular, a model prediction of BP should not be used to exclude PA when clinical suspicion remains high. Future studies should focus on prospective multicenter validation, evaluation within real-world clinical workflows, subtype-specific modeling, and integration of clinical, microbiological, and imaging data before clinical implementation is considered. Formal paired comparison showed that AST demonstrated significantly better discrimination than UST in external validation, whereas the internal AUC difference was not statistically significant. The additional calibration and decision-analytic findings support further evaluation of AST as an adjunctive imaging tool but do not establish readiness for clinical implementation.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Ethics Committee of The First Affiliated Hospital of Guangzhou Medical University (No. ES-2023-017-01). The Second Affiliated Hospital of Guangzhou Medical University was informed of the study and agreed to its conduct. The requirement for informed consent was waived owing to the retrospective nature of the study.

Data Sharing Statement

Available at https://jtd.amegroups.com/article/view/10.21037/jtd-2026-1253/dss

jtd-18-9-1030-dss.pdf ^{(799.5KB, pdf)}

DOI: 10.21037/jtd-2026-1253