Prediction of severe pediatric community-acquired pneumonia using multimodal fusion of chest radiographs and clinical data

Abstract

Background

Community-acquired pneumonia (CAP) is a leading cause of morbidity and mortality in children, particularly in low-income countries. Severe CAP is associated with high mortality risk, making early prediction and intervention critical. Despite advancements in artificial intelligence applications for disease prediction, most models have focused on adults, with limited attention to pediatric populations and the integration of chest radiographs and laboratory tests.

Methods

To develop a predictive model for severe CAP in children by integrating multimodal data, including chest radiographs, laboratory tests, and demographic information. A retrospective cohort of 3,964 pediatric CAP patients was constructed using data from Wuhan Children’s Hospital. A predictive model for severe CAP was developed by integrating chest radiographs, laboratory test results, and demographic data. Key hyperparameters, including model architecture, initialization parameters, loss function weighting, down-sampling strategies, and multimodal fusion were systematically optimized to enhance performance. These approaches were compared to determine the optimal method for predicting severe CAP in children.

Results

The best-performing unimodal model based solely on chest radiographs achieved a PR-AUC of 16.33 ± 1.98% and a ROC-AUC of 78.28 ± 1.48%. Through multimodal fusion, the optimal fusion model significantly improved performance, achieving a PR-AUC of 46.68 ± 4.50% and a ROC-AUC of 92.91 ± 0.64%. SHapley Additive exPlanations values were used to analyze the contributions of individual modalities to the model’s predictions, revealing key predictive factors. Furthermore, the regions activated by the model showed substantial overlap with radiologist-annotated lesion areas on chest radiographs.

Conclusion

This study highlights the potential of multimodal fusion to enhance the prediction of severe CAP in children by integrating chest radiograph and clinical data. The proposed model provides a foundation for future AI-based tools to support early diagnosis and targeted intervention, potentially reducing mortality associated with pediatric CAP.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12887-026-06953-w.

Introduction

Community-acquired pneumonia (CAP) is a prevalent and complex respiratory condition in children [1]. According to data from the World Health Organization, approximately 150 million children under the age of five are affected by pneumonia annually, with 20 million requiring hospitalization and 2 million resulting in death [2]. Pneumonia-related mortality in children under five accounts for over 14% of the total child mortality in this age group [3]. Furthermore, the majority of both confirmed cases and deaths due to CAP in children occur in low-income countries [4]. The risk of mortality from CAP is closely associated with the severity of the disease, with most fatalities occurring in cases of severe CAP [5]. Consequently, early prediction and intervention for severe CAP are essential strategies for reducing the mortality rate associated with pediatric CAP.

In recent years, artificial intelligence (AI), particularly machine learning (ML) and deep learning (DL), has been increasingly applied to diagnosis, outcome prediction, intensive care unit admission prediction, and treatment planning for CAP due to its robust capacity to model complex problems [6]. Recent studies have further highlighted the growing role of multimodal fusion [7, 8], clinically interpretable machine learning [9, 10], and foundation models [11, 12] in pediatric and respiratory disease research. For example, Chang et al. developed ML models to predict acute respiratory infection pathogens in pediatric CAP at admission [13]. Kanwal et al. created a non-invasive, rapid, and cost-effective point-of-care device to assist healthcare workers in diagnosing and screening pediatric CAP [14]. Yang et al. utilized ML algorithms, including AdaBoost, to construct an identification model for severe CAP patients [15], while Ma et al. developed predictive models for acute kidney injury in hospitalized CAP patients using ML techniques [16]. However, most of these models were designed for adult populations, with a limited focus on pediatric populations. Moreover, while some studies included children aged 0–17, these models predominantly relied on laboratory test data, often neglecting other important data modalities.

In addition to laboratory test data, chest radiograph is an essential diagnostic tool for pediatric CAP. Chest radiographs help identify signs of infection, such as lung consolidation and alveolar opacities, and assess the severity and distribution of pneumonia. Leveraging chest radiograph data, Hu et al. developed a context-fusion convolutional neural network model to diagnose the etiology of pediatric CAP noninvasively, achieving notable performance [17]. Similarly, Chen et al. proposed a computer-aided diagnostic framework for chest radiographs of pediatric CAP, achieving an accuracy of 92.47% [18]. However, most AI models developed with chest radiographs have primarily focused on CAP diagnosis rather than the prediction of severe CAP. Additionally, while frontal chest radiographs have been extensively studied [19–21], the potential predictive value of lateral chest radiographs for severe CAP remains unclear. In routine radiologic interpretation, the lateral view is generally used as a complementary projection to the frontal view, particularly for lesion localization and for evaluating regions that may be partially obscured on frontal images.

Although multimodal fusion has shown great promise across various domains [22], its application in pediatric CAP remains limited, particularly for predicting disease severity. Recent studies have begun to explore multimodal approaches for pediatric pneumonia diagnosis. For example, Wang et al. developed a multimodal auxiliary diagnostic framework that integrates frontal chest radiographs, laboratory tests, and clinical text to improve the diagnostic accuracy of pediatric CAP [23]. however, their work focuses on initial diagnosis rather than risk stratification for severe disease. Similarly, Shao et al. proposed a large-scale multimodal integration pipeline for pulmonary infection diagnosis, pathogen identification, and prognosis prediction [24], yet their framework is not specifically targeted toward pediatric CAP and does not address the early identification of severe pediatric cases. To date, no studies have systematically evaluated whether combining chest radiographs from multiple views with laboratory and demographic data can enhance early prediction of severe pediatric CAP. This gap underscores the need for dedicated multimodal fusion models tailored to the pediatric population and focused on severity prediction, where early risk identification is critical for timely intervention and improved outcomes.

In this study, we address this gap by developing a multimodal prediction model for severe pediatric CAP using a retrospectively collected cohort of 3,964 patients. We integrate chest radiographs from multiple views with laboratory tests and demographic information to build a comprehensive predictive framework. The objective of this work is to evaluate whether multimodal fusion can improve early identification of severe CAP in children and thereby support timely clinical intervention.

Methods

In this study, we constructed a retrospective cohort of 3,964 pediatric CAP patients using the electronic medical record system of Wuhan Children’s Hospital. Leveraging this cohort, we developed a predictive model for severe CAP in children by integrating chest radiograph data (frontal and lateral views), laboratory test results, and demographic information (Fig. 1). The effects of various model architectures and pre-trained weights for chest radiograph analysis were compared. To address the imbalance between severe and non-severe CAP cases, we employed loss function weighting and down-sampling techniques. Additionally, the performance of concatenation-based fusion versus attention mechanism-based fusion was evaluated for combining chest radiograph data with clinical features. Following hyperparameter optimization, the optimal fusion model was identified. We further analyzed performance differences in gender and age subgroups, utilized SHapley Additive exPlanations values to elucidate the contributions of different modalities within the prediction model, and identified key predictive factors. Finally, Grad-CAM was employed to visualize regions of the chest radiographs that were most influential in the model’s predictions.

Fig. 1.

The process of constructing and evaluating the severe community-acquired pneumonia prediction model used in this study. CAP: Community-acquired pneumonia, SHAP: SHapley Additive exPlanations, Grad-CAM: Gradient-weighted Class Activation Mapping

Subject characteristics and image datasets

A retrospective cohort study was conducted using data extracted from the electronic medical record system of Wuhan Children’s Hospital. The study was approved by the Medical Ethics Committee of Wuhan Children’s Hospital (approval number: 2024R115-E01). The cohort consisted of children aged 0 to 17 years who were hospitalized for CAP [25] between January 1, 2019, and December 31, 2023. Inclusion criteria were hospitalization with a primary diagnosis of CAP and availability of complete clinical data. Exclusion criteria included congenital immune dysfunction, congenital heart disease, poor respiratory development, pulmonary tuberculosis, history of foreign body aspiration, hospitalization within one month prior to admission, absence of chest radiographic examination within 72 h before or after admission, poor-quality chest radiographs, and admission with a diagnosis of severe CAP (Fig. 2). Finally, 3,964 subjects were included in the cohort. The age distribution of the cohort was skewed toward younger children, which is reflected by the relatively low median age and narrow IQR (Table 1). This pattern is consistent with pediatric CAP hospitalization epidemiology, where infants and toddlers constitute a substantial proportion of admissions.

Fig. 2.

Inclusion and exclusion diagram. The slight difference between the numbers of frontal and lateral chest radiographs reflects real-world pediatric imaging practice, in which younger children are more likely to undergo only a frontal examination and lateral views are obtained selectively. CAP: Community-acquired pneumonia

Table 1.

The demographic characteristics and part clinical symptoms of the subjects. The complete clinical features are shown in Table S1

Total	Non-severe CAP (%)	Severe CAP (%)	Total (%)	Test stat.	P value
	3,776	188	3,964
Sex				Chisq. (1 df) = 0.29	0.592
Female	1,419 (37.6)	67 (35.6)	1,486 (37.5)
Male	2,357 (62.4)	121 (64.4)	2,478 (62.5)
Age (years)				Ranksum test	< 0.001
Median (IQR)	1.59 (0.5,3.56)	0.885 (0.26,2.13)	1.44 (0.46,3.46)
Parity				Chisq. (1 df) = 17.39	< 0.001
1	2,189 (58)	80 (42.6)	2,269 (57.2)
≥2	1,587 (42)	108 (57.4)	1,695 (42.8)
Birth weight (g)				Chisq. (2 df) = 10.79	0.005
Low birth weight	220 (5.8)	22 (11.7)	242 (6.1)
Normal birth weight	3,343 (88.5)	156 (83)	34,99 (88.3)
Large birth weight	213 (5.6)	10 (5.3)	223 (5.6)
Preterm birth				Chisq. (2 df) = 30.72	< 0.001
Preterm birth	215 (5.7)	7 (3.7)	222 (5.6)
Term birth	3,479 (92.1)	165 (87.8)	3,644 (91.9)
Post-term birth	82 (2.2)	16 (8.5)	98 (2.5)
Cough				Chisq. (1 df) = 6.79	0.009
No	169 (4.5)	1 (0.5)	170 (4.3)
Yes	3,607 (95.5)	187 (99.5)	3,794 (95.7)
Temperature (°C)				Ranksum test	0.051
Median (IQR)	37.1 (36.7,38)	37.1 (36.7,38.5)	37.1 (36.7,38)
Respiration (breaths/min)				Ranksum test	< 0.001
Median (IQR)	30 (26,35)	40 (32,48)	30 (26,35)
Heart rate (bpm)				Ranksum test	< 0.001
Median (IQR)	115 (106,125)	130 (120,140)	116 (108,125)
BMI (kg/m²)				Ranksum test	0.766
Median (IQR)	15.9 (14.8,17.5)	16.2 (14.7,17.8)	15.9 (14.8,17.5)
Sputum				Chisq. (1 df) = 0.2	0.657
No	2,131 (56.4)	103 (54.8)	2,234 (56.4)
Yes	1,645 (43.6)	85 (45.2)	1,730 (43.6)

CAP Community-acquired pneumonia, stat. statistic, IQR Interquartile Range, 1/2 df 1/2 degree of freedom

For the 3,964 subjects, chest radiographs taken within 72 h before and after admission were extracted from the PACS. These radiographs were categorized into frontal and lateral projections for analysis. Because the original view labels had been removed during anonymization, we developed a view classifier based on the ConvNeXt architecture to distinguish frontal-view radiographs from lateral-view radiographs (Figure S1). In routine pediatric practice, frontal radiographs may correspond to either anteroposterior or posteroanterior acquisitions, depending on patient age, cooperation, clinical condition, and examination setting. Therefore, in this study, all frontal radiographs were grouped into a frontal dataset (subjects: 3,937; radiographs: 4,447).

For lateral radiographs, right lateral images were horizontally flipped and merged with left lateral images to create a unified lateral dataset (subjects: 3,822; radiographs: 4,033). This preprocessing step was performed to reduce orientation heterogeneity for model development and does not imply that both lateral projections were routinely acquired in the same patient. In clinical practice, lateral radiographs are often obtained selectively rather than universally, particularly in younger children, in accordance with the As Low As Reasonably Achievable (ALARA) principle. All chest radiographs were resized to 512 × 512 and stored in JPG format.

In addition to chest radiographs, the first clinical record of each subject within 48 h before and after admission was collected. This record included basic demographic information, clinical symptoms, signs, and results from biological examinations (such as routine blood tests, blood biochemistry, liver and kidney function tests, and stool and urine analysis), totaling 101 clinical features (Table 1). Like Shi et al. [26], based on clinical expertise, 80 laboratory features in the 101 clinical features were further categorized into discrete normal/abnormal variables. Importantly, all abnormality coding were generated using age-specific pediatric reference ranges embedded within our institutional research data platform, which automatically applies age-appropriate thresholds when deriving categorical values. Age was additionally included as a continuous covariate in all predictive models. As a result, a total of 181 clinical features were included in this study (Table S1). Some subjects did not undergo all the required examinations, leading to missing values for certain clinical features (Table S2). For missing values in continuous variables, median imputation was applied, while mode imputation was used for missing values in discrete variables. We assumed that the missingness mechanism followed a Missing At Random (MAR) pattern [27], which is commonly accepted for clinical observational datasets. For categorical variables, we did not create an additional “missing” category, because these variables were intended to reflect underlying clinical characteristics and a separate missing category could introduce an artificial level without direct clinical interpretation.

Among the 3,964 enrolled subjects, 188 progressed to severe CAP between 72 h after admission and hospital discharge. The diagnosis of severe CAP was determined according to the Guidelines for the Diagnosis and Treatment of CAP in Children (2019 Edition) [25]. Severe CAP in children was defined when any of the following criteria were met: poor general condition; refusal to feed or clinical signs of dehydration; altered level of consciousness; markedly increased respiratory rate (≥ 70 breaths/min in infants and ≥ 50 breaths/min in older children); low oxygen saturation (≤ 0.92 at sea level or < 0.90 at high altitude); radiographic involvement of ≥ two-thirds of one or more pulmonary lobes; presence of pleural effusion; or occurrence of extrapulmonary complications [25]. To train and validate the prediction model, chest radiographs were divided into training and test subsets in an 8:2 split at the patient level, ensuring that all images from the same patient were assigned to a single subset to prevent cross-contamination. The training set included 3,171 patients (147 severe CAP, 3,024 non-severe CAP), and the test set included 793 patients (41 severe CAP, 752 non-severe CAP).

Prediction model architectures and pre-trained weights

In this study, model architectures with proven success in natural image recognition were utilized as feature extractors for chest radiographs, including ResNet50 [28], DenseNet121 [29], ConvNeXt [30], and Vision Transformer (ViT) [31]. To enhance prediction performance, the models were initialized with pre-trained weights. Two types of pre-trained weights were employed: supervised and self-supervised. The supervised weights were derived from training on the ImageNet-1 K dataset [32] for image recognition tasks and were available for all four architectures: ResNet50, DenseNet121, ConvNeXt, and ViT. The self-supervised weights were obtained from Masked Autoencoder (MAE) pre-training on a large-scale chest radiograph dataset [33–35], as reported by Xiao et al. [36]. In that study, MAE-based self-supervised pre-training was developed and released only for DenseNet121 and ViT; corresponding self-supervised weights were not available for ResNet50 or ConvNeXt. Therefore, self-supervised initialization was evaluated only for DenseNet121 and ViT in the present study.

Loss function weighting and down-sampling

In our retrospective cohort, the proportion of subjects who developed severe CAP was 4.74% (188/3,964). This highlights a significant class imbalance between severe and non-severe CAP cases. To enable the prediction model to learn both categories effectively, we implemented two strategies: loss function weighting and down-sampling.

Since the prediction of severe CAP is a binary classification task, we used weighted cross-entropy as the loss function to penalize false negatives more heavily. The weight for each class was calculated using the following equation:

where  N is the total sample size of the dataset,  C is the number of classes, is the sample size of class  m, and ​ is the weight for class  m.

Substituting the calculated weights into the cross-entropy formula results in:

where , and are the weights for the positive and negative classes,  y is the true label, is the predicted output, and  L is the weighted cross-entropy loss.

Another strategy employed was down-sampling of the majority class. Although down-sampling can mitigate class imbalance, it also reduces the effective training sample size. In this study, we evaluated three down-sampling ratios for the non-severe CAP class relative to the severe CAP class: 10:1, 5:1, and 1:1. Here, a 1:1 ratio indicates that the number of non-severe CAP samples was reduced to match the number of severe CAP samples. For each ratio, a separate training subset was constructed and a separate model was trained under the corresponding experimental setting.

Multimodal fusion

To further enhance the performance of the prediction model, we incorporated clinical features alongside chest radiographs. Specifically, 181 clinical variables were first standardized using Z-score normalization and then projected into a 256-dimensional representation through a fully connected layer. The radiographic features and clinical representation were then integrated using two alternative fusion strategies: concatenation-based fusion and attention-based fusion, which were evaluated separately rather than used jointly. These alternative fusion designs were compared to determine the more effective multimodal integration strategy for severe CAP prediction.

Concatenation-based fusion

Concatenation is a straightforward and commonly used method for fusing multimodal features. It involves concatenating the feature vectors of different modalities along a specific dimension, typically the feature dimension. In our case, the chest radiograph features with dimension , denoted as ​, and the clinical features with dimension , denoted as ​, are concatenated to form a unified feature vector.

The concatenation operation is mathematically represented as:

where ​ is the fused feature vector with dimension ​. ​ is then passed through a fully connected layer for further processing and prediction.

Attention mechanism-based fusion

While concatenation captures the raw combination of features, it does not account for the relative importance of different modalities in the final prediction. To address this, we incorporate an attention mechanism to dynamically weigh the importance of chest radiographs and clinical features based on the context of the input data.

The attention mechanism assigns a weight to each modality’s feature vector, which allows the model to focus on more informative features during the fusion process. For each modality  i, the attention weight ​ is computed as:

where is the feature vector from modality  i (i.e., ​or ), ​ is the learnable weight vector for modality  i, is the attention weight for modality  i.

The attention weight reflects the importance of modality  i for the current prediction. Once the attention weights are computed, the weighted features are obtained as:

The fused feature vector ​ is then passed through subsequent layers for further processing and prediction.

Prediction model training configuration

During the training of the prediction model, data augmentation was applied to the input chest radiographs. This included resizing to 256 × 256, random cropping to 224 × 224, random rotation, Gaussian blur, and random affine transformations. The training process used a batch size of 90, a learning rate of 0.001, and ran for 100 epochs.

In addition, several hyperparameters were considered in the prediction model, including chest radiograph view, model architecture, pre-trained weights, loss function, down-sampling ratio, and multimodal fusion strategy. Given the large number of possible hyperparameter combinations, we adopted a sequential one-factor-at-a-time (OFAT) search strategy, in which candidate options for each hyperparameter were compared while the remaining settings were kept at their current default values. In this framework, model architecture was itself treated as a hyperparameter.

We first defined a baseline configuration to initialize the search, consisting of frontal chest radiographs, ResNet50 as the default architecture, no pre-trained weight initialization, weighted loss, no down-sampling, and concatenation-based fusion. ResNet50 was used only as the default starting choice for the baseline configuration, rather than as the sole architecture for hyperparameter optimization. During the architecture-comparison step, ResNet50, DenseNet121, ConvNeXt, and ViT were evaluated as alternative candidate architectures under the same comparison framework. After the best-performing option for each hyperparameter had been identified according to the predefined validation scheme and selection metric, the final model was retrained using the resulting hyperparameter combination and then evaluated on the independent test set.

The model was implemented using PyTorch, and training was conducted on an NVIDIA GeForce RTX 4090 with 24 GB of memory. To mitigate the effects of randomness from model parameter initialization, each configuration of the prediction model was trained five times.

Statistical analysis

Because only 4.74% of the enrolled children progressed to severe CAP, the outcome distribution was highly imbalanced. Therefore, we primarily used the area under the precision–recall curve (PR-AUC) and the area under the receiver operating characteristic curve (ROC-AUC) to evaluate model performance. To further assess the potential clinical utility of the optimal fusion model under different operating conditions, we additionally reported its performance at the threshold yielding the maximum F1 score (F1-max threshold) and at the operating point achieving approximately 90% sensitivity. The results are presented as the mean and standard deviation of each metric across five independent training runs using the same model configuration but different random seeds, thereby quantifying the stability of model performance with respect to stochastic variation during training. In the subgroup analysis, the squared error between the predicted probabilities and true labels was introduced to further evaluate model performance. Additionally, the Mann-Whitney U test was employed to determine whether significant differences existed in the model’s performance across different groups. All statistical analyses were performed using R version 4.3.1 (R Foundation for Statistical Computing, Vienna, Austria). All the code for running the experiments is available at https://github.com/A-Big-Brain/Severe-Pediatric-Community-Acquired-Pneumonia-Prediction.

Results

Hyperparameter optimization for chest radiographs

Before integrating chest radiographs and clinical features, we first optimized the hyperparameters associated with the chest radiographs, including the chest radiograph view, model architecture, pre-trained weights, loss function, and down-sampling ratio. The performance of various hyperparameter configurations in predicting severe CAP is presented in Fig. 3.

Fig. 3.

Hyperparameter optimization and performance of the optimal model based solely on chest radiographs. a-d: Bar plots comparing different options in hyperparameters. The hyperparameters include chest radiograph view (a), model architecture + pre-trained weights (b), and down-sampling ratio + loss function (c, d). The metrics shown are PR-AUC (a-c) and ROC-AUC (d). e-g: Performance curves of the optimal model, including the precision-recall curve (e), the receiver operating characteristic (f), and calibration curve (g). In panel b, the x-axis labels denote: “no-pret” (no pre-trained weights), “imagenet” (weights pre-trained on the ImageNet-1K dataset), and “radi” (weights pre-trained on chest radiograph datasets). In panels c and d, the x-axis labels indicate: “wei” (weighted loss function), “no-wei” (no weighted loss function), “no-sam” (no down-sampling), and “n” (e.g., 1, 5, or 10) referring to different down-sampling ratios

In terms of chest radiograph views, the lateral position demonstrated superior predictive performance, with a PR-AUC of 16.45 ± 5.60%, compared to the frontal view, which yielded a PR-AUC of only 8.56 ± 1.27% (Fig. 3a). Regarding model architecture and pre-trained weights, DenseNet121 combined with ImageNet-1 K pre-trained weights achieved the best performance, with a PR-AUC of 9.84 ± 0.99% (Fig. 3b). For the down-sampling ratio and loss function, the use of a weighted loss function without down-sampling produced the optimal results, with a PR-AUC of 8.56 ± 1.27% and a ROC-AUC of 66.47 ± 1.81% (Fig. 3c, d).

Based on the optimal selection of each hyperparameter, we trained the best-performing model for chest radiographs. The prediction performance of this optimal model is shown in Fig. 3. The PR-AUC was 16.33 ± 1.98% (Fig. 3e), while the ROC-AUC for the optimal model was 78.28 ± 1.48% (Fig. 3f), and the calibration curves show that the model is unstable and generally poorly calibrated (Fig. 3g). Although the ROC-AUC indicates moderate discriminatory ability, the very low PR-AUC highlights that chest radiographs alone have limited predictive value for identifying severe CAP cases. This finding further underscores the necessity of incorporating clinical laboratory features and demographic information through multimodal fusion to achieve more reliable early risk prediction.

Fusion of chest radiographs and clinical features

Building upon the optimal chest radiograph model, 181 clinical features were integrated to further enhance the model’s ability to predict severe CAP. Two different fusion methods were compared, using both frontal and lateral chest radiographs. The corresponding results are presented in Fig. 4.

Fig. 4.

Comparison of fusion methods and performance evaluation of the optimal model based on the fusion of chest radiographs and clinical features. a-b: Bar plots comparing the performance of different fusion methods. The metrics displayed are PR-AUC (a) and ROC-AUC (b). c-e: Performance evaluation of the optimal model, including precision-recall curve (c), the receiver operating characteristic (d), and calibration curve (e). In panels a and b, the x-axis labels represent: “var” (clinical variables), “concat” (concatenation-based fusion), and “atten” (attention mechanism-based fusion)

As shown in Fig. 4a-b, the attention mechanism-based fusion applied to lateral chest radiographs achieved a PR-AUC of 46.68 ± 4.50% and a ROC-AUC of 92.91 ± 0.64%. In comparison, the concatenation-based fusion method on lateral chest radiographs resulted in a PR-AUC of 38.81 ± 7.58% and a ROC-AUC of 92.62 ± 1.49%. These findings indicate that the attention mechanism-based fusion method offers superior prediction performance and greater stability. For the model based solely on clinical features, the PR-AUC was 32.94 ± 6.41%, with a ROC-AUC of 85.49 ± 1.36%. The model based on lateral chest radiographs achieved a PR-AUC of 16.33 ± 1.98% and a ROC-AUC of 78.28 ± 1.48%. The model based on frontal chest radiographs resulted in a PR-AUC of 9.84 ± 0.99% and a ROC-AUC of 73.24 ± 1.80%. These results demonstrate that the clinical features provide richer predictive information than the chest radiographs alone.

The precision-recall curve, receiver operating characteristic curve, and calibration curve for the optimal fusion model are shown in Fig. 4c-e, with the PR-AUC reaching 46.68 ± 4.50%. Compared to the model based solely on chest radiographs (Fig. 3e), the fusion model demonstrated a substantial improvement in prediction performance. Moreover, the fusion model has made progress in the reliability of its predicted probabilities, especially in regions where the model is confident in making positive predictions (P > 0.50) (Fig. 4e). The performance of the optimal fusion model at the F1-max threshold and at the operating point achieving approximately 90% sensitivity is provided in Table S3 and S4, respectively.

To assess whether the performance estimates were sensitive to a single train–test split under the low-prevalence setting, we performed stratified 5-fold cross-validation within the training set. Across folds, the clinical-feature-only model achieved a PR-AUC of 28.69% (95% CI: 22.07–37.04%) and a ROC-AUC of 81.58% (95% CI: 77.20–85.74%). The lateral-chest-radiograph-only model achieved a PR-AUC of 11.69% (95% CI: 9.21–15.79%) and a ROC-AUC of 76.98% (95% CI: 73.48–80.49%). In comparison, the multimodal fusion model achieved a PR-AUC of 40.40% (95% CI: 32.88–49.37%) and a ROC-AUC of 89.59% (95% CI: 87.06–91.98%). These results were consistent with those obtained from the primary hold-out test analysis, indicating that the performance advantage of multimodal fusion was robust to variation in patient partitioning.

Subgroup analysis by sex and age

The anatomical structure of the human chest and the progression of CAP vary across different subgroups [25]. To assess whether the CAP prediction model maintains its effectiveness in the presence of these differences, we conducted a subgroup analysis based on sex and age. For sex, subjects were divided into two groups: male and female; for age, subjects were categorized into two groups: 0–1 year and > 1 year. The performance of the optimal fusion prediction model was compared and analyzed across these subgroups, with the results presented in Fig. 5.

Fig. 5.

Performance of the optimal fusion model in gender and age subgroups. a-f: Comparison of prediction model performance in sex subgroups (a-c) and age subgroups (d-f). Evaluation metrics include PR-AUC (a, d), ROC-AUC (b, e), and squared error (c, f)

In the sex subgroups, the female group included 319 patients (14 severe CAP and 305 non-severe CAP), and the male group included 483 patients (20 severe CAP and 463 non-severe CAP). The PR-AUC was 47.66 ± 14.82% for females and 49.82 ± 4.60% for males, with no significant difference (P = 1.0000, Mann-Whitney U test). The ROC-AUC was 91.88 ± 1.47% for females and 93.83 ± 1.46% for males, with no significant difference (P = 0.1508, Mann-Whitney U test). The mean square errors were 0.2962 ± 0.0361 for females and 0.2935 ± 0.0403 for males, with no significant difference (P = 0.4501, Mann-Whitney U test).

In the age subgroups, the 0–1-year group included 448 patients (26 severe CAP and 422 non-severe CAP), whereas the > 1-year group included 354 patients (8 severe CAP and 346 non-severe CAP). The PR-AUC was 51.36 ± 5.01% for the 0–1-year group and 35.79 ± 4.42% for the > 1-year group, with a significant difference (P = 0.0079, Mann-Whitney U test). The ROC-AUC was 91.90 ± 1.23% for subjects aged 0–1 years and 90.25 ± 3.30% for those aged > 1 year, with no significant difference (P = 0.8413, Mann-Whitney U test). The mean square errors were 0.2727 ± 0.0364 for the 0–1-year group and 0.3088 ± 0.0330 for the > 1-year group, with a significant difference (P < 0.0001, Mann-Whitney U test).

For further details, the receiver operating characteristic curves, precision-recall curves, and calibration curves for each subgroup are shown in Figure S2.

SHapley Additive exPlanations-based interpretability

In this study, SHapley Additive exPlanations values were employed to quantify the contribution of each feature to the model’s predictions. SHapley Additive exPlanations values, introduced by Lundberg and derived from game-theoretically optimal Shapley values [37], offer a robust method for interpreting model predictions. For chest radiograph features, their representations were extracted from the input features of the fully connected classification layer within the prediction model. The SHapley Additive exPlanations values and their distributions for various features are illustrated in Fig. 6.

Fig. 6.

SHapley Additive exPlanations values and their distributions for features from frontal chest radiograph, lateral chest radiograph, and clinical data. a-c: Histograms of SHapley Additive exPlanations values for the unimodal prediction model, including features from frontal chest radiographs (a), lateral chest radiographs (b), and clinical data (c). d: Bee swarm plot of SHapley Additive exPlanations values for clinical features in the unimodal prediction model. The importance of all clinical features is shown in Table S5. e-f: Violin plots of SHapley Additive exPlanations values for the fusion prediction model. In panels e and f, the x-axis label “var” denotes clinical variables. SHAP: SHapley Additive exPlanations

For the unimodal prediction model, SHapley Additive exPlanations values were computed for each feature across all samples, followed by averaging these values across the dataset. As depicted in Fig. 6, the majority of SHapley Additive exPlanations values for the unimodal prediction model are concentrated near zero, suggesting minimal contributions from many features. Additionally, a substantial number of SHapley Additive exPlanations values are negative, likely reflecting the predominance of negative samples in the dataset.

The bee swarm plot in Fig. 6 highlights the top contributing clinical features in the unimodal model. Notable features include respiration, calcium (quant.), Percentage of CD3 + CD4+T lymphocytes (abnorm.), D-dimer (quant.), heart rate, temperature, lactate dehydrogenase (quant.), urinalysis pH (abnorm.), and sex. These nine features demonstrated the most significant impact on model predictions. The predominance of negative SHapley Additive exPlanations values reflects the low prevalence of severe CAP in the cohort, causing most predictions to be closer to the negative class. Since image features in deep learning models lack explicit interpretability, no additional analysis was conducted on these features.

For the fusion prediction model, the mean of the absolute SHapley Additive exPlanations values for each feature was calculated across all samples. As shown in Fig. 6, the average SHapley Additive exPlanations values for image features were smaller compared to clinical features, indicating that clinical features contributed more substantially to the model’s predictions.

Gradient-weighted Class Activation Mapping-based interpretability

In this study, Gradient-weighted Class Activation Mapping (Grad-CAM) [38] was employed to visualize regions of chest radiographs that significantly influenced the model’s prediction scores. Figure 7 presents examples illustrating the lesion areas annotated by radiologists alongside Grad-CAM-generated saliency maps.

Fig. 7.

Grad-CAM saliency map of chest radiographs in the prediction model. a-f: Grad-CAM saliency maps for frontal chest radiographs. g-l: Grad-CAM saliency maps for lateral chest radiographs. In each panel, the left image shows the lesion area annotated with a red bounding box by a radiologist, whereas the right image highlights the regions of the chest radiograph most influential to the prediction score, as identified by the Grad-CAM technique. CAP: Community-acquired pneumonia

As depicted in Fig. 7, the activated regions identified by the prediction model exhibit substantial overlap with the radiologist-annotated lesion areas, with most activations localized within the lung region. However, certain activated regions extend beyond the radiologist-annotated areas, suggesting that the model may be capturing additional information that is not explicitly recognized by the radiologist. Furthermore, compared to frontal chest radiographs (Fig. 7), the activated regions in lateral chest radiographs (Fig. 7) are more confined, indicating that the prediction model leveraging lateral chest radiographs may more effectively identify regions associated with severe CAP.

Discussion

In this study, we developed a prediction model for severe CAP by integrating chest radiographs with clinical features. Hyperparameter optimization was conducted for chest radiograph views, model architectures, and strategies to address data imbalance, resulting in the identification of the optimal model. The final fusion model achieved a PR-AUC of 46.68 ± 4.50% and a ROC-AUC of 92.91 ± 0.64%. Additionally, the model demonstrated significant consistency across different sex subgroups. To enhance model interpretability, we employed SHapley Additive exPlanations values and Grad-CAM, which facilitated the identification of important clinical features and highlighted key regions within the chest radiographs that most influenced the model’s predictions.

In recent years, self-supervised learning and foundation models have garnered substantial attention in the field of medical imaging. Several studies have indicated that leveraging self-supervised pre-trained weights can improve performance in downstream tasks [36, 39–41]. In this study, we evaluated both self-supervised and supervised pre-trained weights. The results indicated that for the DenseNet121 model, self-supervised pre-trained weights performed worse than supervised pre-trained weights, and were similar to models without pre-training (Fig. 3). This discrepancy may be attributed to the smaller dataset used for self-supervised pre-training compared to supervised pre-training, suggesting that researchers should carefully select pre-trained weights based on the specific clinical task rather than defaulting to foundation model pre-trainings.

The age subgroup analysis revealed significant differences in the PR-AUC and squared error between the 0–1 year and > 1-year groups (Fig. 5). This is likely due to the imbalance in the distribution of severe CAP cases between these groups. Additionally, younger children are more prone to developing severe CAP [42, 43], which exacerbates data imbalance in the > 1-year group. In routine radiologic assessment, frontal and lateral chest radiographs are generally interpreted as complementary rather than competing projections. The lateral view is particularly helpful for lesion localization and for assessing regions that may not be fully visualized on frontal imaging, such as retrocardiac or posterior lower-lobe opacities. In our study, the lateral-view model showed higher predictive performance than the frontal-view model (Fig. 3). This finding should not be interpreted as indicating that the lateral projection is inherently more important than frontal imaging in routine diagnosis. Rather, it suggests that, for the specific task of early severity prediction, lateral radiographs may provide additional information that is useful to the model. Several factors may account for this observation. Lateral projections can offer improved visualization of retrocardiac areas and lower-lobe opacities and may reduce the overlap of mediastinal and osseous structures. In addition, because lateral imaging is often obtained selectively in clinical practice, the observed performance advantage may also partly reflect differences in examination patterns and case mix.

In terms of clinical features, respiration was identified as the most influential factor in the model (Fig. 6), aligning with existing research and clinical guidelines on the diagnosis and early decision-making for severe CAP in children [43]. Furthermore, lactate dehydrogenase and D-dimer levels were found to be significant indicators of severe CAP risk, consistent with expert consensus [44]. Additionally, reduced levels of CD3 + CD4+T lymphocytes, which play a key role in immune response coordination, were associated with an increased risk of severe CAP [45]. This finding is in agreement with previous studies indicating that a weakened immune system, due to a decrease in these lymphocytes, may render patients more susceptible to severe infections and poorer outcomes. The SHapley Additive exPlanations analysis also provides clinically actionable insights. Features such as hypoxia, increased respiratory effort, and elevated inflammatory markers—identified as major contributors in the model—are already central components of bedside assessment. Highlighting their relative importance reinforces their value in early triage and suggests that children exhibiting these abnormalities may benefit from closer monitoring or earlier escalation of care.

Our findings also complement and extend prior work on pediatric CAP severity prediction. Traditional risk score models, such as those developed in Ethiopian [46] and Chinese cohorts [47], identified key clinical predictors—including vaccination status, feeding difficulty, impaired consciousness, comorbidities, and laboratory abnormalities—and achieved good discrimination using structured clinical data alone. More recent machine-learning–based studies have incorporated broader clinical variables and have reported strong performance in predicting severe pneumonia or escalation of care, with hypoxia, respiratory distress, and inflammatory markers consistently emerging as important predictors [10, 48, 49]. Compared with these studies, our work contributes two advances: first, we integrate both chest radiographic features and clinical information using a multimodal fusion architecture, enabling the model to capture complementary signals that single-modality models may miss; second, we demonstrate that lateral radiographs—an understudied modality in existing literature—can provide meaningful incremental value. Together, these comparisons highlight the added clinical utility of our multimodal approach and position it as a complementary tool to existing risk prediction models in pediatric CAP.

We note that Grad-CAM provides a coarse, class-discriminative localization map for a classification model rather than a pixel-level delineation of pathology. Accordingly, activations extending beyond the radiologist-annotated ROI should be interpreted with caution and do not necessarily indicate reliance on non-clinical artifacts. First, the annotated ROIs were intended to highlight representative lesion regions and may not capture the full spatial extent of disease, especially for subtle or diffuse pediatric radiographic findings. Second, severity prediction may legitimately depend on broader contextual cues that are not confined to a single focal lesion. Nonetheless, we acknowledge the possibility of spurious correlations in saliency maps. Therefore, in this study we use Grad-CAM as qualitative evidence to support model interpretability, and future work will incorporate additional sanity checks to further assess whether model attention remains clinically plausible.

In this study, we also evaluated whether missingness in clinical variables might relate to disease severity and affect model estimates. Overall, key predictive features showed low missingness (< 3%), with only a few variables, such as temperature (16.45%), percentage of CD3 + CD4+T lymphocytes (20.23%), and urinalysis pH (23.09%) exhibiting moderate missing rates. We used median imputation for continuous variables and mode imputation for categorical variables, assuming a Missing-at-Random mechanism aligned with routine clinical workflows. Although missingness in certain laboratory tests may partially reflect clinician decision-making and therefore weakly correlate with baseline severity, its overall impact is likely limited because the most influential features were largely complete, model performance remained stable across alternative imputation strategies, and the multimodal fusion model can compensate through chest radiograph representations and other fully observed variables. Future prospective studies with standardized testing protocols may help further elucidate the clinical meaning of missingness patterns.

The low prevalence of severe CAP (4.74% in our cohort) has important implications for evaluation and implementation. Under marked class imbalance, ROC-AUC may remain high even when PPV is modest; thus, PR-AUC and operating-point analyses are essential to reflect performance in a screening-oriented setting. In practice, a high-sensitivity operating point may be preferred to minimize missed severe cases, accepting a higher false-positive rate as the cost of early warning. Accordingly, we reported model behavior at clinically relevant thresholds (maximum F1 Score and approximately 90% sensitivity) and recommend that deployment decisions be guided by the intended use case.

Conclusion

In conclusion, while our study demonstrates the promising potential of combining chest radiographs and clinical features for predicting severe CAP, there are several limitations that should be addressed in future research. First, the absence of external and multicenter validation is a major limitation, especially given the class imbalance between severe and non-severe CAP. Without testing on an independent cohort, the model’s generalizability to different populations and clinical environments cannot be fully established. Second, the model’s reliance on retrospective data introduces the possibility of biases that could affect its real-world applicability. Third, although our model integrates a broad set of clinical features, some variables are not routinely collected in standard pediatric CAP care and may not be universally available. This may limit the model’s applicability in healthcare systems where certain laboratory tests are not part of routine practice. Finally, although our model performed well with the selected imaging modalities and clinical features, further investigation is required to explore the integration of additional data sources, which could further improve predictive accuracy.

In the future, we will actively pursue a multicenter prospective cohort within regional pediatric networks to assemble independent data for external validation and to evaluate performance stratified by site and time period. In these planned studies, we will quantify calibration, assess the transportability of clinically relevant operating thresholds, and determine whether lightweight post hoc recalibration is sufficient across centers. The multicenter prospective cohort will be embedded in the admission workflow, in which early-available chest radiographs and routinely collected clinical variables are automatically ingested to generate risk estimates, followed by longitudinal ascertainment of deterioration.

Authors’ contributions

L.H.: Methodology, Software, Writing - Original Draft, Funding acquisition, Visualization. Y.L.: Validation, Data Curation. M.H.: Methodology, Software, Formal analysis. M.K.: Validation. Y.L.: Data Curation. K.C.: Data Curation. Y.Z.: Resources. F.Z.: Data Curation. C.Y.: Data Curation. C.X.: Conceptualization, Resources, Writing - Review & Editing, Supervision. X.L.: Conceptualization, Resources, Writing - Review & Editing, Project administration.

Funding

This study’s financial support was derived from various sources, including the Funding for Scientific Research Projects from Health Commission of Hubei Province (WJ2025Z010), the Young Scientists Fund of the National Natural Science Foundation of China (82402376), the Special Project of the National Natural Science Foundation of China (T2341004), the Knowledge Innovation Specialized Project of Wuhan Science and Technology Bureau (2023020201010198), the Basic and Applied Basic Research Foundation of Guangdong Province (2024A1515011750), Hubei Provincial Preventive Medicine Association’s “Health Management Innovation Talent Cultivation Action” research project (2025SWGKY214), Wuhan Natural Science Foundation-Exploration Program (2025040601020205), and Wuhan Children’s Hospital Doctoral Start-up Fund Project (2024FEBSJJ005).

Data availability

All data generated or analyzed during this study are included in this published article. Due to ethical and legal restrictions related to patient privacy and the confidentiality of clinical records, the underlying individual-level patient data are not publicly available. All the code for running the experiments is available at [https://github.com/A-Big-Brain/Severe-Pediatric-Community-Acquired-Pneumonia-Prediction](https:/github.com/A-Big-Brain/Severe-Pediatric-Community-Acquired-Pneumonia-Prediction) .

Declarations

Ethics approval and consent to participate

All procedures were performed in compliance with relevant laws and institutional guidelines and have been approved by the Medical Ethics Committee of Wuhan Children’s Hospital (approval number: 2024R115-E01).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Clinical trial number

Not applicable.