Semi-supervised multi-class pneumonia classification using a CNN-cascade forest framework

Abstract

Medical imaging diagnosis of pneumonia must be accurate at distinguishing between the various disease subtypes but must be resistant to sparse annotated evidence as well as cross-imaging variability. The current deep learning methods are mostly univariate and binary-classification, which limits their clinical use. As a solution to these shortcomings, this paper suggests a semi-supervised CNN-Enhanced Cascade Forest (CE-Cascade) scheme to classify multi-class pneumonia based on X-ray and CT images of the chest. Based on the approach proposed, a convolutional neural network will first be used to test the medical images to extract discriminative deep features and the resultant features will be refined by a cascade forest to detect hierarchal and multi-scale patterns with great significance in terms of pneumonia detection. A semi-supervised pseudo-labelling strategy is also incorporated in order to competently utilize unlabelled data and better generalization with scarce annotation requirements. The framework is tested over a data set that consists of 4578 chest X-ray and CT images that are divided into categories of bacterial, viral, fungal, general pneumonia, and normal. The experimental findings prove that the given CE-Cascade model reaches the overall classification rate of 98.86 that is higher than some state-of-the-art deep learning systems. Findings validate that semi-supervised learning with the integration of CNN and Cascade Forest to create a robust and clinically meaningful method to classify multi-classes of pneumonia is automated.

Introduction

Unsupervised image characterization is technique used in computer vision and image processing that entails analysing, classifying and interpreting images without having the labelled data or pre-defined categories^1,2. In contrast to supervised approaches, which rely on tagged input data, unsupervised image characterization leverages classical image analysis heuristic schemes, algorithms designed to identify natural patterns, structural elements, characteristics that exist within the images to sort out the similar items. It is most useful when labelled datasets are not available, expensive to acquire, or impossible to generate in quantity³. It will see those applications of unsupervised image characterization are numerous and exist in a wide range of contexts including remote sensing, medical imaging and multimedia content analysis where the technique is used for simple image segmentation for the identification of objects, landscapes, textures or anomalies⁴.

The primary principles of unsupervised image characterization involve mainly two approaches like clustering (i.e., k-means, DBSCAN) and dimensionality reduction (such as PCA, t-SNE) because these strategies seek to provide simplified data visualization⁵. For example, based on remote sensing, unsupervised approach can used in image classification to define areas of various land cover types or in changes detection over time. In medical imaging for example, they can help in the determination of regions of interest of MRI or CT image scans where the labels have not been previously defined and hence, they are very important in the exploratory data analysis^6,7. The future development of generative models based on patterns and self-supervised learning taking place in the most recent years help unsupervised image characterization to go deeper and to learn the hierarchy of features directly from the raw images. The potential of unsupervised characterization for discovering patterns and trends in image data can be particularly valuable for many industries which need to implement automatic image analysis pipelines^8,9.

There are several important limitations associated with the concept of unsupervised image characterization. A significant drawback of this approach is the absence of free labels, which is the primary source of information for unsupervised approaches, and there is no indication of whether the identified patterns have value for the intended task¹⁰. This can result in categorizations that may be imprecise or unrelated to what users perceive as relevant or how they intend to implement them. The other difficulty is algorithm’s sensitivity to parameters: for instance, the number of clusters in clustering algorithms could change the solution dramatically and needs to optimized to meet reasonable level of solution quality^11,12. In addition, unsupervised methods are challenged with high dimensionality data which is common in images, and this may cause expensive computational which if reduced may result to loss of information. Additionally, if something is not labelled, there is no way of assessing or even proving the results and therefore, when defining the best model or, in the iterative improvement of results, there is no way of getting started^13,14. Lastly, interpretability is an issue because while utilizing unsupervised methods, especially deep learning, the resulting patterns or features are less transparent and difficult to explain or trust, especially where the application is sensitive such as pathology or security concerns¹⁵. Figure 1 shows the unsupervised image characterization.

Fig. 1.

Unsupervised image characterization.

Unsupervised image characterization by means of deep learning based on applying neural networks for characterizing images based on their features without using labelled data. Usually, this involves passing the raw image data through a feed-forward neural network such as an AutoEncoder or even a convolutional neural network (CNN) in order to extract relevant features and map complex image data to a smaller feature space¹⁶. The network takes high-level visual information and maps it to a much simpler feature space retaining all the important attributes such as textures, edges, shapes and colour information. After the feature learning process, clustering algorithms such as k- means or hierarchical clustering are used to cluster similar images or regions of images based on algorithm similarities¹⁷. In recent works to improve this process, self-supervision learning aided in feature extraction by using of pretext tasks, where the images rotated or the patches reordered without labels. This makes the model more sensitive at identifying more features in the images especially in areas of medicine and satellite imaging where specific texture or abnormality is required to be detected¹⁸. The process ends with a retrospective qualitative or quantitative analysis where one describes if the clusters make sense or not, capture valid categories or patterns. Unsupervised deep learning is particularly suitable for applications where or when labelled data is hard to come by, or are applications that require the constant discovery of patterns.

Unsupervised image characterization using deep learning have several disadvantages, which may hinder its applicability and efficiency. Another concern is the explainability of deep learning models, especially in the context of unsupervised style, where there are no reference labels¹⁹. This makes it hard to decipher possible meaning or significance of described clusters since decision made by this model is sometimes inexplicable. The last disadvantage is the high computational expense; different deep learning models serve considerable computational load when processing large data sets or in real time modes, especially when the resources are a problem²⁰. However, there are some limitations associated with the existing frameworks, including the rather task-restricted nature of a simple binary system that only categorizes images as pneumonia or non-pneumonia. These systems do not consider the clinical aspects of pneumonia that can be split into bacterial, viral, fungal, and overall pneumonia types. To prevent such a model, our proposed model utilizes the multi-class classification expanding the diagnostic capability to diseased categories and the healthy control group. This classification improves the clinical relevance of the model by affording detailed description of various subclasses of pneumonia. The focus of this work is the proposed CNN-Enhanced Cascade Forest (CE-Cascade) that combines the capability of convolutional neural networks for high dimensional feature learning with the multilayer decision tree structure of Deep Cascade Forest (gcForest). This combination guarantees that both local and global image patterns which are important in the diagnosis of pneumonia are well obtained. Additionally, the model employs a form of pseudo supervision that enhances the training process via additional pooled sample data without the need for large labelled samples and thereby enhances generalisation. Based on the experimental results, we have shown that our proposed CE-Cascade model outperforms other models for both X-ray and CT image modalities. Applying paired datasets, using multiple classification problem paradigm, and implementing new model ensembles, this study presents a reliable foundation for automated pneumonia diagnosis.

Main contributions of the work

• Innovative use of semi-supervised learning: The development of an innovative semi-supervised framework for medical image characterization, effectively leveraging both labelled and unlabelled data.

• CE-cascade model: The proposed CNN-Enhanced Cascade Forest (CE-Cascade) model combines the high-dimensional feature extraction capabilities of Convolutional Neural Networks (CNNs) with the hierarchical and multi-grained scanning capabilities of Deep Cascade Forest (gcForest). This integration ensures comprehensive analysis of both local and global image patterns.

• Pseudo-labelling for data augmentation: Implements a pseudo-labelling technique to expand the training dataset with unlabelled images, reducing dependency on large labelled datasets and enhancing model robustness.

• Multi-Class classification framework: Moving beyond traditional binary classification, the model is designed to classify multiple pneumonia types (bacterial, viral, fungal, and general) along with normal cases. This multi-class framework enhances the diagnostic utility by addressing the clinical diversity of pneumonia.

• Efficiency in training and computational resources: Achieves an efficient balance between high performance and manageable computational complexity, making it suitable for real-world healthcare deployment.

“Related works” provides an overview of related studies on pneumonia diagnosis, focusing on semi-supervised learning, and deep learning architectures in medical imaging. “Methodology” details the proposed methodology, describing the integration of Deep Cascade Forest (gcForest) and CNNs for enhanced representation of pneumonia-related features. “Results and discussions” presents the results and discussion, including quantitative metrics and qualitative analyses, demonstrating the model’s superior accuracy. Finally, the conclusion and future scope discussed, highlighting the model’s impact and potential enhancements for broader medical applications.

Related works

Target registration is crucial in the fast-developing area of critical clinical medical image analysis, especially surgical navigation and tumour diagnosis. Given the imperfections of the presently available non-supervised techniques for registration of single-modal images, an efficient deep learning-based approach presented²¹. The primary enhancement of the algorithm is in implementing the short and long connections within the densely connected U-net architecture. The stated interconnection design helps greatly in improving feature map connectivity and eradicating semantic gaps caused by layered sampling depth. Moreover, an integrated channel attention mechanism at the U-shape AutoEncoder decoder helps to lessen noise, generate more refined deformation fields. This is suitable for refining model sensitivity to subtle details and increasing image registration precision, namely, single-modal brain MRI data. These clinical trials also strongly show the amount of the improvement the algorithm brought into accurate medical image registration.

Use of unmanned aerial vehicles (UAVs) with hyperspectral imagers is relevant for Inland Water Body (IWB) characterization. These UAVs provide data us with high spectral and spatial resolution for water quality parameters using band ratio algorithms with machine learning models. However, this comes with the challenge of needing a huge amount of in situ reference data for developing the model, and which may sometimes be time-consuming and expensive affair. The probabilistic model of self-organizing maps known as Generative Topographic Mapping (GTM) introduced for the visualization of high dimensional hyperspectral images and identification of spectral features required for representing the water endmembers²². With data collected in a North Texas pond, species distributions presented in the GTM provide evidence of spatial heterogeneity in water quality. This approach shows the capacity of hyperspectral imaging by UAV and GTM and their application in precise water quality measurement quickly.

Image segmentation is a typical CV problem that aims to assign each pixel to certain classes, which is different from classification or detection, which gives several sparse outputs. Manually annotating pixel level data is tedious, therefore the desire for unsupervised learning to aid or supplement the annotation. Previous methods of segmentation that include graph theory, clustering, and image processing only made “shallow” classifications where super pixels of similar pixels are made²³. The latest deep unsupervised techniques, however, rely on deep structures that can learn pixels grouping in whatever manner to manage the unstructured image data. It added that with self-supervised learning, various segmentation features could acquire without labels for use in medical imaging, remote sensing and many other tasks. This reviews the fundamentals of unsupervised image segmentation techniques, terms used, ongoing state-of-art works, future trends in the subject area and the applications.

Some of the most valuable methods used in data analysis based on unsupervised machine learning to categorise dimensions and reduce the volume of the dataset. These techniques may thus be useful in health sciences and maybe used to identify new risks factors, improve disease prevention efforts or further individualized patient approaches. A balanced coverage of important topics related to unsupervised machine learning, such as K-means clustering, hierarchical clustering, principal component analysis, and factor analysis²⁴. To this end, by having a thorough understanding of these tools, health researchers can apply unsupervised machine learning to studies to improve data analysis. Their use in health sciences promises to ensure that interventions and treatments provided mirror the patients’ specific needs to enhance improvement through evidence-based research.

Risk prediction of diseases is an essential problem in health care, and there has been a growing interest to solve such problems using machine learning (ML). Although the use of supervised ML methods documented, unsupervised methods are also emerging, especially when dealing with unlabelled or low-incidence diseases. Evaluation of seven unsupervised algorithms, with 15 datasets investigated, related to diseases, including heart failure, diabetes, and breast cancer²⁵. Using the metrics such as Adjusted Rand Index, Homogeneity and Silhouette Coefficient the study has noted DBSCAN (Density-based spatial clustering) to be the most efficient technique followed by Bayesian Gaussian Mixture and Divisive clustering. Consequently, results emphasize that the problem of model and metric choice is best to solved separately for each application. For example, DBSCAN is good in Homogeneity and Completeness, while Bayesian Gaussian Mixture is good in Adjusted Rand Index. The findings of this study extend the use of US ML in healthcare while stressing the need for development of specific models in Healthcare big data to improves diseases and patient outcomes.

Methodology

The proposed methodology, Hybrid CNN-Enhanced Cascade Forest (CE-Cascade), integrates ResNet for feature extraction with a Deep Cascade Forest (gcForest) for hierarchical pattern analysis and classification. A paired dataset comprising 4,578 X-ray and CT images from the same patients, categorized into bacterial, viral, fungal, general pneumonia, and normal cases, utilized to enable cross-modality learning. ResNet employed to extract high-dimensional, discriminative features from the medical images, which then processed by gcForest for multi-grained scanning and ensemble learning. The model incorporates a semi-supervised learning framework using pseudo-labelling to enhance the training process by leveraging both labelled and unlabelled data. Preprocessing steps, including image normalization and augmentation, ensure data consistency and variability. The CE-Cascade model fine-tuned to optimize performance across X-ray and CT modalities, achieving superior accuracy, precision, and adaptability for automated pneumonia diagnosis.

Data collection

A publicly accessible dataset was obtained at Kaggle repositories, namely, the COVID-19 CT Scans Dataset²⁶ and the Chest X-ray Pneumonia Dataset²⁷, which comprise a wide range of medical imaging samples, and are categorized under different disease categories. These groups are general pneumonia, bacterial pneumonia, viral pneumonia, fungal pneumonia and normal (healthy) cases with no observed lung abnormalities. The pathological characteristics of each of the classes are different in order to be fully and equally represented. An example is bacterial pneumonia that is commonly characterized by localized consolidation and dense opacities on CT scans whereas viral pneumonia exhibits diffuse interstitial patterns. Fungal pneumonia is a less frequent disease with distinct radiological appearances, including nodulation and cavities. Normal lung scans should be incorporated as they are necessary to provide an adequate and healthy baseline on which one may compare disease-specific abnormalities. Overall, 4578 paired CT and X-ray images were used, and the data is such that each case of a patient has both modalities with respective features of a diagnosis. This dual-modality design can facilitate cross-modality learning- CT measurements can give more detailed information on the anatomy and X-ray can offer fast diagnostic views.

The chest X-ray and CT images were matched at the patient level to facilitate cross-modality learning and clinical validity through a method of retaining the cases where both modalities were present during the same diagnostic episode. The matching procedure was based on patient-level identifiers and diagnostic metadata offered in the source datasets and made each X-ray image matched to the CT scan taken of the same person. The modalities had temporal consistency since pairs of images were chosen during the same clinical assessment, which reduced the variation due to progression of disease, response to treatment, or recovery. The dataset excluded image pairs having a significant temporal difference or showing a changing clinical status. Pathological consistency was ensured through imposing the same diagnostic labels in the content of both modalities. Image pairs that had different approved categories such as bacterial pneumonia, viral pneumonia, fungal pneumonia, general pneumonia, or normal were eliminated. Those cases which had conflicting, ambiguous or incomplete diagnostic annotations across the modalities were eliminated.

The resulting dataset consisted of 4578 medical images categorized into five clinically relevant sets which included bacterial pneumonia, viral pneumonia, fungal pneumonia, general pneumonia, and normal cases. In order to be transparent, the sample distribution used in this study in terms of the number of classes is summarized in Table 1. The distribution was unbalanced by nature, as it is usually seen in real-world clinical data, with less of the samples being available in the disease of fungal pneumonia than in other types. In order to reduce the possible effects of the class imbalance during the model training, several measures were taken into account. First, data augmentation was more actively used on minority classes, with no change in pathological features and greater sample diversity. Second, the cascade forest element has the intrinsic characteristics of decreasing sensitivity to imbalance, using ensemble-based decision boundaries instead of the one classifier. Third, to make sure that the evaluation of the performance was balanced with the assessment of the categories, the performance assessment paid attention to the class-agnostic indicators (precision, recall, and F1-score).

Table 1.

Distribution of samples across pneumonia categories.

Category	Number of samples
Bacterial pneumonia	1124
Viral pneumonia	1036
Fungal pneumonia	642
General pneumonia	978
Normal	798
Total	4578

To maintain patient-level independence as well as avoid leakage of data across imaging modalities, the dataset was partitioned at the patient level as opposed to image level. All X-ray and CT images of the chest of the same patient were only allocated to one data subset e.g., training, validation or testing so that no patients were represented in more than one split. The strategy ensures that unwanted information sharing between sets is avoided especially in cases where paired modalities are employed. The data set was separated into three sections, training, validation and test, which comprised 70% training, 15% validation and 15% testing. Model learning was done with the training set, which consisted of pseudo-label generation, and hyperparameter tuning, which used the validation set, and tracked pseudo-labelling stability. The test set was not utilized at all and was to be employed during the final assessment of performance.

Data preprocessing

All chest X-ray and CT-images have been preprocessed in a standardized way before model training to guarantee that the imaging modalities and acquisition settings are consistent. They were all down sampled to a fixed resolution and made intensity-normalized to minimize scanner-related effects and make the features more stable. The steps make convolutional neural networks effective learners, and are widely used in medical image analysis pipelines. The methods of data augmentation were used during training to enhance generalization and reduce overfitting due to a small number of annotated data. In particular, rotation, scaling and horizontal flipping were also used to add controlled variability whilst excluding clinically meaningful structural patterns. These augmentation plans assist the model to acquire some invariant representations of the pneumonia-related characteristics across various imaging orientations and scales. The preprocessing and augmentation approach adopted is based on best practices in medical imaging deep learning and aligns with the recent literature that puts more emphasis on conceptual explicitness rather than a low-level mathematical realization. This method showed useful disease classification and stratification based on MRI data through the use of common preprocessing and augmentation approaches without mathematical modeling²⁸. The resultant preprocessed data offers a dependable and sturdy basis to the further feature extraction and classification procedures within the suggested designing.

Feature extraction

ResNet was the main convolutional neural network to be used in the proposed framework to extract the features because it has demonstrated its effectiveness in examining medical images, and because it is capable of learning deep hierarchies using residual learning. ImageNet-pretrained weights were used to initialize the network and the pneumonia imaging dataset was used to fine-tune the network to represent the domain-specific radiological patterns that occur in chest X-ray and CT images. The extraction of features was done by removing the last classification layer of the ResNet architecture and retaining the result of the last convolutional block. This design enables the network to produce high dimensional deep feature vectors providing structural, textural, and intensity-based feature characteristics of subtypes of pneumonia. At this stage, the explicit feature selection or dimensionality reduction methods were not applied to avoid the loss of discriminative information. Before being sent to the deep cascade forest (gcForest), the extracted CNN features were simply flattened and normalized and as a result were compatible with the tree-based ensemble learning process. Fusion of the features was done automatically by means of common representations learned using the unified CNN-gcForest pipeline as opposed to concatenation of features or manually designed fusion techniques. The gcForest model then carries out hierarchical feature refinement and decision learning in several layers and thus allows successful classification without the need to introduce explicit feature reduction mechanisms.

The chest X-ray and CT images are run separately in the feature extraction level because they are modality-specific images in the proposed framework. Every modality is learned individually using the CNN feature extractor enabling the network to be trained on the radiological trends peculiar to X-ray images (e.g., patterns of opacities through projection) and CT scans (e.g., structural features of volumes). This is independent processing and helps to prevent early fusion which may blur the discriminative information that is modality specific. Later a feature integration is carried out with the Deep Cascade Forest (gcForest) as the implicit fusion mechanism. The X-ray and CT feature vectors are extracted to high-level feature vectors and they are learned together in the cascade construction to allow the hierarchical refinement and cross-modality interaction without the explicit concatenation of the features or handcrafted fusion rules. This late-fusion approach enables the gcForest to utilize a complementary information in the two modalities and be resistant to noise that is specific to the modality. The results of this fusion strategy are empirically justified by the results of the experiment. As explained in the Results section, the proposed model is more accurate in classification and generalization in case it is trained on paired X-ray and CT images than single-modality baselines.

For an input image I with dimensions H×W×C (height, width, and channels), the ResNet model processes I through a series of convolutional layers, pooling layers, and residual connections to obtain a feature map. This can mathematically represented as:

1

where is the high-dimensional feature representation of the input image, denotes the ResNet model, and represents the parameters (weights and biases) of the ResNet model. A key component in ResNet is the use of residual connections, which allow the network to “skip” layers and help prevent the vanishing gradient problem. In a residual block, the input x is transformed by two (or more) convolutional layers with weights W₁ and W_2, and then added back to the input:

2

where represents the transformation of through two convolutional layers and is the output of the residual block. For pneumonia related features, fine-tuning is usually required in order to enhance the performance of ResNet. Medical images especially the traditional ones include X-ray and CT scans bear distinctive properties in comparison to what usually recognized as objects such as in ResNet. Fine-tuning enables us to train the weights of the network for these unique medical imaging features on another portion of pneumonia images dataset. When fine-tuning, that is, changing the parameters of ResNet, deeper network layers that detect global features are made to identify certain characteristics of pneumonia, including lung opacities, alteration of texture, and abnormal shape indicating inflammation. The early layers that detect general features the object such as edges and simple shapes can remain frozen, that is, we do not change them after they have extracted while the later layers tweaked to capture features specific to a given domain. This process increases the model’s awareness of pneumonia markers, thereby improving its potential to detect efforts based on images that are not labelled as pneumonia and those that are. The second stage involves fine-tuning ResNet for medical images, for the feature extractor needed to optimize to fit the needs of the analysis that medical images require, to enhance the robustness and accuracy of future semi-supervised learning procedures.

Fine-tuning involves adjusting the parameters θ of the ResNet model to better adapt to the pneumonia dataset. In fine-tuning, we minimize a loss function L, such as cross-entropy loss or mean squared error (for classification or regression tasks, respectively). The updated weights θ′ are obtained by optimizing using backpropagation:

3

where is the learning rate, which controls the step size for updating weights, is the gradient of the loss function with respect to the weights and represents the target output (if labels are available, as in labelled data). Once the model has been fine-tuned, each input image I can passed through the modified ResNet model to obtain its feature vector F, which can be used for downstream tasks:

4

Here, θ′ represents the updated weights after fine-tuning, making the feature vector more tailored to capture pneumonia-related patterns and details specific to medical imaging.

Semi-supervised learning framework

Semi-supervised learning is designed for a situation when the availability of labelled examples is limited or when labelling them is costly. As such, pseudo-labelling is a very useful strategy because the model then learns to assign pseudo-labels to these unlabeled data and continually add them to the labelled set. As the model becomes more confident in its predictions, pseudo-labelling takes the model’s predictions and uses them to label the new, until now unlabeled data, so that it is now pseudo-labelled data. This iterative form of labelling benefits in the application of medical imaging where labelling of data is equally time consuming and requires profuse expertise. We are able to extend the dataset by annotating more images in the Pneumonia X-Ray and CT scan datasets not annotated manually, but through this iterative process.

A confidence-based pseudo-labelling strategy was used to reduce the effects of the noisy or erroneous pseudo-labels during semi-supervised training. In every pseudo-labelling iteration, the model produced scores of class probabilities of the unlabeled samples and only those whose prediction scores surpassed a confidence threshold of 0.90 were considered pseudo-labelled sample. The samples with lower scores in the confidence scale were not put through the pseudo-labelling process and were not labelled. Along with confidence filtering, pseudo-labelled samples were introduced into the training set progressively, that is, the model is allowed to update its parameters in small steps instead of adding a significant amount of pseudo-labeled data simultaneously. This recursive re-use can be used to minimize the error propagation due to initial misclassifications. The effect of pseudo-labelled data on model performance on a held-out validation set was continuously monitored. In case the performance of validation did not improve, or was not stable, then pseudo-labelling expansion was terminated. This validation-based termination criterion made sure that only trustworthy pseudo-labelled samples were used in the model learning.

Pseudo-labelling process

In pseudo-labelling, the general process starts with a model that has trained and has a basic understanding of features from the initially labelled set of data images. In each cycle, the model performs predictions concerning the unlabeled data. They estimate pseudo-labels from these confident predictions where the confidence level of the model for making some certain predictions crosses a specific limit. Let (y∣x) be the model’s predicted probability for a class y given an image x.

5

where τ is the confidence threshold), then the model assigns the pseudo-label y to the image x. This labelled image is then added to the training set, allowing the model to iteratively refine its understanding of pneumonia features. As the model’s performance improves with each cycle, the quality of the pseudo-labels also improves, creating a feedback loop that enhances the overall dataset quality and strengthens the model’s learning process. By incorporating pseudo-labelled images together with the first labelled dataset we are able to expand the training dataset even further, thus increasing the variability of the training sample and, hence, improving the capture of more pneumonia pattern details and variation. The use of pseudo-labelled data provides more diverse samples, which contributes to noticeable difference in the variety of conditions and variations, and the samples missed by the label set during the previous iteration. This process is particularly useful in semi-supervised learning, where having even noisy labels for a lot more images is more helpful to the model since it will force it to learn better through generalization from this data. When combined with the new pseudo-labelled samples, the training set includes a wider variety of examples from a more realistic distribution than that used to generate the labelled samples, which is important for forming a strong model to deal with a diverse range of images of X-ray and CT scan. Finally, the proposed pseudo-labelling approach under a semi-supervised learning formulation helps the model exploit the available data, including both labelled and unlabeled images to improve the model capability of detecting and describing pneumonia in medical images. Besides preventing overdependence on manual labelling, this process provides a more refined model that is flexible, precise and capable of classifying varying pneumonia patterns in different types of imaging.

Proposed methodology hybrid CNN-enhanced cascade forest (CE-cascade)

In pneumonia diagnosis, the proposed CNN-Enhanced Cascade Forest (CE-Cascade) combines the advantages of CNN and cascade forest structure to obtain more efficient feature extraction and classification results. ResNet acts as the hub to yield abstract, multi-layer feature representation from input clinical images, particularly, architectural abnormalities and additional textural characteristics that discern between kinds of pneumonia. These features then passed through the cascade forest component where first single layer decision trees are formed in a hierarchy. Unlike other cascade models, the forest component in CE-Cascade improves feature representation because only the calculated output is concatenated by the input data at each level, and gradually improve and expand the feature space. This iterative perspective is helpful in identifying local and global features, which is highly beneficial when modelling the complexities of medical imaging. Through the cascade’s functional DG structure, the model is able to effectively scale across the different cases due to the dynamic recruitment of circuits as a function of the complexity of the features within the input stream. Specifically, it is applied in this application to first preprocess X-ray and CT images, where ResNet learns modality-specific features and the cascade forest integrates these features into a single representation. This design allows for the construction of a sound and scalable model that can address the hierarchical and multi-scale characteristics of medical imaging data while achieving higher accuracy in the multi-class classification of pneumonia types. The combined framework connects the designs of deep learning and ensemble techniques and checkmating the automated diagnosis system.

Deep Cascade Forest (gcForest) implementation in the Deep Cascade Forest (gcForest) was based on a cascade architecture that is an ensemble-based tree classifier. All cascade layers had two Random Forest and two Completely Random Tree Forest classifiers based on the conventional gcForest design principles. The cascade layers were found empirically by the validation performance. The cascade growth was stopped when the increase in the accuracy of the classification between the successive layers fell beneath a level of significance. The model tended to four cascade layers in the last configuration and this offered a good tradeoff between the refinement of features through hierarchy and the computational efficiency. Every forest in a cascade layer was built on 100 decision trees, and default tree depth parameters were kept to avoid overfitting as well as maintaining model generalization. No feature selection or pruning of the tree was done manually in the gcForest, since hierarchical refinement is already taken care of by the cascade structure itself.

In a gcForest, each layer consists of multiple random forests and completely-random trees that process the input features X and produce an output representation. Let X^l denote the feature representation at layer l. For each layer in the cascade:

6

where is the output feature representation at layer represents the random forest and completely-random tree transformations applied in layer and denotes the parameters (weights, decision splits) of the random forests in layer . The final output of the cascade, , after passing through layers, is the refined feature representation:

7

This refined representation captures high-dimensional, complex patterns in the image data, which are useful for downstream tasks. In each layer, the random forests trained on the labelled data so that the random forest can acquire the patterns related to pneumonia. These forests then make predictions on the unlabeled data, which in fact provides pseudo-label for the new extended labelled set. This allows gcForest to use both labelled and pseudo-labelled data and further allows the injection of some unlabeled information into the system. Every layer of the proposed model makes predictions that are connected to the next layer, and this generates a more complex understanding of the features explained by pneumonia. It discovered that this multi-layered structure enables gcForest to construct a strong and high-ordered feature space of the input data, which is completely beyond the capability of a single layer model. The output from the proposed gcForest that gives us the final feature representation can passed to other processes as a more reliable feature set for the subsequent image classification process.

Similarly, the gcForest algorithm uses a multi-grained scan method that increases the ability of the model to obtain localized fragments of the image. This is particularly useful in medical imaging where certain localized features or textural patterns—for example regions of inflammation in a chest X-ray or CT scan—are often the main diagnostic features. In multi-grained scanning profilometric patterns in the form of a series of small windows slide across each image to give localized patches of data. These patches then passed through the gcForest model to govern them, to learn about the spatial relations and effective patterns characteristic of pneumonia. These windows, though smaller, can combined for effective learning of global and local patterns thus, the function of different regions within the image to the final diagnosis. Such sliding window procedure resembles the convolutional layers of a CNN, but in the forest-based model of gcForest, it allows the model to comprehend the fine as well as the less detailed features that are important for the understanding of medical images.

In multi-grained scanning, the input image is divided into smaller patches or windows. Let each window denote a small section of the image centered at position . For each window, a feature vector is extracted using gcForest:

8

where θ represents the parameters of the gcForest model used to process each window. Sliding windows are applied across the entire image to generate a set of local feature representations:

9

These local features are then aggregated, either by averaging, concatenation, or other pooling methods, to produce a global feature representation F_{multi−grain} that incorporates spatial dependencies and local patterns.

After extracting the described features to dismantle the gcForest, the last step is building a classification network that recognizes and sorts the images depending on the procured features. The high-dimensional visual features extracted by CNN and combined with the results of the gcForest to incorporate both local and global feature patterns. The combined features then passed to a dense classification layer that uses the CNN and gcForest benefits to improve differentiation of pneumonic related features. Thus, the improvements of the gcForest based feature representations incorporated into the characterization of the images fosters a more robust bandwidth for recognizing even latent structural or textural features such as lung opacities and inflammation which may be obscured by the standalone CNN architectures. This architecture could uniquely map the characterized image features into accurate classifications to ensure automatic and reliable diagnostic decision making.

The image classification model integrates features from both the CNN and gcForest outputs to classify the disease. Let F_CNN be the feature representation from the CNN and F_gcForest be the refined features from gcForest. The combined feature F_combined can be represented as:

10

where [] denotes the concatenation of the feature vectors. Figure 2 shows the architecture of proposed model.

Fig. 2.

Architecture of proposed model.

Training and fine-tuning

In training phase of the image-classification model is elaborate process that intended to enable the model to learn how to produce accurate descriptions of X-ray and CT scan images of pneumonia. The model is first trained on labelled set consisting of images of patients with pneumonia obtained from a labelled dataset shelving where the image database has been maintaining for the purpose of training the AI model is carefully selected from such images to enable the model to easily recognize patterns associated with pneumonia. The first step of this training process enables the model’s CNN part to identify primary visual elements, including lung opacity or structural anomalies. Furthermore, the pseudo labelled images created from the semi-supervised pseudo labelling technique are included into the training flow. With the use of pseudo-labels, the model exposed to a larger dataset thus improving on differentiation of variations and general model stability. This training configuration guarantees that the model is adept at explaining all intra and extra-radiological pneumonia characteristics necessary for developing pneumonia AI irrespective of the cases that it was not trained.

To enhance the overall performance, fine-tuning techniques applied to the final CNN-gcForest model. This step involves fine-tuning of the specified model on the X-ray and CT scan datasets to learn finer details. Additional tuning may involve learning rate adjustment, convolutional layer dropout rates, and features of the CNN and gcForest combinations. Additionally, certain hyperparameters related to datasets tuned because X-rays and CT scans have different visual appearances and, therefore, may require different approaches to feature extraction. As the results of tuning process demonstrate, the model learned to fine-tune and identify pneumonia related patterns in both constructed datasets. The fine-tuning is very vital especially to make sure that the new integrated model of CNN-gcForest gives equal and correct performance no matter the modality of the image used.

Evaluation and analysis

For metric assessment, BLEU, CIDEr, ROUGE, and METEOR used in performance evaluation of the model. For instance, BLEU overlap of n-gram, CIDEr likely use term frequent to determine the relevance of specific medical terms. ROUGE measures the precision of the model and METEOR defines the recall of the model. Combined, these measurements provide a quantitative evaluation of how well the model fits the real world and its capacity to identify important attributes of the pneumonia images.

Given an image I, let F_combined represent the combined feature vector obtained from both the CNN and gcForest, and Y = {y₁,y₂,…,y_T} be the sequence of words in the ground-truth label.

The model trained by minimizing the cross-entropy loss L, calculated as:

11

where is the ground- truth word at time step , represents the sequence of words predicated so far and is the probability assigned by the moel to the correct word , given the combined feature vector and previously generated words.

Additionally, to provide the overview of the influence of the pseudo-labelled data, the comparison of the results of the model trained on labelled data only and the model trained on the combination of labelled and pseudo-labelled data provided. This analysis demonstrates the proof of concept of the semi-supervised approach hence displaying an indication of how much actual pseudo-labelled data boost the model’s stability and generalization. The performance differences illustrated in these two setups give the quantitative measure of the benefit pseudo-labelling gives in cases where labelled data is hard to come by, for instance in medical imaging. When fine-tuning, the CNN-gcForest model is adjusted to seek more specific features of pneumonia in both the collected datasets. This involves adjusting the model’s parameters θ by minimizing the loss function using backpropagation with a learning rate η. The weight update for each parameter θ_i is computed as:

12

where is the gradient of the loss function with respect to parameter and is the learning rate controlling the step size for updating the parameters. The qualitative assessment is critical since the primary focus of the proposed model is to use computationally derived attributes to reflect the true underlying medical reality.

Algorithm: CNN-enhanced cascade forest (CE-Cascade)

Novelty of the work

The CNN-Enhanced Cascade Forest (CE-Cascade) model represents a significant advancement in automated pneumonia diagnosis by integrating the strengths of deep learning and ensemble methods. Unlike traditional approaches, this model combines the feature extraction capabilities of ResNet with a novel cascade forest architecture, creating a hybrid framework that effectively captures both local and global patterns in medical images. This integration ensures robust hierarchical feature representation and enables precise classification, setting the model apart from existing methodologies. A key innovation of this work is the creation of a novel dataset comprising 4578 paired X-ray and CT images collected from the same patients. This dataset spans multiple clinically relevant categories, including bacterial, viral, fungal, and general pneumonia, as well as normal cases. The use of paired data addresses a critical gap in existing research by enabling cross-modality learning, where the model can identify shared and complementary features across X-ray and CT images. This approach enhances diagnostic accuracy and ensures consistency in feature representation. The model adopts a multi-class classification framework, moving beyond the traditional binary classification of pneumonia versus non-pneumonia. By distinguishing between different types of pneumonia and normal cases, the CE-Cascade model provides more granular and clinically useful insights, improving its diagnostic utility in diverse medical scenarios. Additionally, the incorporation of semi-supervised learning with pseudo-labelling reduces the reliance on large manually annotated datasets. This innovative approach leverages unlabelled data effectively, expanding the dataset and enhancing the model’s ability to generalize across varied imaging conditions. Another novel aspect of this work lies in the hierarchical and multi-scale feature analysis performed by the cascade forest component. By dynamically refining feature representations at multiple levels, the model captures subtle and complex patterns critical for accurate pneumonia diagnosis. This hierarchical analysis is particularly advantageous for medical imaging, where multi-scale features often play a vital role in identifying disease-specific characteristics.

Results and discussions

The Proposed Model experiment carried out in Google Colab on a machine with an Intel^® Core™ i7-12650 H Processor (24 M Cache, up to 4.70 GHz) and 6GB RAM. This configuration was selected as to provide reasonable computational power and RAM for model training and testing through the notebook environment. Despite having a limited RAM capacity, the semi-supervised approach, usage of the gcForest that enhances the computational runtime, meanings that the model was able to function optimally without using plenty of resources. This representation shows the possibility of the model, as the computing requirements described should not be an issue for systems with similar or a step up in complexity from what is common in health care and research institutions. In the proposed work, the working principle of the model based on the use of CNNs and gcForest in a semi-supervised learning mode to classify pneumonia from medical images. This approach initiated through CNNs used as feature extractors to transform the raw X-ray and CT scan images into high dimensional visual feature space. These extracted features are important shapes and roughness of images including lung opacities and other tissue structure abnormalities that are associated with pneumonia. The output acquired from CNN is then passed to the gcForest where another level of anatomizing feature learning takes place through the employing the multi-grained scanning strategy. This mechanism utilizes the sliding windows to contextualize the image at different resolutions in order to capture both local and global features, which are very important in detecting features related to pneumonia.

To overcome the deficiencies of small-labelled data, the data is augmented using pseudo-labelling method. Pseudo-labelling technique helps the model to predict “pseudo labels” to the images that are initially unlabelled thus enhancing the labels of new data hence aiding in training. The process of reiterating this labelling makes the model more adaptive in learning the new data and we do not require many iterations or labelling to done manually. Each pseudo labelled image makes Infinity contribution towards the feature learning of pneumonia, thus helps in enriching the pseudo labelled training set, and hence, improves the performances. The proposed model integrates CNN-based feature extraction, gcForest multi-scale features extraction and, providing a more extensive solution for automatic image classification in medical applications. This setup not only enhances diagnostic performance but also provides human-interpretable results that facilitate clinical decisions, making it a solution to the current diagnostic limitations in reliable pneumonia diagnosis.

Table 2; Fig. 3 displays results of image classification on labelled-data based on medical X-ray and CT Scan images. For X-ray images, the model records high BLEU scores for all levels right from BLEU-1of 0.92 to BLEU-4 of 0.78, again suggesting that the model captures important descriptive features consistently. CIDEr, which focuses on idea similarity, is also high with a rate of 0.88 showing high correspondence with the reference. Overlapping phrases and semantic similarity of the annotated text are measured using ROUGE and METEOR at 0.83 and 0.8 respectively, vindicating the model’s stability. Precision in X-rays is 0.94 with a recall of 0.91 and an F1 score of 0.92 demonstrating a high level of the algorithms ability to generate accurate classification. Concerning CT Scan images, performance is relatively good but still lower to that of the X-rays. BLEU-1 stands at 0.9 in the beginning, declining to BLEU-4 of 0.76, which shows a drop in the comparison of multiple word phrases. Evaluation metrics: CIDEr − 0.85; ROUGE − 0.81; thus, there is a slightly lower value of semantic similarity. METEOR is 0.78, whereas precision is 0.92, recall 0.89, and F1 score 0.9 to retain the quality of the image classification. In general, the model’s performance on labelled instances for both X-ray and CT images is almost equal, with a slightly improved value of metrics on the X-ray imagery than CT.

Table 2.

Image classification performance on labelled-data.

Metric	X-ray	CT scan
BLEU-1	0.92	0.9
BLEU-2	0.87	0.85
BLEU-3	0.82	0.8
BLEU-4	0.78	0.76
CIDEr	0.88	0.85
ROUGE	0.83	0.81
METEOR	0.8	0.78
Precision	0.94	0.92
Recall	0.91	0.89
F1 Score	0.92	0.9

Fig. 3.

Image classification performance on labelled data.

Table 3; Fig. 4 shows image classification performance in terms of accuracy, precision, recall, F1 score, and BLEU when training with labelled data as well as with the pseudo-labelled data of both X-ray and CT Scan images dataset. Pseudo-labelled data appears to enhance accuracy, richness and relevance compared to only labelled data, meaning that the improvement that comes with enriched training data is ideal. For X-ray images, the model also produces high BLEU scores, with BLEU – 1 at 0.96 and BLEU – 4 at 0.84 proving that the model is more accurate even in single word accuracies not to mention multi-word sequences. The precision, recall, and the F1 score for X-rays also rises, to 0.96, 0.93, and 0.94 respectively, which means the classification generated from this system are highly precise and relevant. Enhancements can also be observed in performance for the CT scan images. BLEU-1 rises to 0.94, and BLEU-4 to 0.82, proving better match of multi-word phrases in comparisons with labelled only data. CIDEr is 0.89 and ROUGE is 0.85, which testifies increased semantic and structural similarity. The recall for METEOR is slightly lower than that for X-rays at 0.83, but phrase matching is better. The precision and recall measures of CT images are 0.94 and 0.91, which indicates that the overall F1 score is 0.92, implying that there is high accuracy as well as efficient recall. Hence, pseudo-labelled data improves the performance of the models for both X-ray and CT scans more where X-ray scans obtain slightly higher gain.

Table 3.

Image classification performance with labelled and pseudo-labelled data.

Metric	X-ray	CT scan
BLEU-1	0.96	0.94
BLEU-2	0.92	0.9
BLEU-3	0.88	0.86
BLEU-4	0.84	0.82
CIDEr	0.92	0.89
ROUGE	0.88	0.85
METEOR	0.85	0.83
Precision	0.96	0.94
Recall	0.93	0.91
F1 Score	0.94	0.92

Fig. 4.

Image classification performance with labelled and pseudo-labelled data.

Table 4; Fig. 5 shows the comparative of CIDEr, METEOR and BLEU scores rises when pseudo-labelled data is combined with labelled data to that of when only labelled data is used. Every figure shows the respective improvement, which underpins the advantages of using pseudo-labelled data. From the BLEU analysis, it is evident that we have the highest gains where BLEU-1 has risen by 0.04, BLEU-2 by 0.05, BLEU-3 by 0.06 and BLEU-4 by 0.06. These gains also accentuate the enhancement of the model in the comprehension of the unique word as well as the multi-word units. The numbers for ROUGE and METEOR both go up by 0.05 that is the improvement for the structural and compatible semantic relevance to the ground truth in terms of phrases and key content. However, all three measures, precision, recall, and F1 score increase by 0.02, indicating that by including the pseudo-labelled data, it is possible to classify the disease that are not only more accurate, but also more comprehensive, in terms of including as many relevant details as possible.

Table 4.

Comparative analysis between labelled-only and pseudo-labelled data.

Metric	Improvement
BLEU-1	0.04
BLEU-2	0.05
BLEU-3	0.06
BLEU-4	0.06
CIDEr	0.04
ROUGE	0.05
METEOR	0.05
Precision	0.02
Recall	0.02
F1 Score	0.02

Fig. 5.

Comparative analysis of metric improvement with pseudo-labelled data.

Table 5; Fig. 6 displays the results of hyperparameter tuning on image classification performance metrics, comparing two sets of hyperparameter values. These parameters as depicted above show a lower learning rate set as 0.0005, dropout as 0.3, and a bigger batch size 64 in Value 2 in contrast to Value 1 that had a higher learning rate set at 0.001, dropout 0.2, and a minor batch size of 32. For BLEU-4, the Value 2 rate is 0.84, which is higher than with Value 1, with 0.82; thus, the fine-tuned hyperparameters increase the model’s performance in producing accurate multi-word phrases. Comparison of the results shows that METEOR and ROUGE speaking scores vary in the same manner. Both structural and semantic quality of the sentences in Value 2 is better since ROUGE increases from 0.83 to 0.85 and METEOR from 0.8 to 0.82. Also, the precision, recall, F1 score enhanced when using hyperparameters tuned in Value 2 with performance perusing 0.94, 0.91 and F1-score equal to 0.92 from 0.92, 0.89 and 0.9 in Value 1. These results indicate that adoption of a lower learning rate, somewhat higher dropping out rate, and larger batch size or Value 2 yields superior performances since in terms of accuracy the model manages to capture a variety of relevant features.

Table 5.

Hyperparameter tuning results (learning rate, dropout, batch size).

Hyperparameter	Value 1	Value 2
Learning rate	0.001	0.0005
Dropout	0.2	0.3
Batch size	32	64
BLEU-4	0.82	0.84
CIDEr	0.87	0.89
ROUGE	0.83	0.85
METEOR	0.8	0.82
Precision	0.92	0.94
Recall	0.89	0.91
F1 score	0.9	0.92

Fig. 6.

Hyperparameter tuning results.

The comparative assessment of the overall accuracy of all architectural models for both X-ray and CT Scan images is shown in Table 6; Fig. 7. In order to have a comprehensive and impartial comparative analysis, several baseline models were chosen to capture popular deep learning architectures in medical image classification. I have chosen CNN-RNN, ResNet + LSTM, EfficientNet + LSTM, DenseNet with Attention, VGG + LSTM, MobileNet + GRU, Inception + RNN, ResNeXt + Transformer, and SE-ResNet + Transformer. These models were selected to represent an extensive variety of architecture such as convolutional, recurrent, attention-based and transformer-driven models which are commonly published in the most recent pneumonia and chest imaging literature. Training and evaluation of all the baseline models were done under the same experimental conditions to be fair and reproducible. Namely, the identical patient-level data divisions (training, validation, and testing), pre-processing and augmentation, and evaluation measures were used with all models. The same validation set was used to hyperparameter tune each baseline, and the same early stopping criteria was used to ensure that an overfit was avoided. Moreover, all experiments were trained on the same computer and with the same training parameters, such as batch size, optimizer parameters, and the number of training epochs wherever possible. This controlled assessment scheme is used to make sure that the observed differences in performance between the proposed CNN Cascade Forest model and the employed baseline methods are credited to the difference in the architectures and methods used and not experimental bias. Semi-supervised learning integrated with CNN, gcForest show the best performance in distinguishing X-Rays with an accuracy of 98.80%, the CT scan with an accuracy of 98.92%, and an overall accuracy of 98.86% which is higher than any model proposed. The first proposed model of CNN + RNN has a lower overall accuracy of 94.35% while its X-ray and CT Scan accuracies are 94.50% and 94.20%, respectively. The performance of the ResNet + LSTM model increases the baseline by achieving an accuracy of 95.90% The Even more noteworthy is that the EfficientNet + LSTM achieves even better results within an overall accuracy of 96.60% % which in turn underlines the generalization capability of the proposed models. DenseNet with Attention achieves a measure of 0.97,15 that is still significantly higher than any other model but again slightly lower than the proposed model. Similar to SE-ResNet + Transformer, the model achieved an overall accuracy of 97.55% with a nearly equal distribution between accuracy on X-ray (97.50%) and CT scan (97.60%) images. The complex image data are mastered relatively poorly by these architectures, which can be observed from rather low accuracies ranging from 92.60 for VGG + LSTM to 93.60 for MobileNet + GRU. Concisely, anticipated from the prior discussion, the Proposed Model yields the least error on both the types of images and thus it exhibits greater capability to learn subtle components of an image and generalize them without significant deviation from the baseline, which makes it ideal for precise medical image analysis.

Table 6.

Overall model performance comparison (accuracy).

Model	X-ray accuracy	CT scan accuracy	Overall accuracy
Proposed model (CE-Cascade)	98.80%	98.92%	98.86%
CNN + RNN (Baseline)29	94.50%	94.20%	94.35%
ResNet + LSTM30	95.80%	96.00%	95.90%
EfficientNet + LSTM30	96.50%	96.70%	96.60%
DenseNet + Attention31	97.10%	97.20%	97.15%
VGG + LSTM32	92.40%	92.80%	92.60%
MobileNet + GRU33	93.50%	93.70%	93.60%
Inception + RNN34	94.30%	94.10%	94.20%
ResNeXt + Transformer35	96.00%	96.20%	96.10%
SE-ResNet + Transformer35	97.50%	97.60%	97.55%

Fig. 7.

Overall model performance comparison (accuracy).

Table 7; Fig. 8 presents the performance of different models in image classification by expressing them through BLEU- 1, BLEU- 2, BLEU- 3, and BLEU-4 scores. The Proposed Model offers highly efficient results, with a BLEU score of 0.96 for the BLEU-1, BLEU-2 of 0.92, BLEU-3 of 0.88, and BLEU-4 of 0.84 suggesting it has a reliable ability of conception detailed. It is statistically evident that the proposed modification to the baseline CNN + RNN has a harvest of around 10% worse BLEU scores with bleu-1 of 0.85 and bleu-4 of 0.68 pointing at uninspiring long phrase match accuracy. When compared to the baseline result, ResNet + LSTM performs even better and achieves BLEU-1 of 0.89 and BLEU-4 of 0.76, which indicates better phrase recognition. The scores improve again by adding EfficientNet + LSTM, where, BLEU-1 = 0.91 and BLEU-4 = 0.79. DenseNet with Attention is also satisfactory from the restricted low BLEU score of 0.93 at BLEU-1 to moderate low score of 0.81 at BLEU-4 that implies good phrase matching. SE-ResNet + Transformer also produces good BLEU scores with 0.95 for BLEU-1 and 0.82 for BLEU-4 model while offering rivalry to the proposed model. VGG + LSTM and MobileNet + GRU compared to the previous models have comparatively low BLEU score, 0.7 on VGG + LSTM and 0.71 on MobileNet + GRU, points toward weaker ability of accurately generating multi-word phrases.

Table 7.

Model performance comparison (BLEU scores).

Model	BLEU-1	BLEU-2	BLEU-3	BLEU-4
Proposed model	0.96	0.92	0.88	0.84
CNN + RNN (baseline)	0.85	0.78	0.72	0.68
ResNet + LSTM	0.89	0.84	0.8	0.76
EfficientNet + LSTM	0.91	0.86	0.82	0.79
DenseNet + attention	0.93	0.88	0.85	0.81
VGG + LSTM	0.82	0.77	0.73	0.7
MobileNet + GRU	0.84	0.79	0.74	0.71
Inception + RNN	0.87	0.82	0.78	0.74
ResNeXt + transformer	0.92	0.87	0.83	0.8
SE-ResNet + transformer	0.95	0.9	0.86	0.82

Fig. 8.

Model performance comparison (BLEU scores).

Table 8; Fig. 9 show the comparison of CIDEr, ROUGE and METEOR score between models to measure the improvement the models have made in bringing accuracy. The Proposed Model is the best performing model with a CIDEr score of 0.92, ROUGE of 0.88, and METEOR of 0.85, proving the model’s semantic relevance and structural similarity of generated labels to the reference labels. Baseline on CNN-RNN has significantly reduced scores: CIDEr: 0.74; ROUGE: 0.72; METEOR: 0.68, meaning that the model has a poor ability to identify the details of the images. The integration of EfficientNet + LSTM yields better results with 0.85 (CIDEr), 0.82 (ROUGE), and 0.78 (METEOR). The modified DenseNet with Attention performs even better and attains 0.89 in CIDEr, 0.85 in ROUGE and 0.82 in METEOR, which proves the model’s excellent content and structural matching ability. The SE-ResNet + Transformer model also has decent results of CIDEr 0.9, ROUGE 0.86, and METEOR 0.83, a little inferior to overall Proposed Model. Hence, the less effective models are VGG + LSTM and MobileNet + GRU that have the following scores: CIDEr – 0.7, ROUGE – 0.69, METEOR – 0.65, respectively.

Table 8.

Model performance comparison (CIDEr, ROUGE, METEOR).

Model	CIDEr	ROUGE	METEOR
Proposed model	0.92	0.88	0.85
CNN + RNN (baseline)	0.74	0.72	0.68
ResNet + LSTM	0.81	0.78	0.74
EfficientNet + LSTM	0.85	0.82	0.78
DenseNet + attention	0.89	0.85	0.82
VGG + LSTM	0.7	0.69	0.65
MobileNet + GRU	0.73	0.71	0.67
Inception + RNN	0.8	0.77	0.73
ResNeXt + transformer	0.87	0.83	0.8
SE-ResNet + transformer	0.9	0.86	0.83

Fig. 9.

Model performance comparison (CIDEr, ROUGE, METEOR).

Table 9; Fig. 10 focuses on comparison of the model generalization by assessing the cross-dataset performance, or in other words cross-validation accuracy when models trained from X-Ray images are tested on CT images and vice versa. Average of above two cross-dataset accuracies are computed to obtain the Generalization Score which depicts the generality of created models to predict the medical images of various types. The Proposed Model has the highest generalization score of 98.40%; for X-ray trained and CT tested, and 98.30% for CT trained and X-ray tested. It can also be seen in this score that it is highly reliable for all kinds of images owing to its good generalized performance across different datasets. The baseline CNN + RNN model has three-time lower Generalization Score of 90.75%, which means its poor ability to generalize to other datasets. ResNet + LSTM and EfficientNet + LSTM are better generalization capability compared with this, which achieves rate of 92.65% and 93.90%, respectively. The models DenseNet with Attention and SE-ResNet + Transformer return the highest Generalization Score, 95.20%, and 96.90%, respectively. These models show relatively good generalization, more specifically, much closer to the Proposed Model is SE-ResNet + Transformer. The generalization is worse: VGG + LSTM and MobileNet + GRU based models scored 90.10% and 91.10% indicating poorer transfer between images in the two sets. In summary, the Proposed Model is characterized by excellent generalization performance that can be beneficial in the shared medical image datasets applications. First, it achieves high accuracy in all five-target databases, which underlines its applicability to different image modalities.

Table 9.

Model generalization comparison (cross-dataset performance).

Model	X-ray trained, CT tested (accuracy)	CT trained, X-ray tested (accuracy)	Generalization score
Proposed model	98.50%	98.30%	98.40%
CNN + RNN (baseline)	91.00%	90.50%	90.75%
ResNet + LSTM	92.50%	92.80%	92.65%
EfficientNet + LSTM	94.00%	93.80%	93.90%
DenseNet + attention	95.30%	95.10%	95.20%
VGG + LSTM	90.30%	89.90%	90.10%
MobileNet + GRU	91.20%	91.00%	91.10%
Inception + RNN	92.00%	91.50%	91.75%
ResNeXt + transformer	94.80%	94.50%	94.65%
SE-ResNet + transformer	97.00%	96.80%	96.90%

Fig. 10.

Model generalization comparison (cross-dataset performance).

Tables 10 and Fig. 11 shows the comparison of computational efficiency between different models in terms of training time per epoch and the number of parameters in millions. These are measures of the complexity of each model and the time taken to train them, factors important in the assessment of resource demands and capacity to scale. With regard to computational efficiency, the Proposed Model has fast training time per epoch at 10’, 45 million parameters. It is observed that the training time per epoch is much shorter than all the other networks and though it contains a high number of parameters. Baseline CNN + RNN takes 12 min per epoch and has twenty million parameters suggesting it’s less complex in terms of layers but takes slightly more time. MobileNet + GRU is also very simplistic with 10 million parameters of the model and took approximately 11 min to train. Using MobileNet + GRU has a great advantage in the sense that it has the lowest number of parameters in it and the training is also faster than the other two but may not be as efficient in complex tasks. The next architectures – ResNet + LSTM and EfficientNet + LSTM look physically more complicated; it takes 15 and 18 min per epoch on the train phase and both models have 25 and 30 million parameters. SE-ResNet + Transformer and DenseNet with Attention are the two models with the highest complexity, as the training time reached 20 and 19 min, and the number of parameters is 40 M and 38 M, correspondingly. In conclusion, a number of aspects favour the Proposed Model as a more adequate representation, owing to a reasonable density of parameters and, at the same time, its sound training time. Given in a large number of parameters to be estimated, its short training time per epoch confirmed the model’s efficient architecture suitable for large-scale use.

Table 10.

Computational efficiency comparison (training time per epoch and parameters).

Model	Training time per epoch (min)	Number of parameters (million)
Proposed model	10	45
CNN + RNN (baseline)	12	20
ResNet + LSTM	15	25
EfficientNet + LSTM	18	30
DenseNet + attention	20	40
VGG + LSTM	14	15
MobileNet + GRU	11	10
Inception + RNN	16	22
ResNeXt + transformer	17	35
SE-ResNet + transformer	19	38

Fig. 11.

Computational efficiency comparison.

In order to test the strength of the suggested framework in the setting with limited labelled data, the model performance was examined with a comparison between fully supervised training (only with labelled samples) and the proposed semi-supervised learning strategy that utilizes pseudo-labelled data. The contribution of semi-supervised learning in the scenario where the volume of the labelled medical data is low is directly quantified in this comparison. Training the model with the help of labelled data only gave it a better baseline performance but the addition of the pseudo-labelled samples gave the same robust classification and semantic consistency in both chest X-ray and CT modalities. The pseudo-labelled data were included, and the gains were measurable in various evaluation measures and included BLEU, CIDEr, ROUGE, METEOR, precision, recall, and F1-score. The overall F1-score was raised by about 2%, and the semantic similarity scores (BLEU-3 and BLEU-4) were raised up to 6%, as it demonstrates greater pneumonia subtype discrimination in the conditions of limited annotation. All these enhancements can prove that the semi-supervised framework is highly effective to employ unlabelled data to improve the feature descriptions and minimize manual labelling.

Conclusion and future work

The proposed CNN-Enhanced Cascade Forest (CE-Cascade) model presents a highly effective and innovative framework for automated pneumonia diagnosis. By integrating ResNet for feature extraction with a novel cascade forest architecture, the model achieves superior hierarchical representation and classification capabilities. The creation of a paired dataset of 4578 X-ray and CT images from the same patients addresses a significant limitation in existing datasets, enabling cross-modality learning and improving diagnostic accuracy. The incorporation of a multi-class classification framework expands the model’s utility to differentiate between bacterial, viral, fungal, and general pneumonia types, as well as healthy cases. Furthermore, the use of semi-supervised learning with pseudo-labelling reduces the reliance on extensive manual annotations, enhancing the model’s generalization and scalability. With an impressive accuracy of 98.86%, the CE-Cascade model sets a new benchmark for performance in medical image analysis, providing a robust, scalable, and clinically relevant tool for pneumonia diagnosis. While the CE-Cascade model demonstrates exceptional performance, there are opportunities to further enhance its capabilities. Future work could focus on extending the framework to additional medical imaging modalities, such as MRI and ultrasound, broadening its applicability to a wider range of diseases. Incorporating active learning techniques could also improve the model’s efficiency by enabling selective data labelling, optimizing training resources. Although the proposed CNN-Enhanced Cascade Forest model can achieve good results when it comes to multi-class pneumonia, the explicit post-hoc interpretability and explainability methods were not considered in the given research. The main focus of the work was on how to enhance the accuracy and robustness of the classification in case of limited labelled data but with the help of a semi-supervised learning strategy. As a key research direction in the future, the methods of explainability like gradient-based class activation mapping (e.g., Grad-CAM), the methods of feature attribution, and the tree-based feature importance analysis will be combined to determine clinically relevant areas and features that drive the classification of various pneumonia subtypes. The implementation of such interpretability mechanisms will increase clinical transparency, contribute to radiological decision-making, and increase confidence in automated diagnostic systems.

Author contributions

P.M. wrote the main manuscript text and T.Y., R.K. prepared Figs. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 and 11. All authors reviewed the manuscript.

Data availability

The datasets used and analysed during the current study available from kaggle repository [https://www.kaggle.com/datasets/andrewmvd/covid19-ct-scans], https://www.kaggle.com/datasets/paultimothymooney/chest-xray-pneumonia.

Declarations

Competing interests

The authors declare no competing interests.

Consent to publish

All the authors gave permission to Consent to publish.