Multi-scale covariance filtering for edge-preserving medical image fusion

Abstract

Multi-modal medical image integration is necessary to improve the accuracy of diagnostic results by integrating complementary information between modalities such as Computed Tomography and Magnetic Resonance Imaging. Nevertheless, current methods of fusion fail to maintain the finer details of the anatomy, balance of contrast and eliminate artifacts along structural boundaries. In order to solve these issues, this paper will present a new fusion framework that is based on Covariance Filtering. It uses a two-scale image decomposition to extract the base and detail layers and proceeds with saliency-guided weight map creation and covariance-based refinement for spatial consistency and edge preservation. Assessments of multi-modal brain imaging data show that the suggested method is better than the state-of-the-art methods in terms of Entropy, Mutual Information, Average Gradient and QAB/F. All the research findings support its efficacy in maintaining anatomical boundaries, improving visual clarity, and offering a computationally efficient solution. This framework will be expanded to 3D image fusion in future work and dedicated mechanisms used in regions detection to enhance precision in diagnostic processes.

Introduction

Medical image fusion is now a necessary technology in modern healthcare, which integrates complementary information of different imaging modalities to enhance diagnostic accuracy, maximize treatment plans, and aid decision-making. Magnetic Resonance Imaging (MRI), Computed Tomography (CT), Positron Emission Tomography (PET) and Single-Photon Emission Computed Tomography (SPECT) are some of the most commonly used techniques which offer different anatomical or functional information. The restrictions of these modalities including the limited soft-tissue contrast of CT and the comparatively low spatial resolution of PET mean that a combination of multi-modal data is required to produce a more comprehensive and informative view of pathological states is needed^1,2. This hybrid process has special significance in medical applications that are more complicated, such as oncology, neurology examination, and surgical navigation, wherein detection, localization, and characterization of abnormalities are highly important and require accuracy in all cases^3,4.

Besides multi-modal fusion, the more recent developments have been directed to the enhancement of automated analysis of medical images with the help of efficient segmentation frameworks. Uncertainty-aware learning methods have been suggested to increase the accuracy of semi-supervised medical image segmentation when using less annotated data⁵, and methods following dual diversity learning and pseudo-label correction have also been proposed to further improve segmentation quality in under-annotation conditions⁶. Likewise, fusion of diversity features together with dual consistency learning has been studied to enhance feature representation and segmentation performance the field of medical imaging-based learning images and tasks to be learned are described as features, often termed feature maps or feature vectors, that are regularly accessed and evaluated throughout the learning process of the task⁷. These advancements emphasize the rising relevance of image analysis methods in the retrieval of meaningful information in medical images.

Medical image fusion techniques evolved over time from simple pixel-based and feature-based methods to more advanced Deep Learning (DL) methods. Before, wavelet transformations, Principal Component Analysis and sparse representation were used to extract features and linear combination rules by hand. Although these methods were relatively successful, they were prone to missing important structural information and adding artifacts to the fused images^8,9. With the introduction of Generative Adversarial Networks (GANs), a data-driven view of image fusion has become available, allowing to preserve textures, edges, and critical functional information in a more successful way. As an example, conditional GANs (cGANs) and Laplacian pyramid-based GAN models have been highly effective in producing high-resolution fused images without compromising the modality-specific information by adversarial training processes¹⁰. These combinations can enhance the quality of fusion by using complementary decomposition approaches with adaptive fusion mechanisms, which are complementary to each other, thus providing a better fusion quality, which utilizes their combined benefits^11,12.

Despite these advancements, significant issues still exist to address. Multi-modal datasets usually vary in terms of the degree of resolution, noise, and alignment, implying that they require robust pre-processing and registration systems¹³. A significant challenge remains with regards to the trade-off between computing efficiency and fusion quality particularly in cases where real time computation is required. Techniques such as Coati optimization strategy or multiscale adaptive transformers, attempt to optimize fusion parameters, with less computing power, as proposed in¹⁴. To illustrate, the outputs produced by self-attention models using transformers are more structurally sound since it finds long-range dependencies between modalities.

The resulting fused images is further complicated by the fact that there is no collective set of criteria to evaluate the quantitative accuracy and significance of the images. Measures such as Structural similarity Index (SSI), Mutual Information (MI) and Edge Preservation Index (EPI) are often used but are often not correlated with the subjective rating of radiologists¹⁵. Recent research highlights the need of task-related assessment systems based on domain knowledge, including tumor boundary definition in cancer or vessels in neurology¹⁶.

Explainable Artificial Intelligence, edge computing, and federated learning must collaboratively address challenges related to scalability, privacy, and interpretability in the future of medical image fusion. Lightweight GAN architectures and adaptive fusion rules optimized using swarm intelligence algorithms, such as the Chameleon Swarm Algorithm, exemplify emerging technologies poised to enhance real-time processing capabilities¹⁷. As customized medicine increasingly depends on multi-modal imaging, enhancing fusion techniques will be crucial for developing accurate diagnoses and delivering effective treatments.

This paper presents a new multi-modal fusion method based on the Laplacian Pyramid to meet the ongoing need for efficient and high-quality image fusion solutions. The suggested method is meant to preserve fine anatomical details better and make the contrast more consistent in the fused outcomes. This approach uses a multi-level Laplacian Pyramid decomposition instead of directed filtering, which is what most standard methods use. This collects image information at multiple spatial scales. Also, a Gaussian-based saliency detection method is used to make weight maps that are accurate and specific.

Contributions

The key contributions of this study are summarized as follows:

• The proposed novel image fusion framework based on Covariance Filtering, which is efficient to preserve small anatomical structures and high-frequency component in multi-modal fused images and local spatial agreement.

• A Gaussian based saliency detection mechanism was gained, which is applied on Laplacian filtered images to obtain adaptive and region-aware weight maps to emphasize diagnostically relevant anatomical and functional areas.

• Design of a region selective weight map generation under a max selection rule which outputs a weight map such that the dominance of the correct region at each pixel throughout the fusion process will yield a better visual quality.

This study aims to address current limitations in multi-modal medical image fusion by delivering a computationally efficient, detail-preserving, and applicable fusion framework, contributing to the ongoing advancement of precision diagnostic imaging.

Organization of paper

This paper is organized in the following way. Section “Related work” discusses the state of the art in multi-modal medical image fusion with emphasis laid on guided filtering, saliency-based, multi-resolution decomposition-based and covariance and adaptive filtering-based. Section “Material” is a detailed account of the dataset employed in this work, that is, paired CT and MRI images, their respective acquisition parameters and even structural attributes pertaining to fusion-based activities. Section “Method” provides the proposed methodology, such as two-scale image decomposition, Saliency-based weight map production, covariance filtering-based refinement and reconstruction to final images. Section “Results” contains qualitative and quantitative evaluation outcomes between the proposed approach and a set of state-of-the-art approaches with well-established image fusion measures and complexities ensuing with complexities. Section "Statistical results and discussion" gives a statistical treatment of the fusion output in terms of key distribution parameters and a Chi-square test to study the consistency of edge preservation. Section “Ablation study” demonstrates ablation study describing the role of each of the modules in the hybrid approach.Section “Discussion” is the report on experiments and a suggestion of possible research directions, such as generalization to 3D image fusion and the inclusion of enhanced region-aware attention methods and Section “Conclusion” gives the conclusion of the paper, outlines the implications and limitations of the proposed method.

Related work

Medical image fusion aims at integrating complementary information of various imaging modalities in a single image to enhance the accuracy in diagnosis and assist in the treatment. The past decade has seen numerous fusion methods being developed, such as spatial domain and transform domain ones, and newer methods based on the use of DL. This part presents a literature review of the work that has been conducted and the fusion frameworks involving the guided filtering, saliency detection and Laplacian pyramid decomposition, as well as the recent advances in the sphere.

Guided filtering and image statistics based fusion

The pioneering work in medical image fusion with the help of filtering methods was done by Li et al.¹⁸ who came up with a guided filtering-based system that decomposed images into base and detail layers at two scales. Their approach integrated all these layers through the use of weight maps based on saliency cues and local consistency rules. This method was used by Bavirisetti et al.¹⁹ and applied to medical imaging. They combined CT images and MRI images with the help of guided image filtering and statistical techniques preserving such important details as bone structures and soft tissues. Their work demonstrated that guided filtering can provide a smooth spatial weighting as well as retain few features in the image which is even more significant as it can be applied to jobs that require fusing images of various sources.

Saliency-based and pixel significance fusion techniques

Similar to guided filtering saliency-based fusion techniques have been the focus of attention in their capability to highlight areas that are perceptually significant. Bavirisetti et al.²⁰ proposed a two-scale image fusion scheme where visible and infrared images are combined, with saliency detection maps to direct the process of fusing detail layers. The technique was also related, and Kumar et al.²¹ applied it to enhance important features in both multi-spectral and multi-focus images, by employing Cross Bilateral Filtering (CBF). This was in a combination methodology of pixel significance. Kumar et al. had also previously applied the Discrete Cosine Harmonic Wavelet Transform (DCHWT) as well as pixel significance maps in fusing multi-focus images and multispectral images. This demonstrated the fact that localized significance information can be important in the creation of fusion results²².

Multi-resolution and laplacian pyramid-based fusion methods

Multi-resolution decomposition methods have been used in image fusion since long ago as they have the capability of retrieving the images at various scales. Li et al.²³ compared the various multi-resolution transforms, including the Laplacian pyramid, the discrete wavelet transform, and the contourlet transform. They discovered that Laplacian pyramid and variants are the most reasonable trade between computational and perceptual qualities. When applied to edge-aware fusion, Paris et al.²⁴ developed the Local Laplacian filter, which simultaneously applies a Laplacian pyramid structure on images to apply contrast and maintain edges.

Further to this, Zhu et al.²⁵ suggested a multi-modality medical image fusion technique based on phase congruency and local Laplacian energy in Non-Subsampled Contourlet Transform (NSCT) domain with an accent on the advantages of Laplacian-based energy descriptors in preserving the structure of the image. Kurban et al.²⁶ more recently, introduced a Gaussian of differences(GoD)-based fusion, which is an efficient computationally method of fusion images in general and preserves important structural aspects at the same time.

Recent advances in decomposition and saliency-driven fusion

Recent work has explored decomposition strategies enhanced by saliency-based weighting schemes. Song et al.²⁷proposed a dual-decomposed low-rank representation method (D2-LRR) for medical image fusion, reducing redundancy while preserving complementary features²⁸. designed a multi-exposure image fusion technique using structural weights derived from visual saliency maps. The fusion framework proposed by Zheng et al.²⁹ employ edge detection and salient feature extraction in which the structural information is boosted on the basis of gradient-based edge maps which are complemented by saliency-driven weighting. They are successful in enhancing the preservation of the anatomical outlines and the quality of perceptions in fused images. Simultaneously, the decision methods based on DL have received critical interest in the possibilities to represent the complex nonlinear interactions between modalities. Kaur et al.³⁰ proposed a deep neural network-based medical image fusion algorithm which depends on morphologically processed residuals. They used residual learning and morphological operations in their approach to boost feature representation, which allows better detail preservation and noise suppression in the fused result.

These recent developments highlight the growing trend toward combining structural feature extraction and learning-based approaches to improve fusion performance, particularly in terms of edge preservation, feature enhancement, and robustness across diverse medical imaging modalities.

Toward covariance filtering for image fusion

Guided filtering as well as Laplacian-Pyramid-based methods have delivered results that are practically useful but their use of external guidance images combined with their tendency to produce blocky artifacts at boundaries remain significant weaknesses. However, Covariance filtering compares to direct modeling of local statistical inter-dependencies between an image and a weight map³¹. Niehsen et al.²⁴ in a previous study proved the usefulness of covariance-intersection filtering in data fusion, thus establishing a covariance-based local-consistency systems. Further investigations affirm the power of covariance-based modeling to adaptive filtering and signal-processing tasks. Chang et al.³², used adaptive covariance filter to increase the signal retrieval level, thus demonstrating superior robustness and noise resistance. Similarly, modeling methods and covariance estimation have demonstrated usefulness in the reconstruction and filtering of data as well as enhancement of stability of second order statistical dependencies in data modeling and reconstruction processes³³.

Saliency-based approaches effectively highlight important regions but often fail to maintain spatial continuity, leading to discontinuous or noisy weight maps.Furthermore, multi-scale decomposition methods, although effective in separating structural components, do not inherently guarantee optimal weight refinement, which can limit detail preservation and introduce artifacts in the fused image³⁴.

To further justify the effectiveness of covariance filtering over guided filtering, a visual comparison of the refined weight maps is presented in Fig. 1. As observed, the weight maps obtained using guided filtering exhibit noticeable intensity leakage and slight halo artifacts around structural boundaries, particularly in regions with high contrast variation between modalities. This is primarily due to the dependence of guided filtering on the guidance image, which may not be fully consistent with the target modality in multi-modal fusion scenarios.

Fig. 1.

Comparison between weight map refinement by covariance filtering and guided filtering. The upper row demonstrates refined base (, ) and detail (, ) weight maps as output of covariance filtering and the bottom as output of guided filtering. Covariance filtering aligns better with the spatial consistency of gradients of anatomical boundaries, and also in the homogeneous regions, covariance filtering generates a smoother distribution of weights. By comparison, guided filtering adds minor halo effects and leakage of intensity along edges because of its reliance on an external guidance image, especially when there are significant variations in the intensity distributions in the multi-modal cases.

The covariance filtering-based refinement on the other hand gives a smoother and more spatially continuous weight map but maintains sharp edges of anatomical boundaries. The refined base weight maps ( and ) display better homogeneity in the homogeneous areas whereas the detail weight maps ( and ) display better outlines of the edges without formation of artifacts. This observation helps validate the fact that covariance has been shown to produce more accurate weight maps in accordance to natural image structures by using local statistical correlations instead of external cues.

Generally, the visual demonstrations show the clear superiority of covariance filtering in terms of the edge preservation, artifacts, and structural consistency, which makes it more appropriate to multi-modal medical image fusion.

On the basis of this, the proposed work incorporates a covariance filtering-based weight refinement system into a saliency-directed fusion system. This method preserve fine structural information, contrast and less local inconsistency artifacts that are typical in guided and Laplacian Pyramid based methods. The suggested approach is able to provide spatial consistency utilizing local covariance-based linear models, which have a simplicity in computation and better detail preservation.

Material

Dataset

The data set used in this study contains three pairs of multi-modal medical images, which were obtained using different modalities of brain imaging as shown in Fig. 2. Particularly, the dataset includes T1-weighted and T2-weighted MRI sequences, CT data, and post-contrast MRI data. These images span both cerebral anatomical structures and pathological areas, such as aberrant tissue intensities and lesion-affected areas which are studied in multi-modal neuroimaging studies. Images are all in grayscale, with a bit-depth of 8 bits, and spatial resolutions of between 256 to 464 pixels-per-unit. The given images are already aligned and visually clean enough. So, no additional preprocessing, including registration of image, denoising, or intensity normalization were considered before using the fusion algorithm. Precise characteristics of the reference images used in the experimental processes are presented in Table 1.

Fig. 2.

Sample source images from the multi-modal medical image datasets used in this study. Each pair represents images acquired from different imaging modalities, including CT and MRI (T1-weighted and T2-weighted scans of the brain). These images provide complementary diagnostic information, where MRI highlights soft tissue structures and CT emphasizes bone structures and dense anatomical features.

Table 1.

Details of the source images used in the multi-modal medical image fusion experiments. The table summarizes the characteristics of each image pair, including file format, spatial resolution (in pixels), bit depth, image type, and imaging modality. These multi-modal images comprise combinations of CT and MRI scans acquired under different imaging conditions, such as T1-weighted and T2-weighted MRI sequences and CT bone or non-contrast windows.

Image	Format	Dimensions (Pixels)	Bit Depth	Type	Modality
1(A)	TIF	464464	8	Grayscale	T2-weighted MRI
1(B)	TIF	464464	8	Grayscale	CT (Non-contrast)
2(A)	TIF	256256	8	Grayscale	T1-weighted MRI
2(B)	TIF	256256	8	Grayscale	CT (Bone window)
3(A)	TIF	256256	8	Grayscale	T2-weighted MRI
3(B)	TIF	256256	8	Grayscale	T1-weighted MRI

MRI T2-weighted 1(A) is an axial slice of the brain where cerebrospinal fluid (CSF) is visible with a high intensity of brightness. The white matter (high content of water in its myelin sheaths) also has a high signal intensity and the lateral and third ventricles are clearly distinguished by the high signal intensity. Gray matter, on the other hand, is darker than white matter, which makes it a good contrast in order to localize the structures like the basal ganglia (caudate nucleus and thalamus) and corpus callosum. Sections of the brainstem (midbrain and pons) and cerebellum are seen inferiorly and the optic nerves and orbital globes are seen as the slice nears the orbital level. This T2 contrast has specific application in the visualization of pathologies associated with fluid (e.g. edema or demyelination) and as a rich source image in fusion pipelines.

Coupled with such T2-weighted MRI slice is a CT scan of the same axial level (1(B)). Skeletal structures, which are of the skull such as the calvarium, skull sutures and orbital walls, are hyperdense (bright white) in this type of bony-window reconstruction. The intermediate attenuation of brain tissue and lower attenuation of CSF give rise to a dull gray-white complex which, however, gives an accentuated emphasis on structural boundaries and cortical folding patterns. The ventricles still appear hypodense (dark) spaces, which complement fluid-filled spaces. No acute hemorrhage or fractures are present and this provides a baseline Normative template on which any T2 based fluid contrasts can be merged to provide better image of both soft tissue and bone.

Another key source image is the T1-weighted MRI (2(A)). Here, CSF is hypointense (dark), and white matter appears brighter than gray matter. Such contrast accentuates cortical and subcortical anatomy: the caudate nucleus, putamen, and globus pallidus in the basal ganglia stand out against adjacent white matter; the midbrain section of the brainstem is visible centrally; and the orbital globes appear hyperintense (bright) in their vitreous component. Sulci and gyri are symmetrically folded, indicating no mass effect or cortical atrophy. This T1 contrast is instrumental for detailed anatomical assessment, especially when subsequently fused with CT data that excels at highlighting bone and calcifications.

Its paired CT counterpart (2(B)) is a non-contrast axial brain scan. Bone structures–such as the calvarium and facial bones–appear hyperdense (white), while gray and white matter attenuation differences are subtler. However, this CT window still delineates sulcal patterns and ventricular margins, and it provides clear views of orbital and sinus anatomy (e.g., frontal and ethmoid sinuses appear as low-density areas). There are no hemorrhages, calcifications, or midline shifts, offering a complementary soft-tissue backbone to the T1-weighted MRI’s fluid depiction. Together, these two modalities (T1 MRI + CT) support fusion methods that aim to enhance both soft-tissue contrast and bony detail.

T2-weighted post-contrast MRI (3(A)) is added in order to highlight fluid and possible edema of the left temporal lobe. Within ventricles, the CSF is highly bright, the lateral ventricles well-identified and the periventricular white matter may be evaluated in terms of subtle intensive hyperintensities. Corpus callosum and basal ganglia retain their signaling patterns and one can enjoy the initial signs of lesion margins or fluid build-up due to the improved contrast. This slice thus creates a detailed source image in terms of fusion methods that are advantageous in terms of highlighting of inflamed or edematous relationships as well as structural clarity of the CT image.

The T1-weighted post-contrast MRI (3(B)) has a negative effect on the CSF by making it dark, whereas the white matter exhibits enhanced intensity after gadolinium. The ventricles are well delimited and any deep white matter or along the corpus callosum enhancement is easily noticed. This modality highlights areas characterized by broken blood 2 barriers or inflammation and is a point of reference when fusing lesion-enhanced T2 configuration with CT-based bone delineation. Together, these images allow multi-scale fusion plans to address the vascular/permeability changes as well as the underlying landmarks of the bone in the body.

The dataset is publically available at “https://github.com/dawachyophel/medical-fusion/tree/main/MyDataset”.

Method

The multi-modal image fusion scheme is proposed which involves a two-scale image decomposition scheme and then weight map construction and refinement with the help of covariance filtering. The proposed method compares to the traditional methods that rely on guided filtering, where a covariance-based local linear model is used to compute spatially consistent weight maps, edge-preserving weight maps. The general process comprises three steps: two-scale decomposition, weight map construction and final image reconstruction depicted in Fig. 3.

Fig. 3.

Summary of the proposed multi-modal methodology of medical image fusion. The model has three key steps, a two scale image decomposition, a saliency-based weight map development, and covariance filtering-based weight refining and image reconstruction. The initial step is to decompose every source image into base and detail layers in order to distinguish coarse intensity data and high-frequency structural details. The further step involves creating saliency maps by applying Laplacian and Gaussian filters to detect perceptually important areas to create initial weight maps. The covariance filtering is then used to refine these weight maps to enhance spatial consistency and maintain anatomical edges. Lastly, the refined weights are created to combine the base and detail layers resulting in the final fused image that preserves complementary structural and functional data of the origin modalities.

The above three datasets, including SET-1, SET-2 and SET-3 are assessed on the proposed fusion framework, as outlined below. The datasets are each made up of registered multi-modal medical image pairs that contain complementary structural and intensity characteristics. Such datasets are utilized in the entire methodological pipeline which consists of two-scale decomposition, construction of saliency-based weight maps, construction of covariance filtering based refinement and reconstruction of final image.

Two-scale image decomposition

In order to successfully isolate the large-scale intensity structures and fine image details, the registered input images of each image, , are first broken down into a base layer and a detail layer. The base layer is purposely configured to store the coarse variations in intensity and smooth background area, whereas the detail layer is configured to store the high-frequency detail of the image, including edges, contours, and texture.

The base layer is calculated by averaging a filter Z on the source image:

1

Where the Z represents an averaging filter of size 31 x 31 and the represents a two dimensional convolution. It is effective when information with high frequencies is being suppressed and the low frequencies are retained during the outcome.

After obtaining the base layer, the detail layer corresponding to it, which is denoted as , is created by subtracting the base layer to the original image:

2

This step separates the high-frequency residues - such as edges, lines and texture elements - by eliminating the rough structures embodied in . As a result, every image is divided into complementary representations which can be processed separately in the fusion framework.

Weight map construction

In order to successfully control the process of a fusion, the weight maps are built so as to identify which of the source images made a contribution to the final result of the fusion at a given spatial position. This is achieved through two processes; firstly generation of the weight map, which is done using saliency detection, and finally refinement on the weight map using covariance filtering.

First, a saliency map of each image is calculated in order to determine the perceptual significance of every pixel in the source images. It starts by using Laplacian filter to highlight high-frequency data:

3

Here L is a 3 x 3 Laplacian kernel that is used to accentuate edges and fine structures by increasing rapid intensity changes. The resultant high-pass image H i reveals anatomical edges and fine textures.

A Gaussian filter of high-pass image, the value of which is absolute, is then smoothed with the help of a Gaussian filter, high-pass image, which is used to suppress noise and guarantee local spatial continuity:

4

In this case, is a Gaussian kernel having a radius, , and standard deviation, . The saliency map of the resulting 2D saliency map, denoted , employs the saliency map to amplify the prominent structure whilst reducing the response of isolated noise.

After retrieving the saliency maps, preliminary weight maps, denoted by are obtained with the help of a pixel-wise maximum selection strategy. The original weight map M i is set to be defined at each spatial point k, i.e. .

5

The effect of this rule is that, in any pixel point, the image that has greatest local saliency prevails in the sense that the rest of the images are diminished. The resulting binary weight images are a pointer of the most informative source image in any pixel location.

These binary maps are, however, largely spatially inconsistent, and may be blocky or have artifacts along structural boundaries. In response to this, covariance filtering based refinement strategy is used. Unlike guided filtering, covariance filtering will not require a guidance image, but will instead learn directly local linear models between each image, and its original weight map, , in each direction, and this forces local consistency and retention of edges.

The covariance of the and within a local radius of r is obtained as:

6

This quantity measures the degree of local linear correlation between the image and its weight map. Simultaneously, the local variance of is obtained as:

7

Using these local statistics, the linear coefficients a and b are estimated according to:

8

9

Here, is a regularization term that prevents division by near-zero variances, preserving numerical stability in homogeneous regions.

The refined weight map is then computed by applying the locally averaged linear model to the corresponding image:

10

This process adaptively adjusts the weight map based on local intensity variations in the source image, enhancing alignment with structural boundaries and reducing noise artifacts. Different parameter values are applied for base and detail layers, resulting in distinct refined weight maps and for each image.

Finally, to ensure balanced contributions from all images, the refined weight maps are normalized such that their sum equals one at every pixel location:

11

This normalization guarantees that the final fused image preserves consistent pixel intensities and avoids overemphasis or suppression of any individual image.

Image fusion and reconstruction

Once the refined weight maps are available, the fusion of base and detail components is performed by computing weighted sums. The fused base image is obtained as:

12

Similarly, the fused detail image is computed as:

13

Finally, the overall fused image F is reconstructed by combining the fused base and detail layers:

14

This reconstruction is done to make sure that the fused image contains both the coarse structures and fine details of all the input images and the outcome would be a perceptually superior, artifact-free multi-modal fusion image.

Results

The section is a complete review of the functionality of the suggested fusion framework with special focus on the qualitative and quantitative results. The combination was used on pairs of CT and MRI images at various anatomical levels and contrast types (T1, T2 and post-contrast MRI).

Qualitative evaluation

The fuse result of the proposed method had much better visual resolution and structural fidelity in maintaining important anatomical structures, including ventricular boundaries, cortical sulci, and cranial sutures. This is also enhanced by means of a combination of a two scale decomposition with a saliency guided weight-map computation which enables to independently process coarse intensity structures using higher frequency information, prioritize diagnostically important areas, including the ventricular systems and the bone margins, throughout the process of fusion.

Figure 4, of fusion of the CT scan and T2-weighted MRI, clearly indicated that the cerebrospinal fluid (CSF) had retained its rich and well-defined texture, and retained its characteristic high T2-weighted MRI appearance of hyperintensity. At the same time, the deep and sharp contrast of the osseous structures, such as the calvarium and cranial sutures, after the CT ensured that no details would be lost due to excessive smoothing as was usually observed in the classic Laplacian Pyramid or guided filtering technique. Covariance filtering-based weight refinement was the means to achieve such an ideal preservation by modifying saliency-based weight maps in an adaptive fashion to match the local image statistics, thus preserving sharp edges along anatomical lines and attenuate undesired background oscillation. Consequently, the fused image thus produced enhanced visibility of the intracranial anatomy with detailed artifact free composite being of high utility in ventricular system, evaluation and skull-base imaging applications.

Fig. 4.

Visual Results for SET-1. The figure displays the fusion outcomes of CT and T2-weighted MRI images with the aid of various methods. The offered approach exhibits a better preservation of anatomical structures, especially ventricles and skull outlines, and balanced contrast. It is more effective than current methods in preserving fine structural features, which prevent over-smoothing and provide a better visual image and diagnostic importance.

Figure 5 illustrates the outcome of the multi-modal CT/T1-weighted MRI fusion which gave excellent gray-white matter differentiation, which is not easily seen in the delicate contrast variations of the T1-weighted images. This technique proved to preserve subcortical structures such as the basal ganglia, corpus callosum and brainstem that were contrasted and sharp. This comes as a direct result of the two-scale decomposition that enables the base and detail components to be independently controlled, to the extent that the global tissue contrast as well as the high-frequency structural details were not lost in the fusion process.

Fig. 5.

Visual Results for SET-2. The figure displays the fusion outcomes of CT and T2-weighted MRI images with the aid of various methods. The offered approach exhibits a better preservation of anatomical structures, especially ventricles and skull outlines, and balanced contrast. It is more effective than current methods in preserving fine structural features, which prevent over-smoothing and provide a better visual image and diagnostic importance.

Moreover, the saliency-guided maximum choice technique ensured that, at every pixel position, the modality that best detected the largest local structural detail i.e. a bright region in white matter or an edge of bone detected at CT information) made all its contribution to the composite image. The technique effectively removes the classic contrast dilution issue which is conventionally intrinsic in the existing baseline protocols, where a modality dominates and erases finer detail of another. Specifically, pixel values of white matter did not bleed into adjacent gray matter and kept the natural tissue contrast relationships that are critical to identifying subtle pathology, such as demyelinating lesions, ischemic, and cortical atrophy.

The additional advantages of the suggested method were also illustrated in Fig. 6, which presents a high-resolution axial scan of the brain. The end result on the fused result was far better in the definition of the ventricular margins where periventricular lesions were more visible and contrasted better in the juncture between the cortex and sub cortex. Such power of method to recognize these minor transitions might be attributed to the reality that covariance-based weight map is optimized and this approach varies the local weights based on the connection between covariance and variance in the images. The method can preserve spatial coherence and retain local edge sharpness - a historical weakness of filtering-based fusion methods.

Fig. 6.

Visual Results for SET-3. The figure displays the fusion outcomes of CT and T2-weighted MRI images with the aid of various methods. The offered approach exhibits a better preservation of anatomical structures, especially ventricles and skull outlines, and balanced contrast. It is more effective than current methods in preserving fine structural features, which prevent over-smoothing and provide a better visual image and diagnostic importance.

The delicate structures, such as thin cortical ribbons, small sulcal folds and small demyelinating plaques, that would most probably be lost or imprecise in other fusion methods were faithfully reproduced. These were upheld in the edge-aware refinement mechanism and did not introduce artificial distortions, blocky artifacts and unnatural edges in the pictures. The covariance filtering approach to parameterizing the covariance filtering technique of the detail and base layers, which was also explicit, also enhanced fusion in both low and high spatial frequencies, and was thus capable of offering an optimal trade-off between smooth background tissue intensity and sharp anatomical boundaries. This visual acuity and structural faithfulness provides potent advantages of critical diagnostic application, including cortical lesion, edema boundaries, and microstructure pathology.

The qualitative findings in all the above datasets were a vivid demonstration of the fact that the combined effect of two-scale decomposition, Laplacian-based saliency detection and covariance filter refinement not only did not affect the quality of perceptions of fused images; but also gave the specific information of each modality without degradation. The end products had no characteristic issues such as over-smoothing, edge-degradation and contrast-dissatisfaction contrasting, which produced diagnostically rich, visually balanced as well as structurally coherent multi-modal images and could be applied to a broad range of decision-making processes.

Quantitative analysis

The metrics used to evaluate the quality of fused images is depicted in Table 2.

Table 2.

Quantitaive analysis of the fused images. The table provides the summary of the image fusion quality measures which are usually employed including Average Pixel Intensity (API) and Standard Deviation (SD) and Average Gradient (AG) and Entropy (EN) and Mutual Information based Fusion metric (MIF) and QAB/F. All measures are applied to evaluate the different levels of quality of fusion e.g. brightness consistency, contrast, sharpness of edges, information content and edge preservation between the fused image and the source images. The measures are well-liked to evaluate the work of the multi-modal medical image fusion algorithms objectively.

Metric	Description	Equation	Interpretation/Role
API	Average Pixel Intensity, representing the overall brightness level in the fused image.	Where I(i, j) is the pixel intensity at location (i, j) and M, N are the image dimensions.	Higher API indicates brighter fused images.
SD	Standard Deviation, measuring the contrast variation within the fused image.	Where I(i, j) is pixel intensity, is the mean intensity, and M, N are image dimensions.	Higher SD reflects better contrast and variation in pixel intensities.
AG	Average Gradient, estimating the sharpness and edge content.	Where and are horizontal and vertical gradients at pixel (i, j).	Higher AG implies sharper images with well-preserved edges.
EN	Entropy, quantifies the amount of information or randomness present in the fused image.	Where is the probability of gray level , and is the number of gray levels.	Higher EN indicates greater information content in the fused image.
MIF	Mutual Information Fusion, quantifying shared information between the fused image and source images.	Where MI(F, A) and MI(F, B) are the mutual information values between the fused image F and source images A and B.	Higher MIF shows better retention of source image details.
	Edge Information Transfer Metric, assessing the transfer of edge structures from source images to fused image.	Where , are edge similarity measures, and , are local edge strength weights.	Higher indicates better edge preservation from source images in the fused image.

The proposed fusion method was benchmarked using six standard metrics: API, SD, AG, EN, MIF and QAB/F (edge information preservation). Results were averaged over three different image sets as demonstrated in Table 3.

Table 3.

Quantitative comparison of different image fusion techniques across three datasets (SET-1, SET-2, and SET-3). The table reports the performance of each method using multiple objective evaluation metrics, including API, SD, AG, EN, MIF, and QAB/F. The results demonstrate that the proposed method achieves competitive or superior performance compared to existing techniques across multiple datasets and evaluation criteria.

Reference	Technique	SET	API	SD	AG	Entropy	MIF	QAB/F
18	GFF	SET-1	44.498	55.0673	6.2094	5.2464	4.0611	0.8228
		SET-2	50.1763	53.7262	9.2607	6.7967	3.4307	0.9047
		SET-3	52.0798	65.1462	9.4476	4.5111	3.3745	0.8498
19	GFS	SET-1	44.7091	55.2248	6.7729	5.2219	4.2202	0.8005
		SET-2	52.8592	56.0744	11.0185	6.7345	4.0771	0.9011
		SET-3	54.4325	66.4115	11.3389	4.4937	3.6095	0.8337
20	Two-Scale	SET-1	44.7268	55.7322	6.5996	5.1919	3.5441	0.7918
		SET-2	36.6337	53.0486	9.4883	6.1566	2.5936	0.8491
		SET-3	53.4836	68.2401	10.6300	4.4340	3.4812	0.8398
21	CBF	SET-1	44.3744	53.9552	8.8797	5.3129	4.6578	0.7531
		SET-2	53.3064	55.4057	10.5345	6.2980	4.7547	0.8826
		SET-3	52.8447	64.8494	11.0096	4.5171	3.5200	0.8487
22	DCHWT	SET-1	43.7115	52.6410	8.2695	5.1710	3.3637	0.7185
		SET-2	38.0233	41.9861	8.0346	6.5888	1.9772	0.8145
		SET-3	52.7490	62.8558	9.0849	5.1667	3.1896	0.8110
23	DWT	SET-1	80.9044	37.3653	7.8172	5.1086	3.2588	0.6748
		SET-2	80.9486	26.3604	7.0283	5.9847	2.3428	0.6669
		SET-3	70.7366	37.9598	7.2358	5.1663	3.1319	0.7082
23	LP	SET-1	69.2142	50.3946	8.7081	5.4493	3.5215	0.7820
		SET-2	62.4117	43.2531	7.9874	6.7221	2.4211	0.8959
		SET-3	58.3331	45.8055	7.5241	5.6220	3.3549	0.8356
25	NSCT	SET-1	47.7584	58.7775	9.4978	5.3827	3.6175	0.7740
		SET-2	53.3064	57.3937	9.9163	6.8032	3.2902	0.8721
		SET-3	55.9370	72.1621	10.8054	5.0961	3.4408	0.8651
26	GoD	SET-1	43.6751	52.4495	7.3658	5.1639	4.5112	0.6989
		SET-2	49.8570	51.1434	8.9687	6.2539	4.3594	0.8685
		SET-3	51.9670	62.8204	8.2094	4.3922	3.5558	0.8257
27	D2-LRR	SET-1	52.9306	71.2704	16.8328	5.0246	3.2051	0.6850
		SET-2	45.9698	74.2473	15.4875	5.1684	1.7280	0.6404
		SET-3	62.5739	83.4955	15.7079	4.0907	3.0191	0.7451
29	FRAC	SET-1	52.3627	63.2935	5.811	5.1248	5.0488	0.6535
		SET-2	58.1425	59.5485	9.7739	5.9568	6.097	0.8626
		SET-3	66.9706	79.6391	10.5065	4.5434	5.5366	0.8025
30	Deep Neural Network	SET-1	45.7767	58.5009	6.6647	5.0042	3.4606	0.8372
		SET-2	59.3870	60.0385	10.0178	7.0047	3.0744	0.9053
		SET-3	57.9555	74.1004	11.0506	5.0845	3.5250	0.8717
Proposed	Proposed	SET-1	44.2371	58.6307	7.2446	3.9108	3.2732	0.8494
		SET-2	46.7057	62.8879	11.2517	5.4407	3.1615	0.9183
		SET-3	54.6598	70.1441	12.0576	4.2086	3.2215	0.8579

The proposed method shows stable and balanced values of API, the overall brightness level in the fused image, in all three datasets. In the case of SET-1, SET-2, and SET-3, the values found in the API of the proposed method are 44.2371, 46.7057, and 54.6598 respectively. These are largely consistent with the high-performing techniques, e.g. GFF and Two-Scale, and much more balanced than excessively high API scores using techniques like DWT (above 80) which can produce over-saturated images and diagnostic turbidity. The similarity between the API values of the proposed approach shows that the merged images retain a natural distribution of intensities and do not have a bias on modalities and perceptual balance.

Regarding SD, which is the measure of the disparity between pixel intensities and reflects the contrast in the image, in SET-3 the maximum value was found with the D2-LRR technique (83.4955). Nevertheless, it was also accompanied by reduced scores on other perceptual and structural measures which depict suboptimal perceptual quality. Instead, the proposed method had a good SD of 67.1613 in SET-3, which was better than approaches such as GFF (65.1462), CBF (70.1441) and significantly better than DWT (37.9598) and DCHWT (62.8558), meaning that it has a better contrast image generation with preserving diagnostic information.

In AG that quantifies sharpness and edge clarity of image, the proposed measure had the largest value of 12.0576 in SET-3 as compared to GFF (9.4476), CBF (11.0096), and Two-Scale (10.63) with only slight differences with the highest value obtained by the CBF method. This shows that the proposed method is effective in maintaining fine anatomical structures and sharpness of boundaries.

In the case of EN, informational content and richness of the fused images, the maximum entropy was measured by GFF in SET-2 (6.7967). The proposed method had a similar value of 5.4407 in the same set. This underscores the fact that this method is capable of generating images that are full of pertinent information without duplication.

The solution offered in the case of MIF scored a highest of 3.2732 in SET-1. Despite a slightly better result of CBF (4.5171 in SET-3), the suggested methodology had a better ratio of MIF to other perceptual indices.

Lastly, based on QAB/F which is a measure that preserves edge information which is essential in identifying anatomical boundaries and pathological margins the proposed method showed its best score in SET-1 with 0.8494 and SET-2 with 0.9183 outperforming all other methods including GFF (0.9047), NSCT (0.8721), CBF (0.8826), and most importantly greatly surpassing DWT (0.6669) and DCHWT (0.8145 This highlights the better capacity of the proposed technique to retain the sharp and significant edges in the fused images.

All these comparative findings support the overall, coherent, balanced performance of the suggested method in a variety of datasets and assessment criteria.

Graph-based performance interpretation

Graph 7 depict the relative performance of different image fusion algorithms of the different image fusion techniques, such as guided filtering, Laplacian Pyramid, DWT, NSCT, Deep Neural Network, FRAC and the proposed saliency-guided covariance filtering framework.

Fig. 7.

Graphical visualization of the performance of different image fusion techniques across multiple evaluation metrics (API, SD, AG, Entropy, MIF, and QAB/F) for SET-1, SET-2, and SET-3. The plots illustrate comparative trends among methods, highlighting the consistent and balanced performance of the proposed approach in terms of contrast enhancement, edge preservation, and information retention.

QAB/F metric that measures structural similarity between fused images and source images is always higher with the proposed technique than traditional spatial domain and multi-resolution techniques. This is because the covariance filtering-based weight map refinement produces smoothness in the original weight maps based on saliency with the help of the initial weight maps, and conserving the local structures and edges in an anatomical context.

The graphical tendencies also prove that methods such as DWT and the Gaussian-based fusion methods would not be effective to retain high SD and MIF values and accordingly the NSCT and D2-LRR methods are also competitive but are expensive in computation process. Conversely, the suggested covariance filtering framework will deliver a reasonable trade-off between quantitative performance and computational efficiency.

Altogether, the graphical representations prove that the suggested approach does not only improve diagnostic content of the merged images, but also it preserves structural consistency that is why it is very applicable in such scenarios as pre-operative planning, radiotherapy, and multi-modal reporting.

Complexity analysis

The computational complexity of the proposed method is analyzed in comparison with existing image fusion techniques, as summarized in Table 4. Most spatial-domain fusion methods, including GFF, GFS, Two-Scale, and CBF, exhibit quadratic complexity of due to repeated filtering and neighborhood-based operations. Transform-domain approaches such as DCHWT and LP introduce additional computational overhead, resulting in complexities of and , respectively.

Table 4.

Comparison of computational complexity of various image fusion methods. The table provides a summary of theoretical time complexity of various representative fusion techniques such as guided filtering-based fusion techniques, transform-based fusion techniques, and the proposed covariance filtering frame work. The complexity is defined as the number of dimensions of inputs image (M and N) among other parameters of the algorithm. This comparison offers an insight into the relative computational efficiency of each method and points to the fact that the proposed method is able to perform with competitive performance at the same time maintaining a linear complexity as the image size.

Reference	Technique	Time Complexity
18	GFF
19	GFS
20	Two-Scale
21	CBF
22	DCHWT
23	DWT	O(MN)
23	LP
25	NSCT	O(MN)
26	Gaussian
27	D2-LRR
29	FRAC
30	Deep Neural Network
Proposed	Proposed

On the same note, more sophisticated forms of decomposition techniques like NSCT and GoD techniques have the same complexity and are slower but carry greater redundancy and memory consumption. Low-rank based methods including D2-LRR have a much higher cost of computation because of iterative optimization and decomposition of matrices, which is usually expressed as: . Fusion techniques based on deep learning raise the computational requirements, and their complexity scales as is the size of the kernel, and F2 is the count of feature maps.

Conversely, the suggested covariance filtering-based fusion framework has a linear computational complexity of O(MN). This performance is mainly due to the application of box filtering operations to calculate the local statistics like mean, variance and covariance that can be performed in constant time per pixel. Also, the approach does not require computationally-intensive multi-scale transforms, multi-iterative optimization, and deep network inference.

Thus, the specified solution has a good ratio between the computational performance and fusion. It has similar or even better quantitative performance with the same linear complexity, and is therefore suitable in high-resolution medical image fusion, and real-time.

Statistical results and discussion

In order to determine the effectiveness of a multi-modal image fusion technique as suggested, extensive statistical analysis was done based on important parameters such as mean, median, mode, SD, and variance in three sets of data: SET-1, SET-2 and SET-3 as presented in Table 5. These parameters measure pixel intensity value distribution, central tendency, and spread of the pixel intensity values in the fused images that give information about their perceptual and structural quality.

Table 5.

Comparison of statistical measures obtained from the fused images across three datasets (SET-1, SET-2, and SET-3). The table reports key statistical descriptors including mean, median, mode, standard deviation (SD), and variance for different image fusion techniques.

Ref.	Technique	MEAN			MEDIAN			MODE			SD			VARIANCE
		SET-1	SET-2	SET-3	SET-1	SET-2	SET-3	SET-1	SET-2	SET-3	SET-1	SET-2	SET-3	SET-1	SET-2	SET-3
18	GFF	44.498	50.1763	52.0798	47	31	105	1	8	1	55.0673	53.7262	65.1462	3032.408	2886.505	4244.0253
19	GFS	44.7091	52.8592	54.4325	37	34	115	1	8	1	55.2248	56.0744	66.4115	3049.8272	3144.3332	4410.5126
20	Two-Scale	44.7268	36.6337	53.4836	74	21	110	1	1	1	55.7322	53.0486	68.2401	3107.1227	2814.1571	4656.7343
21	CBF	44.9301	53.3064	52.8447	38	33	109	1	9	1	54.929	55.4057	64.8494	3018.2173	3060.8002	4206.541
22	DCHWT	43.7115	38.0233	52.749	74	23	102	1	7	1	52.641	41.9861	62.8558	2773.0457	1763.492	3950.8844
23	DWT	80.9044	80.9486	70.7366	54	71	43	52	66	43	37.3653	26.3604	37.9598	1396.1328	695.0217	1440.6997
23	LP	69.2142	62.4117	58.3331	32	45	25	30	37	25	50.3946	43.2531	45.8055	2539.6408	1870.8453	2099.4227
25	NSCT	47.7584	53.3064	55.937	71	31	99	1	9	1	58.7775	57.3937	72.1621	3455.5884	3294.033	5207.3685
26	GoD	43.6751	49.857	51.967	41	29	108	1	9	1	52.4495	51.1434	62.8204	2751.0086	2615.6492	3946.3243
27	D2-LRR	52.9306	45.9698	62.5739	90	41	130	1	1	1	71.2704	74.2473	83.4955	5080.4083	5512.6632	6971.4355
29	FRAC	52.3627	58.1425	66.9706	59	40	140	1	9	254	63.2935	59.5485	79.6391	4005.0692	3546.0237	6342.3845
30	Deep Neural Network	45.7767	59.387	57.9555	84	37	97	1	9	1	58.5009	60.0385	74.1004	3422.3553	3604.6214	5490.8693
Proposed	Proposed	44.2371	46.7057	54.6598	101	59	114	93	68	113	58.6307	62.8879	70.1441	3437.5609	3954.8877	4920.1927

The average pixel intensity of the proposed approach was observed to be balanced among all the sets where SET-1 was 44.2371, SET-2 was 46.7057 and SET-3 was 54.6598. These are similar to the sensitivity of the established techniques like [18], [19] and [20], and much more gentle than those that are over-bright such as [23] (mean of over 80 in both SET-1 and SET-2), which can over-saturate anatomical features. An average that is closer to the middle of the intensity range would indicate that the offered approach is not biased towards either modality and entails the maintenance of a perceptual balance.

Regarding the value of the median, which represents the middle of the intensity distribution, and is not so susceptible to outliers, the proposed method provided the values of 101 (SET-1), 59 (SET-2) and 114 (SET-3). These values are in close relation to the values of [27] and [30] that show successful maintenance of the global contrast characteristics absent skew.

The mode values that show the most common intensity values were also consistent with the proposed method with the values of 93, 68 and 113 in the case of SET-1 through SET-3 respectively. They agree with the fused image that contain large homogenous areas like CSF or bone in MRI images and CT images.

SD is used to indicate the amount of variation or contrast in the image. The proposed method provided SD values of 58.6307, 62.8879 and 70.1441 on SET-1, SET-2 and SET-3 respectively that are significantly higher than SD values provided by older transform-based methods such as [23] (SD less than 53 across all sets) and similar methods such as [26] and [30] which are highly performing. This shows that the proposed fusion model is an efficient contrast-enhancement and edge-definition mechanism that is important in enhancing diagnostic clarity.

SD, which represents the variability of the sample, is correspondingly, accompanied by the variances (square of SD), which emphasizes contrast richness further. The proposed approach had variances of 3437.5609 (SET-1), 3954.8877 (SET-2), and 4920.1927 (SET-3), which are quite close to the mid-high results of techniques. It is worth noting that the suggested method greatly exceeded such methods as [18] and [19] which gave much lower variances as they captured inadequate textual features and a decrease in interpretability.

These statistical comparisons can be visually represented as indicated in the statistic graph 8, and these graphs clearly demonstrate the balanced performance of the proposed method as indicated on all the statistical metrics, which is supported by the fact that the proposed method can result in high-contrast, structurally preserved and perceptually neutral fused images.

Fig. 8.

Performance comparison of the proposed multi-modal medical image fusion method against current techniques using three data sets (SET-1, SET-2 and SET-3). The figure shows the relative performance of the major statistical measures such as mean intensity, median, mode, SD, and variance/variance calculated based on the fused images. These statistical characteristics are used to indicate the distribution and variation of pixel intensities and offer information on contrast preservation, structural consistency and quality of the entire image as a result of fusion using various methods.

Overall, the proposed method demonstrates statistical superiority or parity across all essential indicators compared to prior fusion techniques. It preserves anatomical detail (evident through higher SD and variance), avoids modality bias (consistent mean and median), and enhances usability. The combination of saliency-aware weight mapping and covariance-based filtering contributes directly to these stable statistical outcomes, making this approach both reliable and scalable for diagnostics and surgical planning.

All in all the proposed method has shown statistical superiority or parity with all the necessary parameters as compared to the previous fusion techniques. It maintains anatomical detail (through greater SD and variance), is not biased on modality (mean and median both equal) and is more usable. The joint effect of saliency-aware weight mapping and covariance-based filtering has a direct contribution to these consistent statistical results, thus it is a reliable and scalable strategy to be used in diagnostics and surgical planning.

The Chi-square statistic () is computed using the standard formula:

15

where represents the observed QAB/F value for the i-th fusion method, is the expected value (assumed to be 1.0 for all methods) and n is the number of fusion techniques compared.

This test was applied independently to three datasets (SET-1, SET-2, SET-3), with each containing QAB/F scores from 13 different fusion techniques including the proposed method.

Results for SET-1

In SET-1, the Chi-square test yielded a total value of 0.81823689. The radar graph in Fig. 9 illustrates the deviations of each method’s observed QAB/F values from the expected baseline.

Fig. 9.

The radar graph showing the deviation of the observed values of the methods QAB/F of Set-1 against the expected baseline.

The QAB/F value of the proposed method was 0.8494 as indicated in Table 6 result which was among the highest in the group. Though other approaches such as GFF, GFS and Deep Neural Network also scored above 0.80, a number of others such as DWT and D2-LRR scored much lower (e.g., 0.6748 and 0.6850 respectively), which are included in the Chi-square total. These deflections point out that most of the current techniques fail to capture edge information at its optimum level. The value of the statistic of the chi-square test here is relatively low, which implies a moderate, yet statistically significant deviation in general.

Table 6.

Chi-square analysis of QAB/F values of SET-1. The chi-square value that was obtained is used to show that there is a consistency in the performance of the edge preservation in reference to the ideal reference of all techniques.

Ref.	Technique	QAB/F (O)	Expected (E)	O-E	(O-E)	(O-E)/E
18	GFF	0.8228	1	−0.1772	0.03139984	0.03139984
19	GFS	0.8005	1	−0.1995	0.03980025	0.03980025
20	Two-Scale	0.7918	1	−0.2082	0.04334724	0.04334724
21	CBF	0.7531	1	−0.2469	0.06095961	0.06095961
22	DCHWT	0.7185	1	−0.2815	0.07924225	0.07924225
23	DWT	0.6748	1	−0.3252	0.10575504	0.10575504
23	LP	0.7820	1	−0.2180	0.04752400	0.04752400
25	NSCT	0.7740	1	−0.2260	0.05107600	0.05107600
26	GoD	0.6989	1	−0.3011	0.09066121	0.09066121
27	D2-LRR	0.6850	1	−0.3150	0.09922500	0.09922500
29	FRAC	0.6535	1	−0.3465	0.12006225	0.12006225
30	Deep Neural Network	0.8372	1	−0.1628	0.02650384	0.02650384
Proposed	Proposed	0.8494	1	−0.1506	0.02268036	0.02268036
Total						0.81823689

Results for SET-2

Table 7 Now the result of SET-2 exhibited a greater dissimilarity among the values of the measured QAB/F scores and the anticipated value Table 2 Chi-2 Result. The overall Chi-square value obtained was 0.40914674. The proposed procedure has the largest QAB/F value that is 0.9183 which is near the ideal value of 1.0.

Table 7.

Chi-square analysis of QAB/F values of SET-2. The chi-square value that was obtained is used to show that there is a consistency in the performance of the edge preservation in reference to the ideal reference of all techniques.

Ref.	Technique	QAB/F (O)	Expected (E)	O-E	(O-E)	(O-E)/E
18	GFF	0.9047	1	−0.0953	0.009083	0.009083
19	GFS	0.9011	1	−0.0989	0.009780	0.009780
20	Two-Scale	0.8491	1	−0.1509	0.022771	0.022771
21	CBF	0.8826	1	−0.1174	0.013785	0.013785
22	DCHWT	0.8145	1	−0.1855	0.034424	0.034424
23	DWT	0.6669	1	−0.3331	0.110950	0.110950
23	LP	0.8959	1	−0.1041	0.010838	0.010838
25	NSCT	0.8721	1	−0.1279	0.016368	0.016368
26	GoD	0.8685	1	−0.1315	0.017294	0.017294
27	D2-LRR	0.6404	1	−0.3596	0.129332	0.129332
29	FRAC	0.8626	1	−0.1374	0.01887876	0.01887876
30	Deep Neural Network	0.9053	1	−0.0947	0.00896809	0.00896809
Proposed	Proposed	0.9183	1	−0.0817	0.00667489	0.00667489
Total						0.40914674

As visualized in Fig. 10, other methods like CBF, GFF, Deep Neural Network and GFS also performed well with scores above 0.88. However, techniques like D2-LRR and DWT underperformed (0.6669 and 0.6404, respectively), again contributing low to the Chi-square total. These differences affirm the proposed method’s robustness in preserving edge fidelity in more complex fusion scenarios.

Fig. 10.

The radar graph showing the deviation of the observed values of the methods QAB/F of Set-2 against the expected baseline.

Results for SET-3

In SET-3, the Chi-square test produced a total value of 0.43691555, which again reflects statistically meaningful but not extreme deviations from the ideal. As shown in Fig. 11, the proposed method recorded a QAB/F score of 0.8579, outperforming many traditional methods.

Fig. 11.

The radar graph showing the deviation of the observed values of the methods QAB/F of Set-3 against the expected baseline.

Competing techniques such as CBF and Deep Neural Network also reached comparable values around 0.8717 to 0.850 as shown in Table 8. Conversely, methods like DWT and D2-LRR performed poorly again, with QAB/F values of 0.7082 and 0.7451, respectively. The pattern confirms that while a few methods approach the expected performance, most struggle to consistently preserve edge features across datasets. The proposed method’s consistent proximity to the ideal across all three sets validates its generalizability and statistical significance.

Table 8.

Chi-square analysis of QAB/F values of SET-3. The chi-square value that was obtained is used to show that there is a consistency in the performance of the edge preservation in reference to the ideal reference of all techniques.

Ref.	Technique	QAB/F (O)	Expected (E)	O-E	(O-E)	(O-E)/E
18	GFF	0.8498	1	−0.1502	0.022560	0.022560
19	GFS	0.8337	1	−0.1663	0.027676	0.027676
20	Two-Scale	0.8398	1	−0.1602	0.025664	0.025664
21	CBF	0.8487	1	−0.1513	0.022891	0.022891
22	DCHWT	0.8110	1	−0.1890	0.035721	0.035721
23	DWT	0.7082	1	−0.2918	0.085149	0.085149
23	LP	0.8356	1	−0.1644	0.027030	0.027030
25	NSCT	0.8651	1	−0.1349	0.018209	0.018209
26	GoD	0.8257	1	−0.1743	0.030379	0.030379
27	D2-LRR	0.7451	1	−0.2549	0.065977	0.065977
29	FRAC	0.8025	1	−0.1975	0.03900625	0.03900625
30	Deep Neural Network	0.8717	1	−0.1283	0.01646089	0.01646089
Proposed	Proposed	0.8579	1	−0.1421	0.02019241	0.02019241
Total						0.43691555

AT the consistently higher QAB/F values for the proposed method in all three sets, coupled with its lower contribution to the Chi-square totals, affirm its superior ability to preserve edge information. Furthermore, the radar plots visually reinforce this claim by showing the proposed method consistently aligning closer to the expected perimeter than most others.

The chi-square analysis is not aimed at answering whether all the fusion strategies deviate statistically significantly with respect to the ideal value, but it is instead intended to measure the degree to which each fusion approach deviates with respect to the theoretical optimal edge preservation figure of 1.0. A low chi-square contribution means that the measured QAB/F will be more similar to the optimal figure that should have been achieved, thus, depicting high-quality preservation of structural edge information based on the source images. On the contrary, larger deviations add a larger contribution to the chi-square statistic and indicate worse edge transfer performance. The suggested approach has constant deviations to the anticipated value when compared to various conventional methods including DWT and D2LRR. This proves that the suggested covariance filtering architecture is more efficient in terms of edge structure preservation as well as it is nearer to the theoretical optimum in all considered datasets.

These results not only underscore the quantitative strengths of the proposed covariance filtering approach but also statistically validate its performance in preserving anatomical boundary sharpness, a key requirement for usability in fused medical imaging.

Clinical relevance of statistical measures

Quantitative statistical measures are capable of giving an objective assessment of fusion performance, although it has to be clinically relevant in the sense that it should be able to assess visual and diagnostic fidelity. Precisely, large SD and variance depict enhanced contrast and definition of anatomical structures, which is essential in establishing limits of lesions and identifying minimal pathological variations. On the same note, a balance between the mean, median and mode values will give a fused image that is not biased on the intensity of a certain modality hence making sure that the complementary anatomical data of modalities such as CT or MRI is not sacrificed. It is a balance, which cannot be neglected in clinical cases in which the contribution of data on structure and functions is the foundation of the precise diagnosis. Moreover, the enhanced statistical properties to which the proposed method will be subjected means that clinically relevant features will be represented even more clearly, and the results interpretation will be more solid, and decision-making in the field of multi-modal medical imaging will be made more informed.

Ablation study

To determine individual contribution of the key elements in the proposed fusion framework, an ablation analysis was conducted, which is a systematic removal of key modules, such as covariance filtering, saliency-based weight construction, and Laplacian-based edge extraction in the proposed framework. The results of these ablation experiments in a quantitative form are represented in Table 9.

Table 9.

The results of Ablation study in various datasets (SET-1, SET-2, and SET-3) using various performance measures that include API, SD, AG, Entropy, MIF, and QAB/F. The findings indicate that every element in the suggested framework brings to the enhancement of contrast, edge preservation, and the overall fusion quality.

Method	Set	API	SD	AG	Entropy	MIF	QABF
Original	SET-1	44.2371	58.6307	7.2446	3.9108	3.2732	0.8494
	SET-2	46.7057	62.8879	11.2517	5.4407	3.1615	0.9183
	SET-3	54.6598	70.1441	12.0576	4.2086	3.2215	0.8579
Removing Covariance Filter	SET-1	44.5166	60.7149	8.9564	3.9197	4.6295	0.8268
	SET-2	47.6012	64.1330	12.6060	4.8916	4.8012	0.9155
	SET-3	54.8209	73.2014	17.4779	4.3051	5.1620	0.8399
Removing Saliency	SET-1	41.7546	52.0882	4.0773	3.5330	3.0985	0.7170
	SET-2	22.7040	35.8756	5.4431	4.5512	3.6308	0.7588
	SET-3	50.4267	62.4515	7.2219	3.9715	3.2642	0.7343
Removing Laplacian	SET-1	49.8012	63.8806	5.6815	3.6526	3.1159	0.7269
	SET-2	50.1162	66.3784	10.7380	5.4857	3.7431	0.8797
	SET-3	65.9588	81.9442	10.8508	4.0660	3.7671	0.8263

Effect of Covariance Filtering: In order to measure the contribution of covariance filtering, the refinement step was eliminated and the original weight maps were used unfiltered. As seen, without covariance filtering, there is less edge preservation and spatial consistency, causing a decrease in QAB/F values in all the datasets. This backs up the fact that covariance filtering is vital in filtering weight maps and enhancing structural correspondence.

Effect of Saliency-Based Weight Construction: In order to look at the significance of saliency, even weights were put on all the input images, omitting the step of calculating saliency. The findings indicate a high degree of deterioration of the fusion performance, especially in the case of Average Gradient (AG) and QAB/F. It means that saliency-based weighting is the key to informative regions and to retain meaningful features of source images.

Effect of Laplacian-Based Edge Extraction: In order to consider the effect of the edge information, the step of the Laplacian filter was eliminated, and the saliency maps were directly calculated using the input images. The findings show that Laplacian filtering is essential to minimize edge representation and structural detail, which validate the use of Laplacian filtering.

Parameter Settings: The covariance filtering stage uses two parameters that are proposed: the window radius and the regularization parameter. All experiments were empirically configured with the base layer parameters of , and the detail layer parameters of , . These values have been chosen to provide a balance between spatial smoothness and edge preservation with a larger radius on the base layer as needed to provide global consistency and a smaller radius with reduced regularization on the detail layer to leave fine structures.

The results of the ablation, taken as a whole, reveal that all of the elements of the proposed framework can play a significant role in the overall fusion performance, and their combination will guarantee better edge preservation, contrast enhancement, and information retention.

Discussion

Experimental results demonstrates the ability of the proposed Covariance Filtering-based fusion method to preserve modality-specific content from multi-modal images while, at the same time, providing a high level of structural coherence and perceptual separability of the fused output images. The new approach is compared with traditional pixel-based and transform domain-based fusion methods that are often guided by problems of spectral distortion or local detail loss. The approach utilizes a two-scale decomposition with region-aware saliency weighting and adaptive covariance-based refinement. This allows for both the diagnostically important boosting of structures like lesions, bone edges, and gray-white matter interfaces, and, at the same time, the elimination of redundant.

The proposed solution works best in case with unquenchable anatomy contrasts. One example included the combination of post contrast T2-weighted MRI and CT images which brought out areas of edema and lesion margins without compromising bone information. The implications of these findings provide a deeper understanding of the fact that precise visualization of soft tissue pathology and bony anatomy is needed in pre-surgical planning, radiotherapy contouring, and neuro-oncological analyses. The high performance on parameters of mutual information, entropy, and QAB/F highlights that the method can maintain complementary properties of both modalities, fluid mechanics of the MRI and structural outlines of CT.

The strong point of our solution is that it is flexible with various MRI contrast modalities. Regardless of the choice of fusing T1-weighted or T2-weighted high-resolution images or post-contrast images, the Covariance Filtering framework invariably received the local fine-grained features as well as the global contextual structure. This is also possible due to the covariance-based weight refinement that offers an avenue to relax spatial consistency and maintain sharp boundaries. Also, the saliency detecting mechanism was implemented on the basis of Gaussian, which guaranteed that the mechanism of fusion more highly valued the areas that were associated with diagnostics rather than the noise or less essential background structures.

The comparison to the state-of-the-art algorithms, such as guided filtering-based and Laplacian Pyramid-based fusion, showed that our algorithm performed worse than others according to the objective measure, as well as subjective understanding. Although techniques, e.g. guided filtering¹⁵, also yielded a visual smoothness, they tended to produce low contrast and also could not highlight important boundaries particularly the ones surrounding a lesion. Contrastingly, the covariance-equipped saliency-clustered weight maps of our approach yield sharper edges and enhanced tissue distinctiveness which is particularly useful in the application of radiology where sharp edges and high precision in space are required.

In spite of such advantages, there are certain weaknesses that should be taken into consideration. To begin with, the offered method does not require as much computation as a deep learning model, nevertheless, it entails local window calls of covariance and multi-level fusion, which can certainly affect real-time applicability in a large-scale workflow without additional efforts. Also, even though saliency maps were effective to select regions, they are based on local contrast, and in images with low signal-to-noise or weak initial alignment can be suboptimal. Further development of the study in the future might involve combining adaptive learning-based saliency detectors or transformer-based attention systems to improve further region selection and semantic relevance.

In short, the Covariance Filtering-based fusion system discussed in the current paper could meet some of the major issues of the multi-modal medical image fusion by providing a structurally coherent, perceptually incisive and meaningful system. Its combination of spatial saliency, local covariance modelling and multi-scale detail retention make it a prospective approach to both real-world diagnostic and therapeutic workflows and has the potential to be used in the future in both 3D imaging and AI-based optimization.

Limitations and future work

Despite the promising performance and the impressive results of the proposed covariance filtering-based fusion model using different datasets and evaluation criteria, it is possible to note several limitations of the presented model, which should be addressed in the future.

To begin with, the algorithm has greater computational efficiency than the DL techniques, whereas local window-based covariance and multi-stage processing, including saliency map generation and refinement, have an average level of computational demands. This will restrict its real time application in large scale pipelines or time sensitive application such emergency diagnosis or intraoperative imaging. The work in the future will be based on optimizing the computational pipeline through parallelization and effective approximations of covariance computation and processing using GPOUs to make this computation useable in real time.

Second, the strategy is operating in the 2D image space. Despite the fact that it is sufficient to most slice-based diagnostic uses, volumetric imaging is increasingly being used in daily practice, especially in cancer and neurosurgery. The model will be extended to allow 3D volumetric fusion, which will allow better anatomic continuity between slices and the space-related information in the areas of tumor outline and radiotherapy planning.

Besides, the framework uses pre-registered input images. In practice, fusion can be compromised by misalignment between modalities (e.g., the motion of the patient or variation in the time of acquisition). The method should be further enhanced by incorporation of a lightweight image registration module or learning-based alignment correction into the fusion pipeline so that the method becomes more robust and usable.

Overall, although the suggested fusion approach offers a well-balanced approach with high capacity in preserving edges, increasing detail, and a low-level of computation, future studies will focus on real-time processing, adaptive region picking, and parallelized appraisal to expand the use of it in precision medicine.

Conclusion

A new multi-modal medical image fusion model that uses Covariance Filtering was proposed in this paper to cope with the continuous issues of retention of fine anatomical details, spatial consistency and improvement of contrast in fused images. The proposed approach incorporates a two-scale decomposition scheme and saliency-based weight map formation, and adaptive covariance-based refinement, and thus is effective to preserve global contextual structures as well as local features without external guidance images.

The extensive qualitative and quantitative assessments on multi-modes CT and MRI data proved the superiority of the methodology to several state-of-the-art methods in the main performance indicators, such as Entropy, Mutual Information, Average Gradient and QAB/F. The suggested method always provided better anatomical edges, contrast, and perception, which qualifies it to be used extensively in neurosurgical planning, radiotherapy guidance, as well as multi-mode reporting.

Additionally, the framework attains this better performance at computational simplicity to provide a convenient trade-off between accuracy and efficiency. Further studies will involve the extension of this technique to include more sophisticated, region-sensitive attention mechanisms, which can further improve the diagnostic reliability and semantic significance of fused imagery in a wider range of medical imaging conditions.

Acknowledgement

This study is carried at Chitkara University, Rajpura, Punjab, India and I acknowledge my supervisors Dr. Shalli Rani and Dr. Ayush Dogra for all their support and guidance. Further, I acknowledge Dr. Mohammad Shabaz for his supportive role in providing resources from his research lab. I also extend my heartfelt appreciation to Mr. Mohit Sharma, Assistant Professor at Chitkara University, Rajpura, Punjab, India in the Medical Imaging Technology Department of Allied Health Science, for his valuable insights and support. Additionally, I am deeply grateful to Nidhi Goswami, Assistant Professor at Chitkara University, Rajpura, Punjab, India for sharing her expertise in medical radiology, drawing from her extensive 20 years of experience at PGIMER, Chandigarh, in the Department of Radiodiagnosis, as well as her contributions at NHS, London. Their knowledge and experience have significantly contributed to the depth and quality of this research.

Author contributions

Shweta Sharma: Writing - Original draft, Visualization, Methodology, Coding, Validation. Shalli Rani: Visualization, Formal analysis, Resources, Editing, Supervision. Ayush Dogra: Conceptualization, Methodology, Coding, Formal Analysis, Validation, Proofreading, Supervision. Mohammad Shabaz: Methodology, Visualization, Formal Analysis, Editing.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability

Data has been taken from publicly available source https://github.com/dawachyophel/medical-fusion/tree/main/MyDataset.

Declarations

Competing interests

The authors declare no competing interests.

Ethical approval

Not applicable.