Quantifying the Interaction and Contribution of Multiple Datasets in Fusion: Application to the Detection of Schizophrenia

Abstract

The extraction of information from multiple sets of data is a problem inherent to many disciplines. This is possible by either analyzing the datasets jointly as in data fusion, or separately and then combining, as in data integration. However, selecting the optimal method to combine and analyze multiset data is an ever-present challenge. The primary reason for this is the difficulty in determining the optimal contribution of each dataset to an analysis as well as the amount of potentially exploitable complementary information among datasets. In this paper, we propose a novel classification rate based technique to unambiguously quantify the contribution of each dataset to a fusion result as well as facilitate direct comparisons of fusion methods on real data and apply a new method, independent vector analysis (IVA), to multiset fusion. This classification rate based technique is used on functional magnetic resonance imaging data collected from 121 patients with schizophrenia and 150 healthy controls during the performance of three tasks. Through this application, we find that though optimal performance is achieved by exploiting all tasks, each task does not contribute equally to the result and this framework enables effective quantification of the value added by each task. Our results also demonstrate that data fusion methods are more powerful than data integration methods, with the former achieving a classification rate of 73.5 percent and the latter achieving one of 70.9 percent, a difference which we show is significant, when all three tasks are analyzed together. Finally, we show that IVA, due to its flexibility, has equivalent or superior performance compared with the popular data fusion method joint independent component analysis.

I. Introduction

The collection of data from multiple sensors has become common in many fields, since each sensor is expected to provide a complementary view of the system under study [1], [2]. This raises the issue of how best to utilize each of these datasets in order to maximize the use of available information for the given task. When applied to medical imaging, the methods used to analyze data from multiple sensors fall into two broad categories: data integration and data fusion [3], [4]. In data integration, the datasets are analyzed separately and the results are combined [5]–[7]. Data fusion methods, on the other hand, analyze the datasets together, enabling exploitation of the information shared across datasets, see e.g., [4], [8]–[10]. However, there are several issues that prevent straightforward selection of the ideal method when analyzing multiple real world datasets, these include: quantification of the additive value of each dataset, determining the combination of datasets that achieves the best performance, and quantification of the joint information between datasets within an analysis.

Due to its non-invasive nature and high spatial resolution, functional magnetic resonance imaging (fMRI) data has become increasingly utilized to study both human and animal neurological activity [11]. To facilitate understanding of such a complicated system, fMRI data has been combined with multimodal data, i.e., data of different types, such as electroencephalography [12]–[17] or diffusion tensor imaging [18], [19], or with multiset data, i.e., fMRI data from different stimuli or tasks [20]–[23]. Since little is known about the relationships between datasets, it is important to minimize the underlying assumptions placed on the data while allowing the datasets to fully interact. This has lead to the popularity of data-driven fusion methods for jointly analyzing multiset fMRI data, such as the popular joint independent component analysis (jICA) [24], an extension of independent component analysis (ICA) to multiple datasets, that has been used in many other medical applications, see e.g., [25]–[27]. Despite its widespread use, jICA is a fairly constrained model, since it assumes that all datasets have identical mixing matrices, and thus may perform poorly when the modeling assumptions are not met [28]. This assumption also means that jICA inherently requires each task to contribute similarly to the result, though this may not always be the case in practice [29], [30].

In this paper, we propose a novel classification rate based technique to assess the performance of different multiset analysis methods for various combinations of datasets. We should note that perfect classification is not the goal of this work, especially since the patients with schizophrenia were receiving antipsychotic and/or mood stabilizing medication during the scanning sessions, thus making it difficult to attribute observed differences to the disease or to the medication [31]–[33]. Instead, our goal is to enable unambiguous quantification of the additive value of each dataset, determination of the combination of datasets that achieves the greatest performance, and quantification of the interaction among datasets within an analysis. Note that the proposed technique differs fundamentally from the nonparametric prediction, activation, influence, and reproducibility resampling framework [34], whose goal is to assess how well a model generalizes to new data rather than how useful a dataset is to a fusion analysis. We apply this technique to fMRI data from three tasks: an auditory oddball (AOD) task, a Sternberg item recognition paradigm (SIRP) task, and a sensorimotor (SM) task, drawn from 121 patients with schizophrenia and 150 healthy controls. In addition, we also propose the use of independent vector analysis (IVA) [35], a more recent and flexible multiset extension of ICA that exploits similarities across different datasets to achieve a successful decomposition, for fusing multitask fMRI datasets.

The remainder of the paper is organized as follows. In Section II, we introduce our classification procedure, the fMRI tasks, as well as the the features used in this study. Section III contains the results of the classification procedure on the multitask fMRI data and associated discussions. We present our conclusions in Section IV.

II. Materials and Methods

A. Feature Extraction

Since the timing of each task is different, it is difficult to directly fuse multitask fMRI data. Instead, for each task and subject, the data from each voxel is analyzed using a simple linear regression using the statistical parametric mapping toolbox (SPM) [36], where the regressors are created by convolving the hemodynamic response function (HRF) in SPM with the desired predictors. Note that for these regressions, the noise was assumed to be Gaussian and uncorrelated. The resulting regression coefficient maps or “features” are then used for each subject and task. This reduction using a lower-dimensional yet still multivariate representation of the data facilitates exploration of the associations across these feature sets, see e.g., [4], [20], [37]. Such an investigation provides a natural way to discover links across tasks as well as simplifying the identification of biomarkers of disease. See Section II-D for greater detail about the features used in this study.

B. jICA/IVA

Given an fMRI feature dataset from M subjects, X ∈ ℝ^M×V, where the mth row of X is formed by flattening the feature of V voxels from the mth subject, the noiseless ICA model can be written as

$$x\left(v\right)=As\left(v\right),1\le v\le V,$$ (1)

where the C latent sources or neural activation patterns, s ∈ ℝ^C, are linearly mixed by the mixing matrix, A ∈ ℝ^M×C. By assuming spatial independence on the part of the latent sources, ICA seeks to estimate a demixing matrix, W, such that the estimated components are given by ŝ(v) = Wx(v). For simplicity, we will drop the sample indices in the rest of the discussion. Due to the fact that we seek to maximize independence between the components, ŝ₁, …, ŝ_C, estimation of the demixing matrix can be accomplished through the minimization of their mutual information, written as

$${ℐ}_{\text{ICA}}\left(W\right)={\displaystyle \sum _{i=1}^{C}H\left({\widehat{s}}_i\right)-\log \left|\text{det}\left(W\right)\right|-H\left(x\right)},$$ (2)

where H(·) is the entropy.

Since it is usually assumed that the number of subjects is larger than the number of latent sources, ICA is generally performed following preprocessing using principal component analysis (PCA). Determining the appropriate order for the PCA preprocessing step for fMRI data is an active area of research, see e.g, [38]–[43]. Despite its widespread study, the majority of order selection techniques have been optimized for single subject or single task analyses, potentially limiting their applicability to multiset fusion. In this work, we used the order estimated using a modification to the procedure in [44] because, to the best of our knowledge, it is the only method that has shown desirable performance for the task of order estimation for data fusion of medical imaging data. The technique, named PCA and canonical correlation analysis (PCA-CCA), estimates a common order for two datasets through a series of binary hypothesis tests. The process is begun by assuming that the common order is 0 and the null hypothesis is that the current common order is appropriate. If the null hypothesis is rejected, the common order is increased by 1 and the binary hypothesis test is performed again. This process is repeated until the null hypothesis cannot be rejected or the common dimension is equal to M/2. In this application, we estimated the order for each pairwise combination of datasets and then selected the highest estimated order to enable the retention of the most signal.

Though there are many ICA algorithms, in this work, we used ICA-EBM, due to the fact that it has shown superior performance in both simulated and real neurological data when compared with the popular Infomax algorithm [45], see e.g., [30], [46], [47]. This performance is due to the fact ICA-EBM does not assume any distribution for the latent sources and instead attempts to upper bound their entropy through the use of several measuring functions [46]. These measuring functions describe a wide variety of distributions, including those that are: unimodal, bimodal, symmetric, and skewed [46], thus generally leading to accurate estimation of all sources within the mixture.

Note that the columns of the estimated mixing matrix, Â, provide the loadings of each component across subjects. Thus, the pth column of the estimated mixing matrix, â_p, represents the relative weights of the pth source estimate, ŝ_p, for each corresponding subject. Since the dataset has been reduced to a feature for each subject, to look for differences in the expression of components between two groups, a two-sample t-test can be performed on the subject covariations, where one group is represented by the subject covariations from the patients with schizophrenia and the other by the subject covariations from the healthy controls [4].

The model in (1) can be extended in a straightforward manner to K datasets, or tasks, as

$$x^{\left[k\right]}=A^{\left[k\right]}{s}^{\left[k\right]},1\le k\le K.$$ (3)

Due to the inherent scaling and permutation ambiguities of ICA, running an ICA individually for each task and aligning the results is impractical and suboptimal. For this reason and to exploit interactions across different tasks, one may assume that each task is mixed with the same mixing matrix, A, which enables the solution of the problem posed in (3) through the performance of a single ICA in which the sources from the disparate datasets form underlying “joint sources” [13], [20]. Joint estimation of the sources, and hence, the determination of the individual components, can then be achieved through the performance of a single ICA on the horizontally concatenated X^[k] defined as

$$\underset{\_}{X}=[X^{\left[1\right]},X^{\left[2\right]},\dots ,X^{\left[K\right]}]=A[S^{\left[1\right]},S^{\left[2\right]},\dots ,S^{\left[k\right]}]=A\underset{\_}{S}.$$ (4)

This method, referred to as jICA [24], is one of the most popular multitask fMRI data fusion methods, see e.g., [20], [29]. However, jICA is reliant on the assumption that each dataset has the same mixing matrix, thus it may perform poorly when this is not the case [28].

An alternative to jICA for two datasets is canonical correlation analysis (CCA) [48] or its extension to multiple datasets, multiset CCA (MCCA) [49], which maximizes the correlation of sources across datasets. Neither CCA nor MCCA restrict their solution space by assuming that each mixing matrix is identical, but rather estimate a separate mixing matrix, Â^[k], for each dataset simultaneously. Because of their less restrictive forms, CCA, MCCA, and related methods have successfully been applied to multisubject analyses, see e.g., [50]–[54], as well as the fusion of multimodal neurological data, see e.g., [10], [18], [55]. Despite their widespread use, CCA and MCCA have two important limitations that can hurt their applicability to real data: first, both CCA and MCCA only exploit second-order statistics and second, both CCA and MCCA limit the solution space by assuming that the demixing matrices are orthogonal. A way to overcome the shortcomings of CCA and MCCA is through the use of IVA, a recent multiset extension of ICA, which exploits similarities across datasets to achieve a successful decomposition. Unlike CCA or MCCA, IVA does not limit the demixing matrices to be orthogonal and can take advantage of second as well as higher order statistics [56]. This flexibility has lead to the use of IVA in multisubject fMRI analyses, see e.g., [57]–[59], however, to our knowledge, it has not yet been applied to the problem of fusing multitask feature data.

The IVA formulation requires the definition of the cth source component vector (SCV) as ${\widehat{s}}_c={\left[{\widehat{s}}_c^{\left[1\right]}\dots {\widehat{s}}_c^{\left[K\right]}\right]}^{\mathrm{T}}\in {ℝ}^K$, i.e., by concatenating the cth component from each of the datasets. Given the definition of an SCV we can write the mutual information cost function for IVA in a similar manner to that of ICA,

$${ℐ}_{\text{IVA}}(W^{\left[k\right]})={\displaystyle \sum _{c=1}^{C}H\left({\widehat{s}}_c\right)-{\displaystyle \sum _{k=1}^K\text{log}\left|\text{det}\left(W^{\left[k\right]}\right)\right|-{\displaystyle \sum _{k=1}^{K}H(x^{\left[k\right]})}}}.$$ (5)

The difference between (2) and (5) is that we are now minimizing the mutual information between SCVs and not components. Rewriting (5), we have

$${ℐ}_{\text{IVA}}(W^{\left[k\right]})={\displaystyle \sum _{c=1}^C{\displaystyle \sum _{k=1}^{K}H\left({\widehat{\mathit{s}}}_c^{\left[k\right]}\right)}-ℐ\left({\widehat{s}}_c\right)-{\displaystyle \sum _{k=1}^K\text{log}\left|\text{det}\left(W^{\left[k\right]}\right)\right|-{\displaystyle \sum _{k=1}^{K}H(x^{\left[k\right]})}}}.$$ (6)

The new term, $ℐ\left({\widehat{s}}_c\right)$, in (6) is the mutual information within the SCV and is being maximized through the minimization of (6). Thus, we see how IVA exploits the shared information between across datasets, when it exists, since if the term is not present, (6) reduces to individual ICAs on each dataset separately [56].

C. Classification Procedure

Since there is no ground truth for the components or subject covariations, we propose to use classification rate to determine the relative performance of different techniques on real multitask fMRI datasets. We use this as our metric for two reasons: first, classification rate has an unambiguous meaning that can be defined for different combinations of datasets and analysis techniques, and second, classification rate effectively encapsulates the total discriminative power, between patients and controls. A diagram of the classification procedure is shown in Figure 1 and the process is described below.

Fig. 1.

Classification process for a single feature dataset. For the case where multiple datasets are analyzed, the ICA step is replaced with either jICA or IVA, performed on the concatenated feature datasets or the collection of feature datasets, respectively. The procedure is as follows: (a) the data is split into a training set, X_Train, and a test set, X_Test. (b) the training dataset is dimension reduced using PCA, ICA is run, and the discriminatory components, ${\stackrel{\sim }{S}}_{\text{Train}}$, and corresponding subject covariations, Ã_Train, are selected. (c) in the final stage, Ã_Train is used to train the classifier, ${\stackrel{\sim }{S}}_{\text{Train}}$ is regressed onto X_Test producing Ã_Test, and Ã_Test is used to test the classifier. This process is repeated N times and the mean classification rate is evaluated. Note that for jICA there is a single joint set of subject covariations, ${\stackrel{\simeq }{A}}_{\text{Train}}$ and ${\stackrel{\simeq }{A}}_{\text{Test}}$. for IVA and well as the data integration technique, there are dataset specific subject covariations ${\stackrel{\sim }{A}}_{\text{Train}}^{\left[k\right]}$ and ${\stackrel{\sim }{A}}_{\text{Test}}^{\left[k\right]}$; however, for IVA the subject covariations are derived by fusing information across all datasets whereas for the data integration technique the subject covariations are extracted from each dataset individually.

We will describe the classification process for a single dataset using ICA, since the extension to multiple datasets and different analysis techniques is, as we will show, straightforward. The first step is randomly subsampling 190 subjects from the original fMRI feature dataset, X, in order to produce a dataset to train the classifier, X_Train. The feature data from the remaining 81 subjects is formed into X_Test, which will be used to test the trained classifier. In order to reduce any bias introduced in the random selection of subjects, the proportion of patients and controls is kept the same in X, X_Train, and X_Test. In the next step, PCA, using the order specified using [44], then ICA, using the entropy bound minimization (ICA-EBM) algorithm [46], are performed on X_Train. Following ICA-EBM, a two-sample t-test is performed on each column of the estimated subject covariations, Â_Train, and those that are declared significant, i.e., p < 0.05, as well as the corresponding spatial maps are formed into Ã_Train and ${\stackrel{\sim }{S}}_{\text{Train}}$, respectively.

The final stage of the classification process begins with the training of a classifier, such as a radial basis function kernel support vector machine (KSVM) using the rows of Ã_Train. Next, Ã_Test is found by regressing ${\stackrel{\sim }{S}}_{\text{Train}}$ onto X_Test and is used to test the classifier produced using Ã_Train. The value of the kernel parameter, if any, is selected by computing the average classification rate for 800 independent Monte-Carlo subsamplings of the data with different kernel parameters and finding the value with the highest average classification rate. Once the optimal value of the kernel parameter is selected, the classification procedure is run on N independent Monte-Carlo samplings of the data and the mean classification rate is computed. In our results, we use 200 as the value of N.

For jICA, the process is identical to the one described above, except X is composed of the concatenated feature datasets. With IVA, the process is similar to the one described above, except the two-sample t-test is run on every column of all estimated mixing matrices, ${\widehat{A}}_{\text{Train}}^{\left[k\right]}$, and the columns showing a significant difference between the patients and controls are all concatenated together when training the classifier. In order to compare these two data fusion methods with a technique that does not analyze the data sets jointly, i.e., data integration, we took the same setup from IVA, but replaced the IVA step with a separate ICA analysis for each feature dataset individually. In this work, we use the IVA using a multivariate Gaussian and Laplacian (IVA-GL) algorithm [60] since it has been shown to be an effective IVA algorithm for analysis of fMRI data, see e.g., [60], [61]. Note that to facilitate comparisons between methods and remove possible confounding from the use of different orders for different methods, the same order, 24, was used in the PCA step for all methods. Possible effects of the order on classification performance were studied by repeating the ICA analyses for different orders and no statistically significant differences, after a Bonferroni correction, were observed between the classification rates at different orders.

Note that if the classifier requires the determination of a parameter, such as the value of kernel parameter for KSVM, the selection of the optimal value is done for each method individually. One final note regarding the computational complexity, since T ≫ C, K, by far the most time consuming step in the proposed framework is the jICA/IVA step. The computational complexity is approximately $\mathcal{O}(KT{N}^2)$ per iteration and approximately $\mathcal{O}(KT{N}^2+K^{2}TN)$ per iteration for jICA using EBM and IVA-GL, respectively.

D. FMRI Tasks and Features

The datasets used in this study are from the Mind Research Network Clinical Imaging Consortium Collection (publicly available at http://coins.mrn.org). These datasets were obtained from 271 subjects, 150 healthy controls and 121 patients with schizophrenia. Next, we briefly introduce the tasks used in this study as well as the multivariate features extracted from each task.

1) Auditory Oddball Task (AOD)

This auditory task involved subjects listening to three different types of auditory stimuli: standard (1 kHz tones occurring with probability 0.82), novel (computer generated, complex sounds occurring with probability 0.09), and target (1.2 kHz tones with probability 0.09, to which a right thumb button press was required), in a pseudo-random order [62], [63]. Each run consisting of 90 stimuli each with a 200 ms duration and a randomly changing interstimulus interval of 550–2,050 ms. The order of novel and target stimuli was changed between runs to ensure that the responses did not depend on the stimulus order [63]. In addition, the sequences of stimuli were designed such that they would produce orthogonal blood-oxygen-level dependent responses for each of the three stimuli [64], [65]. For this task, the regressor was created by modeling the target and standard stimuli as delta functions convolved with the default SPM HRF in addition to their temporal derivatives [22]. Subject averaged contrast images between the target versus the standard tones were used as the feature for this task.

2) Sternberg Item Recognition Paradigm Task (SIRP)

In this visual task, the subjects had to remember a set of 1, 3, or 5 randomly chosen integers between 0 and 9. The task paradigm consisted of: a 1.5 second learn condition, a blank screen for 0.5 seconds, a 6 second encode condition, where the sequence of digits was presented together, and a 38 second probe condition, where the subject was shown a series of integers and had to indicate, with a button press with the right thumb, whether it was a member of the memorized set or not [63]. Each probe digit was presented for up to 1.1 seconds in a pseudo-randomly jittered fashion within a 2.7 second interval [63]. A total of 84 probes, 42 targets and 42 foils was obtained per scan and the prompt-encode-probe conditions were run twice for each set size in a pseudo-random order [63]. For this task, the regressor was created by convolving the probe response block for the three digit set with the default SPM HRF [22]. This was done for both runs of the probe response and the average map was used as the feature for this task.

3) Sensory Motor Task (SM)

In this auditory task, the subjects were presented with a sequence of auditory stimuli consisting of 16 different tones each lasting 200 ms and ranging in frequency from 236 Hz to 1,318 Hz with a 500 ms inter-stimulus interval [63]. The first tone presented was set at the lowest pitch and each subsequent tone was higher than the previous one until the highest tone was reached and which point the order of the tones was reversed [63]. Each tonal change required a button press with the right thumb. A total of 15 increase-and-decrease blocks were alternated with 15 fixation blocks, with each block lasting 16 seconds in duration [63]. For this task, the regressor was created by convolving the entire increase-and-decrease block with the default SPM HRF [22]. The responses for each subject were averaged over the separate runs and the average map was used as the feature for this task.

III. Results and Discussion

The classification results for the individual multitask datasets as well as their combinations, using KSVM, are shown in Figure 2. We note that though we only show the results of KSVM in this paper, the trends that we observe are similar to those observed when either k-nearest neighbors or Fisher discriminant analysis are used as the classifier, though the actual classification rates are different. It also should be noted that perfect classification is not primary goal of this work, but rather to determine the comparative advantages of different combinations of fMRI tasks and multiset analysis techniques. Several noteworthy trends can be observed in Figure 2. First, we can see that equivalent or improved classification rate is achieved when multiple datasets are jointly analyzed than when only a single dataset is analyzed. This is due to the fact that there is more total discriminative power available in a combination of multiple datasets than there is in any of the datasets individually. Additionally, the AOD dataset has the highest classification rate of all of the individual dataset analyses, while the SM dataset has the lowest. This means that, when analyzed separately, the AOD dataset is best able to differentiate between patients and controls, while the SM dataset has more difficulty. We also note that there is a clear advantage to be found through the use of data fusion, i.e., jICA and IVA-GL, over the use of data integration, i.e., combined ICAs, in all cases, except where only the AOD and SIRP datasets are used, where equivalent performance is achieved. This shows that the results for jICA and IVA-GL are more than the “sum of their parts,” hence, emphasizing importance of allowing datasets to fully interact with each other in an analysis [2], [66]. Additionally, in all cases IVA-GL performs better than or is statistically equivalent to jICA, showing the advantages of using a less constrained model when the signal-to-noise ratio is relatively high [28]. The strong performance of IVA-GL when compared with the ICA-based techniques is noteworthy since the algorithm used for the ICA-based techniques was EBM, which is able to fit a much larger range of latent source distributions than IVA-GL, thus showing the power of methods that exploit complementary information across datasets. Since IVA-GL is best able to summarize the total discriminative power of a combination of datasets, we can use the difference in classification rate between a two dataset analysis and the three dataset analysis to assess the contribution of the third dataset on the fusion result. For example, the additional contribution of the SM dataset is the difference between the classification rate of IVA-GL using the AOD and SIRP datasets and the classification rate of IVA-GL using all three datasets. Based on this, the additional contribution of AOD is 4.5 percent, the additional contribution of SIRP is 1.7 percent, and the additional contribution of SM is 5.5 percent. Thus, the SM and AOD datasets contribute much more exploitable information than the SIRP dataset. In order to provide a qualitative assessment of the differences observed in Figure 2, 2-sample t-tests were performed on the classification rates and the results are shown in Table I.

Fig. 2.

Average classification results using KSVM for individual datasets and combinations of datasets using either data fusion, with jICA and IVA-GL, or data integration using combined ICAs. The first three points from the left refer to the case where only one dataset is analyzed. The fourth, fifth, and sixth points from the left refer to combinations of two datasets. The rightmost point shows the classification performance when all three task datasets are jointly analyzed. Note that error bars are omitted for clarity, since the largest value of the standard error is 0.0035.

TABLE I.

Significance of the difference in classification rate using KSVM of IVA-GL compared to the data integration technique, combined ICAs, and the popular data fusion method, jICA. Statistical significance is assessed through a two-sample t-test performed on the classification rates obtained for 200 independent subsamplings of the fMRI feature datasets. If the difference corresponds to p ≥ 0.05, the result is marked “not significant” (NS). Note that there is no correction for multiple comparisons, but all significant differences would remain so after the conservative Bonferroni correction.

Datasets	Significance of Difference (p-values)
	IVA-GL to Combined ICAs	IVA-GL to JICA
AOD and SIRP	NS	NS
AOD and SM	1.0×10−10	NS
SIRP and SM	1.4 × 10−16	9.5 ×10−5
All Three Datasets	2.3 × 10−9	8.5 × 10−4

As can be seen from Table I, all of the major differences observed visually in Figure 2 are statistically significant. No correction for multiple comparisons is performed, however we note that all of the observed differences remain statistically significant even after the conservative Bonferroni correction. An interesting result that can be seen in both Figure 2 and Table I is that for the combination of AOD and SIRP feature datasets, there is no statistical difference in the classification rates of IVA-GL compared with that of the combined ICAs. This means that there is no measurable difference in performing data fusion or data integration for this combination of datasets. These results suggest that there is little joint information that is exploited between the AOD and SIRP datasets. On the other hand, there is significant improvement in performance when fusing the AOD and SM datasets, implying that there is a significant amount of joint information between the AOD and SM datasets.

In order to probe this difference, we ran both IVA-GL and the data integration technique on all of the data 10 times and selected a run from each method for comparison using the minimum spanning tree (MST)-based method described in [67]. The MST-based method works by aligning components across different runs of the algorithm, then selecting the run to be used for comparison by finding the run that has the highest correlation with the T-map constructed from a one-sample t-test run on the aligned components. The reason that the components from this run are used is two-fold. First, each of the results of a method on the subsampled data may be seen as rough approximations of the results of the same method on the whole dataset, since we are greatly reducing the dimension of the datasets using PCA prior to performing the analysis. Second, since both the ICA and IVA algorithms are iterative in nature, there is a natural need to determine a result that best represents the average decomposition of a given set of data. For this reason, we use the MST-based method described in [67], since it has shown superior performance to the popular method of ICASSO [68]. Using the run selected by the MST-based method, we find the p-values by performing a two-sample t-test on each column of the estimated mixing matrices of that run, where one group is the subject covariations of the controls and the other is the subject covariations of the patients. Those p-values that pass a significance threshold of p < 0.05 are declared statistically significant and the corresponding z-scored components are aligned between the data fusion and data integration techniques. Two components are considered aligned if they have a sample Pearson correlation coefficient above 0.5, which we find matches with the results produced through visual alignment. The results are shown in Figures 3 and 4.¹

Fig. 3.

Statistically significant components for the combination of the AOD and SIRP datasets. (a) the significant components obtained through the use of data fusion, using IVA-GL, are shown in the first two rows. The third and fourth rows contain those significant components obtained through the use of data integration, using ICA-EBM, that have a correlation above 0.5 with the components obtained using IVA-GL. The aligned components are in the same column. Those components obtained using ICA-EBM that do not have a correlation above 0.5 with any of the components obtained using IVA-GL are shown in the (b) and (c) for the AOD and SIRP datasets, respectively. The maps have been flipped such that the activation (red and orange) represents an increase in controls over patients and deactivation (blue) corresponds to a decrease in controls versus patients. The p-values for each component are located above the corresponding spatial map and those that remain significant after a Bonferroni correction are displayed in green. All spatial maps are Z-maps thresholded at Z=2.7.

Fig. 4.

Statistically significant components for the combination of the AOD and SM datasets. (a) the significant components obtained through the use of data fusion, using IVA-GL, are shown in the first two rows. The third and fourth rows contain those significant components obtained through the use of data integration, using ICA-EBM, that have a correlation above 0.5 with the components obtained using IVA-GL. The aligned components are in the same column. Those components obtained using ICA-EBM that do not have a correlation above 0.5 with any of the components obtained using IVA-GL are shown in the (b) and (c) for the AOD and SM datasets, respectively. The maps have been flipped such that the activation (red and orange) represents an increase in controls over patients and deactivation (blue) corresponds to a decrease in controls versus patients. The p-values for each component are located above the corresponding spatial map and those that remain significant after a Bonferroni correction are displayed in green. All spatial maps are Z-maps thresholded at Z=2.7.

There are two trends which become clear from the comparison of Figures 3 and 4. The first is that the proportion of components that are declared significant in the IVA-GL results which are not seen in the ICA results is much higher for the combination of AOD and SM than it is for AOD and SIRP. This would imply that the IVA-GL decomposition shows greater difference for the combination of AOD and SM, i.e., there is greater joint information that is exploited. This helps explain the significant improvement in the classification rate of the data fusion technique over the data integration technique for the combination of AOD and SM as well as the reason for its absence in the combination of AOD and SIRP. We should note that the regions found to be significantly different in patients and controls that do not correspond to the ventricles and are found for both IVA and the individual ICAs in Figure 3 correspond to: medial visual cortex, fronto-insular cortex, thalamus, cerebellum, left fronto-parietal cortex, motor cortex, visual cortex, and sensorimotor cortex. These regions have previously been shown to differentiate patients with schizophrenia from controls, see e.g., [69]–[74]. Similarly, the regions found activating significantly differently in patients and controls in Figure 4 correspond to the: medial visual cortex, default mode network (DMN), motor cortex, thalamus, left fronto-parietal cortex, cerebellum, visual cortex, and sensorimotor cortex. These regions have also been previously noted as being impacted by schizophrenia, see e.g., [69]–[75], consistent the idea of cortico-cerebellar-thalamic-cortical circuit disruption or cognitive dysmetria in schizophrenia [76], thus increasing our overall confidence in the results of this analysis. In addition to these, we also have differentiating components with spatial activations corresponding to the posterior parietal lobe in Figure 3 and the temporal lobe and right fronto-parietal lobe in Figure 4 that are seen in the IVA results and not in the ICA results. These regions have also been shown previously to differentiate patients from controls, see e.g., [69], [74], and they, coupled with the higher, on average, significance values for the components show the advantages of a fusion analysis over individual analyses.

The second trend that can be seen from both Figure 3 and Figure 4 is that there are multiple significant components that are found in the single dataset analyses and not found in the fusion results. This is due to the fact that the fusion results report joint components, thus they may mask weaker individual components [77]. The regions found in the ICA results and not in the IVA results correspond to the cerebellum, DMN, fronto-parietal cortex, and medial-visual cortex in Figure 3 and to the cerebellum, DMN, temporal cortex, and frontal lobe in Figure 4. These regions have been shown to be affected by schizophrenia, see e.g., [69], [70], [74], [75] and their absence from the IVA results demonstrates the importance of performing a comparison between the fusion results with the ICA results in order determine the joint as well as dataset-specific information.

These results also motivate the use of the proposed framework for automated feature selection. Determining optimal features from different modalities\tasks is a continuing challenge in many data fusion studies, see e.g., [4], [20], [23], [24], [62], [78]. If the goal of a fusion study is the determination of differences between a set of groups, the technique developed in this paper can be used for automated selection of optimal features based on the combination of features that achieves the highest classification rate.

IV. Conclusions

It has become increasingly common for fMRI data from multiple stimuli or tasks to be collected in medical studies, since each condition is expected to provide a complementary view of the problem under study. This raises the issue of how to most effectively combine datasets from different subjects and tasks. In this paper, we have developed a data-driven classification procedure to determine the performance of different multitask analysis techniques as well as the discriminatory information that a new dataset introduces. We have applied this new method to real multitask fMRI data drawn form patients with schizophrenia as well as healthy controls. Through this application, we find that though we achieve the best performance by exploiting all tasks, each task does not contribute equally to the result and we quantify the value added by each task. Our results show that data fusion methods, through their exploitation of shared information across datasets, are more powerful than data integration methods, which do not take advantage of such information. We also show that the more flexible IVA-GL has equivalent or superior performance compared with the popular data fusion method joint independent component analysis (jICA). Finally, it is important to note that the proposed framework can be applied, not only to fMRI data from multiple tasks, but also to explore the interactions among data from different sensors, i.e., multimodal data, thus enabling quantification of the additive value of each modality in the analysis. However, for such data, rather than modeling the spatial maps directly, the subject covariations should instead be modeled using the tIVA framework [28].