7T‐guided super‐resolution of 3T MRI

Abstract

Purpose

High‐resolution MR images can depict rich details of brain anatomical structures and show subtle changes in longitudinal data. 7T MRI scanners can acquire MR images with higher resolution and better tissue contrast than the routine 3T MRI scanners. However, 7T MRI scanners are currently more expensive and less available in clinical and research centers. To this end, we propose a method to generate super‐resolution 3T MRI that resembles 7T MRI, which is called as 7T‐like MR image in this paper.

Methods

First, we propose a mapping from 3T MRI to 7T MRI space, using regression random forest. The mapped 3T MR images serve as intermediate results with similar appearance as 7T MR images. Second, we predict the final higher resolution 7T‐like MR images based on sparse representation, using paired local dictionaries for both the mapped 3T MR images and 7T MR images.

Results

Based on 15 subjects with both 3T and 7T MR images, the predicted 7T‐like MR images by our method can best match the ground‐truth 7T MR images, compared to other methods. Meanwhile, the experiment on brain tissue segmentation shows that our 7T‐like MR images lead to the highest accuracy in the segmentation of WM, GM, and CSF brain tissues, compared to segmentations of 3T MR images as well as the reconstructed 7T‐like MR images by other methods.

Conclusions

We propose a novel method for prediction of high‐resolution 7T‐like MR images from low‐resolution 3T MR images. Our predicted 7T‐like MR images demonstrate better spatial resolution compared to 3T MR images, as well as prediction results by other comparison methods. Such high‐quality 7T‐like MR images could better facilitate disease diagnosis and intervention.

1. Introduction

In the last decade, several technical improvements propelled strengthening the magnetic field of Magnetic resonance imaging (MRI) to produce higher resolution MR images. The outcome is the introduction of ultra‐high‐field (7T) MRI scanners that have many advantages in comparison to the routine MRI such as 3T MRI.¹ For instance, there has been an increase in the signal‐to‐noise ratio (SNR) of 7T MRI, which allows imaging with higher spatial resolution. In addition, 7T MRI has higher sensitivity to tissue changes and anatomical details, which thus produces higher spatial resolution and clearer tissue boundaries.

Therefore, 7T MRI contains more pathophysiological information compared to 3T MRI. Such information could help better diagnose and treat multiple sclerosis, brain tumors, aging‐related changes, and cerebrovascular diseases.^{2, 3} In addition, 7T MRI is more accurate in the segmentation of MRI brain tissues such as white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF), in comparison to 3T MRI. In addition, the segmentation of small brain structures such as hippocampal subfields is often difficult in 3T MR images, but more practical in 7T MR images.^{3, 4, 5, 6, 7} Besides, the cortical lesions and atrophies have higher quality in terms of resolution in 7T MRI in comparison to 3T MRI.³ However, at the current stage, 7T MRI scanners are more expensive and less available in hospitals and clinical centers, and thus most MR images are still acquired using routine 3T scanners.⁸ Therefore, it is an alternative solution to predict high‐resolution 7T MR images from 3T MR images with a computerized postprocessing. In this paper, we propose a method to predict images at the resolution of 7T MRI, referred to as 7T‐like MRI, from the input 3T MRI, with applications to brain tissue segmentation.

In medical imaging, a number of resolution enhancement (RE) methods have been proposed to reconstruct a high‐resolution (HR) image from one or more low‐resolution (LR) images.^{9, 10} The traditional RE methods use interpolation functions such as bilinear or bicubic. For instance, Peled et al.¹¹ introduced subpixel shifting strategy in in‐plane dimension to reduce slice thickness. Some other researchers (Greenspan et al.,¹² Peeters et al.,¹³ and Kim et al.¹⁴) proposed methods to reduce thickness in the slice direction of volumetric images. Other methods (Herment et al.,¹⁵ Shilling et al.,¹⁶ Carmi et al.¹⁷) combined multiple shifted scans to achieve a HR result. Woo et al.¹⁸ proposed an interpolation method for enhancement of tongue by generating a 3D image volume using three orthogonal images. However, HR reconstruction based on interpolation may cause some visual artifacts such as smoothing and ringing.

Another set of RE approaches are based on the image degradation models that try to solve a highly ill‐posed inverse problem of recovering a HR image from one or more LR images. For example, Plenge et al.¹⁹ investigated the RE methods in the slice selection direction, as an alternative to high‐resolution acquisition in terms of the SNR and acquisition time trade‐offs. They found that RE methods could improve the resolution and signal‐to‐noise ratio compared to the direct HR acquisition. Islam et al.²⁰ solved the inverse problem in the wavelet domain to enhance the resolution of 3D MRI volumes by using prior models of the MRI data. In another work, Shi et al.²¹ combined low‐rank and total variation as regularization terms in the spatial domain to better solve the inverse problem with the application to 3D MR images. To reduce the ill‐posedness of the problem, Manjon et al.²² proposed a RE method by using self‐similarity as an additional constraint. Otazo et al.²³ also introduced a RE method based on a collection of LR images, which is applicable to functional and spectroscopic imaging.

A recent trend in the RE of medical images is the use of learning‐based methods that are more effective than the traditional methods, as they are able to produce novel details that do not appear in the LR image. The learning‐based methods use paired LR and HR dictionaries, so that the high‐frequency details lost in the LR image can be predicted from the corresponding HR dictionaries. Several works^{24, 25, 26, 27, 28, 29, 30, 31} aimed to reconstruct super‐resolution of natural images based on learning‐based approaches. For instance, Yang et al.²⁷ proposed a method for super‐resolution of natural images and face images by assuming that the LR and HR image patch pairs share the same sparse representation with respect to LR and HR dictionaries, respectively.

In medical imaging, learning‐based methods have been investigated for HR reconstruction in different image modalities. For instance, Zhang et al.³² proposed a method for resolution enhancement of CT lung images based on a hierarchical patch‐based sparse representation. In another work, Bhavsar et al.³³ proposed an approach for enhancement of lung 4D‐CT based on group sparse representation. Rueda et al.³⁴ presented a sparse‐based super‐resolution method with paired high‐ and low‐resolution images, so that a HR version of a LR brain MR image can be generated. In another work, Coupe et al.³⁵ proposed a method to use a nondiffusion image to constrain the reconstruction of diffusion‐weighted images based on nonlocal patch‐based strategy. Roy et al.³⁶ introduced an MR image example‐based contrast synthesis (MIMECS) approach to recover images with both the desired tissue contrast and a normalized intensity profile by using an atlas containing patches of the acquired and desired tissue contrasts. Konokuglu et al.³⁸ proposed a restoration method for improving the quality of MRI acquisitions based on pairs of high‐ and low‐resolution images. Besides, some other example‐based methods have been proposed for transferring the images from one modality to another. For instance, Iglesias et al.³⁷ proposed a patch‐matching driven, multimodality synthesis work to synthesize T1 data from PD images. Some regression‐based synthesis methods have also been proposed for transferring the images from MRI to CT modality,^{41, 42} or quality transfer between T1 and T2.⁴³ Some other patch‐based methods have been proposed for modality propagation from T1 to T2 and DTI⁴⁴ or from T1 and T2 to FLAIR.⁴⁵ While these methods need paired scans for training from input modality and to‐be‐estimated modality, in,⁴⁶ a patch‐based unsupervised cross‐modal synthesis (PUCS) approach was proposed that works without paired training data.

However, the drawback of learning‐based sparse representation methods is that they assume a high correlation between LR and HR images, which may not be completely satisfied in practice. In these methods, first, the dictionaries of LR and HR image patch pairs are constructed. Then, the LR image patches are represented by the LR dictionary to estimate the sparse coefficients and the estimated sparse coefficients are further used together with the HR dictionary to estimate the desired HR image patches. In this case, the assumption is that the sparse coefficients estimated in the LR space can be used in the HR space, while this may not be true if there is a large distribution gap between LR and HR spaces. To address this problem, in our previous work Bahrami et al.,^{39, 40} we proposed multilevel canonical correlation analysis (CCA) transform, called M‐CCA, to increase the correlation between LR and HR spaces. However, as the mapping from LR to HR space is nonlinear, CCA as a linear transformation is limited to increase the correlation of these two spaces. Furthermore, this method uses a one‐step mapping, which may be insufficient to capture large structural gap between low‐resolution and high‐resolution MR images. In addition, it assumes that the sparse representation in the 3T space can be directly transferred to the 7T MR space, which limits the performance due to generally the low correlation between 3T MRI and 7T MRI spaces.

To overcome this weakness, we propose a method that bridges the large distribution gap between LR (3T) and HR (7T) spaces. Our method has two main steps. In the first step, we generate a direct mapping from 3T to 7T MRI based on random forest regression to create the initial 7T‐like images which have higher correlation and similarity with the ground‐truth 7T MRI. In such case, we reduce the large distribution gap between the 3T and 7T MR spaces. Although some methods^{41, 47, 48} have been proposed for image quality transfer from LR to HR space, such direct mapping causes fuzzy results due to the following two reasons. First, the training step of random forest regression generates an average mapping of the patches from LR to HR space. Second, the training of the random forest is done using all the patches, which may not be good enough for the representation of the input patch. To overcome this problem, following the generation of initial 7T‐like images, in the second step, we use sparse representation between initial 7T‐like images and 7T MR images to achieve the final high‐resolution 7T‐like images.

Our main contributions include: (a) We proposed to sequentially use random forest regression and sparse representation to improve the reconstruction performance. Random forest is used to reduce the distribution gap between 3T and 7T spaces by generating a new space for the 3T images, while the sparse representation is used to achieve high‐quality result by representing each input patch with the best representation patches from the new space of 3T images. (b) We introduced a new ensemble strategy based on the distribution of the outputs of random forest trees to remove outliers and avoid unreliable results. (c) We introduced a weighting scheme by assigning higher weights to the more representative training samples in random forest trees. (d) We proposed a preselection scheme based on the images in the high‐resolution space and further incorporate the group sparsity to share the sparsity among the outputs of regression random forests for more reliable and accurate 7T‐like image reconstruction.

The rest of this paper is organized as follows. In Section 2, we introduce materials and methods. Experimental results are given in Section 3. Section 4 presents the conclusion.

2. Materials and methods

2.A. MR image acquisition and preprocessing

In this work, we recruited 15 volunteers, including 10 healthy subjects, 2 mild cognitive impairment (MCI) subjects, and 3 epilepsy patients, who were scanned using both a 3T Siemens Trio scanner and a 7T Siemens scanner. For 3T images, 224 coronal slices were obtained in a 3T Siemens Trio scanner with a 3D MP‐RAGE sequence using image parameters: TR = 1900 ms, TE = 2.16 ms, TI = 900 ms, Flip angle = 9${}^{\circ }$, and resolution=1×1×1 $m{m}^3$. For 7T images, 191 sagittal slices were obtained in a Siemens Magnetom 7T whole‐body MR scanner with a 3D MP2‐RAGE sequence using image parameters: TR = 6000 ms, TE = 2.95 ms, TI  =  800/2700 ms, Flip angle = 4${}^{\circ }$/4${}^{\circ }$, and resolution=0.65×0.65×0.65 $m{m}^3$. Based on our observations, there are only minimal distortions between the obtained 3T and 7T images as these images were acquired with a gradient echo sequence. As for alignment, we registered the corresponding 3T and 7T images using FLIRT in FSL package with a 9‐DOF transformation, to minimize the possible global distortions. To do so, all the 7T images were linearly aligned to the MNI space by using an individual template.⁵¹ The 3T image of each subject was rigidly aligned to its corresponding 7T image, which is sufficient as they are from the same subject and thus have the same anatomy. We also estimated and corrected the bias field of both 3T and 7T MR images by using the N3 method,⁴⁹ and then performed the skull stripping⁵⁰ to remove the nonbrain tissues.

2.B. Outline of our proposed method

Followed by the collection of the dataset including 15 pairs of 3T and 7T MR images, we use leave‐one‐out strategy to use one pair as testing, where the 7T image is served as ground‐truth, and the rest N = 14 pairs as the training. Our proposed method for the prediction of 7T‐like images from 3T MRI is summarized in Fig. 1. First, using the training set, we propose a direct mapping between the training 3T MR and 7T MR images based on random forest regression. To further improve the result of random forest, we propose a weighted sampling scheme and a new ensemble technique. The outcome of such mapping is a new image with higher correlation with the ground‐truth 7T images. Within the training dataset, we use the leave‐one‐out strategy to learn a mapping for each training image by using the rest N−1 training images. Each resulted mapping is applied to generate the initial 7T‐like training image for each 3T training image. Given a testing 3T image, the N learned mappings from the training images are respectively applied to construct N initial 7T‐like testing images. In such case, we can achieve more reliable results for the reconstruction of local structures and local anatomical details by using N initial 7T‐like testing images. Next, for each patch of the initial 7T‐like testing images, a local initial 7T‐like dictionary and a local 7T dictionary are constructed from the training dataset. Then, each initial 7T‐like testing patch is sparsely represented by the local initial 7T‐like dictionary, while the obtained sparse coefficients are utilized to reconstruct the final 7T‐like patch with the corresponding ground‐truth 7T dictionary. To do so, a new preselection strategy based on K‐means clustering is proposed and used to further improve the reconstruction. Furthermore, the similarity of sparse representation from the patches at the same location in N initial 7T‐like images is also enforced using group sparsity. The following sections provide detailed steps of our proposed method.

Figure 1.

Proposed framework for the prediction of 7T MRI from 3T MRI. [Color figure can be viewed at wileyonlinelibrary.com]

2.C. Constructing paired 3T‐7T local dictionaries

In the learning‐based methods, usually two strategies are utilized to generate the pairs of LR and HR dictionaries, namely dictionary learning strategy and local dictionary strategy. In the dictionary learning strategy, with all of the patches of training LR and HR images, a pair of global LR and HR dictionaries are jointly learned.²⁷ In contrast, in the local dictionary strategy, for each input LR test patch, LR and HR patches are extracted from a local neighborhood in the training sets of LR and HR images to generate the local LR and HR dictionaries. This strategy is applicable when the images have similar structures and contents, thus benefiting from similar local structures for generating local dictionaries, instead of a pair of global LR and HR dictionaries for the whole image.

In this paper, we use the local dictionary strategy. The reason is that all of the 3T and 7T images are aligned and have similar local anatomical structures. So the patches at the same location across different subjects have similar structures. Also, the neighboring patches are more likely to be similar, while faraway patches have less similarity. These issues motivate us to use local dictionary strategy to benefit such similarities for representation of input 3T patch with the local dictionary.

To construct the local 3T and 7T dictionaries, at a given voxel location, the overlapped patches are extracted from the 3T and 7T training images by considering a search window with size of W around the same location in the input 3T patch. We define x to be a column vector representing a 3T patch of size m× m × m extracted from the input 3T MR image X. Then, for 3T patch x, we build the local 3T dictionary ${\mathbf{D}}_{3T}$ and local 7T dictionary ${\mathbf{D}}_{7T}$ by extracting the overlapped patches at the same location and also neighboring locations from all N pairs of aligned 3T and 7T training MR images.

2.D. Direct mapping from 3T to 7T MRI

Followed by the generation of local dictionaries ${\mathbf{D}}_{3T}$ and ${\mathbf{D}}_{7T}$ for each input 3T patch, in this step the objective is to reconstruct the initial 7T‐like image from the input 3T MRI by directly mapping from ${\mathbf{D}}_{3T}$ to ${\mathbf{D}}_{7T}$ dictionaries using random forest regression.

2.D.1. Random forest regression

Random forest is a learning‐based approach proposed for classification and regression problems, and has been widely used in computer vision⁴⁸ and medical imaging⁴⁷ areas. In the current work, we use the random forest for the regression purpose to generate a mapping from 3T to 7T space. Generally, random forest is composed of multiple decision trees, which are trained independently, and the outputs of all the trees are combined using a mechanism, for instance, averaging, to provide the final result. A training sample consists of an input feature vector and its output target value. Here, the input feature is the 3T MRI patch in vector representation, and the output target value is the intensity of a voxel in the corresponding 7T MRI patch. With such a mapping, we use the local 3T and 7T dictionaries to train the random forest regressors. Then, in the testing stage, we use the trained models to estimate each patch of the to‐be‐estimated initial 7T‐like image.

Random forest includes a set of decision trees. For each tree, a subset of patches from ${\mathbf{D}}_{3T}$ and their corresponding target values v ∈ v from ${\mathbf{D}}_{7T}$ are randomly selected, denoted as S. An internal node j in each tree includes a set of samples $S_j\subset S$. At node j, the information gain of $S_j$ is achieved by choosing the k‐th feature to split the samples via

$$I_j^k=H(S_j)-\frac{|S_{j,L}^k|}{|S_j|}H(S_{j,L}^k)-\frac{|S_{j,R}^k|}{|S_j|}H(S_{j,R}^k)$$ (1)

where L and R denote the left and right child nodes, $S_{j,L}^k=\{(\mathbf{u},v)\in S_j|{\mathbf{u}}^k<{\mathit{\theta }}_j^k\},S_{j,R}^k=S_j\backslash S_{j,L}^k$, ${\mathbf{u}}^k$ is the k‐th feature of feature vector u, ${\mathit{\theta }}_j^k$ is the splitting threshold chosen to maximize the information gain $I_j^k$ for the k‐th feature ${\mathbf{u}}^k$, and |.| is the cardinality of the set. $H(S_j)$ denotes the average variance of all target values in $S_j$, and can be calculated by the variance of all target values in node j.

In the training stage, the splitting process is performed recursively until the information gain is not significant, or the number of training samples falling into one node is less than a predefined threshold. After training, each leaf node of the decision trees stores multiple training samples, which can be used to derive the final prediction result. The traditional regression random forest methods consider the same importance for all the training samples. However, in practice, the patches in the local dictionary may not have the same contribution in the generation of the mapping from 3T to 7T images. To address this weakness, we propose a weighting scheme to increase the performance of random forest regression by assigning higher weights to the patches with higher similarity to the input 3T patch as will be explained in the next section.

In the testing stage, the feature vector of each 3T image patch is passed through each tree, and finally reaches one particular leaf node per decision tree. To derive the final prediction result, we need a mechanism to combine all information stored in those arrived leaf nodes of all decision trees. A common ensemble approach is to average all the training samples in the arrived leaf nodes across all decision trees. However, this ensemble method may not be so much robust and accurate, especially when there are some outliers or the large variation of the results from different decision trees. To address this problem, we propose a new ensemble model, called averaging‐voting ensembling, by reducing the effect of the outliers and unreliable results.

2.D.2. Weighting scheme for input training patches

Let $\mathbf{u}\in {\mathbf{D}}_{3T}$ and $\mathbf{v}\in {\mathbf{D}}_{7T}$ be input feature vector and its corresponding target values for regression. Random forest regression generates a mapping from the atoms of the local 3T dictionary ${\mathbf{D}}_{3T}$ to the atoms of the local 7T dictionary ${\mathbf{D}}_{7T}$. In such mapping, all the patches in the local dictionary are assumed to have the same importance in the generation of mapping. However, as all the patches in the local dictionary may not have equal importance in generation of the mapping from 3T to 7T MR images, we propose a weighting scheme for the patches in the local dictionary based on the similarity to the input 3T patch. To calculate the weights, we define an exponential function using $L_2$‐norm distance between the input 3T patch x and each atom u in the local 3T dictionary ${\mathbf{D}}_{3T}$,

$$w=\text{exp}(-\frac{||\mathbf{x}-{\mathbf{u}||}_2^2}{h})$$ (2)

where h is used for adjusting the weight decay speed and calculated based on the variance of the distribution of the similarity of 3T patch x with each atom u. To apply these weighting scheme, we repeat the training patches in the dictionary proportionally to the weights. First, for each atom u in ${\mathbf{D}}_{3T}$, we normalize the weight w to be an integer in the range of [0,5]. Then, proportionally on the weight, we repeat the patch in the dictionary. In such case, the patches with higher similarity to the input patch will be repeated a lot in the trees of random forest and thus the final result of random forest will be affected highly by these patches. In contrast, the patches with less similarity to the input patch will be rare in the final dictionary and the final result of random forest will be less affected by these patches.

2.D.3. Proposed ensemble model

To address the problem of traditional ensembling as discussed before, we propose a new ensemble model, called averaging‐voting ensembling. Let T be the total number of decision trees in the random forest. In our approach, first, we separate the T trees into G groups $T_1,\dots ,T_G$ based on K‐means clustering. To do so, in the testing stage, the feature vector of each 3T image patch is passed through each tree and finally reaches one particular leaf node per decision tree. We use the target values of that leaf node as feature to separate the trees into groups based on K‐means clustering. Then, we take the average of arrived leaf nodes within each group g (g = 1,…,G) via

$$y_g=\frac{1}{{\sum }_{t\in T_g}{L}_t}\sum _{t\in T_g}\sum _{l\in \{1,\dots ,L_t\}}{v}_l^t$$ (3)

where $v_l^t$ is the 7T MRI voxel value of the l‐th training sample contained in the arrived leaf node of decision tree $t\in T_g$ of the g‐th group, $L_t$ is the number of training samples contained in the arrived leaf node of the decision tree t, and $y_g$ is the estimated initial 7T‐like voxel for the g‐th group.

Then, we compose the distribution of the outputs of all groups of decision trees by generating the histogram of the values with B bins. Finally, we select the bin with the maximum value in the distribution, which shows the value that the majority of groups of trees achieved, as the output of the regression random forest. If two or more bins have the maximum value, we take the average of all the bins with the maximum value.

The output of such prediction is the initial 7T‐like patch. Followed by the estimation of all the 7T‐like patches, the initial 7T‐like image can be reconstructed by taking the average of the overlapped patches. This procedure is performed for all the N training 3T MR images using leave‐one‐out strategy by considering one 3T MRI as testing and the rest N−1 as training set to construct N initial 7T‐like training images, which can be used as the initial 7T‐like training set for the second step. This same procedure is also repeated N times for the input 3T MRI by considering N−1 training 3T MRI as the training set to construct N initial 7T‐like testing images ${\mathbf{Y}}_i(i=1,\dots ,N)$. In fact, by generating N initial 7T‐like testing images, in the next step we can use multitask representation to reconstruct the final 7T‐like image. In such case, we can achieve more reliable results in the reconstruction of local structures and local anatomical details by enforcing the same sparse representation among N initial 7T‐like testing images.

2.E. Reconstruction based on initial 7T‐like images

After generation of all N initial 7T‐like images ${\mathbf{Y}}_i(i=1,\dots ,N)$, we generate local dictionary ${\mathbf{D}}_{7T}^{\prime }$ for each patch ${\mathbf{y}}_i$ in each initial 7T‐like image ${\mathbf{Y}}_i$ by extracting the patches from all the initial 7T‐like training images using a search window around the same location.

2.E.1. Preselection based on K‐means clustering

In the local dictionary, all the patches may not necessarily have high importance in representation of the input 3T patch. To select the best patches, we propose a method based on a K‐means clustering to divide the local dictionary ${\mathbf{D}}_{7T}^{\prime }$ into K local subdictionaries ${\mathbf{D}}_{7T,k}^{\prime }$ (k = 1,…,K) by minimizing the intracluster variance. In our method, we use both 3T and 7T patches as input features for clustering the local dictionary into local subdictionaries. In such case, we benefit the high‐quality 7T patches for better result in clustering. Correspondingly, we divide ${\mathbf{D}}_{7T}$ into K local subdictionaries ${\mathbf{D}}_{7T,k}$ (k = 1,…,K) using the same indices as ${\mathbf{D}}_{7T,k}^{\prime }$. In this way, K paired local subdictionaries ${\{{\mathbf{D}}_{7T,k}^{\prime },{\mathbf{D}}_{7T,k}\}}_{k=1}^K$ are established.

Then, the local subdictionary whose elements have the minimum distance to the initial 7T‐like patch is selected for the sparse representation of the initial 7T‐like patch, denoted as ${\mathbf{D}}_{7T,k_{min}}^{\prime }(k_{min}\in \{1,\dots ,K\})$. The distance is defined based on the average distance of the intensity value of all voxels. Different from the previous preselection methods which often use only the similarity of the training LR patches to the input LR patch, in our proposed method, we benefit from the high‐quality HR patches for better preselection by using both LR patches (${\mathbf{D}}_{7T,k_{min}}^{\prime }$) and HR patches (${\mathbf{D}}_{7T,k_{min}}$) as input features for clustering of local dictionary into local subdictionaries.

2.E.2. Group sparse representation

Followed by the construction of the local subdictionaries, in this step, the objective is to reconstruct the final 7T‐like image Z from the initial 7T‐like images ${\mathbf{Y}}_i(i=1,\dots ,N)$. In the learning‐based sparse representation methods,²⁷ each initial 7T‐like image patch y can be sparsely represented with respect to the local subdictionary ${\mathbf{D}}_{7T,k_{min}}^{\prime }$ via

$$\widehat{\mathit{\alpha }}=\text{arg min}||\mathbf{y}-{\mathbf{D}}_{7T,k_{min}}^{\prime }{\mathit{\alpha }||}_2^2+{\mathit{\lambda }||\mathit{\alpha }||}_1{k}_{min}\in \{1,\dots ,K\}$$ (4)

where $\widehat{\mathit{\alpha }}$ is the column vector of the sparse coefficients. Then, the estimated sparse coefficients $\widehat{\mathit{\alpha }}$ can be utilized to reconstruct the final 7T‐like patch z, as follows

$$\mathbf{z}={\mathbf{D}}_{7T,k_{min}}\widehat{\mathit{\alpha }}$$ (5)

Different from the previous methods, here we incorporate N initial 7T‐like images ${\mathbf{Y}}_i(i=1,\dots ,N)$ for reconstruction of the final 7T‐like image Z. To do so, we enforce group sparsity to the patches at the same location of all the initial 7T‐like images ${\mathbf{Y}}_i(i=1,\dots ,N)$ for more reliable local structure reconstruction, by defining all the patches as a group. Instead of sparse representation of a single initial 7T‐like patch, we incorporate the group sparsity using all initial 7T‐like patches in order to let them share the same sparsity and thus make local structure consistent for the reconstructed patches. We reformulate the Eq. (2) to have group sparsity as below

$$\widehat{\mathbf{A}}=\text{arg min}\sum _{i=1}^N||{\mathbf{y}}_i-{\mathbf{D}}_{7T,k_{min},i}^{\prime }{\mathit{\alpha }}_i{||}_2^2+{\mathit{\lambda }||\mathbf{A}||}_{2,1}$$ (6)

where the first term is multitask least square minimizer for N patches from N initial 7T‐like images ${\mathbf{Y}}_i$, with ${\mathbf{D}}_{7T,k_{min},i}^{\prime }$, ${\mathbf{y}}_i$, and ${\mathit{\alpha }}_i$ denoting the initial local 7T subdictionary, initial 7T‐like patch of image ${\mathbf{Y}}_i$, and sparse coefficient vector of the i‐th patch in the group, respectively. The second term is regularizer, which is a combination of $L_1$ and $L_2$ norms on $\widehat{\mathbf{A}}=[{\mathit{\alpha }}_1,\dots {\mathit{\alpha }}_N]$, where each column of $\widehat{\mathbf{A}}$ includes the sparse coefficients of a patch in the group. The $L_2$ norm is imposed to each row of $\widehat{\mathbf{A}}$ for making the patches at the same location have similar sparsity, while the $L_1$ norm is imposed to the all rows of $\widehat{\mathbf{A}}$ to ensure sparsity.

3. Experimental results

In order to evaluate the proposed method for the prediction of 7T‐like image from 3T MRI, we perform an extensive set of experiments. First, we study the effects of the different parameters used in our proposed method. Then, we compare the proposed method with the previous methods, including histogram‐based, sparse representation,²⁷ random forest regression,⁴⁷ MIMECS,³⁶ PUCS,⁴⁶ and M‐CCA,³⁹ both visually and quantitatively. Next, we add the evaluation of intermediate results of 7T‐like image reconstruction. We also discuss the major components of the proposed method. Finally, we investigate the postprocessing results of brain tissue segmentation from these reconstructed 7T‐like images.

3.A. Parameters settings

In this section, we study on the parameters used in the experiments. We used leave‐one‐out cross‐validation strategy to evaluate our proposed method by considering one image for testing at each time and the rest of the images for training. We consider 100 trees in the random forest regression. For the averaging‐voting ensembling, we use the K‐means clustering with G = 10 clusters and the histogram with B = 5 bins. For sparse representation, we set λ = 0.1.

To select the patch size and neighborhood size W for dictionary, we evaluate the effect of these parameters on the performance of our proposed method. In Fig. 2, we show the result of our method by changing the patch size from 7 × 7 × 7 to 3 × 3 × 3 to see the effect of patch size on the performance of our proposed method. From Fig. 2, it can be observed that, by decreasing the patch size, the performance is decreased. By considering the larger patch size, we could capture more structure of the local regions, but this also increases the computational complexity. To balance between high performance and computational time, we chose patch size of 5 × 5 × 5.

Figure 2.

Performance of our proposed method for patch sizes of 7×7×7, 5×5×5, and 3×3×3 for all the 15 subjects. [Color figure can be viewed at wileyonlinelibrary.com]

In Fig. 3, we further show the result of our method by changing the neighborhood size for local dictionary from 11 × 11 × 11 to 7 × 7 × 7. It can be observed that, generally, by increasing the search window size, the performance is increased. The reason is that, in regression random forest and sparse representation, for smaller search window size, the number of samples is not enough for estimation. For larger search window size, there is higher chance to find the patches with the high similarity to the input 3T patch. However, increasing the search window size also increases the computational time. By considering a trade‐off between performance and computational time, we chose the search window size of 9 × 9 × 9. Also, we consider the overlap of 1 voxel between the adjacent patches and average results in the overlapping voxels to obtain the final predicted 7T‐like image.

Figure 3.

Performance of our proposed method for window sizes of 11×11×11, 9×9×9, and 7×7×7 for all the 15 subjects. [Color figure can be viewed at wileyonlinelibrary.com]

The experimental settings for the comparison methods are also optimized to achieve their best performances. For instance, the histogram‐based method is applied to the whole image and we consider the patch size of 5 × 5 × 5 for both sparse representation and random forest regression.

3.B. Comparison with the state‐of‐the‐art methods

In this section, we evaluate our proposed method by comparing, visually and quantitatively, with six state‐of‐the‐art methods, i.e., (a) the histogram‐based, (b) sparse representation,²⁷ (c) random forest regression,⁴⁷ 4) MIMECS,³⁶ 5) PUCS,⁴⁶ and 6) M‐CCA³⁹ methods, as briefed below one‐by‐one.

1. A simple and straightforward idea of mapping 3T images to 7T images is to change the contrast. This could be implemented by histogram matching, which has been used for contrast matching between medical images with different intensities.^{52, 53, 54, 55, 56} Here, we use the histogram matching to change the intensity level of 3T to be like 7T. First, the histogram of the input 3T image is represented by the histograms of all the training 3T MR images by estimating the representation weights. To do so, the histograms of the training 3T and 7T images are used to generate the dictionaries of histograms of 3T and 7T MR images, respectively, by defining each atom of the dictionary as the vector of histogram bins. Then, the same estimated representation weights are used together with the dictionary of histograms of 7T MR images to generate the histogram of to‐be‐estimated 7T‐like image. Finally, by matching the input 3T image to the generated histogram, the 7T‐like image can be reconstructed.

2. The sparse representation method²⁷ originally has been proposed for super‐resolution of natural images and face images. In Yang et al.,²⁷ using a large set of training images, LR and HR dictionaries are jointly learned for reconstruction of a HR image from a LR image. Here, we use the local dictionary strategy as discussed before, as the brain MR images have been aligned and have similar structure. Specifically, we generate local 3T and 7T dictionaries for each patch of the input test 3T image.

3. Alexander et al.⁴⁷ used random forest for image quality transfer for diffusion MRI. Here, we use random forest regression for quality transfer from 3T to 7T MRI, i.e., mapping 3T to 7T MR images. Note that mapping is done from each cubic patch from the 3T space to one voxel in the 7T space. Learning is done using local dictionary, and the learned random forest is used for reconstruction of 7T voxels from the input 3T cubic patches.

4. Roy et al.³⁶ introduced a restoration technique (MIMECS) to recover MR images with both the desired tissue contrast and normalized intensity. Here, we used this method for reconstruction of 7T‐like MRI from 3T MRI.

5. Vemulapalli et al.⁴⁶ proposed a method (PUCS) that works with/without paired training data for modality transfer from T1 to T2 and vice versa. Here, we used this method with paired training data for modality transfer from 3T to 7T MRI.

6. In our previous work Bahrami et al.,³⁹ we proposed a hierarchical framework for reconstruction of 7T‐like images from 3T MRI based on multilevel CCA and group sparsity.

Figure 4 shows the reconstructed 7T‐like images by the six comparison methods and our proposed method, together with the difference maps. It should be mentioned that the difference maps is more useful in highlighting the errors of the local regions. The cross‐views and also their respective close‐up views are shown in Figs. 4(a), (b), and 4(c).

Figure 4.

Visual comparison of (a) axial, (b) sagittal, and (c) coronal views of reconstructed 7T‐like images, together with close‐up views of selected regions. From left to right columns: histogram‐based method, sparse representation,²⁷ random forest,⁴⁷ MIMECS,³⁶ PUCS,⁴⁶ M‐CCA,³⁹ and our proposed method.

It can be seen that the reconstructed 7T‐like image by our proposed method is the closest to the ground‐truth 7T MRI, compared to all of six previous methods. Our results have clearer tissue boundaries and contain more structural details, while the results by other methods are fuzzy. For instance, the results by the histogram‐based method shows blocky boundary between the white matter and gray matter. Results by sparse representation are also fuzzy and tissue boundaries are not clear. The main reason is that the sparse representation method assumes high correlation between LR and HR spaces. But, such assumption may not be valid in practice, especially for the regions with tissue boundaries. Regression random forest produces better results than sparse representation method due to more effective learning of mapping from 3T to 7T space. MIMECS also produces fuzzy results because it assumes the LR and HR dictionaries share the same sparse representation. PUCS has fuzzy result and has lower performance compared to our method. M‐CCA offers much better results, compared to the PUCS, MIMECS, random forest, and sparse representation methods; however, in some regions the results are also blocky and lack of details.

In another experiment, we show the results of our proposed method for six axial slices of three subjects 1, 4 and 8 in Figs. 5 (a), (b), and 5(c), respectively. For each subject, from top to bottom rows, the 3T MRI, reconstructed 7T‐like image by our proposed method, and ground‐truth 7T MRI are shown. Compared to the 3T MRI, our reconstructed 7T‐like images are much closer to the actual 7T MRI, which also shows the ability of our proposed method in reconstruction of different brain regions.

Figure 5.

Results by our proposed method for 6 axial slices of subjects 1, 4 and 8, together with the corresponding 3T and 7T images for visual comparison. (a) Subject 1; (b) Subject 4; (c) Subject 8.

We also evaluate our proposed method numerically by comparing the reconstructed 7T‐like image with the ground‐truth 7T image of the same subject. We use three image similarity evaluation measures, including peak‐signal‐to‐noise ratio (PSNR), structural similarity (SSIM), and universal quality index (UQI). PSNR is defined as the ratio between the maximum intensity value of the ground‐truth image and the power of corrupting noise. SSIM is defined to measure the quality of an image based on mean, variance and cross‐correlation. UQI measures the image quality based on image distortions such as loss of correlation, luminance distortion, and contrast distortion. There are other metrics for evaluation of resolution enhancement such as point spread function. However, we would like to mention that generally, global point spread function is used for the estimation of the blur kernel of the blurred images, while the estimation of global point spread function is not very accurate to be used as a metric for resolution enhancement evaluation for 3T and 7T MRI.

In general, higher PSNR, SSIM, and UQI indicate that the reconstructed 7T‐like images closer to the ground‐truth 7T MR images. A comparison between the reconstruction results of 15 subjects by histogram‐based, sparse representation,²⁷ random forest regression,⁴⁷ MIMECS,³⁶ PUCS,⁴⁶ M‐CCA,³⁹ and our proposed method is given in Table 1. Our proposed method has a much higher PSNR, SSIM, and UQI for all 15 subjects, compared to the six previous methods. In the last row of Table1, we also show the averages of PSNR, SSIM, and UQI for all the 15 subjects.

Table 1.

Performance comparison of the histogram‐based, sparse representation,²⁷ random forest regression,⁴⁷ PUCS,⁴⁶ MIMECS,³⁶ M‐CCA,³⁹ and the proposed method in terms of PSNR, SSIM, and UQI

Subject	Histogram‐based	Sparse	Random forest	MIMECS	PUCS	M‐CCA	Proposed
(a) PSNR
1	21.7	23.6	25.7	24.5	24.8	25.9	26.8
2	22.1	24.9	25.8	25.5	25.7	27.0	27.4
3	21.7	24.0	24.4	24.1	24.7	26.0	26.3
4	21.6	22.1	25.0	23.3	23.5	25.4	26.1
5	21.6	24.5	25.4	24.4	24.8	26.7	27.1
6	20.8	21.3	23.1	22.2	22.3	23.6	26.0
7	21.2	23.5	25.5	24.0	24.3	25.9	27.6
8	20.0	21.9	23.7	22.5	22.3	24.3	26.7
9	22.3	23.8	24.7	25.9	24.7	25.8	27.2
10	19.6	21.1	23.5	21.7	22.0	24.0	26.5
11	17.0	22.7	22.8	22.3	22.1	23.1	23.2
12	19.9	23.2	22.5	23.5	23.0	23.8	24.2
13	21.0	22.4	23.1	22.7	22.2	24.1	24.5
14	24.2	26.2	27.7	26.8	26.7	28.7	29.1
15	26.9	26.9	27.9	27.2	27.5	28.5	29.6
Average	21.4	22.2	24.7	24.0	24.1	25.5	26.6
(b) SSIM
1	0.30	0.32	0.40	0.41	0.38	0.46	0.49
2	0.31	0.36	0.42	0.44	0.42	0.52	0.57
3	0.39	0.35	0.35	0.41	0.39	0.51	0.58
4	0.31	0.36	0.46	0.39	0.42	0.51	0.59
5	0.39	0.32	0.45	0.38	0.40	0.48	0.60
6	0.37	0.33	0.40	0.36	0.39	0.47	0.54
7	0.26	0.36	0.45	0.42	0.38	0.51	0.58
8	0.29	0.34	0.34	0.42	0.43	0.50	0.62
9	0.28	0.35	0.39	0.45	0.44	0.53	0.61
10	0.24	0.28	0.38	0.32	0.35	0.45	0.59
11	0.20	0.25	0.32	0.29	0.28	0.36	0.48
12	0.29	0.32	0.33	0.33	0.35	0.46	0.57
13	0.28	0.31	0.37	0.31	0.30	0.46	0.57
14	0.38	0.38	0.40	0.38	0.41	0.51	0.60
15	0.40	0.39	0.41	0.41	0.42	0.53	0.62
Average	0.31	0.32	0.39	0.38	0.39	0.48	0.57
(c) UQI
1	0.22	0.30	0.38	0.39	0.40	0.44	0.52
2	0.26	0.34	0.40	0.44	0.43	0.50	0.63
3	0.23	0.38	0.43	0.40	0.42	0.49	0.60
4	0.31	0.34	0.42	0.46	0.39	0.49	0.61
5	0.22	0.31	0.40	0.41	0.39	0.46	0.58
6	0.33	0.33	0.38	0.38	0.37	0.46	0.57
7	0.30	0.35	0.43	0.43	0.41	0.50	0.61
8	0.28	0.30	0.42	0.39	0.40	0.48	0.55
9	0.33	0.34	0.46	0.48	0.45	0.51	0.59
10	0.19	0.26	0.38	0.35	0.37	0.43	0.55
11	0.21	0.29	0.25	0.29	0.30	0.35	0.46
12	0.32	0.26	0.35	0.39	0.38	0.44	0.54
13	0.30	0.37	0.39	0.38	0.32	0.45	0.53
14	0.34	0.39	0.42	0.41	0.43	0.48	0.55
15	0.33	0.40	0.41	0.42	0.42	0.49	0.58
Average	0.27	0.33	0.36	0.40	0.39	0.46	0.56

3.C. Initial and final 7T‐like reconstruction results

In this experiment, we show the outcomes of the two main steps in our method, i.e., (a) random forest regression and (b) sparse representation, to show the impact of each step in the 7T‐like image reconstruction.

Figure 6(a) shows some selected patches from the axial view of a 3T MRI. In Fig. 6(b), we show the initial 7T‐like image estimated by random forest regression. From this initial 7T‐like image, the final 7T‐like image is produced by sparse representation, as shown in Fig. 6(c). It can be observed that by reconstructing an initial 7T‐like image using random forest and then the final 7T‐like image using sparse representation, the final result is much closer to the ground‐truth 7T as shown in Fig. 6(d). The main reason is that, random forest can generate the initial 7T‐like images by mapping from 3T to 7T MRI and making the initial 7T‐like image highly correlated with the actual 7T MRI. Then, sparse representation method can reconstruct the final 7T‐like image just based on the same sparse representation estimated for the initial 7T‐like image. In such case, the reconstructed final 7T‐like image is much closer to the ground‐truth 7T MR image.

Figure 6.

Evaluation of intermediate results for some axial patches of a selected subject: (a) 3T image, (b) output of random forest, (c) output of sparse representation based on the result of random forest, (d) 7T image. [Color figure can be viewed at wileyonlinelibrary.com]

3.D. Super‐resolution of low‐resolution 7T MRI

In this section, we perform an experiment to evaluate our method for super‐resolution of low‐resolution 7T MRI. To do so, we resample all 7T MR images with a factor of 3 (i.e., down‐sampling by a factor of 3, followed by up‐sampling by a factor of 3), to reduce the quality of 7T MR images. By doing so, down‐sampling decreases the number of voxels and also causes the loss of image details, while up‐sampling increases the number of voxels by interpolation on the existing voxels but does not recover the lost information. We repeat this experiment by considering the resampling rate of 2, and 4, also. The result of super‐resolution based on average PSNr over 15 subjects for different resampling rates has been shown in Table 2.

Table 2.

Average PSNR over 15 subjects for super‐resolution of low‐resolution 7T MRI when the resolution of 7T MR images is decreased with the resampling rate of 2, 3, 4

Resampling rate	Ave. PSNR for low‐resolution 7T MRI	Ave. PSNR for reconstructed 7T‐like MR images
2	27.2	33.4
3	25.4	30.1
4	22.8	26.8

Figure 7 visualizes the super‐resolution results by our method for the factor of 3. Notably, our reconstructed 7T‐like MRI improves the low‐resolution 7T MRI and also looks much similar to the ground‐truth 7T MRI.

Figure 7.

Super‐resolution of low‐resolution 7T MRI. [Color figure can be viewed at wileyonlinelibrary.com]

3.E. Resolution enhancement of 3T MRI

In this section, we evaluate our method for resolution enhancement of 3T MRI. Figure 8 shows the generated 7T‐like MR image of a selected slice of a representative subject, together with the 3T MRI and ground‐truth 7T MRI. Also, we display a close‐up on middle brain region to better show the anatomical details. These visual results indicate that our generated 7T‐like MR image clearly improves the resolution of the 3T MRI.

Figure 8.

Super‐resolution of 3T MRI. [Color figure can be viewed at wileyonlinelibrary.com]

3.F. Discussion on the contributions of our proposed method

There are several contributions in the proposed method for the reconstruction of super‐resolution 7T‐like images from 3T. In this section, we discuss the impact of these contributions on the result of our proposed method in terms of PSNR, as shown in Fig. 9.

Figure 9.

Evaluation of different contributions on performance of our proposed method. RF+SP: random forest + sparse representation. RF+SP+WI: random forest + sparse representation + weighted input. RF+SP+WI+AV: random forest + sparse representation + weighted input + averaging‐voting ensembling. RF+SP+WI+AV+CP: random forest + sparse representation + weighted input + averaging‐voting ensembling+ clustering‐based preselection. [Color figure can be viewed at wileyonlinelibrary.com]

The first contribution is the successive use of regression random forest mapping followed by sparse representation. As we discussed earlier, random forest regression is used to establish direct mapping relationships between the 3T and 7T images. The outcome of such mapping improves correlation with the ground‐truth 7T MR images. Then, each patch of the initial 7T‐like images can be sparsely represented by the local initial 7T‐like dictionary, and then the obtained sparse coefficients can be directly utilized to reconstruct the 7T‐like patch with the corresponding local 7T dictionary. In Fig. 9, the result of this contribution is denoted as 'Random Forest + Sparse Representation (RF+SP)', which shows improvement compared to 'Random Forest (RF)' itself.

In the regression random forest, as all the patches in the local dictionary may not have equal importance in generation of the mapping from 3T to 7T images, we proposed a weighting scheme for the patches in the local dictionary based on their similarity to the input 3T patch. The output of our method by adding this contribution is denoted as 'Random Forest + Sparse Representation + Weighted Input (RF+SP+WI)'. The result shows that by using weighting scheme the result is improved, because the patches with higher similarity to the input 3T patch are repeated more in building trees of random forest and then eventually guide the final reconstruction result.

As additional contribution, we introduced a new ensembling method based on the averaging and voting of the outputs of all decision trees, called 'averaging‐voting' ensembling. The output of our method with this contribution is denoted as 'Random Forest + Sparse Representation + Weighted Input + Averaging‐Voting Ensembling (RF+SP+WI+AV)'. The result shows that this averaging‐voting ensembling improves the reconstruction result by reducing the effect from outliers and unreliable results.

Finally, we evaluate our proposed patch preselection strategy, based on the K‐means clustering, by dividing the local dictionary into local subdictionaries. The output of our method by adding this contribution is denoted as 'Random Forest + Sparse Representation + Weighted Input + Averaging‐Voting Ensembling + Clustering based Preselection (RF+SP+WI+AV+CP)'. The results reveal that, by selecting the best patches using the proposed patch preselection method, the final reconstruction result can be further improved.

4. Application to tissue segmentation

In this experiment, we evaluate the influence of the reconstructed 7T‐like image on postprocessing steps, such as brain tissue segmentation, where brain tissues are separated into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). We compare the segmentation results of our proposed method with those by the 3T MRI, histogram‐based method, sparse representation,²⁷ random forest regression,⁴⁷ MIMECS,³⁶ PUCS,⁴⁶ and M‐CCA,³⁹ by considering the segmentation result of 7T MRI as ground‐truth. Here, we employ a widely‐used FAST in FSL package (Zhang et al.⁵⁷) for tissue segmentation.

The segmentation results for the axial view of a subject, together with the close‐up view of selected regions, based on 3T MRI, 7T MRI, and the reconstructed 7T‐like images by six different methods are shown in Fig. 10. Compared to six previous methods, the segmentation result by our proposed method is closer to the segmentation of the ground‐truth 7T image.

Figure 10.

Comparison of tissue segmentation results by different methods. (a) from left to right: axial view of a subject for 3T MRI and the reconstructed 7T‐like MRI generated by histogram‐based method, sparse representation,²⁷ random forest regression,⁴⁷ PUCS,⁴⁶ MIMECS,³⁶ M‐CCA,³⁹ our proposed method, and 7T MRI. (b) Segmentation results from 3T MRI, and the reconstructed 7T‐like MRI generated by histogram‐based method, sparse representation, random forest regression, our proposed method, and 7T MRI, together with the close‐up views of selected regions. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 11 shows the average dice ratio of segmentation maps of WM, GM, and CSF for all 15 subjects. As can be observed, the segmentation result by our method significantly outperforms the direct use of 3T MRI and also other six comparison methods, for all WM, GM, and CSF tissues (P < 0.01 by two‐sample tests).

Figure 11.

Box plot of the dice ratio for segmentation of (a) GM, (b) WM, and (c) CSF by different methods. From left to right: 3T MRI and the 7T‐like images generated by histogram‐based, sparse representation, random forest, MIMECS, PUCS, M‐CCA, and our proposed method, respectively.

5. Conclusion

In this paper, we have proposed a new framework to predict high‐resolution 7T‐like MR image from 3T MR image by incorporating both the advantages of regression random forest‐based mapping and sparse representation into the two main steps. In the first step, we propose a direct mapping from 3T MRI to 7T MRI based on a training set of paired 3T and 7T MR images using regression random forest, to reduce the large distribution gap between 3T and 7T MR image spaces. To further improve the result of such mapping, a weighting scheme and a new ensemble technique are proposed. The output of this step is the estimation of the initial 7T‐like MR image with higher correlation and similarity with the actual 7T MRI. In the second step, followed by patch‐based image partitioning in the new space of the initial 7T‐like MR image, we use the same sparse representation in the new space and the 7T MRI space to reconstruct the final 7T‐like MR image. To do so, a new patch preselection based on K‐means clustering is also proposed to select the best patches for sparse representation.

We have conducted a comprehensive set of experiments to show the superiority of our proposed method both visually and numerically, compared to several comparison methods including histogram‐based method, sparse representation, random forest regression, MIMECS, PUCS, and M‐CCA. The results based on pairs of 3T and 7T MR images of 15 subjects demonstrated that our proposed method is better in recovering anatomical details, compared to the previous methods. Then, as a medical application, we have investigated the performance of the high‐resolution 7T‐like MR images in segmenting WM, GM, and CSF brain tissues. We have compared the segmentation maps generated using the proposed 7T‐like MR images to those generated using the original 3T MRI, as well the 7T‐like MR images produced by the histogram‐based method, sparse representation, random forest regression, MIMECS, PUCS, and M‐CCA. We considered the segmentation of the actual 7T MRI as ground truth. Based on both visual and numerical results, our proposed method outperforms all other methods under comparison.

It should be mentioned that beside reconstructing 7T‐like image with higher resolution in comparison to the input 3T image, our method is also suitable for many applications. For instance, more accurate tissue segmentation can be achieved using 7T‐like images. Specifically, the segmentation of WM, GM, and CSF from reconstructed 7T‐like MR images has higher accuracy, compared to the original 3T MRI, as we have shown in this paper.

However, there are some other applications that will be our future work such as 7T MRI atlas construction based on 3T MRI, multimodal registration based on 3T and 7T‐like MRI, and more accurate segmentation of small brain regions, such as segmentation of hippocampus subfields regions which is generally difficult in the 3T MR images due to low resolution, while it is relatively easier in 7T MR images.

Despite the high accuracy of our method for super‐resolution, it has some weaknesses. For instance, our method fails to generate high‐resolution patches with any abnormality, if the training 3T patches are all normal. For instance, if cortical lesions are not visible in the 3T MRI, it will be hard to be reconstructed in the 7T‐like images. However, if such information exists in the 3T MRI with low‐resolution, it could be reconstructed with high‐resolution in the 7T‐like MR images. On the other hand, the sample size of this study is still limited. A larger number of samples could provide more training subjects, make the model more representative, and has better generalizability. The use of different MRI acquisition protocols and scanners may also affect the performance, which needs further research.

Of note, although we evaluated our proposed method by using a dataset of 3T and 7T MR images from brain, however, the same method can also be applied to other body organs as well.

We will release our software and the datasets of 3T and 7T MR images later in our software website (http://www.med.unc.edu/bric/ideagroup/free-softwares).

Conflict of interest

The authors have no relevant conflicts of interest to disclose.