From simultaneous to synergistic MR‐PET brain imaging: A review of hybrid MR‐PET imaging methodologies

Abstract

Simultaneous Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) scanning is a recent major development in biomedical imaging. The full integration of the PET detector ring and electronics within the MR system has been a technologically challenging design to develop but provides capacity for simultaneous imaging and the potential for new diagnostic and research capability. This article reviews state‐of‐the‐art MR‐PET hardware and software, and discusses future developments focusing on neuroimaging methodologies for MR‐PET scanning. We particularly focus on the methodologies that lead to an improved synergy between MRI and PET, including optimal data acquisition, PET attenuation and motion correction, and joint image reconstruction and processing methods based on the underlying complementary and mutual information. We further review the current and potential future applications of simultaneous MR‐PET in both systems neuroscience and clinical neuroimaging research. We demonstrate a simultaneous data acquisition protocol to highlight new applications of MR‐PET neuroimaging research studies.

1. INTRODUCTION

Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) are widely used in vivo imaging modalities that provide complementary information about the brain. MRI has excellent sensitivity in capturing even very small changes in brain structure and function. PET is highly specific in measuring numerous molecular targets in the brain including glucose metabolism, oxygen usage and neurotransmitter distribution and concentration. Although PET provides unmatched ability to image numerous biochemical and molecular targets, its ability to capture detailed spatial and anatomical information is limited. Thus, the integration of two modalities in a simultaneous MR‐PET system, also referred as PET/MR, offers a unique opportunity for multidimensional neuroimaging (Judenhofer et al., 2008). In this article, we review the current state‐of‐play of simultaneous MR‐PET imaging, with a particular focus on applications in neuroimaging and cognitive neuroscience.

Although the idea of an integrated MR‐PET scanner arose more than 25‐year ago, physical integration of the two systems has proven to be a difficult task (Herzog & Lerche, 2016). The mutual interference of MR and PET technologies represented a major technical challenge for the development of hybrid systems and required several generations of technology development. The first generation of simultaneous MR‐PET scanners that were developed utilised an MR compatible PET detector insert with PET technology sized to fit within the bore of a standard MRI scanner. The success of this approach led to further developments of MR‐compatible PET detector technology (see Section 2.1). While the insert technology does provide simultaneous acquisition of MR and PET images, the reduced MR scanner bore size severely limits the size of the sample (i.e., the patient, specific body parts) that can be scanned using this architecture. Nevertheless, the smaller effective MR bore size with the PET insert results in higher PET sensitivity and image spatial resolution than for fully integrated systems. Fully integrated systems that place the PET detector ring and electronics within the MR system have been the most technologically challenging designs to develop but provide capacity for simultaneous whole‐body imaging and the potential for new diagnostic and research capability (Delso & Ziegler, 2009). Integrated system architecture was long the ‘holy grail’ of MR‐PET technology and with the development of novel PET detector technologies, commercial systems are now available. Although the current generation of hardware is now capable of providing simultaneously acquired MR and PET data sets, the development of integrated data processing and analyses is currently an area of active research. Much research is now focused on developing processing pipelines that lead to an improved synergy of the two data sets, including optimal data acquisition and joint post‐processing methods for both modalities, and novel applications based on the underlying complementary and mutual information.

From the perspective of applications in neuroimaging, simultaneous MR‐PET can be useful in many situations. Brain PET is unrivalled in the neuroimaging field for the quantification of molecular targets, and so the combination of PET data with MR greatly expands the physiological targets that can be examined by neuroscientists. For example, in fundamental neuroscience, the dual modality system can be used to understand the inter‐relationship between cerebral glucose metabolism and BOLD‐fMRI activity (Wehrl et al., 2013). In cognitive neuroscience, simultaneous MR‐PET can be used to understand the neurochemical bases of cognition, human brain development and age‐related degeneration and disease states (Jones, Rabiner, & PET Research Advisory Company, 2012). In clinical neuroimaging, simultaneous MR‐PET using [18‐F]FDG and amyloid imaging (e.g., [11‐C]PiB) can provide diagnostic information of structural, functional, metabolic and amyloid deposits in brain regions in the work‐up of patients with dementia. In neuro‐oncological cases, MRI together with [18‐F]FDG PET can help to assess the identification, localisation and characterisation of tumours, particularly after treatment. Complementary to MRI, PET imaging can be used to elucidate the metabolic status of a tumour including transportation of amino acids, cellular proliferation and tissue hypoxia (Catana, Drzezga, Heiss, & Rosen, 2012). With the simultaneous MR‐PET system, more effective staging and monitoring of diseases can be achieved by combining the diagnostic features of both MR and PET.

In the current literature, several papers provide excellent overviews of the state‐of‐the‐art of MR‐PET instrumentation (Herzog & Lerche, 2016; Vandenberghe & Marsden, 2015), comparison of existing attenuation correction methods (Ladefoged et al., 2017), review of motion correction strategies in whole body MR‐PET (Furst et al., 2015), as well as an overview of neurologic applications (Catana et al., 2012) and whole body oncological applications (Antoch & Bockisch, 2009). In this review, we comprehensively review the challenges and opportunities in simultaneous MR‐PET instrumentation, data acquisition and processing with a particular focus on neuroimaging and its applications in systems neuroscience and clinical neurosciences. Using a simultaneous functional MRI‐functional FDG‐PET example, we will highlight the unique opportunities that MR‐PET now provide for neuroimaging research applications.

2. SIMULTANEOUS MR‐PET SYSTEM DESIGN

A PET scanner detects annihilation photons via a scintillating crystal that creates a burst of light that is detected by photon detectors (Bailey, 2005). An MRI system operates in an elegant interaction of a strong static magnetic field, a spatially varying magnetic field and a radio frequency (RF) produced magnetic field. Thus, inclusion of a PET detector inside the MRI bore can affect the operation of both MRI and PET subsystems: the high static magnetic field and switching gradients can interfere with the electronics of the PET detectors, and the presence of the PET detector causes field inhomogeneities, eddy currents and electromagnetic interference to the MR system (Vandenberghe & Marsden, 2015). However, studies suggest that it is possible to obtain MR and PET images with limited artefacts using a simultaneous system (Delso et al., 2011; Grant et al., 2016). This section reviews the major design considerations for a simultaneous MR‐PET system for imaging the brain, focusing first on the development of novel PET detectors suitable for the MR environment, then examining the effects of PET detector technology on the MR system.

2.1. MR compatible PET detectors

It is a nontrivial task to develop PET detectors that can operate inside an MR system without interference to the signal generation and detection of both modalities. The initial hurdle of hybrid MR‐PET scanner design was the separation of photomultiplier tubes (PMTs) and scintillators, as PMTs cannot operate within the strong magnetic field. As such, early efforts to develop a hybrid system focused on transporting the scintillation light from scintillator crystals inside the MR scanner to PMTs located outside the main magnetic field via optical fibres (Shao et al., 1997).

Unlike PMTs, avalanche photodiodes (APDs) have low gains and are more sensitive to temperature changes, but can operate inside a strong magnetic field and this has been shown up to 9.4 T. Thus, the second generation of hybrid systems used APD PET detectors inserted into the MR magnet bore (Catana et al., 2006; Grazioso et al., 2006; Judenhofer et al., 2008; Woody et al., 2007). Insert designs resulted in MR signal to noise ratio decreases of 15%, and APD PET detector sensitivity decline of approximately 5–20%; however, images were obtained without significant artefact or distortion (Grazioso et al., 2006). Furthermore, the insert design is complex to setup and operate. These image quality and workflow issues led to the development of fully integrated MR‐PET systems which integrated the PET detector between the MR scanner gradient and the RF coils. Using a Siemens Biograph mMR, Delso et al. (2011), reported no significant distortions or visible interferences on 3D T2 weighted turbo spin‐echo MR or [18‐F]Fluoride PET images.

Recently, silicon photomultipliers have been introduced for integrated simultaneous MR‐PET for both clinical and preclinical imaging. Compared with APD, SiPMs provide a binary response to excitation (Geiger mode) which results in significantly better timing resolution (below 1 ns FWHM) and reduction of background counts (Otte et al., 2005). The improved timing resolution enables time‐of‐flight (TOF) imaging in MR‐PET, which can improve image quality in large field of view (e.g., whole‐body PET). However, compared with PMT, both APD and SiPM are more sensitive to temperature changes (Table 2 in Spanoudaki & Levin, 2010), and a sophisticated cooling design is therefore necessary in the MR‐PET scanner.

Table 2.

Summary of the advantages and disadvantages of the AC approaches (Izquierdo‐Garcia & Catana, 2016)

MR‐based methods	Atlas‐based methods	PET‐based methods
Advantages
‐ Easy implementation ‐ Low computational cost ‐ Whole‐body applicability	‐ Bone identification ‐ Continuous LACs ‐ Noise, bias	‐ Robust to artefacts ‐ MR time for clinical use ‐ Whole‐body applicability
Disadvantages
‐ Bone/air segmentation ‐ Discrete LACs ‐ Noise, bias	‐ High computational cost ‐ Subject‐specific variability ‐ Whole‐body application	‐ High computational cost ‐ Accuracy ‐ Extended acquisition time what caused higher exposition

Another recent development in PET detector technology is the development of digital SiPMs, in which SiPMs are fabricated into microcell arrays with the signals from each microcell summed digitally. A fully digital SiPM (dSiPM) can potentially offer excellent spatial and temporal resolution. These characteristics make dSiPM a desirable solution for simultaneous MR‐PET imaging. However, as a prototype technology, it has been demonstrated that dSiPMs tend to generate electromagnetic noise, potentially degrading MR image quality (Weissler et al., 2015). dSiPMs are also potentially affected by gradient switching in the MR subsystem (Wehner et al., 2014). Thus, further work is required to develop dSiPM technology for reliable and noise free applications in simultaneous MR‐PET imaging.

Table 1 summarises the basic performance characteristics of different PET detector technologies (Hergert, 2016). One important motivation behind the development of new photodetector technologies for MR‐PET is improvement in spatial and temporal resolution. While there is an inherent physical limit to the spatial resolution in PET, the intrinsic resolution of the technology is also closely related to the PET detector gamma ray positioning accuracy, and the distance between coincident detectors (Lecomte, 2009). Current state‐of‐the‐art MR‐PET scanners offer a ~4 mm isotropic PET resolution for imaging the brain (even though most clinical images are reconstructed with 1.5–3 mm isotropic voxel size for MR‐PET brain imaging) (Delso et al., 2011), while anatomical 3 T MR can easily achieve ~1 mm isotropic resolution.

Table 1.

Characteristics of photodetectors for PET (Hergert, 2016)

Photodetectors	PMT	APD	SiPM
Spectral coverage (nm)	115–1,700	190–1,700	320–900
Gain (μ)	105–106	<100	105–106
Active area (mm2)	<12,000	<100	<10
NEP (W/$\sqrt{\mathit{Hz}}$)	>2 × 10−17	>1 × 10−15	>6 × 10−16
Rise time (ns)	>0.15	>0.35	>1
Time jitter (ns)	>0.05	>0.2	>0.2
Peak QE (%)	<40	<90	< 40 (PDE)
Bandwidth (Hz)	<2x109	<1x109	NA

2.2. MR static magnetic field and gradient components

Much research into the issue of between‐modality interference has focused on the effects of MR on PET, with the general consensus that the effect of MR on PET can be minimised as long as the PET detector and read‐out electronics are properly shielded (Wehrl et al., 2011). On the other hand, the effect of PET on MR is a more complex issue to examine, as the effects are likely to differ according to the MR imaging sequences that may generate differential susceptibility artefacts and RF interference (Vandenberghe & Marsden, 2015; Wehrl et al., 2011). The physical presence of the PET scanner components within the magnet bore can affect the homogeneity of the static magnetic (B ₀) field and degrade the linearity of the MR gradients, due to the magnetic susceptibility of the PET detectors and their associated shielding electronics. Therefore, only MR compatible PET detectors can be used, and the static B ₀ field must be recalibrated when the PET detectors are fully integrated into the MR magnet bore. Furthermore, to avoid operating frequency interference, filters must be placed between the MR gradient and radiofrequency components and the PET detector electronics.

Spatial and temporal stability of the integrated system are critical factors, especially for functional MRI (fMRI) and dynamic PET imaging in neuroimaging experiments. Because of the confined area in which the PET detectors are positioned within the MR system, the PET components must be isolated from the gradient coil. Advanced functional and diffusion MRI of the brain requires rapid gradient switching, running under full duty cycle over long‐time periods. The rapidly switching magnetic fields can induce eddy current loops in any conductive components in the PET circuitry. In addition, the gradient switching may increase the temperature of components, which can induce significant drifts in the gain of solid‐state photodetectors such as APDs and SiPMs. These effects degrade the energy resolution of PET detectors, cause sensitivity variations and result in errors when quantifying the PET images (Espana et al., 2010). Lastly, fast switching of gradients causes mechanical vibrations of the coils which can compromise the robustness and lead to a reduced lifespan of the PET detectors and electronics. Thus, to ensure image quality and stability of the PET data, the PET detector rings placed inside the gradient coils must be physically well separated from the coils.

2.3. RF components and hardware attenuation

The MR receive pathways are highly sensitive and shielded within a Faraday cage to detect the very small nuclear magnetic resonance (NMR) signals in tissue, and to minimise frequency interference. Integration of PET and the MR RF components requires special consideration to avoid crosstalk between the operating frequencies in both systems. For example, RF interference to PET front‐end electronics may cause an increase in PET random count rates, triple coincidence count rates and dead time problems (Delso & Ziegler, 2009). To avoid interference PET detectors and electronics within the RF pathways, most system designs place PET components outside RF transmit and receive pathways. Recent development on RF‐penetrable PET insert by electrically floating PET detectors relative to MRI system shows potential utilisation of build‐in body coil for MR imaging (Grant, Lee, Chang, & Levin, 2017). In general, integration of the two components requires additional shielding which must be carefully considered, as the shielding itself may degrade the uniformity of the static B ₀ magnetic field or interact with the gradients of the scanner (Truhn, Kiessling, & Schulz, 2011).

Any object placed within the field‐of‐view of the PET scanner will require accurate correction for attenuation. In an integrated system, it is inevitable that MR hardware components are placed between the patient and PET detectors. Fixed hardware structures, such as the scanner bed and the embedded RF coils, and removable elements such as surface RF coils, headphones, head coil mirror and medical positioning foam, all contribute significantly to attenuation of the PET signal (Wagenknecht, Kaiser, Mottaghy, & Herzog, 2013). Correction for attenuation from fixed hardware can be standardised using a precalculated attenuation map and the normalisation factors during daily calibration. For human neuroimaging, MR head/neck coils are rigid and positioned at known locations with respect to the magnet system, so a predetermined attenuation correction map can be used for correction. However, the conventional high‐density head coils (i.e., coils with more than 32 elements and those manufactured using high‐density materials) can significantly attenuate PET signals, and hence are not commonly used in MR‐PET imaging. This limits the usage of advanced MR acquisitions that require high acceleration factors in parallel MR imaging and/or multiband imaging. Research is currently focused on developing alternate designs of these advanced coils for use with integrated MR‐PET scanners (Sander et al., 2015).

3. MR‐PET DATA ACQUISITION AND IMAGE RECONSTRUCTION

The integrated MR‐PET scanner enables acquisition of information from both modalities simultaneously. This, however, adds complexity in data acquisition, correction and image reconstruction. In a clinical setting, most of the imaging workflow needs to be completed within an hour to be cost‐effective. However, in research settings such workflow time constraints are a less important issue. Figure 1 shows an overview of data flow during a simultaneous MR‐PET imaging experiment and highlights the unique modules associated with the dual modality system. A detailed review on patient centric workflow consideration in MR‐PET imaging can be found in (von Schulthess & Veit‐Haibach, 2014). Here, we will focus on technical aspects of the data acquisition and reconstruction.

Figure 1.

Data flow in simultaneous MR‐PET imaging. The following modules are unique to the dual modality: MR based AC; MR derived motion parameters; joint MR‐PET image reconstruction

3.1. MR‐PET data acquisition strategies for brain imaging

Protocol optimisation requires careful consideration of the anatomy of interest and specific imaging target (e.g., tumours, glucose metabolism, etc.). For imaging of the head alone, most protocols can be completed in a single bed position. For head and neck studies two bed positions may be required to cover the required extended field of view.

Optimal simultaneous acquisition of MR and PET data requires the development of novel acquisition methods to fully capitalise on the mutual and unique information provided by each modality. Merely implementing existing techniques that have been developed for discrete MR and PET technologies to simultaneously acquired data does not provide significantly greater value beyond sequential MR and PET acquisitions. Thus, it is important to evolve the data acquisition protocols from simultaneous MR‐PET to synergistic MR‐PET techniques. By developing image acquisition procedures that leverage the complementary information of simultaneous acquisitions, it is possible to provide new information that cannot be obtained using each method alone. For example, the recent development of simultaneous fMRI and fPET data acquisition protocols (see Section 4) enables acquisition of BOLD and dynamic [18‐F]FDG metabolism images from a unified stimulation paradigm.

3.2. Attenuation correction (AC) in MR‐PET

In order to obtain quantitative PET images, it is necessary to correct the PET data for photon attenuation and scatter [for a review see (Martinez‐Moller & Nekolla, 2012)]. Attenuation of photons in a PET image occurs due to scattering throughout the biological tissue of the patient (Compton scattering), or through attenuation of the photons through the scanner hardware. Attenuation due to scanner hardware is a more significant problem in MR‐PET than in other technologies (e.g., PET/CT), as additional items such as coils and positioning foam are present within the FOV (see Section 2.2).

Determination of the patient attenuation map is a challenging problem in MR‐PET and an area of active research development. Unlike CT scans, which directly measure the tissue absorption properties of X‐rays, MR signals are based on proton density and relaxation properties of tissues, and do not provide information on tissue electron density and photon attenuation properties. Since bone has a very short T2* relaxation time, it is very difficult to distinguish bone from air using the most commonly available MR sequences. This means that magnetic resonance based attenuation correction (MR‐AC) estimation is more difficult than CT‐based attenuation correction methods (CT‐AC). Three are currently three main categories of AC methods applied in simultaneous MR‐PET: MR image segmentation based, atlas‐based and PET emission‐based methods. The first two categories are widely used in brain imaging.

3.2.1. MR based attenuation correction (AC)

MR based attenuation correction methods rely only on MR images to segment voxels into tissue classes: for example, air, bones, tissue and water. Linear attenuation coefficients (LACs) are then assigned to each tissue class. Ultrashort Echo Time (UTE), Zero Echo Time (ZTE) and Dixon MR sequences are most commonly used. UTE and ZTE methods provide images of tissues and organs with very short T2 values and provide contrast for bones and ligaments. Dixon methods make use of the resonant frequency difference between water and fat to generate water images and fat images. For neuroimaging, the skull results in significant attenuation, so UTE, ZTE and Dixon in combination of bone atlas are widely used, whereas the Dixon methods are routinely applied in whole body imaging due to their scan efficiency and calculation of multiple tissue classes.

For UTE methods, two echo times are used to calculate a T2 map of the head from which bone maps are then segmented and mapped to attenuation coefficients (Cabello et al., 2015; Catana et al., 2010; Juttukonda et al., 2015; Keereman et al., 2010; Ladefoged, Benoit, Law, Holm, Kjaer, et al., 2015). These methods provide reliable bone segmentation images, but nevertheless segmentation errors can occur at air‐tissue interfaces. Advanced algorithms can also be used to map UTE images to the attenuation coefficients. For example, the feed forward neural network (FFNN) algorithm (Ribeiro, Kops, Herzog, & Almeida, 2014). Berker et al. (2012) proposed a UTE triple‐echo (UTILE) MR sequence combining UTE for bone detection with Dixon water‐fat separation. The combination of UTE and Dixon enables classification of air, bone, soft tissue and adipose tissue, but may result in bone misclassification at the paranasal sinus area. Recently ZTE methods have been applied for the purpose of attenuation correction (e.g., Huang et al., 2015; Wiesinger et al., 2016). In contrast to UTE, ZTE is a proton density‐based method, which enables gradients during RF, resulting in a nominal echo time of 0. The proton density image is then used to obtain attenuation coefficients of air cavities, bone and soft tissue.

3.2.2. Atlas and template based AC

Typically, an atlas consists of pairs of anatomical MR images (or an MR template) and the corresponding CT images (or an AC template). The acquired MR anatomical images (e.g., T1‐weighted images) are co‐registered with the MR atlas/template using nonrigid registration methods or probabilistic methods, and the same transformation is applied to the atlas CT images to obtain a subject‐specific AC map (Wagenknecht et al., 2013). Schreibmann et al. proposed a multimodality optical flow deformable model where AC maps are created by deforming a standard CT template to match the subject's MR anatomical images (Schreibmann et al., 2010). This method provided geometric accuracy with less than 3 mm discrepancy between the MR‐based method and the CT‐based method for bone and skin. Pseudo‐CT methods (Burgos, Cardoso, Thielemans, Modat, 2014; Hofmann et al., 2008; Koesters et al., 2016) register the atlas/template images into the subject's MR image. The resultant deformation is then used to warp the atlas CT into the subject space. Pseudo‐CT AC is obtained by combination of all the atlas CT images. Roy et al. proposed a patch matching method to synthesise the AC map from dual‐echo UTE images and CT images (Roy et al., 2014). The combination of the matched patches is done using a Bayesian framework and no explicit registration is needed. This method provides AC maps that have an excellent correlation (0.99) with CT‐based AC maps. Izquierdo‐Garcia et al. proposed a segmentation and template method based on SPM8 (Statistical Parametric Mapping 8) software (Izquierdo‐Garcia et al., 2014). To obtain an MR‐CT template, MR images were first segmented into six tissue classes, and then nonrigidly co‐registered these tissue classes to form a template space. Finally, CT images were transformed into the template space, and the AC template was generated by averaging all transformed CT images. This method resulted in reconstructed PET images having less than 4% mean relative errors compared to the CT based approach.

3.2.3. PET‐based AC

In the early PET only systems that included germanium‐68 rod sources, attenuation information could be estimated from the PET transmission scan. Later (and more recently) the maximum likelihood reconstruction of attenuation and activity (MLAA) algorithm estimated attenuation from the emission data (Benoit et al., 2016; Mehranian & Zaidi, 2015; Nuyts et al., 1999; Salomon, Goedicke, Schweizer, Aach, & Schulz, 2011). Although these methods show promise for accurate attenuation correction of PET images from MR‐PET scanners in the presence of MR metallic artefacts, they still produce noticeable quantification bias in the context of brain PET imaging (Mehranian, Arabi, & Zaidi, 2016). Berker and colleagues (Berker, Kiessling, & Schulz, 2014) showed further improvements in the accuracy of attenuation correction during back‐projection image reconstruction by taking into account scatter coincidences. Including an MR prior in the joint estimation of TOF PET activity and attenuation coefficients have recently shown promising results (Ahn et al., 2018; Mehranian et al., 2017). Mehranian et al. (2017) demonstrated a less than 5% error in brain PET quantification using their algorithm.

PET transmission data can also be used to obtain attenuation information. Using an annulus transmission source, Mollet et al. have demonstrated more accurate estimation attenuation maps can be obtained compared with MR based methods (Mollet et al., 2014; Mollet, Keereman, Clementel, & Vandenberghe, 2012), but this technique requires additional source and complex experiment setup. A more promising approach is to use the background radiation of the LSO scintillators to simultaneous acquire both the emission and transmission data (Rothfuss et al., 2014).

3.2.4. Methods comparison and validation

Attenuation correction is a priority area for development in MR‐PET, as noncorrected images show substantial spatially variant artefacts across the brain, and significant [up to 25%; (Andersen et al., 2014)] underestimation of radiotracer uptake. A comparison of the advantages and disadvantages of each AC method category is presented in Table 2. It is widely acknowledged that there is much room for improvement in the vendor‐supplied AC routines. For example, the Dixon and UTE AC methods have been shown to yield segmentation errors near the base of the skull and frontal sinuses [UTE; (Dickson, O'Meara, & Barnes, 2014)] and underestimate the activity concentration near the cortical bones (Andersen et al., 2014) compared to CT‐AC reconstructed images. Similarly, the atlas‐based correction methods have led to errors in the cerebellar regions (Sekine et al., 2016) which is a significant issue for kinetic studies that use the cerebellum as a reference region.

MR based AC methods are particularly challenged in accurately segmenting bone from air. Segmentation typically requires that all voxels within the same tissue class have the same linear attenuation coefficients (LAC). However, more accurate correction may be possible using continuous LAC values within tissue types. For example, we (Baran et al., 2017) recently showed that a continuous LAC UTE‐based approach showed more accurate bone attenuation and lower overall reconstruction error, compared to a fixed LAC UTE‐based approach. For atlas‐based methods, the correction accuracy critically depends on the accuracy of the atlas itself, and different demographic and disease groups require different atlases to minimise bias. For example, the density of the cortical bone changes significantly across the adult lifespan, with different trajectories for males and females (Thompson, 1980). Furthermore, atlas‐based methods need to account for inter‐individual variability to achieve high accuracy and are computationally demanding, which limits their application in the clinical environment.

Recently, machine learning methods have been developed to generate pseudo‐CT AC maps (Leynes et al., 2018; Liu, Jang, Kijowski, Bradshaw, & McMillan, 2018). Deep learning neural networks are trained using pairs of MR (e.g., UTE/ZTE, T1 weighted, Dixon) and CT images. Liu et al. applied a convolutional auto‐encoder network for segmentation of human head T1 weighted images into three AC classes (i.e., bones, soft tissue and air). Similarly, Leynes et al. use deep convolutional neural networks to directly estimate continuous AC values from ZTE and Dixon images. These deep learning‐based methods have shown significantly improved PET quantification accuracy when compared with MR based AC methods.

Thus, development of new simultaneous MR‐PET AC methods is an area of intense focus, with new methods being developed, validated and benchmarked against CT‐AC. In a dementia study, (Cabello et al., 2016) compared the performance of attenuation in various methods and found that the quantification accuracy amongst recent methods was similar (i.e., ±5%) except for the UTE and Dixon techniques provided by the scanner vendor which provided quantitative regional errors of 10–20%. In a recently published multicentre evaluation of 11 MR‐AC methods, (Ladefoged et al., 2017) demonstrated that all MR‐AC methods have an average global performance within ±5% of the gold‐standard CT‐AC for [18‐F]FDG PET. The best results were reported for the atlas‐based methods (Burgos, Cardoso, Thielemans, Modat, 2014; Izquierdo‐Garcia et al., 2014; Merida, Costes, Heckemann, & Hammers, 2015) and for the segmentation‐based method (Ladefoged, Benoit, Law, Holm, Hojgaard, et al., 2015). Note although these studies are comprehensive, the use of CT‐AC as the gold standard for validating PET reconstructions from MR‐PET data sets is debatable. There may be registration errors between the CT and MR images, and there are differences between PET and CT in photon detection energy (Burger et al., 2002). Therefore, further clinical evaluation of various attenuation correction methods is still needed.

3.3. Head motion correction in the simultaneous MR‐PET

During an imaging session subject motion can lead to degradation of the acquired images and the occurrence of false positives in the analytical results. In most situations, head motion is a rigid‐body motion. The current motion correction routines in MR‐PET imaging can be grouped into three categories: external hardware based, PET data based and MR data based. These methods have been developed for each modality independently, and to date little work has been done in the area of multimodality motion correction.

3.3.1. Prospective motion correction using external hardware

External cameras or sensors can be used to track head motion and communicate motion parameters to the imaging system for prospective motion correction. Measuring motion and updating the imaging acquisition protocol in real time is very useful for MR imaging, but it is somewhat less critical for PET due to lower image resolution, and it is unnecessary to update PET detector geometry for prospective motion correction due to its symmetrical layout. The mathematical basis of prospective motion correction for any imaging modality relies upon the maintenance of a constant relationship between the imaged object and the image itself, even under motion (Maclaren, Herbst, Speck, & Zaitsev, 2013). For example, Picard and Thompson (1997) use two CCD cameras together with three LEDs fixed on the subject's head to monitor head movement for PET imaging. In MRI, Callaghan et al. (2015) evaluated an optical system based prospective motion correction to achieve high resolution MR images. Alternatives to such optical methods include field detection methods, for example, Ooi, Krueger, Thomas, Swaminathan, and Brown (2009) use markers attached to the subject to measure the scanner gradients to track motion, and MR navigators, for example, tracking signals from rigid fat tissue motion (Engstrom, Martensson, Avventi, Norbeck, & Skare, 2015), which quantify translations and rotations of the object during data acquisition. Huang et al. (2014) developed MR micro‐coils attached to the brain and used dedicated MR sequences for tracking motion. In their detailed review of prospective motion correction methods, Maclaren et al. (2013) argued that external hardware‐based methods provide greater flexibility, improved correction of large motion and better compensation for spin‐history effects compared to retrospective correction techniques in MRI. Despite these advantages, Maclaren et al. (2013) argued that a number of challenges must still be addressed before prospective motion correction routines can be widely implemented, including data quality issues, the choice of a motion tracking marker and its position on a subject, and the inability to correct for higher‐order motion and static B ₀ magnetic field inhomogeneities introduced by the movement itself. Another practical issue with the use of external hardware‐based motion correction technologies is the complexity of integrating them into either MRI or PET, with their incorporation into dual modality systems being even more challenging. Last but not least, patient compliance also need to be addressed for routine usage of hardware‐based approaches.

3.3.2. PET data driven motion correction methods

The relatively poor spatial resolution of traditional PET scanners has meant that the technique was previously considered relatively robust against subject movement. However, with the increased spatial resolution provided by the current generation of PET detectors, even small motion can degrade PET image quality. Most PET data driven methods have been developed mostly to correct for respiratory and cardiac motion in whole‐body PET (Catana, 2015; Furst et al., 2015), but some of the recent work has introduced motion correction for brain studies. Thielemans, Schleyer, Dunn, Marsden, and Manjeshwar (2013) applied principal component analysis (PCA) on the raw dynamic PET list‐mode data to estimate head motion based on principal components. This method successfully detected sudden movements retrospectively in list‐mode PET data. Thielemans et al. also examined PCA‐based motion correction in TOF‐PET list mode data and concluded that PCA‐correction provides retrospective detection of head motion in raw TOF‐PET data, improving image sharpness and potentially reducing intra‐frame motion.

Another approach to derive motion parameters is to rigidly co‐register the reconstructed PET images. To achieve fine motion modelling, it is desirable to have shorter temporal PET frames (i.e., higher temporal resolution). However, there is always a tradeoff between signal to noise (SNR) and temporal resolution. Mukherjee et al. (2016) proposed a pre‐processing pipeline to achieve high registration accuracy with short frames (e.g., 5 s). These processing steps combine Gaussian smoothing for noise reduction, median filtering for edges preservation and Gamma contrast enhancement for each short temporal frame. This approach is promising for retrospective motion correction of nonattenuation corrected PET brain images.

3.3.3. MR‐informed motion correction

MR informed motion correction can be superior compared to PET data driven methods, especially at low dose PET experiments or dynamic PET image reconstruction, due to high SNR and anatomical accuracy in MR images [Furst et al. (2015) demonstrated this in whole‐body applications]. A simple approach to MR informed motion correction for the brain is to perform a rigid‐body co‐registration of MRI EPI volumetric images that can be acquired periodically throughout a hybrid imaging protocol. The derived motion parameters can then be used to correct motion in the PET data acquisition by either realignment of PET frames (Catana et al., 2011; Keller et al., 2015; Ullisch et al., 2012) or by directly correcting the raw list mode PET data prior to reconstruction (Ullisch et al., 2012). This simple approach is complicated by the fact that simultaneously acquired MRI and PET data may be spatially mis‐registered, as the centre of the PET FOV may not fall within the isocentre of the MR scanner (Catana et al., 2011). In this case, an additional transformation matrix must be obtained by scanning a phantom visible to both modalities for application during the motion correction routine. This calibration matrix together with other post‐processing steps such as EPI distortion correction are needed to achieve accurate motion correction. Another issue with this approach is that EPI scans cannot be used to track motion during anatomical MR acquisitions such as T1 or T2 and inserting multiple EPIs into the MR protocol lengthens the total acquisition time unless used in EPI based protocols, for example, BOLD fMRI.

MR navigators are also widely used to track head motion in order to achieve prospective motion correction. These navigators are embedded into the MR pulse sequences. For example, in the embedded cloverleaf navigator approach (van der Kouwe, Benner, & Dale, 2006), a navigator is inserted at every TR of a 3D‐FLASH sequence, providing an estimate of the rigid‐body motion relative to an initial k‐space map. Navigator methods can be used to compensate for motion in real time. Other examples of navigator sequences include FatNav (Engstrom et al., 2015), vNav EPI navigators (Tisdall et al., 2016), PROMO which is a 3D spiral navigator (White et al., 2010), k‐space orbital navigators (Fu et al., 1995) and k‐space spherical navigators (Johnson, Liu, Wade, Tavallaei, & Drangova, 2016; Welch, Manduca, Grimm, Ward, & Jack, 2002). Another technique used for fMRI prospective motion correction is the Prospective Acquisition CorrEction (PACE) method (Thesen, Heid, Mueller, & Schad, 2000). This method estimates motion matrices between two consecutive fMRI volumes and then applies motion adjustments in real time for the subsequent scans. In comparison to EPI volumes, MR navigators in general provide faster motion sampling (e.g., in every TR of milliseconds to seconds range) and have minimal impact on total acquisition and processing time. However, many of these navigators are not available in the vendor provided sequences due to their complexity in implementation and validation.

Each of the approaches discussed so far have a number of advantages and disadvantages (summarised in Table 3). While MR informed motion correction is considered the gold standard for motion correction of MR and PET images, existing methods are limited in their application. EPI co‐registration based correction methods are relevant only for simultaneous MR‐PET acquisitions that use EPI acquisitions. Navigator approaches are in general not available from the vendors. Our group (Sforazzini et al., 2017) has recently developed a method that extracts motion parameters from all available image contrasts (EPI, T1, T2, diffusion, etc.). The technique uses existing neuroimaging toolboxes (FSL, MRtrix3), and results in significantly improved image quality and reduced motion artefact in PET images simultaneously acquired with MR images. In the future, a combination of both MR informed motion correction and PET data driven approaches could take advantage of each modality and provide a complete motion correction strategy.

Table 3.

Summary of the advantages and disadvantages of difference motion correction methods

External hardware based methods	PET data driven methods	MR based methods
Advantages
‐ Excellent temporal resolution of motion ‐ High spatial accuracy ‐ No additional scan times ‐ Prospective motion correction	‐ No additional scan times ‐ No acquisition change needed	‐ Very good temporal resolution ‐ No additional acquisition needed (if MR images are directly used) ‐ Capable of prospective motion correction
Disadvantages
‐ Complex to setup ‐ Need to make sure no interference to MR and PET ‐ Cost of the hardware	‐ Temporal resolution often limited by PET statistics ‐ Spatial accuracy is limited by PET resolution ‐ Only retrospective correction, and no real‐time correction	‐ MR sequence modification needed if navigators are used ‐ Spatial accuracy is limited by MR image/navigator resolution

3.4. Recent developments in MR‐PET image reconstruction

3.4.1. Joint MR‐PET image reconstruction

The goal of synergistic joint MR‐PET image reconstruction is to improve the image quality of both modalities, such as reduction of MR image artefacts and reduction of PET partial volume effects.

Essentially, the optimization problem of joint MR‐PET image reconstruction can be written as (Ehrhardt et al., 2015; Knoll et al., 2016; Mehranian et al., 2018; Sudarshan, Chen, & Awate, 2018): $\underset{u,v}{\min \mathrm{\alpha }}{‖\mathit{Mu}-u_0‖}_2^2+\mathrm{\beta }\int \left(\mathit{Pv}-v_0\log \left(\mathit{Pv}\right)\right)+R\left(u,v\right)$, where u and v are objective MR and PET images, respectively u₀ and v₀ are the acquired raw MR and PET data, α and β are scaling factors, $\mathcal{R}\left(u,v\right)$ models anatomical prior, and M and P are the MR encoding matrix and the PET system matrix, respectively. The first term in the above equation reconstructs MR images, u, the second term reconstructs PET images, v, and the last term models the mutual information between the two and provides a regularisation term during reconstruction. The image reconstruction simultaneously seeks optimal solutions for both u and v, and takes advantage of the synergy of the two data sets.

To date, several joint reconstruction methods for static MR‐PET acquisitions have been proposed. Ehrhardt et al. (2015) worked from the premise that MR and PET data are likely to show similar anatomical structures, such that u and v depend on a common object and are related to each other. The prior knowledge of joint images is measured by the parallelism of the image gradients at each spatial location using the parallel level set (PLS) approach. In 2D phantom studies, the proposed method demonstrated less MR and PET reconstruction errors and sharper edges compared with separate reconstructions. Knoll et al. proposed a joint reconstruction using a multichannel image regularisation approach (Knoll et al., 2016). Specifically, this method uses a second order Total Generalised Variation (TGV) regulariser to exploit the anatomical correlations between the two different imaging modalities. The method shares information about the underlying anatomy during the reconstruction while also potentially preserving information unique to each modality. Another advantage of this method is the parameterisation of the optimisation problem is set to be convex, and therefore, the global optimisation is guaranteed. Results from [18‐F]FDG simulations and a range of MR image contrasts in two human subjects using the joint TGV prior demonstrated improved PET image quality, resolution and quantitative accuracy and reduced MR parallel imaging artefact in comparison with conventional separate reconstruction of two modalities (Knoll et al., 2016). Mehranian et al. (2018) extended the joint total variation prior to a nonconvex one, demonstrating better preserved common boundaries between the two images.

While conventional PET image reconstruction produces a static 3‐dimensional volume of radiotracer distribution, dynamic imaging requires the reconstruction of a 4‐dimensional tracer distribution to allow for kinetic parameter estimation (Reader & Verhaeghe, 2014). Novosad and Reader proposed a method for dynamic MR‐PET reconstruction that incorporates spectral temporal basis functions and spatial basis functions derived using the kernel method, with anatomical MR images used to constrain the spatial basis function (Novosad & Reader, 2016). Using [18‐F]FDG simulations and [11‐C]SCH23390 (D1 dopamine antagonist) human data, the proposed method resulted in reduced reconstruction errors and decreased spatial error on kinetic parametric maps in the grey and white matter regions of the brain, compared to conventional reconstruction methods.

Joint MR‐PET image reconstruction is an area of active development, and further work with validation and rigorous testing is still needed. While joint reconstruction methods aim to capitalise on mutual information between the modalities, the methods should ensure that structures visible in one modality but not the other should neither be dampened nor transferred to the other modality during reconstruction. This is particularly the case for PET images, where quantitative information could be compromised by inappropriate reconstruction techniques.

4. NEW OPPORTUNITIES USING SIMULTANEOUS MR‐PET IN BRAIN IMAGING

4.1. Neuroimaging in neuroscience

In this section, we will review some of the major developments in the use of simultaneous MR‐PET imaging in neuroscience. Most research in this area has been focused towards using PET to understand the neurovascular coupling that underlies the BOLD response. fMRI relies on the haemodynamic BOLD response, which is a composite of all neurochemical events in response to neuronal activity. In contrast, PET provides neurochemical mapping of multiple processes including synthesis, reuptake and release, with high specificity. Thus, simultaneous MR‐PET offers great potential for disentangling neurochemical contributors to the haemodynamic response. We will also review studies that have used metabolic or receptor imaging to understand the molecular components of brain function, and how MR‐PET can be incorporated with EEG for advanced neuroscience studies.

4.1.1. Utility of simultaneous MR‐PET in neuroscience

The goal of biomedical imaging technologies in neuroscience is to understand the neural bases of cognition and disease in the human brain in vivo. To this end, MRI, particularly BOLD‐fMRI, represents the cornerstone of human neuroimaging research techniques. It is a noninvasive technique (i.e., does not require exposure to ionising radiation or insertion of electrodes into tissue) that provides excellent spatial resolution of the brain with acceptable temporal resolution (~0.5–3 s; c.f. PET ~1 min; electroencephalography [EEG] ~ 10–20 ms). Despite its widespread use and the increased sophistication of the technique, BOLD‐fMRI has a number of limitations the most important of which is that it is neither a quantitative nor a direct measure of neuronal activity. BOLD‐fMRI infers neuronal activity indirectly from a complex interaction between the cerebral metabolic rate of oxygen, cerebral blood flow and cerebral blood volume which is still incompletely understood (Logothetis, 2008). Without the use of calibrated BOLD fMRI techniques involving hypercapnia, it is not possible to quantify the magnitude of BOLD‐fMRI response between MR scanners, between individuals or even between sessions of the same individual.

The higher spatial resolution and the absence of ionising radiation in MR imaging meant that it quickly surpassed the use of PET to study human brain function. However, the advent of simultaneous MR‐PET technology may lead to a re‐emergence of PET imaging in neuroscience. It has long been recognised that PET provides unsurpassed ability to quantitatively measure brain metabolism and function, using radiotracers such as [18F]FDG and [15O]H₂O, respectively. PET imaging is also unrivalled in its capacity to image a broad range of physiological and neurotransmitter systems critical to our understanding of brain function in health and disease, including drug pharmacokinetics, amino acid transportation, protein synthesis, neuroinflammation and neurotransmitter synthesis and receptor density (Table 4 provides a selective list of radiotracers used in neuroscience research to date).

Table 4.

Selective review of PET tracers used in neuroscience studies

Target	Radiotracer	Reference
Glucose utilisation	[18‐F]FDG
Oxygen utilisation	[15‐O]oxygen	e.g., Frackowiak, Lenzi, Jones, and Heather (1980)
Blood flow	[15‐O]water	e.g., Silbersweig et al. (1993)
Dopamine
D1 receptor antagonist	[11‐C]SCH23390	Derlet, Albertson, and Rice (1990)
	[11‐C]NNC112	Halldin et al. (1998)
D2 receptor antagonist	[11‐C]raclopride	Ehrin et al. (1985)
D2/D3 receptor antagonist	[11‐C]FLB457	Halldin et al. (1995)
	[18‐F]fallypride	Mukherjee et al. (1999)
	[18‐F]desmethoxyfallypride	Dobrossy et al. (2012)
Transporter (reuptake inhibitors)	[11‐C]nomifensine	Aquilonius et al. (1987)
	[11‐C]β‐CIT‐FE	Halldin et al. (1996)
	[11‐C]altropane	Elmaleh et al. (1996)
	[11‐C]d‐threo‐MP	Ding et al. (1995)
	[11‐C]CFT	Hantraye et al. (1992)
Synthesis	[18‐F]F‐DOPA	Garnett, Firnau, and Nahmias (1983)
Other	[11‐C]methyl‐spiperone (dopamine and serotonin receptors)	Arnett, Fowler, Wolf, Shiue, and Mcpherson (1985)
	[11‐C]FBL457	Halldin et al. (1995)
Serotonin
5HT‐1A receptor antagonist	[carbonyl‐11C]WAY‐100635	Hume et al. (1994)
	[11‐C]cis‐FCWAY	Choi et al. (2014)
	[11‐C] MPPF	Shiue et al. (1997)
	[11‐C]trans‐MeFWAY	Saigal et al. (2006)
5HT‐1A receptor partial agonist	[11‐C]CUMI‐101	Milak et al. (2010)
5HT‐1B receptor antagonist	[11‐C]AZ10419369	Varnas et al. (2011)
	[11‐C]P943	Nabulsi et al. (2010)
5HT‐2A receptor agonist	[11‐C]Cimbi‐5	Ettrup et al. (2010)
	[11‐C]Cimbi‐36	Ettrup et al. (2013)
5HT‐2A receptor antagonist	[18‐F]altanserin	Lemaire, Cantineau, Guillaume, Plenevaux, and Christiaens (1991)
	[18‐F]setoperone	Blin, Pappata, Kiyosawa, Crouzel, and Baron (1988)
	[11‐C]NMSP	Lyon et al. (1986)
	[11‐C]volinanserin	Schmidt, Fadayel, Sullivan, and Taylor (1992)
5HT‐4 receptor antagonist	[11‐C]SB207145	Kornum et al. (2009)
5HT‐6 receptor antagonist	[11‐C]GSK215083	Parker et al. (2012)
Neuroinflammation
Translocator protein (TSPO)	[18‐F]DPA‐714	Reynolds et al. (2010)
	[11‐C]Ro 5–4,864	Farges et al. (1994)
	[11‐C](R)‐PK11195	Trapani, Palazzo, de Candia, Lasorsa, and Trapani (2013)
	[18‐F]FEMPA	Hellberg et al. (2017)
	[11‐C]DPA‐713	Scarf and Kassiou (2011)
	6‐[18‐F]PBR28	Damont et al. (2011)
	[11‐C]PBR01	Briard et al. (2008)
Protein
Amyloid	[18‐F]florbetaben	Barthel et al. (2011)
	[18‐F]florbetapir	Choi et al. (2009)
	[18‐F]flutemetamol	Nelissen et al. (2009)
	[18‐F]NAV4694	Cselenyi et al. (2012)
	[11‐C]PiB	Price et al. (2005)
Tau	[11‐C]PBB3	Hashimoto et al. (2014)
	[3‐H]THK5117	Lemoine et al. (2015)
	[18‐F]THK5351	Ng et al. (2017)
	[18‐F]AV1451 (flortaucipir)	Okamura et al. (2013)
	[18‐F]T807	Chien et al. (2013)

Of course, it has always been possible to acquire both PET and MRI data in the same individuals, and PET neuroimaging studies usually acquire MRI scans to provide the anatomical information required to interpret low spatial resolution PET images. For example, many studies implement the same experimental protocol across two separate scanning sessions using independent MR and PET scanners. However, nonsimultaneous MR‐PET acquisition provides a number of advantages over sequential acquisitions in neuroscience. Firstly, sequential acquisitions that occur hours, days or weeks apart result in significant intra‐individual differences in attention, motivation, sleep status, caffeine and nutritional intake and blood chemistry. This is especially problematic for some cognitive paradigms such as memory and learning tasks that are simply unable to be repeated across testing sessions without learning effects. It is therefore very difficult to control for all experimental confounds when comparing an individual's performance and brain activity between imaging sessions. Secondly, some subject groups such as patients with dementia or advanced neurodegenerative disorders resulting in ataxia or Parkinsonism, may be incapable of withstanding multiple imaging sessions of an hour or more. Simultaneous acquisition of PET and MR data effectively halves the total imaging time for a single subject. Thirdly, mis‐registration of images between modalities becomes a significant issue in nonsimultaneous acquisitions, due to inter‐session differences in subject placement. Many of these issues can be eliminated using integrated scanners that provide nonsimultaneous imaging, where MR and PET acquisitions can be interleaved within a scanning session, eliminating the between‐session differences in participant state (fatigue, motivation) and subject positioning. However, integrated nonsimultaneous acquisitions cannot match the temporal dynamics of brain function between MR and PET as is possible using simultaneous acquisitions.

4.1.2. Simultaneous BOLD‐fMRI/FDG‐PET imaging

A number of studies have used simultaneous MR‐PET to quantify brain activity via cerebral glucose metabolism using [18‐F]FDG PET. The use of glucose metabolism as a direct, quantifiable and metabolically well understood measure of neuronal activity is superior to the use of the blood oxygenation level dependent (BOLD) measure used in fMRI (for a review see (Mergenthaler, Lindauer, Dienel, & Meisel, 2013). Simultaneous FDG‐PET/BOLD‐fMRI offers the opportunity to develop improved models of the physiological processes that underlie the BOLD fMRI signal (Wehrl et al., 2013). Most glucose metabolism in the brain occurs at the synapses (Harris, Jolivet, & Attwell, 2012), and the regional cerebral metabolic rate of glucose metabolism (rCMRGlc) increases linearly with spike frequency (Sokoloff, 1999). While the BOLD‐fMRI signal is more heavily weighted towards the draining veins and macrovessels, the FDG‐PET signal arises primarily from excitatory synaptic activity, which is localised to the grey matter tissue compartment, with a relatively low signal in white matter and blood (Wehrl et al., 2013). As a result of these differences, comparisons of simultaneously acquired FDG‐PET/BOLD‐fMRI have found a spatial mismatch in activity between the two techniques. In a study of the whisker barrel field in the rat, Wehrl et al. found that whisker stimulation resulted in a larger network of activity in FDG‐PET than BOLD‐fMRI, with FDG‐PET showing smaller activation foci than BOLD‐fMRI. These results suggest that FDG‐PET and BOLD‐fMRI provide partially complementary information about neural activity, and that some activated brain regions may show FDG signal increase that is not indexed by the BOLD response. The smaller activation foci in FDG‐PET may indicate improved localisation of neural activity in PET activation maps than in BOLD‐fMRI maps.

Other projects have focused on improving the temporal resolution of FDG‐PET. Standard FDG‐PET imaging in human neuroscience research uses the bolus administration method, where an FDG dose is administered to the participant s a bolus injection, which is then followed by ~30mins quiet rest for the glucose uptake. Following the uptake period, the subject is positioned in the scanner for imaging. Thus, the measured FDG‐PET signal largely reflects the integral of neural activity during the uptake and positioning procedure, and not cognitive activity during the scan (Aiello et al., 2015; Villien et al., 2014). Given that the BOLD‐fMRI signal indexes recent neural activity with a temporal resolution of 0.5–3 s, there is a significant temporal confound when comparing simultaneously acquired FDG‐PET and BOLD‐fMRI activation maps. A current focus of research in simultaneous FDG‐PET/BOLD‐fMRI acquisition is to improve the temporal resolution of FDG‐PET imaging to overcome this confound. One procedure is to dynamically track FDG uptake by slowly infusing FDG over the course of the scan. In a proof‐of‐concept study, Villien et al. (2014) demonstrated that a slow continuous infusion technique yields sufficient FDG‐PET signal in the occipital cortex during a checkerboard stimulation to be able to identify activation in the primary visual cortex. Comparable results were recently obtained by (Hahn et al., 2016) in the motor cortex.

Even with the slow continuous infusion technique, the difference in temporal resolution between FDG‐PET and BOLD‐fMRI signals represents a significant challenge for designing brain functional experiments that optimise the use of both modalities. One recent study (Hahn et al., 2017) circumvented this problem by using endogenously triggered (or self‐paced) finger‐tapping and eyes open/closed tasks. During eyes open versus closed, CMRGlc changes in the visual cortex was associated with reduced fMRI activity in the angular gyrus and middle temporal gyrus and reduced axial diffusivity in area V1 white matter tracts. During the finger tapping task, CMRGlc in the motor cortex was associated with reduced fMRI activity in the supplementary motor area and contralateral M1. Interestingly, they found no correlation between task‐specific changes in CMRGlc and BOLD‐fMRI response. Li et al. (2018) demonstrated task‐based experimental designs that are capable of yielding sufficient signal to noise in both FDG functional PET (fPET) and BOLD‐fMRI data sets when acquired using simultaneous experimental protocols. Recent preliminary results indicate significant neural activity can be simultaneously identified in both imaging modalities (see Figure 2), with a temporal resolution of around 1 min for fPET and 2 s for fMRI (Li et al., 2018).

Figure 2.

Simultaneous BOLD‐fMRI/FDG‐fPET activation using a checkboard visual stimulation in a single subject (adapted from Li et al., 2018). Consistent with the results of (Wehrl et al., 2013) in the rat, the results from the two modalities are comparable, but with more focused activity in the FDG‐fPET than BOLD‐fMRI data [Color figure can be viewed at http://wileyonlinelibrary.com]

4.1.3. MR‐PET receptor imaging

Exogenous and endogenous stimuli, including behaviourally relevant stimuli, pharmacological challenges and mood changes, evoke widespread changes in the neurotransmitter system of the brain. These changes in neurotransmission are important for understanding neural function in health and disease. For example, dynamic changes in the dopaminergic system are known to contribute to wide range behaviours including affect, decision‐making and inhibitory control (Cools, Nakamura, & Daw, 2011). Dynamic changes in neurotransmission are also known to contribute to CBV and BOLD changes (e.g., dopamine, Mandeville et al., 2013; GABA, Muthukumaraswamy, Evans, Edden, Wise, & Singh, 2012). However, it has proved challenging to clarify the contributions of specific neurotransmitter systems to dynamic changes in the BOLD response. Simultaneous MR‐PET offers the ability to concurrently measure changes in neuroreceptor occupancy and haemodynamic parameters, and thus disentangle the neurotransmitter contributions to the BOLD response.

Efforts to understand the dynamic contributions of neurotransmitter systems on the fMRI response using simultaneous MR‐PET has been led by the MGH group (e.g., Sander et al., 2013, 2017; Sander, Hooker, Catana, Rosen, & Mandeville, 2016). Much of their work has focused on establishing the contribution of the dopaminergic system, specifically D2/D3 receptors, on CBV and the fMRI response. Sander et al. (2013) showed that PET signals from D2/D3 receptor antagonism with [11‐C]raclopride are spatially and temporally matched to fMRI signals in the basal ganglia. These results suggest that the haemodynamic response is directly linked to D2/D3 receptor occupancy by neurovascular coupling mechanisms. Subsequent studies have used this relationship to understand the mechanisms of dopaminergic receptor agonism. Using the D2/D3 receptor agonist quinpirole, Sander et al. (2016) found an fMRI response following the first, but not the second injection of quinpirole. This suggests that once D2/D3 receptors are internalised, this remain so for several hours.

In another study, Wey et al. (2014) examined the opioid system using simultaneous MR‐PET with [11C] diprenorphine and an experimental manipulation of pain. They found that the majority of regions in the BOLD network in response to pain were not co‐localised with the PET opioid receptor binding, suggesting that they are not sites for direct opioid mediated pain modulation. This result demonstrates that both similarities and differences between the modalities can be informative: regions with both fMRI and PET signal can indicate molecular/neurotransmitter contributors to the BOLD response, and regions showing fMRI or PET signal but not both shows regions where the two signals measure different components of neurovascular coupling.

In summary, while the use of simultaneous MR‐PET to understand dynamic neurotransmitter changes during brain function is still in its infancy, preliminary results show great promise for the technique to inform comprehensive models of brain function beyond the traditional haemodynamic based functional imaging methods.

4.1.4. MR‐PET and EEG

Trimodal brain imaging methods extend the philosophy that underlies bimodal imaging methods and involves the acquisition of three modalities simultaneously. Efforts to develop trimodal neuroimaging have focused on acquiring EEG concurrently with MRI and PET data. EEG data has superior temporal resolution (milliseconds compared to the temporal resolution of fMRI being seconds, and minutes for PET imaging) and measures the summed electrical activity of populations of (mostly pyramidal) neurons. Thus, EEG represents a direct measure of neuronal electrical activity available from the BOLD‐fMRI/FDG‐PET/EEG trimodal neuroimaging approach. However, the poor spatial resolution of EEG limits the inferences that can be made regarding the spatial specificity of brain electrical activity. The inclusion of EEG into tri‐modality brain imaging therefore is motivated by the requirement for a direct index of brain activity at the same temporal resolution as neuronal metabolic activity, which is not possible using fMRI or PET.

The TRIMAGE project is an initiative to develop simultaneous trimodal neuroimaging for biomarker discovery. Biomarker discovery projects have been largely focused towards schizophrenia, since putative schizophrenia biomarkers have been developed using each technology independently, without showing sufficient specificity to be diagnostic. Using the TRIMAGE trimodal system, Shah et al. (2017) examined the relationship between BOLD‐fMRI activity in the default mode network to FDG‐PET uptake in the network and resting‐state EEG. BOLD‐fMRI activity in the default mode network was positively related to FDG uptake in the network, but neither BOLD nor FDG was related to EEG connectivity across frequency bands (δ, θ, α, β1 – 4). BOLD, FDG, EEG and DWI of the default mode were also not related to magnetic resonance spectroscopy measures of glutamate or GABA in the posterior component of the network (precuneus/posterior cingulate). Overall, these results are consistent with results obtained previously (e.g., Aiello et al., 2015; Riedl et al., 2014; Savio et al., 2017) showing that the BOLD‐fMRI response is closely related to metabolic activity as measured by FDG‐PET. While BOLD‐fMRI activity within the default mode network was not related to EEG oscillatory activity, BOLD‐fMRI connectivity between the default mode and sensorimotor network was related to EEG activity (δ, θ, α, β − 1), suggesting that the EEG measures were more sensitive to between network connectivity than within network connectivity.

In a proof‐of‐concept study, Del Guerra et al. (2018) used the TRIMAGE trimodal system to examine BOLD‐fMRI, EEG (specifically, event‐related potentials, ERPs) and glutamate receptor PET ([11‐C]ABP688) in one healthy person and one person with schizophrenia. A mismatch negativity (MMN) task was used, as the EEG signature of MMN is a putative biomarker for schizophrenia (Michie, 2001). The person with schizophrenia showed reduced MMN amplitude, reduced BOLD‐fMRI connectivity in the auditory network and reduced [11‐C]ABP688 uptake relative to the healthy control. While these results are only preliminary, they are important for a number of reasons. Firstly, they demonstrate that high‐quality data can be obtained when acquiring all three modalities simultaneously, which is important given that each modality could introduce measurements errors and signal noise in the other measurements. Secondly, the results demonstrate the potential of the trimodal neuroimaging system to acquire comprehensive data sets that may aid in the early diagnosis of complex and multifactorial psychiatric illnesses including schizophrenia.

Trimodal neuroimaging is also possible on commercially available simultaneous MR‐PET scanners. Golkowski et al. (2017) acquired BOLD‐fMRI, FDG‐PET and EEG in people with disorders of consciousness (minimally conscious/comatose). They found that FDG‐PET but not EEG or BOLD‐fMRI was significantly higher in minimally‐conscious people compared to people with unresponsive wakefulness syndrome, and FDG‐PET was related to the clinical rating of consciousness. On the other hand, EEG band power (δ) but not BOLD‐fMRI or FDG‐PET was significantly higher in patients that improved from unresponsive wakefulness to a minimally conscious state compared to those who did not improve. These results show the power of the complementary information available using the trimodal brain imaging systems in the development of diagnostic and prognostic markers.

In sum, simultaneous MR‐PET imaging may re‐introduce the significant advantages of PET to the neuroscience community. Despite continued improvements in PET imaging methodologies over the last two decades, contemporary neuroscience researchers have been slow to take advantage of the many molecular imaging measures that PET can provide (Jones et al., 2012). Simultaneous MR‐PET offers many advantages to sequential MR and PET acquisitions, and recent improvements in technology, acquisition procedures and analysis pipelines are likely to enable more widespread use of this technology.

4.2. Clinical neuroimaging

MRI has been the key modality for clinical brain imaging for over a quarter of a century. PET has been shown to provide additional molecular sensitivity and specificity to key aspects of clinical brain imaging, particularly tumour characterisation and the monitoring of tumour responses to treatment. PET and SPECT are also invaluable tools for locating hypermetabolic foci associated with epileptic activity. More recently, the development of new PET radiotracers has led to increasing interest in the use of PET for brain imaging research in the context of dementia or suspected dementing illnesses.

The brain is an almost immobile body part and surrounded by anatomical features that yield high contrast in MR and CT imaging modalities, facilitating the use of co‐registration procedures that have been applied to sequentially obtained MRI and PET for many years (Ge et al., 1994; Woods, Mazziotta, & Cherry, 1993). The full potential of simultaneously acquired MR‐PET images will only be realised when simultaneous dynamic imaging with the two modalities is practicable, for example, tissue perfusion with MRI and tracer uptake with PET.

4.2.1. Brain tumours

MRI is very sensitive to the presence of brain tumours but has limited specificity in differentiating high and low‐grade gliomas, and in distinguishing recurrent disease from treatment‐induced necrosis. Early work using [18‐F]FDG showed some increase in specificity with the addition of PET information, but was limited by the very high background uptake of FDG in the brain and the relatively low uptake in some tumours, notably small metastases [reviewed in (Wong, van der Westhuizen, & Coleman, 2002)]. This led to the development of more specific tracers for brain tumour imaging, such as [11‐C]methionine (MET), and the longer‐lived [18‐F]fluoroethyltyrosine (FET), and [18F]fluorodopa (Galldiks, Law, Pope, Arbizu, & Langen, 2017; Langen, Galldiks, Hattingen, & Shah, 2017). These are selectively taken up by neoplastic tissue, with relatively little uptake in normal brain, and essentially no uptake in necrotic tissue. These radiotracers have been critical in differentiating recurrent enhancing tumour from radionecrosis, a common clinical dilemma in tumour monitoring. While MET‐PET was found to be the more useful tracer for identifying tumour recurrence or progression after treatment, FDG‐PET offered some complementary information, and best results were obtained by combining findings from both tracers (Van Laere et al., 2005). Dynamic studies with FET PET have shown promise in glioma grading, distinction of primary gliomas from metastases, identification of the highest‐grade components of tumours (reviewed in Muoio, Giovanella, & Treglia, 2017), and assessment of response to treatment (Albert et al., 2016).

An important issue in the clinical management of brain tumours is to accurately identify the maximal amount of tumour that can be safely resected during surgical procedures, without causing unacceptable neurological deficits post‐surgically. It is known that high‐grade tumour often lies outside the contrast‐enhancing region when using contrast enhanced MR images, and that not all T2 hyperintense tissue surrounding the enhancing component necessarily represents tumour. Recent work combining MR‐PET and transcranial magnetic stimulation (Neuschmelting et al., 2016) suggests complementary roles for contrast enhanced MRI and FDG‐PET for the identification of areas of eloquent brain that are compromised by tumour, and where resection would be likely to lead to significant deficits. While much early work in hybrid imaging of brain tumours was done with sequential scans, the increasing use of highly targeted surgical and radiotherapeutic interventions requires the most accurate possible co‐localisation of the MRI and PET images; this is most likely to be achieved with simultaneous acquisition, which can also improve the overall diagnostic accuracy of the procedure, as shown by Deuschl et al. (2018) with [11‐C]‐MET‐PET‐MRI.

4.2.2. Epilepsy

Epilepsy is a very common condition in the community, with a lifetime incidence of 1–2%. Neuroimaging is routinely employed to exclude major structural causes, such as a tumour or vascular anomaly, but most cases are not associated with a visible focal lesion. For most patients, anti‐epileptic medications provide adequate control of seizures. Patients with refractory epilepsy may have subtle structural lesions that are difficult to identify even with high‐resolution MRI techniques. However, resection of these can often greatly improve seizure control. Metabolic tracers such as [18‐F]FDG, administered around the time of seizure (particularly if the seizure is sustained) can show foci of dramatically increased radiotracer uptake at the site of the lesion. Co‐registration with MRI, which is more readily achieved in a hybrid scanner, facilitates accurate lesion localisation, and surgical removal when appropriate (Kumar & Chugani, 2017). Even if radiotracer administration around the time of seizure is not feasible, metabolic abnormalities (often hypometabolism) may be seen at the site of the epileptogenic lesion for scans acquired interictally. With the development of electrophysiological equipment (such as EEG) that can be used while the patient is in an MRI scanner, the possibility of adding simultaneous electrophysiological data to the metabolic information from PET and the high‐resolution structural imaging of MRI has been realised (Grouiller et al., 2015).

4.2.3. Stroke

Stroke is a major cause of morbidity and mortality, but it is known that in some cases early treatment, involving thrombolysis and or clot retrieval, can lead to excellent recovery. Diffusion weighted MRI is highly sensitive to acute cerebral infarction and appears to be a reliable guide to the volume of irretrievably damaged tissue early in the course of a stroke. MRI perfusion techniques can identify regions of decreased cerebral blood flow, but these techniques are semi‐quantitative and have not been shown to reliably identify areas of critical but recoverable ischaemia. The addition of quantitative perfusion imaging with PET and [15‐O]water has been proposed as a means of improving patient selection for aggressive treatment (Werner et al., 2015). However, CT‐based perfusion techniques are less cumbersome, much more widely available, and have shown clear benefit in multiple clinical trials. MR‐PET is unlikely have a substantial role in this field unless a tracer (or tracers) can be developed that definitively identifies tissue that will progress to infarction without an improvement in blood supply and can distinguish tissue at risk of infarction from irretrievably infarcted brain tissue.

4.2.4. Neurodegenerative conditions

MRI findings in many neurodegenerative conditions are often subtle, with global or regional parenchymal volume loss often the main feature even at advanced stages of the disease, and particularly difficult to detect in the early stage of neurodegenerative disease. Distinction of a primary neurodegenerative condition from lesions caused by vascular disease or other causes may be difficult. Molecular radiotracers for neurodegenerative disease pathologies offer the prospect of highly sensitive and specific assessments of these conditions. Initial work used commonly available radiotracers including [18‐F]FDG (Shivamurthy, Tahari, Marcus, & Subramaniam, 2015) and [18‐F]fluorodopa. Regional volume loss in dementing conditions is associated with decreased regional perfusion (demonstrable with many forms of perfusion imaging, including arterial spin labelling with MRI; (Chen et al., 2011), and also with decreased metabolic activity as demonstrated with [18‐F]FDG PET. [18‐F]fluorodopa and analogous molecules that bind to the dopamine transporter protein have been licenced in some countries for some years for the assessment of Parkinsonian conditions, in which dopaminergic function is impaired, and there is markedly reduced uptake of the precursor molecule dopamine (and its molecular analogues) in specific brain sub‐cortical nuclei (Booth et al., 2015).

Many neurodegenerative conditions are associated with the deposition of abnormal proteins in brain tissue, notably beta amyloid (amyloid plaques), tau (neurofibrillary tangles) and alpha synuclein. Molecular tracers that selectively label these proteins are under active development, with the aims of: (a) early identification of patients at risk of, or in the early stages of, conditions such as Alzheimer and fronto‐temporal dementia, and (b) of differentiating dementing conditions on the basis of their underlying molecular neuropathology. A recent systematic review of the use of [18‐F]florbetapir (a marker for beta amyloid deposition) in patients with mild cognitive impairment concluded that more evidence was needed to establish its value in predicting progression to Alzheimer disease (Martínez et al., 2017). Recent work has also suggested that tracers that demonstrate tau protein deposition may be more sensitive and specific in the diagnosis of dementias (Bejanin et al., 2017). In a recent report by Mattsson et al., [18‐F]AV1451 PET was claimed to be superior to CSF tau protein measurements in the diagnosis of established Alzheimer's dementia, though not in the early clinical phase of the disease (Mattsson et al., 2018). Currently, the main potential benefits to patients from the increasing use of amyloid and tau radiotracers are likely to be more accurate diagnoses, access to support and counselling services, and prognostication, rather than changes in outcome, as there are few if any effective treatments for these insidious conditions.

5. CONCLUSIONS

Given the complementary nature of MR and PET imaging, the development of an integrated MR‐PET scanner has been a focal area of research into novel biomedical imaging technologies. In the past decade, the rapid progress in PET detector hardware development has led the introduction of the current generation of simultaneous MR‐PET scanners. Much research is now needed towards fully integrating the two systems at the software level, including MR based attenuation correction, motion correction and image reconstruction. These developments are targeted towards achieving fully quantitative PET imaging using MR‐PET imaging systems. While the systems neuroscience and clinical neuroscience imaging research communities are still at early stage in the adoption of MR‐PET technologies, the full potential of the simultaneous MR‐PET scanner lies in a synergistic combination of dynamic imaging of both systems (e.g., BOLD‐MRI and dynamic FDG‐PET). The full realisation of simultaneous MR‐PET promises to provide highly novel ways to investigate important questions in fundamental neuroscience as well as in the clinical neurosciences.

CONFLICT OF INTEREST

None.

Chen Z, Jamadar SD, Li S, et al. From simultaneous to synergistic MR‐PET brain imaging: A review of hybrid MR‐PET imaging methodologies. Hum Brain Mapp. 2018;39:5126–5144. 10.1002/hbm.24314