Quality Control for 4D Flow MR Imaging

Abstract

In recent years, 4D flow MRI has become increasingly important in clinical applications for the blood vessels in the whole body, heart, and cerebrospinal fluid. 4D flow MRI has advantages over 2D cine phase-contrast (PC) MRI in that any targeted area of interest can be analyzed post-hoc, but there are some factors to be considered, such as ensuring measurement accuracy, a long imaging time and post-processing complexity, and interobserver variability.

Due to the partial volume phenomenon caused by low spatial and temporal resolutions, the accuracy of flow measurement in 4D flow MRI is reduced. For spatial resolution, it is recommended to include at least four voxels in the vessel of interest, and if possible, six voxels. In large vessels such as the aorta, large voxels can be secured and SNR can be maintained, but in small cerebral vessels, SNR is reduced, resulting in reduced accuracy. A temporal resolution of less than 40 ms is recommended. The velocity-to-noise ratio (VNR) of low-velocity blood flow is low, resulting in poor measurement accuracy. The use of dual velocity encoding (VENC) or multi-VENC is recommended to avoid velocity wrap around and to increase VNR. In order to maintain sufficient spatio-temporal resolution, a longer imaging time is required, leading to potential patient movement during examination and a corresponding decrease in measurement accuracy.

For the clinical application of new technologies, including various acceleration techniques, in vitro and in vivo accuracy verification based on existing accuracy-validated 2D cine PC MRI and 4D flow MRI, as well as accuracy verification on the conservation of mass’ principle, should be performed, and intraobserver repeatability, interobserver reproducibility, and test–retest reproducibility should be checked.

Introduction

3D cine phase-contrast (PC) MRI with electrocardiogram (ECG) gating and velocity encoding (VENC) of three velocity components allows us to obtain velocity vectors in three spatial dimensions and a time axis (total of four dimensions). Therefore, this imaging technique is called 4D flow MRI.¹ The target of this imaging method is fluids and structures that move periodically in synchronization with the heartbeat, and its application to cerebral blood vessels, carotid arteries, cardiovascular and abdominal vessels, as well as cerebrospinal fluid, is being vigorously pursued. Using datasets obtained by this imaging technique, we can visualize time-resolved 3D hemodynamics and cerebrospinal fluid dynamics, and quantify fluid velocity, flow rate, various biomarkers that measure the effect of fluid on the wall (e.g., wall shear stress [WSS] and related indices)^2–8 and energy-related indices (e.g., turbulent kinetic energy,⁹ kinetic energy¹⁰ and energy loss¹⁰), and so on.

Compared with the previously developed 2D cine PC MRI,¹¹ the advantage of 4D flow MRI is that after imaging the blood vessels and spinal fluid cavity in the entire ROI, the flow velocity, flow rate, and WSS of the lumen can be analyzed at any location (flexible retrospective quantification). In addition, it is possible to examine the consistency of the measured flow rate in 4D flow MRI using the conservation of mass’ principle based on the flow rate information of successive lumens.

One of the biggest disadvantages of 4D flow MRI is the longer acquisition time as compared with 2D cine PC MRI. This increases the possibility of patient motion and makes the image worse, making it difficult to apply this imaging method clinically. In addition, spatially and temporally averaged values of the analytical values obtained by 4D flow MRI are lower as compared with other modalities, such as ultrasonography (US), computational fluid dynamics (CFD), particle image velocimetry (PIV), and laser Doppler velocimetry (LDV). Increasing the spatial resolution of 4D flow MRI results in lower SNR and longer acquisition time. Increasing the temporal resolution also increases the acquisition time.

There are also issues to be solved in analysis. First of all, there are a variety of analysis software, and there are a variety of analysis biomarkers, such as WSS, for which the analysis method is not standardized. Segmentation and analysis also take time. In addition, the geometry of the blood vessel and the subarachnoid space may vary depending on the conditions of segmentation (images and their quality used for segmentation, threshold value, analyst, and so on), and as a result, the analyzed results of biomarkers such as flow velocity, flow rate, and WSS may vary. Furthermore, the position and angle of the analyzed cross-section and the position and size of the analyzed region set by the observer can affect the results of the analysis. Therefore, intraobserver and interobserver variability cannot be ignored. With these disadvantages in mind, it is important to check the accuracy, repeatability, and reproducibility of the current 4D flow MRI and to conduct quality control.

We searched 343 documents by PubMED in November 2021 under the terms “4D flow MRI or 4D flow” and “accuracy or reproducibility or validation or test retest or intraobserver or interobserver”. In this review, from these documents, we cite representative literatures on the accuracy, repeatability and reproducibility of 4D flow MRI in each organ region (cerebrovascular vessels, cerebrospinal fluid, carotid artery, cardiovascular system, and abdominal vessels) and phantom to confirm the current status and discuss quality control for 4D flow MRI. The summary of some of the representative literature is shown in supplementary Tables 1 and 2. There are many biomarkers for blood flow analysis based on the velocity vector information of each voxel, but in this paper, we mainly focus on cross-sectional mean blood flow velocity (systolic and mean), cross-sectional flow rate (systolic and mean), and WSS, which are considered important biomarkers. However, there are various ways to obtain the shear stress depending on the analysis software used by the researcher, and the methodology will not be discussed in depth in this review paper. For the shear stress, the values obtained in each study were used. We also discussed accuracy and precision separately (Fig. 1). Accuracy is the degree of closeness to true value. In addition, precision is a measure of the degree of mutual variation among values such as multiple measurements, and their repeatability performed by the exact same method and reproducibility in cases where the process is different. Accuracy is important in comparison with other modalities, while precision is important in follow-up and comparison of patients with the same modality.

Fig. 1.

Concepts of accuracy and precision. Accuracy and precision are likened to the distribution of arrows when they are shot at the target. (a) When the arrows are gathered in the center of the target, there is high accuracy and high precision. (b) When the arrows are scattered and distributed around the center of the target, there is high accuracy and low precision. (c) When the arrows are gathered outside the center of the target, there is low accuracy and high precision. (d) When the arrows are scattered and distributed throughout the target, there is low accuracy and low precision.

4D Flow MRI Accuracy Verification on How to Verify the Measurement Accuracy of 4D Flow MRI for Blood and Cerebrospinal Fluid Flow

In vivo measurements, in vitro experiments, and in silico modelling are used to verify the measurement accuracy of 4D flow MRI for blood and spinal fluid flow. In silico modelling refers to CFD. In vivo measurements and in vitro experiments can be used to measure the velocity using 4D flow MRI, and the combinations of the examination target and the true value to verify the accuracy are as follows.

In vivo (human and animal) validation methods for 4D flow MRI include human CFD,^{3–5,12–19} animal CFD,²⁰ human US,^21–23 human transcranial Doppler sonography (TCD), transcranial color-coded duplex sonography (TCCD),²⁴ human intra-arterial Doppler sonography wire,²⁵ invasive flow probe used to measure aortic blood flow in swine,²⁶ conventional 2D cine PC MRI (many papers), existing 4D flow MRI with confirmed accuracy (many papers), and the conservation of mass’ principle.^27–29 The conservation of mass’ principle is a method of measuring the flow rate upstream and downstream of a continuous vessel and checking the rate of agreement.³⁰ The flow rate can be compared before and after a bifurcation with different vessel diameters or at different distant sites. If the accuracy is high, the flow rate can be saved.

When 4D flow MRI is performed on in vitro models (e.g., straight tube models or human vascular models) or on isolated human or animal vessels, validation methods include the time-beaker flow measurements,³¹ rotating phantom,³² Coriolis flowmeters,^33,34 theoretical values obtained from the Hagen-Poiseuille equation,^33,34 pump flow rate,³⁵ particle image velocimetry (PIV),^17,36,37 LDV,^38,39 CFD,^36,38,40,41 conventional 2D cine PC MRI (many references), existing conventional 4D flow MRI with confirmed accuracy (many references), and the conservation of mass’ principle (many references). In some studies, in vivo measurements, in vitro experiments, and CFD have been performed.¹²

In the statistical processing described above, the accuracy of the flow velocity, flow rate, WSS, etc. obtained by 4D flow MRI is compared with the true value obtained by other modalities using paired t tests, Pearson’s correlation (r), or error rates. Many authors consider accuracy to be good when there is no statistical difference in paired t test between the measured and true values. When examining precision, Bland-Altman analysis, interclass correlation coefficient (ICC), Cohen’s kappa, or the coefficient of variation (COV) are used. Many authors consider precision to be good when the Bland-Altman analysis does not exceed the limit of agreement or when the ICC or Cohen’s kappa value is 0.7 or higher. We follow their description in this review.

Technological Factors Affecting the Measurement Accuracy of 4D Flow MRI

There are various factors that affect the measurement accuracy of 4D flow MRI, such as patient preparation, patient cooperation, magnetic field strength, imaging equipment of the MR system manufacturer, receiver coils, 4D flow MRI sequences provided by the manufacturer, imaging parameters, ECG gating, respiratory gating, data processing before analysis, analysis software, analysis methods, and observers. In this section, we will discuss magnetic field strength, spatial resolution, temporal resolution, VENC, and the imaging acceleration technique as factors affecting the accuracy of 4D flow MRI flow velocity, flow rate, and WSS.

Magnetic field strength

For intracranial arteries, SNR of 4D flow MRI at 3T is about twice that of 1.5T, and the advantage of using acceleration techniques under the higher magnetic field strength has been reported.⁴² Tajima et al. performed 4D flow MRI with 3T and 7T scanner on an in vitro intracranial aneurysm model and 10 healthy volunteers, calculated angular similarity index (ASI)⁴³ and magnitude similarity index (MSI)⁴³ using CFD results as the true value, and reported that 7T was superior than 3T (P < 0.05).⁴⁴

In non-contrast 4D flow MRI of the aorta, SNR at 7T was reported to be 2.2 times greater than at 3T, and SNR at 3T was 1.7 times greater than at 1.5T.⁴⁵ In the same study, the improvement in SNR with contrast media was 1.8-fold at 1.5T, 1.7-fold at 3T, and 1.4-fold at 7T. On the other hand, evaluation of the image quality of aorta, peak velocity, flow volume, and WSS by 4D flow MRI of 1.5T, 3T, and 7T systems was made, and it was reported that the image quality of the 7T system was the worst, with variations in measurements due to magnetic field strength exceeding intraobserver variability.⁴⁶ However, for the data that could be evaluated, it was reported that the peak velocity did not depend on the strength of the magnetic field as compared with WSS and forward flow. The authors stated that the 7T 4D flow MRI had sufficient quality despite issues such as the navigator, the positioning, the ECG, and the coil placement.⁴⁶ It has also been reported that visual assessment (semiquantitative grading) for 3D PC MRA and time-resolved 3D particle traces obtained from 4D flow MRI of the thoracic aorta with a spatial resolution of about 2 mm and a temporal resolution of about 41 ms was better at 3T than at 1.5T, but there was no statistical difference in systolic peak velocities, net flow, and WSS.⁴⁷ In addition, of the recent papers on cardiovascular 4D flow MRI using acceleration techniques, four^23,31,48,49 were performed on 1.5T systems and four^50–53 on 3T systems. Another paper⁵⁴ evaluated pulmonary arteries at 1.5T and 3T but did not provide a detailed comparison. Since the cardiovascular structure is relatively large, a relatively large voxel can be secured in it. Therefore, it is considered that the SNR of 4D flow MRI of the cardiovascular system is good, and the accuracy of 1.5T, 3T, and 7T is maintained.

One paper⁵⁵ on 4D flow MRI of hepatic vessels using acceleration techniques was performed on a 1.5T system, and two papers^27,29 were performed on a 3T system. Spiral 4D flow MRI on a 1.5T system (acquired spatial resolution, 2.5 × 2.5 × 5 mm³ [reconstructed spatial resolution, 1.3 × 1.3 × 2.5 mm³]; acquisition time, 22 s; temporal resolution, 66–71 ms; VENC, 60 cm/s; and contrast material use) showed poor test–retest repeatability of small vessels.⁵⁵ In addition, a study in which hepatic vessels were examined with 4D flow MRI (voxel size, about 2.5 mm³; temporal resolution, 61–82 ms; VENC, 100 cm/s; and acquisition time, 8.2–14.6 min) on a 3T MR system reported inaccuracies in hepatic arterial mean velocity and intrahepatic portal velocity due to low spatial and temporal resolution.²⁹ Based on these findings, 3T with higher SNR may be more advantageous than 1.5T systems because smaller abdominal vessels have a correspondingly smaller appropriate voxel size, and slower flow velocities have a lower velocity-to-noise ratio (VNR).⁵⁶

Spatial resolution

In 2D cine PC MRI, the effect of spatial resolution on accuracy has already been investigated in detail. Hofman et al. measured the femoral artery of a canine with a diameter of 3.5 ± 0.7 mm using 2D cine PC MRI (spatial resolution, 0.98 × 0.98 × 6 mm) on a 1.5T MR system and compared the results with ultrasonic flow measurement.⁵⁷ Their 2D cine PC MRI results showed that if 3 pixels were included in the vessel, the flow rate could be accurately measured with improved results at 5 pixels. Debatin et al. compared blood flow velocities in human renal arteries and veins using 2D cine PC MRI with para-amino hippurate-clearance renal blood flow data, and reported that they were able to accurately measure small renal arterial blood flow, as well as large renal veins.⁵⁸ From the average area of the renal arteries and the size of the pixels at this time, about 5.5 pixels were thought to be included in the blood vessels. Tang et al. performed an in vitro experiment in which eight different tubes ranging from 4.55 mm to 12.45 mm flowing with water mixed with copper sulfate (T1 of about 700 msec) were examined with 1.5T 2D cine PC MRI and reported that if the vessel contained more than four voxels, the error was within 10%.⁵⁹ They also stated that 16 voxels were needed in the vessel for detailed flow detection in a study using simulation.

Validation of the measurement accuracy of 4D flow MRI has also been carried out. In the first paper on 4D flow MRI, Markl et al. scanned stenoses of 8.27 mm in diameter, as well as tubes 15.9 and 19.1 mm in diameter with 4D flow MRI (spatial resolution; 0.94 × 0.94 × 3 mm³, 0.63 × 0.71 × 1 mm³) on a 1.5T MR system and compared the flow velocity with 2D cine PC MRI, showing the good accuracy of 4D flow MRI.¹ In this paper, the tubes contained more than eight voxels. Fukuyama et al. circulated blood mimicking fluid (an aqueous solution with 40% weight glycerol; viscosity, 3.57 mPa•s) into acrylic pipes (inner diameters of 15, 12, 9, 6, and 3 mm) with constant flow and performed 4D flow MRI on a 3T machine (isovoxel size; 0.67 mm³, 1.00 mm³, 1.25 mm³, 1.67 mm³, and 2.00 mm³) for a phantom.³³ They determined the true value of the maximum velocity at the center of the tube theoretically from the Hagen-Poiseuille equation⁶⁰ (the maximum velocity is equal to twice the cross-sectional mean velocity), and the spatial-averaged velocity was calculated based on the Coriolis flow meter flow value and the tube cross-sectional area. The measured values were compared with the reference values, and the temporal changes were calculated. As a result, the error rate of the maximum flow velocity at the center of the tube was less than 10% when the ratio of the pixel size to the inner diameter was less than 10% for tubes larger than 6 mm, and the error rate of the cross-sectional mean velocity was less than 10% when the ratio of the pixel size to the inner diameter was less than 30% for tubes larger than 6 mm (Fig. 2). It was thought that the 3-mm tube did not meet the above error rates during this experiment due to its low SNR. New generation MR systems have led to improved measurement accuracy mainly due to comprehensive factors, such as the stability of the static magnetic field, the linearity of the gradient magnetic field, and the improvement of the SNR of the receiving coil, so that the above error rate is attainable.⁶¹ The aorta to carotid artery has a diameter of 6 mm or more, which should ensure a higher SNR and more accuracy than the smaller cerebral vessels. David et al. used 4D flow MRI (spatial resolution; 1.5 × 1.5 × 1.5 mm³, 1.9 × 1.9 × 1.9 mm³ and 2.8 × 2.8 × 2.2 mm³) on a 1.5T MR system to image a 20-mm inner diameter straight tube with water flowing in a pulsatile flow.⁶² Mean flow and peak velocity were measured and compared with 2D cine PC MRI. The voxel size ranged from 7.5% to 14% of the vessel diameter and the measurement error of all spatial resolutions was less than 10%. The accuracy was the best with 1.5-mm voxels. Aristova et al. reported 4D flow MRI measurements on a 3T system using a straight bifurcation model with different internal diameters (4, 6, and 8 mm in diameter) simulating AVMs and flowing water mixed with Gd contrast agent.⁶³ They reported that error in conservation of mass’ principle was within 15% for four–five voxels in the blood vessel and within 10% for more than six voxels. Ebel et al. performed 4D flow MRI (3T; k-t-GeneRalized Autocalibrating Partial Parallel Acquisition (GRAPPA)5 [acquisition time about 2 min] and GRAPPA2 [acquisition time about 4 min]; spatial resolution, 2.5 mm isotropic; VENC, 140–350 cm/s) on a 1.27-cm diameter straight tube of tap water in steady and pulsating flow, and the flow rate and velocity were compared with 2D cine PC MRI.⁶⁴ The results showed that the accuracy of flow and velocity measurements in 4D flow MRI (k-t-GRAPPA5 and GRAPPA2) was good and test–retest reproducibility was good. Since VNR is inversely proportional to VENC, a high VENC will result in a low VNR,⁵⁶ but in this study, the tube contained five voxels with 2.5 mm isotropy, and a 3T device was used, which resulted in a high SNR, so setting a high VENC did not affect the measurement accuracy. In addition, consensus statement by 4D flow cardiovascular magnetic resonance (CMR) consensus group³⁰ cited Hofman et al.’s literature⁵⁷ and stated that at least five to six voxels are required in the blood vessel, and that the voxel size of 4D flow MRI for aorta and pulmonary artery is recommended to be 2.5 × 2.5 × 2.5 mm³ or less, and that for heart and great vessels is recommended to be 3.0 × 3.0 × 3.0 mm³ or less. Based on the above, in order to obtain accurate cross-sectional mean flow velocity and flow rate, at least four voxels, and if possible, six voxels, are recommended for the vessel diameter of the vessel of interest. However, it is also necessary to consider SNR.

Fig. 2.

Error rates of 4D flow MRI with different voxel sizes for straight tube phantoms of different diameters with blood mimicking fluid flowing in steady flow. (a) Error rates of spatially averaged velocity averaged temporally obtained by 4D flow MRI. (b) Error rates of maximum velocity averaged temporally obtained by 4D flow MRI (Cited from the previously published paper³³).

Temporal resolution

The consensus statement by 4D flow CMR consensus group³⁰ is that the temporal resolution should be less than 40 ms. In fact, when examining whether 2D cine PC MR can capture the systolic peak, it is possible to do so with a temporal resolution of less than 40 ms. However, in many cases, such a setting may extend the imaging time or may not be set to an ideal value due to sequence restrictions. This situation may be improved by the acceleration technique described later. In addition, as discussed below, spatial resolution is considered to be more important than temporal resolution for the measurement accuracy of 4D flow MRI.

VENC

To prevent velocity wrap around, VENC should be set about 10% larger than the highest flow velocity.³⁰ However, since VNR is proportional to velocity and inversely proportional to VENC,⁵⁶ when VENC is set large enough for high velocities in the case of stenotic lesions with high velocities and arteriovenous malformations (AVMs) with low to high velocities, the VNR for low velocities will decrease further, resulting in poor accuracy. Therefore, it is best to use dual-VENC,⁶³ multi-VENC,⁶⁵ or variable VENC,⁵⁶ which can improve the dynamic velocity range and VNR. The acquisition time is shorter than that of multiple single VENCs, since TR is 7TR instead of 8TR because of sharing the reference data. Dual-VENC 4D flow MRI improved the image quality and accuracy of flow velocity measurement of cerebral arteries from cerebral veins at low-to-high flow velocities,³² and detectability of abnormal blood flow in the left atrium, which had caused thrombus formation after left upper lobectomy.^66,67 Multi-VENC 4D flow MRI imaging was reported to show improved accuracy of low flow in the aorta and improved accuracy of pathlines and streamlines,⁵¹ and good accuracy, good conservation of mass, good test–retest reproducibility, and reduced interobserver variation in the arteriovenous flow measurement near the heart.⁶⁵

Imaging acceleration technique

It is best to keep the acquisition time as short as possible because the longer the acquisition time, the more the images deteriorate due to the patient’s body movement. In recent years, the development of acceleration techniques has progressed, and there are various research reports on this subject. Sequences that allow imaging of blood vessels in a few minutes have also been developed. The problem is that it takes time for image reconstruction, but some studies have shown how to solve this problem. In the cerebral blood vessels, Cartesian y-z radial sampling, k-t sensitivity encoding (SENSE),⁶⁸ a pseudo-random variable-density Cartesian undersampling strategy (CIRCUS) with the combination of k-t, parallel imaging and compressed sensing image reconstruction techniques (k-t SPARSE-SENSE),⁶⁹ and pseudo-spiral Cartesian undersampling with compressed sensing reconstruction⁷⁰ were reported. In the cervical vessels, compressed sensing (CS),⁷¹ hybrid one- and two-sided flow-encoding and velocity spectrum separation (HOTSPA),⁷² and principal component analysis (PCA)⁷³ were reported. In the cardiovascular system, spiral,²³ CS,⁴⁸ k-t Broad-use Linear Acquisition Speed-up Technique (k-t BLAST),⁵⁰ and echo planar imaging (EPI)³¹ have been reported. Spiral⁵⁵ in the liver has been reported. The following are representative examples. The imaging time can be shortened and the accuracy is good. Some of them have reported reproducibility.

In cerebral blood vessels, van Ooij et al. performed non-accelerated, k-t BLAST, and SENSE 4D flow MRI (3T; spatial resolution, 0.78 × 0.78 × 0.8 mm³; VENC, 100 cm/s; acquisition time for k-t BLAST, about 1/4 of non-accelerated; and acquisition time for SENSE, 6/4 of k-t BLAST) on in vitro and in vivo cerebral aneurysms, and reported that k-t BLAST underestimated the systolic velocity and SENSE was superior.⁷⁴ Sekine et al. performed two types of 4D flow MR imaging (Cartesian y-z radial sampling and k-t SENSE; spatial resolution, 0.82 × 0.82 × 1.4 mm³) with a 3T scanner on the intracranial arteries of 16 normal volunteers. They reported that Cartesian y-z radial sampling was compatible in flow measurement and k-t SENSE was comparable in pathline delineation to the standard scan using SENSE. The imaging time was also reduced by half.⁶⁸ Liu et al. performed 4D flow MRI with CIRCUS with k-t SPARSE-SENSE (3T; spatial resolution, 1.25 × 1.25 × 1.25 mm³ to 1.4 × 1.4 × 1.4 mm³; temporal resolution, within 30 ms; and acceleration factor (AF), 4 and 12) on intracranial arteries of five normal subjects and eight patients with cerebral aneurysms within 5 min.⁶⁹ Their results showed that when comparing 4D flow MRI with a temporal resolution of 24 ms and 73 ms, the connection of the path lines in the aneurysmal vortex of the former was well recognized. Gottwald et al. imaged 15 healthy subjects cerebral vessels and 5 cerebral aneurysms using 4D flow MRI with pseudospiral Cartesian undersampling with compressed sensing reconstruction with high spatial resolution (0.5-mm isotropic), high temporal resolution (about 30 ms), and a scan time of 10 min in a 7T MR system.⁷⁰ The results showed an improvement in accuracy, but the improvement in temporal resolution was not effective in improving the accuracy of WSS, although it was effective in improving the accuracy of oscillatory shear index (OSI). As described above, there are many reports of 4D flow MRI using acceleration techniques for cerebral blood vessels that can shorten the acquisition time by up to a quarter, increase the spatial and temporal resolution, and maintain accuracy. However, since each acceleration technique has its own characteristics, the accuracy and visualization of hemodynamics of 4D flow MRI should be evaluated for clinical application based on existing 2D cine PC MRI or 4D flow MRI with confirmed accuracy.

There are varied reports on acceleration techniques related to the carotid artery. Tao et al. scanned the carotid arteries of 16 volunteers with 4D flow MRI with compressed sampling (1.5T, spatial resolution, 0.83 × 0.83 × 2 mm³; VENC, 100 cm/s; and acquisition time, 49 seconds [147 seconds for full sampling time]) and reported that the accuracy of flow velocity and flow rate was maintained based on the comparison with 2D cine PC MRI, and the imaging time was reduced to 1/3.⁷¹ Wang et al. introduced 4D flow MRI with hybrid one- and two-sided flow-encoding and velocity spectrum separation (HOTSPA) (3T; spatial resolution, 0.78 × 1.0 × 2.5 mm³; full sampling). They imaged the common carotid artery of 12 healthy volunteers and compared the results with conventional methods.⁷² By encoding velocities in three directions using only 2TR instead of 4TR as in the conventional method, the acquisition time was reduced to half of the conventional method. They reported a blood flow error of 0.1% and a flow velocity error of 1.1%. Pagé et al. used 4D flow MRI with PCA (spatial resolution: Y-shaped phantom, 0.25-mm isotropic voxels; human, 0.65 mm isotropic) on a 3T MR system to image Y-shaped bifurcation phantoms with internal diameters of 2–5 mm, carotid bifurcation phantoms, and human carotid artery.⁷³ The accuracy of the transverse imaging was good, and the imaging time could be reduced up to 1/6. As described above, 4D flow MRI using acceleration technique for the carotid artery can shorten the acquisition time up to 1/6, and the accuracy is good.

There are also reports focusing on the cardiovascular system. Negahdar et al. used spiral imaging for 4D flow MRI (1.5T; AF, 36; spatial resolution, 2.5 × 2.5 × 5 mm³; and TE, 1.68 ms) to image four patients with aortic stenosis and five normal subjects within 4 min, with a TE of 1.68 ms, which is shorter than usual, and reported good accuracy of peak velocity assessment.²³ Ma et al. performed 4D flow MRI (1.5T; spatial resolution, 2.4–2.5 × 3.5–3.8 × 3.3–3.8 mm³ [interpolated spatial resolution, 2.4–2.5 × 2.4–2.5 × 2.4–2.9 mm³]; temporal resolution, 38.1–44.5 ms) with compressed sensing (CS) (AF = 7.7) on the aorta of 11 thoracic aortic disease patients and 20 normal subjects under free breathing in less than 2 min on average and reconstructed by the scanner in 5 min.⁴⁸ Compared with the existing 4D flow MRI (GRAPPA; AF, 2), the authors reported that the error in net flow values was less than 3.5%, and the error in peak velocities and peak flow was 10–15%. Zaman et al. performed 4D flow MRI using k-t BLAST (3T; spatial resolution, 2.5 × 2.5 × 2.5 mm³; acquisition time, about 5 min) to image the thoracic aorta in 15 healthy subjects and 8 patients, and reported that the acquisition time was reduced by a factor of 5 and the accuracy of flow, peak velocity, and stroke volume was comparable to existing 4D flow MRI using SENSE.⁵⁰ Pruitt et al. used fully self-gated whole-heart 4D flow MRI (3T; spatial resolution, 2.3–2.6 mm [isotropic]; AF, 19–20; acquisition time, 5 min) to acquire images including the hearts of two patients and eight volunteers, and reconstructed the images with Reconstructing Velocity Encoded MRI with Approximate message passing aLgorithms (ReVEAL).⁵³ Compared with 2D PC MRI, the accuracy of peak velocity, peak flow rate, and net flow of this method was reported to be good. Garg et al. used 4D flow MRI with 4D-SPGR, 4D-EPI, and 4D-k-t BLAST (1.5T; spatial resolution, 3 × 3 × 3 mm³ [reconstructed spatial resolution, 2 × 2 × 3 mm³]) to image mitral inflow and aortic outflow in 25 normal subjects, and reported that 4D flow MRI with EPI (acquisition time around 8 min) was the best in terms of accuracy, image quality, and acquisition time.³¹ Neuhaus et al. performed 4D flow MRI with CS (3T for healthy subjects, 1.5T for patients; spatial resolution, 2.5-mm isotropic [reconstructed spatial resolution, 1.25-mm isotropic]; acquisition time, about 5 min) on 20 healthy subjects and 3 subjects with aortic disease, and CS image reconstruction was performed online in 9 min.⁵² The flow curve, net flow, peak flow, and peak velocity were compared with SENSE, and the accuracy was shown to be maintained up to an AF of 6 for CS. As described above, for 4D flow MRI of cardiovascular system, accelerated techniques that can acquire images within 2 min and reconstruct images within 5 min have been developed, and many reports indicate that the imaging time is reduced and the accuracy is good.

We found one report where the focus was on the liver. Bane et al. used spiral 4D flow MRI (1.5T; acquired spatial resolution, 2.5 × 2.5 × 5 mm³ [reconstructed spatial resolution 1.3 × 1.3 × 2.5 mm³]) to evaluate the maximum and time-averaged through-plane velocities, and net flows of 13 hepatic vessels in 52 patients with chronic liver injury in 22 seconds with a single breath-hold.⁵⁵ Although they did not verify the accuracy by comparing with existing 2D cine PC MRI or 4D flow MRI, the interobserver variability was consistent within an acceptable range. On the other hand, they reported that test–retest repeatability of large vessels was good, but that of small vessels was not good. For abdominal blood vessels such as the liver, the imaging time can be shortened by acceleration techniques, but the reproducibility may be considered to be reduced because the vessel diameter is smaller than that of the cardiovascular system. Although the positions of the intracranial vessels are barely affected by respiration, it is considered that the movement due to respiration also reduces the measurement accuracy of the hepatic blood vessel using 4D flow MRI.

The acceleration technique can shorten the acquisition time of 4D flow MRI and improve the spatial and temporal resolutions, making it a very promising technology for clinical applications. It may also reduce the time required for image reconstruction, which has been a problem in research imaging sequences. It is necessary to investigate the effect of each acceleration technique on accuracy and blood flow visualization before clinical application. In addition, the concepts of spatial resolution, SNR, and VNR are important as mentioned earlier.

Trigger time

Recently, Fukuyama et al. reported that the effect of the differences in the trigger time, which is the timing when VENC is applied in three directions in 4D flow MRI, affects the WSS analysis result more than the flow rate and flow velocity.⁷⁵ Schmidt et al. modified a velocity-encoded sequence with synchronized encoding (SYNC SPI) to allow direct acceleration mapping by replacing the bipolar encoding gradients with tripolar gradient waveforms for the purpose of synchronization of the spatial and acceleration encoding time points.⁷⁶ Although it is thought that the effect depends on the design of the imaging sequence, this is a research topic that seems to be noteworthy in order to accurately measure the flow velocity, flow rate, and WSS.

Organ Factors Affecting the Measurement Accuracy of 4D Flow MRI

Accuracy is the degree of closeness to true value.

Cerebral blood vessels

There are normal variations and individual differences in cerebral blood vessels, and the diameter of blood vessels varies depending on the measurement location. The average value of intracranial artery diameter by MR angiography is reported as follows: petrous segment of internal carotid artery (ICA), 4.08 mm; cavernous segment of ICA, 3.90 mm; supraclinoid segment of ICA, 3.25 mm; precommunical segment of anterior cerebral artery, 1.67 mm; horizontal segment of middle cerebral artery, 2.46 mm; vertebral artery, 2.78 mm; basilar artery, 2.76 mm; and interpeduncular segment of posterior cerebral artery, 1.60 mm.⁷⁷ In order to maintain measurement accuracy, the vessel should contain at least four voxels and preferably six voxels as described above. For example, if 5 voxels are to be included in the vessel, the recommended voxel size is 0.8 mm for the internal carotid artery, 0.5 mm for the middle cerebral artery, basilar artery, and vertebral artery, and 0.3 mm for the anterior cerebral artery and posterior cerebral artery. However, due to the constraints of acquisition time and other factors, the typical voxel size reported in the literature for 4D flow MRI of the brain is approximately 0.5 to 1 mm (Supplementary Table 1). Image reconstructions, such as zero-filled interpolation (ZIP), make the voxels even smaller. In a study using ZIP to measure the flow velocity of the aorta, it has been reported that the accuracy and precision are maintained while shortening the imaging time by 2/3.⁷⁸ However, it was not a study of small blood vessels or high spatial resolution.

Since the diameter of the internal carotid artery in the cranium is about 4 mm, the accuracy of blood flow measurement in the internal carotid artery is expected to be maintained when the spatial resolution of 4D flow MRI is about 0.8 mm. However, since other cerebral blood vessels are thinner than the internal carotid artery and have relatively low spatial resolution, the measurement accuracy of 4D flow MRI of cerebral blood vessels is considered to be low. On the other hand, although some intracranial aneurysms are larger than the diameter of cerebral blood vessels, they have spatially complex flow, and the partial volume phenomenon is presumed to reduce the accuracy. Furthermore, when aiming for high spatial resolution, the voxel size becomes even smaller, and it is speculated that the SNR decreases and the accuracy decreases due to the low temporal resolution.

So far, there are many basic reports using in vivo 2D cine PC MRI and US, in silico CFD, in vitro PIV and LDV, and so on, as true value for accuracy verification of intracranial arteries by 4D flow MRI. Meckel et al. compared cerebrovascular flow velocity with 4D flow MRI (3T; spatial resolution, 1.0 × 0.7 × 1.2 mm³; and temporal resolution, 54.4–56.0 ms; VENC, 100 cm/s) and 2D cine PC MRI, TCD, and TCCD, and reported a moderate or greater correlation and an underestimation (23.5% in average velocity as compared with TCCD) in small vessels such as the middle cerebral artery.²⁴ Hollnagel et al. used 4D flow MRI (acquired spatial resolution, 0.78 × 0.47 × 0.7 mm³ [reconstructed spatial resolution, 0.49 × 0.49 × 0.7 mm³], number of cardiac phases, 40 phases) to image an internal carotid-posterior communicating artery aneurysm model with flowing fluid of 59.4% weight glycerol solution and compared the results with those of CFD and LDV.³⁸ The results were qualitatively similar, but quantitatively different. On the other hand, CFD and LDV were in good quantitative agreement. Jiang et al. performed 4D flow MRI (vastly undersampled isotropic projection reconstruction [PC-VIPR]; voxel size, 0.525 mm to 0.62 mm isovoxel; temporal resolution, 35 m; VENC, 50 cm/s and 150 cm/s; acquisition time, 8 m) on two aneurysms created in two canines and compared the results with CFD.²⁰ The results showed that the flow pattern in the aneurysms were almost the same, but the differences in the similarity index of the magnitude and angle of the velocity vector were 10% to 20%, and that the correlation of time-averaged WSS between the two methods was low (0.22 and 0.31).²⁰ Berg et al. measured the flow velocity of the circle of Willis and intracranial aneurysms with 4D flow MRI (spatial resolution, 0.75 × 0.75 × 0.8 to 0.8 × 0.8 × 0.8 mm³; VENC, 80 cm/s) on 3T and 7T MR systems and compared them with CFD, and reported that the flow patterns within the aneurysm were similar, but there were differences in the magnitude and angular similarity indices.⁴³ As shown above, there are many reports of low values of flow velocity of 4D flow MRI when compared with the reference value of CFD, but with moderate or higher correlations and well-correlated streamline patterns.^3,4,13,18,79 There are also many reports where WSS is low when compared with the reference value of CFD, but is moderately or more correlated, and the relative distributions are similar.^3–5,13,18

On the other hand, van Ooij et al. acquired data on an in vitro experimental intracranial aneurysm model with 4D flow MRI (3T; receiver coil, solenoid rat coil; spatial resolution, 0.2 × 0.33 × 0.2 mm³) and compared the velocity and velocity vector angles with PIV at its spatial resolution of 0.33 × 0.33 × 0.21 mm³, and with CFD at its spatial resolution of 0.2 × 0.33 × 0.2 mm³, showing that 4D flow MRI was accurate.³⁶ Furthermore, they reported that the accuracy of WSS vectors improved with an increase in spatial resolution (from 0.94-mm to 0.19-mm isotropic) of 4D flow MRI (using a solenoid rat coil) in data collection of an in vitro experimental intracranial aneurysm model.⁵ Similarly, when the spatial resolutions of CFD and PIV were downsized and compared with those of 4D flow MRI, the results of blood flow measurements in 4D flow MR agreed well with those of CFD and PIV in many reports.^37,80 The reason for the inaccuracy of 4D flow MRI of cerebral blood vessels may be due to its low spatial resolution. Compared with the true value, which has the same spatial resolution, 4D flow MRI is more accurate. When the spatial resolution of 4D flow MRI is improved in the future, the accuracy will be further improved.

Cerebrospinal fluid

Bunck et al. compared 4D flow MRI (1.5T; voxel size, 1.5-mm isotropic [reconstructed voxel size, 1-mm isotropic]; VENC, 10–15 cm/s; and acquisition time, 8–14 min, sagittal slab) and 2D PC MRI of the cervical spinal canal from the craniocervical junction in 10 healthy subjects. The peak flow velocities of spinal fluid at the central level of each vertebral body in C1, C2, C5, and C7 were shown to be in good agreement.⁸¹ Since the cervical spinal subarachnoid space (SSS) was relatively wide (150 mm² in a narrow area) compared to the voxel size, the accuracy of the peak flow velocity was considered to be good. Yiallourou et al. compared 4D flow MRI and CFD of the flow of the cervical SSS in three healthy subjects and four Chiari malformations, and reported different flow results between 4D flow MRI and CFD, partly because the shape of the CFD could not reflect the small anatomical structures and tissue movements of the cervical SSS.¹⁴ Thyagaraj described the importance of adding anatomically realistic spinal nerve roots (NRs) to the CFD geometry.¹⁶ Heidari Pahlavian et al. performed an in vitro experiment using a model of the cervical SSS and performed 4D flow MRI (1.5T; spatial resolution, 1 × 1 × 1.5 mm³; VENC, 15 cm/s; and temporal resolution, 52 ms) and compared the results with CFD analysis (spatial resolution matched to MR).⁴⁰ Their results showed that the mean difference between systolic and diastolic peak velocities was 7.5% and 10.6%, respectively. However, 4D flow measurements were less accurate during the times of low CSF flow, and the correlation between CFD and 4D flow in-plane velocities was low. The authors speculate that the accuracy of CFD is poor because the fine geometry of the SSS cannot be reproduced in the model. Williams et al. performed 4D flow MRI (3T; spatial resolution, 1.5 × 1.5 × 1.5 mm³; and VENC, 15 cm/s) on an in vitro model of Chiari malformation and compared the CSF velocities with those obtained by 2D cine PC MRI, and reported that the CSF velocities showed good accuracy in each case.⁸² Currently, it is difficult to accurately reproduce the geometry of the SSS in CFD and in vitro models due to its detailed and complex structure, and these are not considered to be the ideal true value for human 4D flow MRI measurements for SSS. Therefore, at this time, the measurement accuracy of 4D flow MRI for the human SSS should be examined using existing 2D cine PC MR, whose accuracy has been confirmed.

Cervical vessels

Medero et al. compared stereo PIV and tomo PIV with 4D flow MRI (3T; isotropic spatial resolution, 0.625 mm; temporal resolution, 64 ms; VENC, 75 cm/s; and acquisition time, around 10 min) in an in vitro experiment for the carotid artery and reported good accuracy agreement.⁸³ Harloff et al. performed 4D flow MRI (3T; spatial resolution, 1.1 × 0.9 × 1.4 mm³; and temporal resolution, 52 ms) of the common carotid artery and internal carotid artery and reported that systolic blood flow velocity was underestimated (26% and 19%, respectively) and diastolic blood flow velocity was almost as good (9% and 3%, respectively) as compared to US.²¹ The reason for the underestimation of 4D flow MRI compared to US is that US has higher spatial and temporal resolution (spatial resolution/temporal resolution: MR, 1.1 mm³/ 45 ms; US, 0.3 mm²/1.5 ms). The diameter of the internal carotid artery near the carotid bifurcation and the common carotid artery has been reported to be 4–5.2 mm and 6–7.8 mm, respectively,⁸⁴ so a spatial resolution of 1.1 mm would include at least four voxels and maintain accuracy, but this is considered an underestimation compared to the high spatiotemporal resolution US.

Cardiovascular system

Markl et al. were the first to report 4D flow MRI (1.5T; spatial resolution, 0.94 × 0.94 × 3 mm,³ 0.63 × 0.71 × 1mm³; and VENC, 100 cm/s) on an in vitro fluid experiment to measure flow velocity in an 8.27-mm diameter stenosis, 15.9-mm diameter, and 19.1-mm diameter tubes. They compared those results with 2D cine PC MRI.¹ The results showed that the accuracy of velocity profiles at peak flow, waveform along the time, and mean velocities was good. Takehara et al. compared the flow velocities of the descending aorta of the non-diastolic part of 18 patients obtained by 4D flow MRI and 2D cine PC MRI and reported that there was no difference in the diastole and a difference of up to 8.6% in the middle contraction period.⁸⁵ Stam et al. used 4D flow MRI (1.5T; acquired spatial resolution, 2.1 × 1.7 × 2.8 mm³; reconstructed spatial resolution, 2.1 × 1.7 × 1.4 mm³; VENC, 250 cm/s; and scan time, 5.57–8.51 min) to image the aorta of nine swine, and compared flow velocity measurements using a flow probe guided into the aorta with existing 2D cine PC MRI flow velocity measurements.²⁶ In their study, 4D flow MRI showed higher accuracy than 2D cine PC MRI. Frydrychowicz et al. performed 4D flow MRI (3T or 1.5T; spatial resolution, 2.0–3.0 × 1.58–1.95 × 2.0–3.5 mm³; number of cardiac phases, 10–24; VENC, 150–200 cm/s) on 24 patients after aortic coarctation (CoA) surgery, 4 patients before CoA surgery, and 19 healthy subjects and reported that the peak velocities of 4D flow MRI were underestimated, although they correlated with echocardiographic results.⁸⁶ They thought that the reason for the underestimation may have been the low spatial resolutions of 4D flow MRI. Puiseux et al. examined in vitro aortic models with 4D flow MRI (1.5T; isotropic voxel size, 2 mm, 3.1 mm, and 3.4 mm; and temporal resolution, 49–52 ms) and compared the results with those of CFD and reported that downsizing the spatial resolution of CFD to that of 4D flow MRI improved the correlation of flow velocity and WSS between the two methods.⁴¹ These results indicate that the accuracy of cardiovascular 4D flow MRI is equal to or better than 2D cine PC MRI, and that the accuracy is lower than that of US or CFD with high spatiotemporal resolution, but better when the spatial resolution is made comparable.

When comparing the measured values of 4D flow MRI with the measured values and/or calculated values of other modalities, it is necessary to pay attention to the pulsation of the blood vessel wall. Although blood vessels in the body beat with the heartbeat, intracranial vessel walls have little elastin and beat only slightly. Therefore, in CFD for in vivo and in vitro use, treating the intracranial vessel wall as a rigid body is considered to be less problematic.^3,4,87 However, there is more pulsation in the aorta so the rigid wall assumption may be problematic. This should be kept in mind in cases when CFD cannot simulate vessel wall pulsatility. For example, in blood flow analysis of in vitro models, both 4D flow MRI and validation modalities treat the vessel wall as a rigid body, so there is no inconsistency.¹² However, in the case of in vivo blood flow analysis, human blood vessels are beating, whereas the vessel wall in CFD modeling is often calculated as a rigid body. Therefore, CFD calculations in this case cannot fully simulate the hemodynamics of the body.¹² The same is true for 4D flow MRI of in vitro models of blood vessels that have the same geometry as the in vivo 4D flow MRI examination. Therefore, the validation of the accuracy of in vivo 4D flow MRI measurements of the cardiovascular system should be based on comparisons with well-established 2D cine PC MRI and 4D flow MRI, which have already been validated for accuracy, or the conservation of mass’ principle.

Abdominal blood vessels

Roldán-Alzate et al. used radial 4D flow MRI (3T; PC-VIPR; spatial resolution, 1.4 mm isovoxel; VENC, 50–100 cm/s; and 0.1 mmol/kg gadobenate dimeglumine intravenous injection) to measure flow velocities of the hepatic artery, portal vein, and splenic vein in seven cases of portal hypertension and 17 normal subjects and reported that the accuracy based on conservation of mass’ principle was good.²⁷ Stankovic measured flow velocity and flow rate of the hepatic artery and portal vein in 10 normal subjects with different spatiotemporal resolutions of 4D flow MRI (3T; spatial resolution/temporal resolution, 2.4 × 2.0 × 2.4 mm³/61.2 ms, 2.5 × 2.0 × 2.4 mm³/81.6 ms, and 2.6 × 2.5 × 2.6 mm³/80 ms) to examine the accuracy based on conservation of mass’ principle and reported a decrease in the accuracy of measurement of hepatic arterial blood flow and portal vein blood flow at low spatiotemporal resolution.²⁹ As mentioned above, when the diameter of the abdominal vessel is small, the measurement accuracy may decrease due to the fact that more than five voxels cannot be secured in the vessel and the VNR decreases at low velocities.

Summary of accuracy of 4D Flow MRI

MR systems with high magnetic fields have high SNR, which is advantageous for maintaining the measurement accuracy of 4D flow MRI. However, for the cardiovascular system and the abdomen, 7T MR systems are not sufficient at present. For the cardiovascular system, 1.5T and 3T are considered to have almost the same accuracy. Spatial resolution is recommended to be at least four voxels, and if possible, six voxels for the diameter of the vessel of interest. Keep in mind that cerebral and abdominal vessels are thinner than cervical vessels, the cardiovascular system, and the SSS, so the accuracy of flow velocity measurement will be lower. VNR should also be considered for arteries and veins with low flow velocity. The acceleration technique of 4D flow MRI is very promising for shortening acquisition time, increasing spatial resolution, and increasing temporal resolution, but it has different effects on the accuracy and the results of hemodynamic visualization, so these should be checked when introducing a new technology into the clinical practice.

Repeatability and Reproducibility of 4D Flow MRI

As with any medical device, the higher the accuracy of the 4D flow MRI measurements, the better, but what is more important clinically is the precision. Precision is the degree to which an instrument or process will repeat the same value. It is important to have good test–retest repeatability, test-retest reproducibility, intraobserver repeatability, and interobserver reproducibility. The reason for this is that even if the measured value is different from the true value, if the repeatability and reproducibility are high, it is possible to distinguish between normal and abnormal values, and in the case of abnormal values, it is possible to classify them into grades. In addition, reproducibility is important for follow-up of cases. This is essential for single-center studies. In addition, in order to conduct a multi-center study, reproducibility needs to be assessed using different instruments and different facilities. Furthermore, there are some differences between test-retest repeatability and test-retest reproducibility. Test-retest repeatability should include the first and second set of scans done by the same operator, without repositioning or replanning, within the same scan session. Test-retest reproducibility, however, uses data obtained by MR examinations done on two or more different occasions. Because of the different MR conditions, test-retest reproducibility is thought to be affected by the MR operator, repositioning of the patient, replanning, patient’s heart rate and arrhythmia. Most of the papers we evaluated focused on test-retest reproducibility rather than test-retest repeatability. Test-retest reproducibility is more important in the clinical setting.

In order to confirm reproducibility, it is first necessary to establish a protocol for the imaging method, analysis method, and analysis values required for each imaging target and disease. Also, in 4D flow MRI, the geometry and size of blood vessels or SSS may vary depending on the image quality, method, and threshold used for segmentation. In addition, the measured values may vary depending on the position and angle of the flow velocity measurement cross-section. For these reasons, it is especially important to consider intraobserver repeatability and interobserver reproducibility in the analysis, as well as in the data collection in order to promote clinical application.

In the following, we introduce the papers on repeatability and reproducibility of 4D flow MRI, mainly for flow rate and velocity (Supplementary Table 2).

Cerebrovascular vessels

The test–retest reproducibility of 4D flow MRI for flow and velocity measurements of the main intracranial arteries and sagittal sinus,⁸⁸ the superior sagittal, straight and transverse sinuses,²² and intracranial arteries and aneurysms⁷⁰ was reported to be good. In these papers, the interobserver reproducibility^22,88 and intraobserver repeatability²² were also reported to be good. Multicenter reproducibility was also reported to be good for the same model.⁸⁸

Cerebrospinal fluid

The test–retest repeatability of 4D flow MRI measurements of cerebrospinal fluid flow and velocity has been reported to be good.⁸⁹

Cardiovascular system

Markl et al. first reported reproducibility of 4D flow MRI.⁹⁰ They used 4D flow MRI (3T; spatial resolution, 1.7 × 2.0 × 2.2 mm³; temporal resolution, 40.8 ms; VENC, 150 cm/s; and acquisition time, 12–20 min) to image the thoracic aorta in 12 normal volunteers and showed that scan–rescan, interobserver, and intraobserver agreement for the calculation of total flow and peak systolic velocity were excellent, and interobserver reproducibility and test–retest reproducibility for the segmental distribution of WSS in the thoracic aorta were good. 4D flow MRI imaging target of cardiovascular system where interobserver reproducibility was reported included the aorta,^{46,52,86,90–94} the aorta and pulmonary artery,^65,95 tricuspid valve regurgitation,⁹⁶ the right heart,⁹⁷ and left atrial (LA) blood flow,⁹⁸ all of which had good interobserver reproducibility. In these studies, some papers reported good intraobserver repeatability.^{52,86,90–98} There are, however, reports of aorta and pulmonary artery evaluations that did not assess interobserver reproducibility but reported good intraobserver repeatability.^54,99 Some of the above papers reported good test–retest repeatability or reproducibility.^{52,65,90,92–94,98,99} However, Spartera reported that longitudinal changes in heart rate and blood pressure at the interval scan in the same subjects were associated with significantly higher variability for LA stasis.⁹⁸ In addition, good multicenter reproducibility of aorta and pulmonary artery evaluation by 4D Flow MRI of the same model at three different centers has been reported.⁹⁹ In vitro volume flow in the great vessel model showed good inter-vendor reproducibility, while in vivo kinetic energy in the left ventricle showed poor inter-vendor reproducibility.⁴⁹ The authors of that paper thought that velocity offset varied between sites having the same vendor and sequence as a reason. In addition, there are reports of poor inter-vendor reproducibility in aortic evaluations using devices from different manufacturers.⁹³ The authors stated that this was caused by differences in imaging parameters between vendors. Furthermore, poor intra-vendor reproducibility has been reported when aortas were evaluated using different sequences on devices from the same company with different field strengths (1.5T, 3T, and 7T).⁴⁶

Abdominal blood vessels

In 4D flow MRI of the liver, interobserver reproducibility is good, but poor test–retest repeatability or reproducibility of arteries or small vessels has been reported.^29,55 In the evaluation of transplant renal artery and vein, interobserver reproducibility was good, but test–retest reproducibility was reported to be worse in veins than in arteries.¹⁰⁰

In vitro experiments

The test–retest reproducibility was good in a study of a straight tube, assuming an aorta.⁶⁴ 4D flow MRI measurements in a 7–mm straight tube showed that the inter-vendor reproducibility of three different devices from three different manufacturers was generally good.³⁴ On the other hand, in an in vitro experiment using a subject-specific CSF flow model of a Chiari malformation patient, 4D flow MRI was performed three times with five different devices at five centers, and the test–retest repeatability was good, but inter-scanner reproducibility was poor.⁸² The reason raised in that paper was that the imaging parameters could not be sufficiently matched.

Segmentation

Segmentation has an effect on the precision of 4D flow MRI. When creating the shape of a blood vessel or subarachnoid space, the analysis result of flow rate changes depending on the cross-sectional area, and the analysis result of shear stress changes depending on the position of the blood vessel wall. Therefore, reproducibility within and between analysts is important. Juffermans et al. attempted to clarify the aortic lumen segmentation in 4D flow MRI by interexamination, interobserver reproducibility, and intraobserver repeatability.⁹⁴ The results showed that the reproducibility and repeatability were good (volume, surface area, and centerline length [COV = 10%–32%, r = 0.54–0.95 and ICC = 0.65–0.99], maximal diameter [COV = 3%–11%, r = 0.94–0.99, ICC = 0.94–0.99], curvature radius [COV = 25%–62%, r = 0.73–0.95, ICC = 0.84–0.97]). In addition, there is a report by Dunås et al. on Centerline Processing Scheme for semi-automatic segmentation and quantification of cerebral blood vessels,¹⁰¹ and a report by Schrauben et al. for evaluation and optimization of several types of cerebral blood vessel segmentation.¹⁰² As for AI-based 3D segmentation, Berhane et al. reported a fully automated, fast 4D flow MRI-based 3D segmentation of the aorta using deep learning to show excellent agreement for flow, peak velocity, and dimensions with low bias and limits of agreement with a less than 10% difference compared with manual analysis.¹⁰³ The time taken for this segmentation was 0.438 ± 0.355 seconds, compared to 630 ± 254 seconds for manual human segmentation, a significant time saving. AI will be a valuable tool in the future.

Summary of repeatability and reproducibility of 4D Flow MRI

4D flow MRI has good test–retest reproducibility, intraobserver repeatability, and interobserver reproducibility. Multicenter reproducibility can be expected for the same model. On the other hand, intra-vendor reproducibility is poor even for the same manufacturer’s equipment due to differences in the equipment model, magnetic field, and acquisition sequence. In some cases, inter-vendor reproducibility is good even with different devices from different manufacturers, but this is not always the case. For this reason, it is safer to use the same model for follow-up of patients in the same institution or for multicenter studies. In addition, test–retest repeatability or reproducibility may be poor for small vessels due to their relatively low spatial resolution and for vessels with slow flow velocity due to their low VNR. For segmentation, automation and AI support are desired in the future.

Future of 4D Flow MRI

For cardiovascular system, standardization of imaging methods and confirmation of repeatability and reproducibility have progressed, and clinical application has advanced considerably. For cerebral, cervical vessels, and SSS, it is important to establish the usefulness of 4D flow MRI and standardize appropriate analysis methods for each disease, and to check the repeatability and reproducibility of these methods. For the abdominal vessels, in order to further improve accuracy and reproducibility, it is necessary to take measures against the movement of blood vessels due to respiration, low SNR and securing the number of voxels in small blood vessels, and low VNR due to low flow velocity. Successful implementation of these measures will greatly enhance clinical application of 4D flow MRI.

The development of various acceleration methods is expected to shorten the acquisition time, increase the spatial resolution and the temporal resolution, and improve the measurement accuracy of 4D flow MRI. In addition, post-processing software equipped with AI is expected to further develop reproducible analysis methods that can be processed in a short time. In addition, the development of various biomarkers that can differentiate and stratify various diseases is expected to advance further in the future. These are expected to be more useful in the diagnosis and care of patients.

In the future, it would be ideal if hemodynamic biomarkers could be expressed as numerical values, like a complete blood count in a blood test.

Conclusion

In the quality control for 4D flow MRI, it is important to maintain accuracy, as well as precision.

It should be noted that the imaging time is extended by increasing the spatiotemporal resolution in order to maintain accuracy and precision, while the SNR and VNR decrease in small blood vessels and blood vessels with low flow velocity. Therefore, it is necessary to maintain the SNR and VNR by setting the appropriate voxel size and VENC according to the diameter and flow velocity of the vessel to be imaged, or by introducing multi-VENC.

For the clinical application of new imaging methods, including acceleration techniques, it is desirable to perform in vitro and in vivo accuracy verification based on existing accuracy-verified 2D cine PC MRI and/or 4D flow MRI, as well as accuracy verification by the conservation of mass’ principle. It is also prudent to check test–retest reproducibility, interobserver reproducibility, and intraobserver repeatability. Using the same model for follow-up study and multicenter studies is advisable.

Supplementary Information

Supplementary files below are available online.

Supplementary table 1

Representative publications about their imaging parameters and accuracy of 4D flow MRI.

Supplementary table 2

Representative publications about repeatability and reproducibility of 4D flow MRI.