Recommendations Towards Standards for Quantitative MRI (qMRI) and Outstanding Needs

RECOMMENDATIONS

Magnetic resonance imaging (MRI) has made it possible to image soft tissues within the human body, allowing non-invasive determination of information. Quantitative MRI (qMRI) can be defined as the extraction of a characteristic from an MR image that has a magnitude that can be expressed as a number with units (e.g., distance) or relative to a reference material (e.g., proton density as % of water signal). This definition includes measurement of length and volume; relaxation properties (T₁, T₂, T₂*); flow; phase and more. Quantitative MRI has the potential to make a great clinical impact on diagnostics by enabling earlier detection of disease, complementing or replacing biopsy, providing clear numeric differentiation of disease states, and increasing the quality of information available to artificial intelligence algorithms. However, the incredible amount of variability in clinically-used image acquisition and post-processing techniques hinders current efforts to extract reliable, consistent and accurate quantitative information from routine MRI exams. Reference materials and reference objects (phantoms), along with associated image acquisition protocols and software for image and data analysis can be developed to address some of these obstacles. Each of the components: reference object, acquisition protocol and software, should be developed with the challenges of the particular anatomy and method being considered. The reference objects and test methods need to be transferred into the clinic, such that reliable and reproducible qMRI becomes routine. Then, the value of qMRI can make a definitive impact.

The National Institute of Standards and Technology (NIST) hosted workshops in 2014 and 2017 with participants from several organizations working towards standards in quantitative MRI (Table 1). Here we summarize the recommendations of the workshop.

Table 1:

Organizations working towards standards in quantitative MRI who participated in the 2014 and 2017 NIST workshops

Acronym	Organization
AAPM	American Association of Physicists in Medicine
ACR	American College of Radiology
ACRIN	American College of Radiology Imaging Network
DoD	Department of Defense
FDA	Food and Drug Administration
IAC	Intersocietal Accreditation Commission
ISMRM	International Society of Magnetic Resonance in Medicine
MITA/NEMA	Medical Imaging and Technology Alliance/ National Electrical Manufacturers Association
NIH	National Institutes of Health
NCI	National Cancer Institute
NIST	National Institute of Standards and Technology
PhRMA	Pharmaceutical Research and Manufacturers of America
RSNA	Radiological Society of North America
QIBA	Quantitative Imaging Biomarkers Alliance
QIN	Quantitative Imaging Network (under the NCI Cancer Imaging Program)
VA	Department of Veteran’s Affairs

While there is a great deal of interest within the clinical community regarding qMRI, all workshop attendees agreed that clinical adoption of quantitative imaging metrics will only be successful if methods are reliable, repeatable, accurate, and, perhaps most importantly, have an impact on the decision-making process for patient care. To ensure repeatable and accurate methods, reference objects and analysis software should be developed concurrently with the development of the quantitative imaging method. A recent example includes an anthropomorphic, prostate, body-imaging reference object(1) developed by the University of Minnesota in collaboration with High Precision Devices, Inc (Boulder, CO). The UMN/HPD body phantom (Figure 1) uses materials for T₁ and T₂ relaxation times and ADC values that were previously used in the International Society of Magnetic Resonance in Medicine (ISMRM)/NIST MR system phantom and the Quantitative Imaging Biomarkers Alliance (QIBA^®)/National Institutes of Health (NIH)/NIST isotropic diffusion phantom, to support the broader translation of a prostate cancer prediction model based solely on qMRI data(2). Additionally, for ease-of-use by clinical sites, development of a given reference object should include associated analysis software packages or a digital package akin to an application programming interface (API) that would include locations of the relevant regions of interest and reference property values (e.g., T₁ relaxation times). This “full package” for method development can ease the transition from research to clinic for qMRI applications.

Figure 1:

Top-view of the UMN/HPD anthropomorphic, prostate, body-imaging reference object. This three-sectioned body phantom has a quantitative prostate mimic at its center (i.e. yellow, green and red structure) and can accommodate surface and endorectal coil configurations.

Reference objects can serve multiple important roles in the development and deployment of a qMRI method. The most common role for a qMRI phantom is as a reference sample that can be imaged to measure and compare the performance of a combined imaging and analysis method across instruments and over time. Some standardized reference objects exist(3–12), as described later in this manuscript, but there is a need for more types of phantoms to evaluate the performance of different types of sequences and MRI contrasts, such as physiologically relevant values of T₁, T₂ and ADC in a single voxel, multi-modality (e.g., MR-PET), magnetization transfer (MT), and others listed in Table 2. Reference objects may also serve important roles in the design of quantitative imaging methods from acquisition strategy through analysis pipeline. First, ex vivo tissues, animal models, tissue extracts, or chemical solutions may be used as phantom representations of in vivo human tissue. Experimental studies of such phantoms have helped establish current qMRI tissue models, and as we aim to probe ever more specific tissue characteristics with qMRI, corresponding experimental studies of suitable tissue phantoms will be needed. Finally, in silico phantoms can be used to evaluate the propagation-of-errors in qMRI methods, considering error from image noise, as well as from both the variance and bias of parameter constraints or assumptions, such as deviations from the assumption of homogeneous and static radiofrequency (RF) fields. For example, the MR extended Cardiac-Torso (MRXCAT) phantom for realistic simulation of cardic MR(13) and the Radiological Society of North America (RSNA) QIBA^® digital reference objects (DRO) for dynamic contrast-enhanced (DCE) MRI(14). These computational phantom studies are necessary to define potential performance of a qMRI method and can serve to inform on both model and pulse sequence design. While this manuscript focuses on reference objects for use in comparing acquisition data from various imaging platforms to identify and mitigate sources of bias and various to the degree possible and to define confidence intervals on measures obtained from the acquired data, the workshop attendees appreciated that the data analysis packages may also contribute significantly to measurement bias and variance. Application-specific DROs allow the assessment of such sources of error for a given analysis package as well as levels of variance across analysis packages and across time. Finally, at the workshop, several advanced reference objects were discussed, and needs identified by participants. We summarize those suggestions in Table 2.

Table 2:

Specific reference object and measurement needs identified at the 2017 NIST workshop¹

Low-cost/home-made phantom recipes to help get sites started on standardized quantitative MRI  Make reference standards more biologic (e.g., including magnetization transfer, microstructure, etc) Improve ability of reference objects to account for spatial variations in B1 and B0 (e.g., distribute samples for given parameter throughout phantom or vary all parameters within samples) Consider more advanced contrast media (e.g., T1 simultaneous with T2 or multi-exponential diffusion with realistic T1 and T2) Next generation phantom to include physiologically relevant values of T1, T2 and ADC in each voxel Phantom sensitive to magnetization transfer (MT) effects, both to measure MT and to optimize fast T2 mapping techniques so that T2 values are not affected by MT Consider next generation of diffusion phantoms (DTI applications) Phantom with compartments that each simultaneously have diffusion-T1-T2-R2*-PDFF (proton density fat fraction) characteristics and span relevant physiological values R2*/fat fraction phantom Dynamic phantom (e.g., physiological motion)  Pre-clinical (animal) phantom Multi-modality (MR-PET) phantom Quantitative flow phantom  Musculoskeletal (MSK) phantoms (by considering the coil shape; can start with knee coil); T1rho measurement; the tissue relaxation properties of long T1 and short T2; temperature monitoring and control etc.

¹The order of the list should not be interpreted as a ranking of priority.

BACKGROUND

In light of the many positive contributions qualitative MRI already makes, existing workflows and decades of experience with image interpretation, a legitimate first question regarding quantitative MRI is “Why do we need qMRI?”. One answer lies in the difference between clinical and sub-clinical presentation of symptoms – or, the absolute minimum deviation of a physiological or anatomic target from normal. A concrete example would be assessing the presence of a tumor, visible by eye (clinical presentation) vs. detecting the presence of that same tumor earlier as a small collection of cells with abnormal shape or physiology (sub-clinical presentation). The treatment options and patient outcome are typically better for the latter scenario than the former. However, to excel at identifying sub-clinical presentation of disease requires numbers (quantitative analysis) in addition to images.

As a case study, consider the effect of quantitative analysis on the determination of pediatric blood lead levels over the past 60 years. Prior to 1960, the prevailing sentiment was typified by toxicologists like Dr. Robert Kehoe who had “never seen a case of lead poisoning with lead level < 80 μg/dL”(15). New analytical methods, especially those based on atomic absorption spectroscopy, were developed in the 1960s and allowed scientists and clinicians to begin addressing the relationship between blood lead level and toxic manifestations quantitatively. Documented method (quality) control, interlaboratory comparisons and improved instrument performance led to agreement between experts on measured lead values and reduction of the measurement error. Without quantitative laboratory tests, Dr. Kehoe relied on clinical manifestations like colic, encephalopathy or death, which occur when lead blood concentrations exceed 50 μg/dL. Implementation of quantitative analytical techniques have now shown specific sub-clinical manifestation of lead concentration in the blood with deleterious impact on hemoglobin synthesis (40 μg/dL), vitamin D metabolism (30 μg/dL), nerve conduction velocity (20 μg/dL), and IQ level, hearing and growth at values as low as 10 μg/dL(16). Recent reviews have even suggested there is sufficient evidence to lower the Centers for Disease Control (CDC) limit for lead in blood to 2 μg/dL(17).

Until 1960, leading clinicians and toxicologists were certain they had an excellent understanding of the lead toxicology and epidemiological impact. However, only after the application of advanced analytical instrumentation in a standardized way did the community see how little they understood. We may ask the same question about the way MRI is currently practiced. What new patterns and sub-clinical presentations of disease states will reveal themselves, once we have removed (through quality control and standardization) variation in image information and quality due to scanner, site and operator influence? Can we use earlier knowledge of disease state, acquired with qMRI, to treat earlier and therefore more efficiently or effectively? Could we avoid unnecessary treatment or interventions with improved quantitative metrics to better risk stratify patients? Many experts believe the answer to these questions is unequivocally “Yes”.

Lead levels are one example of a biomarker, which is a defined characteristic that is measured using an assay, as an indicator of normal biological processes, pathogenic processes or responses to an exposure or intervention(18,19). The term “biomarker” is often assumed to imply a measurand of a laboratory test (e.g., blood cholesterol levels), but it can also refer to a clinical measurand like blood pressure or the output of a clinical imaging scan. Imaging methods that provide information about the nature and amount of matter or activities present can be considered biomarkers and are conceptually similar to laboratory assay targets. Image-based biomarkers can be considered as a type of in vivo assay. An assay is a procedure for measuring the presence, amount, or functional activity of a target entity (the analyte, measurand or target of the assay). The analyte can be a drug, a biochemical substance, or a structure or process in an organism or organic sample. A biomarker is a characteristic (of normal or disease), and an assay is the test that measures some parameter that reflects the “amount”, extent, or degree of biomarker that is present. Common terminology, procedures, and methods have become established in medicine to describe, evaluate and validate laboratory assays through the work of the Clinical and Laboratory Standards Institute (CLSI). Many of the same concepts and approaches can and should be applied to radiological diagnostic assays, and this has begun to occur in an organized way over the past decade through the efforts of many groups (Table 1).

The use of imaging-based biomarkers for clinical applications, drug development and safety evaluation has increased considerably in recent years and has stimulated a large number of efforts to develop methods for accuracy and consistency across medical imaging platforms, especially in the setting of multi-center trials. The National Institute of Standards and Technology hosted workshops in 2006(20), 2014 and 2017 to drive discussion and collaboration amongst the various stakeholders and groups working on standardization for quantitative medical imaging through tools such as reference objects and protocols. Standards range from common understanding or common practices to common requirements, and standards can be established through consensus groups such as the RSNA QIBA^® or National Electrical Manufacturers Association (NEMA) (Table 1). A documentary standard is a classification, guide, specification, or test method developed and established according to principles of consensus, such as those of NEMA. Here, we review some of the achievements of the standardization efforts of the past decade and discuss some needed steps to overcome the obstacles inhibiting clinical diagnostics based on qMRI.

DEPLOYED NIST REFERENCE OBJECTS

NIST created three reference objects that have been commercialized and are being actively used by the MRI community: the ISMRM/NIST system phantom(7,21), the QIBA^®/NIH/NIST isotropic diffusion phantom(3,22,23), and the University of California San Francisco (UCSF)/NIST breast phantom(4,5). The NIST reference objects have been used to identify measurement errors and then improve the stability of acquisitions in technique/pulse sequence development and protocol development for multi-site studies. These reference objects were necessary to aid standardization, but they are insufficient given the wide range of qMRI applications – that is, there is still a need for additional reference objects. For example, the breast phantom was created in part because the existing phantoms were not physically compatible with a breast coil.

In general, the reference object (or phantom) is an outer shell in an idealized geometry (e.g., a sphere for a head), which contains reference solutions in specified configuration. The outer container is dependent on both the anatomy of interest and the RF hardware in use (e.g., a 15.0 cm diameter sphere is not a good match for a breast coil). Previous work demonstrated that using phantoms with correct geometry for the anatomy of interest, which is also appropriately sized for the RF hardware, is important for assessing quantitative imaging performance(5,24). The reference solutions within the phantom are dictated by the quantitative parameters of interest. For example, the UCSF/NIST breast phantom contains fat-tissue mimics, while the other reference objects do not.

The ISMRM/NIST system phantom was designed to address many of the needs of qMRI including volumetric distortions, slice profile assessment, high contrast spatial resolution, proton density and T₁ and T₂ relaxation times. The system phantom design includes important features from the Alzheimer’s Disease Neuroimaging (ADNI) MagPhan phantom (The Phantom Laboratory, Salem, New York, USA) used to assess geometric distortion in three dimensions (Figure 2) (25,26) and the American College of Radiology (ACR) phantom used to assess slice profile and resolution(27,28). The system phantom reference solutions have various T₁ and T₂ properties that have been fully characterized using NIST traceable methods(29). This phantom has been, for example, used to assess day-to-day variation of MR fingerprinting (MRF) measurements of T₁ and T₂ relaxation times(30). In this study, the ISMRM/NIST system phantom revealed variation in T₂ measurement, which could be a result of temperature variation (such measurements when using MnCl₂ solutions are known to be temperature sensitive). However, this finding also led to improvements in the T₂ measurement technique, addressing measurement error due to transmit B₁ variations and reducing measured variation.

Figure 2:

Sagittal slices of a 3D T₁-weighted spoiled gradient echo sequence from the ISMRM/NIST system phantom (A) and the ADNI MagPhan phantom (B), which can be used to assess geometric distortions. An advantage of the ADNI MagPhan phantom in this orientation is the lack of symmetry.

Based on the work of Pierpaoli et al. to develop aqueous solutions of polyvinylpyrrolidone (PVP) (22,23) the National Institutes of Health (NIH) and Chenevert et al. ice-water phantom(31,32), Boss et al.(3) developed an isotropic diffusion phantom (Figure 3). The QIBA^®/NIH/NIST isotropic diffusion phantom, filled with ice-water, was used by the TRACK-TBI trial to qualify 19 clinical sites as enrolling centers. Across the 11 initial sites, for the 0 to 40 % weight-by-weight (w/w) PVP concentrations, the coefficient of variation increased from 2.2 % to 4.5 %. The 50 % w/w PVP sample had a considerably higher coefficient of variation, 11.9 %, likely due to the chosen protocol that used b-values of 0, 500 and 900 s/mm²(33).

Figure 3:

NIST developed an isotropic diffusion phantom with support from NIH and RSNA-QIBA® based on the work of Pierpaoli et al. to identify an appropriate polymer, PVP, (22,23) and the ice-water phantom of Chenevert et al (31,32). A photo from the inside of the prototype diffusion phantom (A), a spin-echo image of the phantom filled with the ice water bath (B), and an apparent diffusion coefficient map with values in the phantom from approximately 0.2 to 1.1 × 10⁻³ mm²/s (C).

Finally, the UCSF/NIST breast phantom, which has T₁ and ADC values representative of fibroglandular and fat tissue, is used by the I-SPY 2 clinical trial(5). The phantom is designed to test both sides of breast coils, which requires a unique geometry. An initial study with the phantom revealed geometric distortions in the right-left orientation of the axial plane with echo planar diffusion-weighted images (Figure 4) across systems (fields and platforms) dependent on which side of the coil the object was located(4). The distortions remained in the right/left orientation regardless of phase-encoding direction, did not change with b-value, and thus are most likely due to B₀ inhomogeneity(34). The B₀ inhomogeneity can be due to the magnet itself; the RF or gradient coil hardware; and objects within the scanner (e.g., the phantom or foreign objects in the body). In addition, the distortions were present in patient data as well and were not identified prior to using the phantom with the geometric distortion array. The geometric distortion, which varies both day-to-day and scanner-to-scanner, limits the precision with which tumor sizes can be measured and as a result limits the ability to identify changes in tumor size due to intervention (e.g., radiation or chemotherapy).

Figure 4:

Axial images from a central slice of the UCSF/NIST prototype breast phantom. The T₁-weighted images (A, E) have no geometric distortion compared to the engineering designs, regardless of position in the coil. The EPI images (B-D, F) all demonstrate a geometric distortion in the right-left orientation: a stretch on the patient (and image) left and a shrinking on the patient (and image) right. In the ADC map (B), b-value = 0 image (C) and b-value = 800 s/mm² image (D), the object on patient (image) left is larger than the object on patient (image) right. Similarly, when the objects are reversed (E, F), the object on patient (image) left is wider in the b-value = 0 s/mm² image (F) compared to the T₁-weighted image (E).

The NIST reference objects have a demonstrated benefit for qMRI protocol development across manufacturer systems and an ability to find technical errors in system software and hardware. Reference objects should be used not just as a tool to qualify sites and qualitative data for clinical trials, but additionally to qualify systems to provide qMRI data in order to demonstrate the potential added value and for ongoing quality control programs. Other reference objects that are used by the MR community include the ACR phantom(27,28); structural brain imaging phantom developed by ADNI(26); a dynamic contrast-enhanced MRI perfusion phantom developed by RSNA QIBA^®(35); other isotropic diffusion phantoms(31,32); static tissue phantoms for phase-contrast flow applications(36–38); cardiac tissue phantoms(39); MnCl₂ phantoms for iron measurement applications(40,41); and proton-density fat fraction phantom(42–44). Finally, the three reference objects reviewed here and those mentioned in the previous sentence are not able to address all the needs of the MRI community. In the sections that follow, we highlight some outstanding needs as identified at the workshop.

STAKEHOLDER PERSPECTIVES FROM THE 2017 WORKSHOP

During the 2017 workshop, several stakeholders presented their perspectives centered around the question “How is qMRI being used and what are the associated needs or challenges?” Summaries of these perspectives are provided below.

Present Status of Standards (Perspective from Organizations Developing Technical Standards)

Documentary standards for MRI are established by several organizations, some of which are based in the United States and some internationally. Documentary standards are beneficial because a proven, robust, well-documented and relatively simple method is identified that is known to produce a reliable output. Documentary standards for image performance (e.g., SNR, uniformity), ghosting, resolution and concept of quality assurance testing are developed and maintained by the NEMA Medical Imaging and Technology Alliance (NEMA/MITA) as found in the NEMA MS series 1–12(45–47) and the International Electrotechnical Commission (IEC) (62464–1)(48). Safety (e.g., whole body, local specific absorption rate (SAR); acoustic noise) standards for MRI are developed and maintained by both the IEC as IEC 60601–2–33(49) and some standards in the NEMA MS series. Currently, adoption of all MRI test standards is voluntary in the United States, and these standards may be used by MRI system manufacturers when obtaining marketing authorization, assessing safety, or performing basic system quality assurance tests. The Digital Imaging and Communications in Medicine (DICOM) standard image file format is also maintained by NEMA/MITA. To receive reimbursements from Medicare, MRI systems, at least at outpatient facilities, must annually pass a Medicare-approved certification process (such as, but not limited to, the ACR MRI Accreditation Program)(6,50). However, the tests in these Medicare-approved accreditation processes are only partially quantitative and do not include evaluation of T₁ or T₂ relaxation times or ADC measurements.

Documentary standards may be a useful mechanism for enabling a unified approach to specific qMRI methodologies. However, prior to creating new documentary standards or tests, we caution the community to ask: “what new information does an additional test or standard deliver?”. If the goal is an imaging protocol that works across systems, a documentary standard may not be the best way to achieve that goal. Without agreement on one method to make a measurement, guidelines or recommendations, such as those promulgated by RSNA QIBA^® (Table 1), may be the best path toward meeting the community’s needs for standardized acquisition protocols.

Members of the community can lead the adoption of de facto standards by championing their own cause. A necessary step is to collect and publish test-retest data that leads to identifying repeatable protocols and the creation of a standardized protocol across manufacturers’ systems. For example, the magnetic resonance elastography (MRE) community conducted several test-retest studies(51–53) and leveraged those to create a QIBA^® profile for performing MRE in the liver(54). With non-trivial effort, the quantitative MRI community itself can lead the development and adoption of standardized practices. However, consensus agreement must still be reached across the community to support a particular standard.

In addition to the work of QIBA^®, other efforts toward consensus methods come from specific communities. For example, the cardiac MR community regularly publishes consensus methods(55–57), which are available at https://scmr.org/page/guidelines. ISMRM organized or endorsed workshops led to consensus statements on diffusion outside the brain(58), fat-water standardization(59,60), speech MRI(61), and a proposed standard for raw data format(62). The ISMRM perfusion study group along with the European consortium for Arterial Spin Labeled (ASL) in dementia wrote a consensus statement on recommended implementation of ASL perfusion MRI(63), which led to inclusion in the ACR practice parameters documentation(64). As discussed earlier in this section, a standard should be adopted with consensus and because it addresses an unmet need or leads to the sharing of new information.

Seeing the Same: the Need for Platform Independence in Imaging (Perspective from the Veterans Health Administration)

The principal mission of the Veterans Health Administration is the health of the military Veteran in the United States. One example relates to brain disorders. While improvements in outer tactical vests (body armor) and helmets have reduced fatal injuries, many Servicemembers return with a traumatic brain injury (TBI), post-traumatic stress disorder (PTSD), suicidal thoughts or behaviors, and/or related co-morbidities(65). These co-morbidities or co-occurring conditions are defined as mental health disorders by the National Research Action Plan (NRAP)(66). One of the main objectives of the NRAP is to develop methods that could detect TBI and PTSD in Servicemembers and Veterans and enable the monitoring of progression of the condition and its response to therapy.

MRI could provide safe, noninvasive imaging techniques to meet many of the goals of the NRAP. For example, diffusion tensor imaging (DTI) could be used to visualize white matter tracts, which are known to be susceptible to progressive aspects of TBI(67). However, differences across manufacturers, models, and software versions as well as possible “drift over time” have hindered use of the technology, even for clinical trials(68). Differences across manufacturers, models, post-processing techniques and time were recognized at the workshop as a key hinderance to clinical adoption. Differences exist for simple geometric measurements, as documented by the ADNI trials(26), and for model-based measurements such as T1 relaxation time measurement(69). Reference objects were used to identify the geometric distortions in ADNI trials(26) and assess T1 measurement variation(69). Reference objects are critical to ensure quantitative MRI data is comparable between systems (both hardware and software variations) for neurodegenerative disorders and other applications. Developing DTI into an accurate, reliable diagnostic that can be used to assess the health of white matter over time could lead to a noninvasive tool that not only can track neurodegenerative disease, but also the efficacy of therapies designed to halt progressive degeneration or to repair the damaged brain for Veterans, Servicemembers, and civilians.

Quantitative Imaging Biomarkers in the Era of Precision Medicine (Perspective from Clinical Research)

In the era of precision (personalized) medicine, quantitative imaging biomarkers have become a pre-requisite. Quantitative imaging develops and optimizes anatomical, functional and molecular imaging protocols, data analyses, display methods and reporting structures. Quantitative imaging biomarkers can address unmet medical needs and help in providing personalized medicine.

In MRI, there are several relevant quantitative imaging biomarkers derived from specific MRI techniques, and few have been incorporated in the clinical workflow. One such technique is diffusion-weighted MRI (DW-MRI), which is sensitive to random motion of water molecules. The quantitative imaging biomarker derived from mono-exponential modeling of DW-MRI data is the apparent diffusion coefficient (ADC). In the clinic, ADC has been used to stage tumors(70–73), assess treatment response(74,75), and predict tumor aggressiveness(76,77). However, at present, confidence currently doesn’t exist to compare data across scanners and populations, preventing the use of quantitative ADC measures in clinical workflow. Qualification and validation would reduce measurement variability across sites and lead to comparable data across scanners, sites, and populations so that ADC and other biomarkers could be reliably assessed, and data aggregated from multiple clinical trials. Once that step is taken, it is conceivable that ADC could be used to replace biopsy in some cases or better inform the ‘watch and wait’ path.

Therefore, there is an urgent need to validate and qualify quantitative imaging biomarkers, such as ADC, through a rigorous process, which involves a partnership between manufacturers and academic research centers, such as the work by RSNA QIBA^®(78).

Quantitative Assessment of MRI-Guided Interventions (Perspective from Academic Research)

Magnetic resonance imaging (MRI)-guided interventions, also known as interventional MRI (iMRI), takes advantage of the unique strengths of MRI to plan, guide, monitor, and assess diagnostic and therapeutic interventions. In light of the increasing clinical applications of iMRI and continual development of new technologies for iMRI, quantitative assessment is critical to characterize performance, understand limitations, facilitate and benchmark new developments, and ensure procedural safety, efficiency, and success.

To enable quantitative assessment of iMRI, it is informative to classify iMRI procedures as delivery (e.g., of a device, energy, or agent) vs. extraction (e.g., of tissue or a device), or device-based (e.g., targeted needle biopsy) vs. energy-based (e.g., focal ablation). The technical parameters to quantify include: position and geometry of the device/tissue, effects of the device (e.g., image quality, SNR, contrast-to-noise ratio (CNR), geometric distortion, tissue properties, safety), position and amount of the energy/agent (e.g., temperature, concentration), and effects of the energy/agent (e.g., image quality, SNR and CNR, tissue properties, safety). In addition, the application aspects to quantify include: procedural workflow (e.g., number of steps, time per step, failed/repeated steps, repeatability of each step, learning curve and operator dependency), technical success (e.g., accuracy of targeted device placement, accuracy of location and dose of energy/agent delivery), and clinical success (e.g., patient outcomes).

Each of the components for quantitative assessment should be standardized by the community, including scientists, clinicians, and industry experts for specific iMRI applications. As an example, the American Association of Physicists in Medicine (AAPM) Task Group on MRI-Guided Robotics-Assisted Interventions is developing standardized 3D-printed reference objects, experimental configurations, MRI protocols, performance metrics, and analysis software to quantify the impact of MRI-guided robots on image SNR and geometric distortion, as well as the accuracy and precision of MRI-guided robotic-assisted targeted device placement. Previous efforts have developed an MRI-compatible motion platform to emulate respiratory motion(79) for application in dynamic MRI(80) and real-time MRI-guided procedures(81) (Figure 5).

Figure 5:

A programmable motion phantom for quantitative assessment of dynamic and interventional MRI (79). (A) The motion platform is actuated by a pair of master-slave hydrostatic actuators (slave actuator shown in photo). The slave and master actuators are connected by a fluid-filled line, with the master actuator positioned outside of the MRI scanner room and connected to a computer-controlled motor. The gel-filled phantom contains a plate with geometric features of known dimensions for assessment of dynamic MRI sequences. These features can also serve as targets for MRI-guided targeted device placement. The platform and phantom are constructed using plastic materials. (B) A representative real-time gradient echo MRI frame depicting the phantom and its geometric features. Note that the fluid (water) in the actuator is also visible. (C) By using a learning-based algorithm, the motion platform is programmed to accurately and repeatably generate motion as specified by a reference input waveform. In this example, the reference input was an actual respiratory waveform pre-recorded from a human subject.

The impact of robotic systems and interventional devices on MRI SNR can be determined by measuring SNR (according to ACR and NEMA guidelines) with and without the device and then calculating the percentage difference. While it is obviously desirable to design systems and devices that do not degrade SNR, acceptable levels of SNR percentage difference should be considered in the context of specific clinical applications. Similarly, the impact of devices on geometric distortion can be assessed by imaging a phantom with known structural features (e.g., a grid plate) with and without the device and then calculating the observed mis-alignment of features on MRI (e.g., displacement in mm and rotation in degrees). The amount of geometric distortion depends on the choice of MRI sequence (e.g., balanced steady-state free precession, gradient echo, or spin echo) and imaging parameters (e.g., readout bandwidth, echo time (TE), repetition time (TR)), as each sequence has a different degree of sensitivity to perturbations caused by devices such as susceptibility gradients and off-resonance effects. The acceptable level of geometric distortion depends on the dimensions of the tissue of interest and the definition of technical/clinical success for the procedure. When possible, the MRI sequence with most appropriate CNR for the anatomy of interest should be used, and the imaging parameters should be prescribed to reduce geometric distortion to acceptable levels. In cases where this is not readily achievable, a second MRI sequence with acceptable levels of geometric distortion can be used to provide complementary information for guidance. Image artifacts due to systems and devices can also be assessed and managed using similar methods. In addition, the technical accuracy and precision of a particular interventional workflow and system, such as in targeted needle placement, can be characterized by performing repeated experiments in a standardized phantom with known target locations and then analyzing the distribution of an error metric (e.g., 3D Euclidean distance to measure needle-to-target error in mm). For energy delivery applications, the temperature measurement accuracy of MRI (in degrees Celsius) could be assessed with respect to a known reference (e.g., stable body temperature in a subject without heating or fiber-optic temperature probes). The acceptable accuracy and precision of the temperature measurement would need to be evaluated in the context of required therapeutic dose for treatment in diseased tissues and safety bounds for avoiding damage to normal tissues.

In summary, quantitative assessment is crucial for the research and application of iMRI. Additional efforts are needed for community building and standardization in iMRI, and there will be much to gain by leveraging and joining related endeavors in diagnostic MRI and MRI safety.

Survey of Current Reference Object Use in Multi-Site Studies (Perspective from Pharmaceutical Research)

The biopharmaceutical industry uses qMRI to guide the determination of drug efficacy and selection. At the level of multi-site trials, imaging clinical research organizations (iCROs) are often engaged to conduct the clinical trials. The iCROs conduct the imaging component of clinical trials in partnership with pharmaceutical companies and contribute to the study design, the training and qualification of sites, and harmonization of protocols, and provide a central image quality control and analysis. For this workshop, Jeffrey Evelhoch informally surveyed five iCROs (BioClinica, ICON, IXICO, PAREXEL and VirtualScopics) on their use of reference objects for MRI clinical trials. In general, when the clinical trial uses qMRI as an endpoint, MRI reference objects are used for site qualification, site monitoring(40,41), QA/QC(6) or as a tissue reference (e.g., fat fraction(42–44)). However, while all of the polled iCROs were aware of the NIST reference objects, none of them used the NIST reference objects for their qMRI trials. There are at least two reasons for that: first, the iCRO customers have not requested the use of NIST reference objects; second, some trials employed qMRI to assess metrics that were not included in the NIST phantoms (e.g., iron). There may be a need for NIST or other national metrology institutes to be involved in the creation of reference objects to contribute expertise, such as the contributions of NIST and Physikalisch-Technische Bundesanstalt (PTB) for the T₁ Mapping and ECV Standardization (T1MES) cardiac phantom(39).

The Need for Reference Objects in Multi-Site Studies for “Killer Applications” (Perspective from Medical Imaging Research)

Utilization of imaging for neurological disease is evolving rapidly. While structural (qualitative) imaging remains the mainstay of MRI referrals for diseases of the brain, researchers and clinicians are exploring the value qMRI brings to diagnosis, prognosis, treatment optimization and monitoring for patients suffering from neurological disease. Standard of care for TBI today is CT imaging to rule out brain bleeds, which often require immediate surgical intervention. However, CT imaging is normal in the vast majority of mild TBI cases, even for subjects with significant symptoms and who eventually have a complex and prolonged recovery. TRACK-TBI has demonstrated that 3T MRI is more sensitive than CT in discovering soft tissue pathology in the brain following trauma, but even with this sensitive modality, qualitative structural MRI is still of limited use in the clinical management of mild TBI patients(82,83). Several studies have explored the value of qMRI techniques such as diffusion imaging, functional MRI, perfusion arterial spin labeling, susceptibility mapping, MR spectroscopy, and other approaches; however, a complete picture has not yet emerged.

GE Healthcare and the National Football League (NFL) have partnered to study the value of advanced qMRI in a clinical research study in mild TBI(84). To overcome the recognized limitations of existing MR techniques, a state-of-the-art, multi-contrast MRI protocol was developed and distributed to the seven participating sites using uniform scanner hardware (3T MR750 with Nova Medical 32 channel RF brain coil) and software. The same imaging protocol was deployed and monitored at all sites, data was processed by a core lab, and analyzed by a team of GE scientists. Despite these stringent controls, subtle differences were detected across most sites, which could be attributed primarily to differences in the patient populations. One site was recognized as a statistical outlier for quantitative diffusion MRI data analysis. A root cause analysis is in progress to determine the source of these significant but small, quantitative differences that were limited to the high b-value shell and that do not affect routine, clinical imaging. A multi-shell (up to b=800, 1200, 2800 s/mm², 147 q-space points distributed on three shells) diffusion imaging protocol was run at all sites(85). Comparison of the data across sites on a single, traveling human phantom showed a coefficient of variation for mean fractional anisotropy (averaged over all whole white matter) of 1.6 % and 2.0 %, and mean orthogonal diffusional kurtosis of 1.9 % and 3.3 % respectively, excluding and including the statistical outlier site. At the time of the data collection, there was no appropriate phantom. Existence of an appropriate phantom for the specific quantitative metric under study (for example, restricted anisotropic diffusion) would improve harmonization of the data across sites as well as a testing paradigm to recognize some of these statistical anomalies.

Even with these technical considerations, qMRI biomarkers were identified and related to clinical measures of severity and progression of the disease. Resting state functional MRI results demonstrated remarkable correlation with clinical presentation and symptomatology(86,87), perhaps creating an opportunity to replace some of the subjective symptom inventories with objective imaging-based measures of the severity of the disease. Perfusion arterial spin labeling demonstrated decoupling of the clinical and physiological recovery of the brain(88). Symptoms disappeared sooner than physiological blood flow changes and could point to the necessity to re-evaluate return-to-activity criteria in light of the persisting abnormal blood-flow physiology in certain individuals. Finally, diffusion imaging techniques beyond current routinely used diffusion tensor models unlocked additional sensitivity to subtle axonal pathology following trauma. Not only could traumatic axonal injury can be quantified and monitored, but also axonal swelling potentially could be determined via diffusion kurtosis imaging, which characterizes tissue microstructure(89). An outlier analysis compared each individual subject to a population of normal controls and identified regions of the brain where these axonal swelling effects were preferentially noticed(85).

Deep Learning and Medical Imaging (Perspective from Industrial Research)

Deep learning techniques are used in a wide variety of fields for discovery of features in very large data sets. In medical imaging, these techniques help to identify and measure structures within images to follow the progress of disease. In order to use the output of deep learning models to make medical decisions, it is essential to be able to quantify their accuracy. Arterys (San Francisco, CA) presented an update on their deep learning models for cardiac medical images, one of which was approved by FDA for medical use. Arterys’ first commercial product is a deep learning model that was trained for segmentation of the left and right ventricles in cardiac MR images. The key to this success lies in careful work to obtain the right data to train, validate, and assess accuracy of the model. Large data sets of annotated images were screened and collected to train their model and address missing data or inconsistencies in the training annotations. Models were validated during construction using a collection of separate images annotated by radiologists. Determining the accuracy required further testing to a reference data set. The model was evaluated on two sets of data: one with hundreds of studies, where the goal was to show < 10% volume error and another with a few dozen studies, where the goal was to show accuracy within the expected range of expert annotators. By accessing industry-accepted consensus data, Arterys was able to obtain quantitative comparison of accuracy and precision compared to expert annotators. These reference data sets are not always available, yet there is a real need for a consensus ground truth in all applications. The lack of reference data limits the use of deep learning models in medicine, as it becomes difficult to validate algorithms and compare them to physician’s output.

Pre-clinical MRI (Perspective from Pre-Clinical Research)

Pre-clinical (animal) scanners, historically, have been subject to fewer regulations and standards than their clinical counterparts. However, animal MRI is considered a quantitative technique since the NIH and other research funding agencies expect to see quantitative biomarkers as the end result of a preclinical study to be readily translated into clinical trials. Most animal studies, similarly to clinical trials, are based on a longitudinal study design with repetitive MRI acquisitions on the same animal. Ensuring that the data obtained are reliable and accurate requires good image quality and quality controls. To achieve reliable results from pre-clinical scanners and imaging protocols over time, quality assurance programs, industry/manufacturer standards and qMRI reference objects are desperately needed. Even the most sophisticated, high-field pre-clinical MRI scanners are installed and validated with simplified manufacturer phantoms, which are often based on conical 50-cc tubes filled with fluids for contrast (no relaxation time measurements) and small, interlocking, plastic, toy “bricks” for volumetric measurements and anatomical localization (Figure 6). There are only a few published reports on home-built pre-clinical MRI reference objects(90,91), which are assembled without defined standard operating procedures. None of those prototypes have been broadly adopted outside the individual laboratories involved in the design. Nevertheless, these individual efforts show the existing demand for and tendency towards designing and implementing anatomical and tissue specific multi-modal, multi-parametric, preclinical qMRI phantoms(91,92). One suggestion from the workshop was to create an open-source design of a reference object for pre-clinical imaging, such that a site could make a reliable reference object particular to their magnetic field strength and bore and coil sizes. The availability of such phantoms would facilitate the promise of translation to human imaging studies.

Figure 6:

Photo of a manufacturer-supplied reference object for a 9.4 T pre-clinical imaging system from August 2018 (A) and an acceptance image using a similar manufacturer-supplied reference object for a 4.7 T pre-clinical imaging system from June 2005 (B). The pre-clinical imaging reference objects are simply conical 50-cc tubes filled with fluids for contrast (no relaxation time measurements) and small, interlocking, plastic, toy “bricks” for volumetric measurements. There was no significant improvement in reference object design for over a decade.

DISCUSSION

Several of the perspectives presented here demonstrate that qMRI applications can produce highly valuable measures for applications in TBI, cancer diagnostics and treatment planning, clinical trials and preclinical research. However, technical obstacles can prevent accurate measurements, longitudinal comparisons, and multi-site comparisons for single patients and populations. For example, qMRI measurements are only as good as the model implementation and overly simplified models could result in misleading qMRI results (e.g., assignment of a single T1 and T2 value to macroscopic voxels that contain a mix of multiple T1 and T2 values). Clinical and technical validation of qMRI techniques is needed, and the workshop attendees proposed and discussed appropriate best practices and next steps.

The increased emphasis on evidence-based and precision medicine requires physicians to integrate data from clinical examinations, laboratory tests, and imaging studies when deciding on patient care, and to assess and alter the plan as necessary. Such integration of data from multiple sources is becoming increasingly automated, and this requires that input data be interoperable, machine-readable and, ideally, quantitative. Decision-support tools and artificial intelligence algorithms need to perform their tasks using reproducible data. Quantitative MRI procedures must move systematically toward a standardized, reproducible, high-throughput format for clinical diagnostic implementation. Clinical validation builds upon the technical performance of the qMRI method and requires that the technical performance of the test meets the needed performance specifications; poor reproducibility may hinder clinical validation. Similarly, there cannot be a high level of confidence in conclusions drawn from clinical trials that depend on qMRI methods with poor reproducibility.

Reference objects can contribute to validation and reproducibility of qMRI, and in this report, we reviewed the impact of reference objects on qMRI technique development and multi-site studies. Looking forward, the GE-NFL multi-site study and preclinical MRI studies, as examples, clearly demonstrate the need for additional reference objects. In some cases, the NIST reference objects can be used as an initial source for the design of application-specific reference objects. A valid concern is that reference objects, even if there is a demonstrated need, will not be imaged on a regular basis if they are not easy-to-use and developed with software for automated analysis. Finally, when considering clinical use of qMRI, we need to know how often quality assurance or characterization of the systems are needed.

The clinical impact of differences in scanner software and hardware, as well as scanner performance over time, is not yet known; however, there is evidence that scanner variability can hinder the implementation of qMRI(21,93). With an appropriate reference object and a standard protocol, MRI systems can be characterized at regular intervals to develop appropriate quality assurance protocols that can mitigate the effects of scanner variability, such as has been done for specific studies(26,39,94). This recommendation is beyond the typical single-time site qualification. It can be challenging to understand the clinical utility of rapidly evolving technology coupled with instrument variability. Maximal scientific advancement is hindered in the absence of standards and reference objects.

Beyond quality assurance, to see the impact of quantitative MRI, we will need population level measurements of “normal” results. Bojorquez et al. present a compilation of the available T₁ and T₂ relaxation times at 3 T in a broad selection of tissues that to date is defined by the authors as in vivo measurements on normal subjects(95). Based on the range of relaxation times reported both within and across these studies, it is evident that more data collection in normal subjects is needed. Without such data characterizing normal results across a population, how do we know if a measure is in fact pathological? Without standardized data collection practices, including the use of reference objects, it will be hard, if not impossible, to compare data across centers, limiting the value of qMRI measurements.

In conclusion, to realize the value of qMRI for clinical diagnostics, treatment planning and outcomes assessment, the community needs to address several issues before or simultaneously with clinical validation, such as the technical verification of the MRI systems, acquisition techniques and analysis methods used to obtain them. Reference objects, both physical and computational, are a necessary component to the implementation of qMRI in a clinical setting, and a key conclusion from the workshop was that the widespread clinical adoption of qMRI requires ease-of-use, including easy reference object set-up, the smallest practical set of multi-parametric and possibly multi-modal reference objects for many techniques, and software for automatic analyses. Without addressing each of these components, the ability to implement qMRI in the clinic is likely to be limited.