Multisite Concordance of DSC-MRI Analysis for Brain Tumors: Results of a National Cancer Institute Quantitative Imaging Network Collaborative Project

DSC-MR imaging data were collected after a preload and during a bolus injection of gadolinium contrast agent using a gradient recalled-echo-EPI sequence. Forty-nine low-grade and high-grade glioma datasets were uploaded to The Cancer Imaging Archive. Datasets included a predetermined arterial input function, enhancing tumor ROIs, and ROIs necessary to create normalized relative CBV and CBF maps. Seven sites computed 20 different perfusion metrics. For normalized relative CBV and normalized CBF, 93% and 94% of entries showed good or excellent cross-site agreement. All metrics could distinguish low- from high-grade tumors.

Abstract

BACKGROUND AND PURPOSE:

Standard assessment criteria for brain tumors that only include anatomic imaging continue to be insufficient. While numerous studies have demonstrated the value of DSC-MR imaging perfusion metrics for this purpose, they have not been incorporated due to a lack of confidence in the consistency of DSC-MR imaging metrics across sites and platforms. This study addresses this limitation with a comparison of multisite/multiplatform analyses of shared DSC-MR imaging datasets of patients with brain tumors.

MATERIALS AND METHODS:

DSC-MR imaging data were collected after a preload and during a bolus injection of gadolinium contrast agent using a gradient recalled-echo–EPI sequence (TE/TR = 30/1200 ms; flip angle = 72°). Forty-nine low-grade (n = 13) and high-grade (n = 36) glioma datasets were uploaded to The Cancer Imaging Archive. Datasets included a predetermined arterial input function, enhancing tumor ROIs, and ROIs necessary to create normalized relative CBV and CBF maps. Seven sites computed 20 different perfusion metrics. Pair-wise agreement among sites was assessed with the Lin concordance correlation coefficient. Distinction of low- from high-grade tumors was evaluated with the Wilcoxon rank sum test followed by receiver operating characteristic analysis to identify the optimal thresholds based on sensitivity and specificity.

RESULTS:

For normalized relative CBV and normalized CBF, 93% and 94% of entries showed good or excellent cross-site agreement (0.8 ≤ Lin concordance correlation coefficient ≤ 1.0). All metrics could distinguish low- from high-grade tumors. Optimum thresholds were determined for pooled data (normalized relative CBV = 1.4, sensitivity/specificity = 90%:77%; normalized CBF = 1.58, sensitivity/specificity = 86%:77%).

CONCLUSIONS:

By means of DSC-MR imaging data obtained after a preload of contrast agent, substantial consistency resulted across sites for brain tumor perfusion metrics with a common threshold discoverable for distinguishing low- from high-grade tumors.

Current Response Assessment in Neuro-Oncology criteria,¹ which include anatomic imaging only, are often insufficient for determining tumor response and progression after therapy. Frequently, imaging changes on postcontrast MR imaging or FLAIR are not the result of increased tumor activity but rather a consequence of the treatment itself. These changes, which can include edema, inflammation, or increased vascular or blood-brain barrier permeability, make it difficult to distinguish true tumor progression from treatment response.^2–6 Because of these difficulties, patients are often precluded from switching to potentially more effective therapies within treatment windows of 3–5 months.¹ Clearly, better indications of tumor response that are not confounded by these treatment adverse effects are needed.

Perfusion MR imaging methods, which have repeatedly demonstrated the ability to provide biologically relevant information for treatment management, have the potential to overcome these limitations. For brain perfusion, the DSC-MR imaging methods have been most commonly used. With DSC-MR imaging, T2- or T2*-weighted images are acquired with high temporal resolution during a bolus administration of a gadolinium contrast agent.⁷ The derived relative CBV (rCBV) maps have demonstrated the ability to predict tumor grade^8,9 and survival,¹⁰ distinguish treatment effects from recurrent tumor,^11,12 and predict response to antiangiogenic therapy more reliably than standard MR imaging.^13–15

Despite this promise, the translation of DSC-MR imaging for routine clinical use has been hindered by a lack of consistency in the methods used and the rCBV values reported to make the noted distinctions. However, often a threshold determined for one purpose, such as distinguishing low- from high-grade tumor,¹⁶ is used for another purpose such as predicting outcomes.¹⁷ Consequently, the present confusion may be due to the lack of well-defined studies performed under carefully controlled conditions that test a specific outcome. This study addresses these limitations by providing carefully curated DSC-MR imaging datasets of low-grade glioma (LGG) and high-grade glioma (HGG) to multiple sites that participate in the National Cancer Institute Quantitative Imaging Network. With this approach, variations in image acquisition and preprocessing are eliminated and postprocessing methods can be directly compared in their ability to distinguish LGGs from HGGs. In addition, the threshold for this distinction can be identified both for each individual site and as a consensus recommendation.

Materials and Methods

Patients

All subjects recruited from a single site provided informed written consent according to institutional review board policy. Subjects with histologically confirmed, newly diagnosed, and treatment-naïve glial tumors who had preoperative DSC-MR imaging were included in this study. Subjects with purely oligodendroglial lesions were not included due to demonstrated differences in rCBV compared with astrocytic tumors.¹⁸ Due to the disparity in the number of subjects histologically diagnosed with low- and high-grade tumors, consecutive subjects with low-grade tumors between 2008 and 2014 and high-grade tumors from 2010 to 2014 were identified. Subjects were excluded if anatomic images were not available for lesion delineation or when DSC-MR imaging data were of poor quality.

Imaging

Studies were performed on 1.5T or 3T MR imaging systems. Standard precontrast MRIs were acquired including FLAIR (TE/TR/NEX/matrix = 126/9000 ms/2200–2500 ms/2/252 × 215) and T1WI spin-echo (TE/TR/NEX/matrix = 11 ms/650 ms/2/256), after which gadolinium contrast agent was administered. Either 0.05 mmol/kg of gadobenate dimeglumine (MultiHance; Bracco Diagnostics, Princeton, New Jersey) (n = 48) or 0.1 mmol/kg of gadodiamide (Omniscan; GE Healthcare, Piscataway, New Jersey) (n = 1) was administered before obtaining the postcontrast T1-weighted images. This initial gadolinium administration serves as a preload for the subsequent DSC-MR imaging acquisition. A preload results in more accurate rCBV because it reduces the T1 leakage effects that can confound the DSC-MR imaging data analysis.^9,19–21 After collecting the postcontrast T1WI (T1+C), using parameters equivalent to the precontrast settings, we acquired DSC-MR imaging data (gradient recalled-echo–EPI: TR = 1100–1250 ms, TE = 30 ms, flip angle = 70°–80°, 12–17 slices, slice thickness = 4–5 mm, interslice gap = 0–1.5 mm, matrix = 96 × 96/128 × 128, FOV = 165–240 × 165–240 mm²). The gradient recalled-echo–EPI data were collected for 120 seconds with 0.1 mmol/kg of gadolinium injected at 60 seconds. For ease of coregistration of the DSC-MR images to anatomic images, a spin-echo “reference” scan (TE/TR/NEX/matrix = 14 ms/500 ms/1/256 × 192) was obtained using a slice prescription matching the more limited slice prescription of the DSC-MR imaging examination.

Central Preprocessing

The preprocessing workflow is schematized in Fig 1. All preprocessing was performed in OsiriX Imaging Software (http://www.osirix-viewer.com) using the IB Delta Suite (Imaging Biometrics, Elm Grove, Wisconsin). Six datasets provided for each case included the following: 1) T1+C images; 2) DSC-MR imaging time-series co-registered to the T1+C images; 3) an arterial input function (AIF), which included 3 AIF locations in each file; 4) a whole-brain mask and ROIs of 5) normal-appearing white matter (NAWM) and 6) tumor. The NAWM was used to compute normalized rCBV (nRCBV) and normalized CBF (nCBF) maps.

Fig 1.

Preprocessing workflow. Forty-nine subjects were identified, 13 of whom had a diagnosis of low-grade glioma and 36 with a diagnosis of high-grade glioma. The DSC-MR image volume was co-registered to the T1+C images via the reference scan obtained with the same slice prescription as the DSC-MR imaging. Six datasets were provided for each case including the following: 1) T1+C images; 2) DSC-MR imaging time-series registered to the T1+C images; 3) an AIF, which included 3 AIF locations in each file; 4) a whole brain (WB) mask and ROIs of 5) normal-appearing white matter (NAWM), and 6) tumor. Each ROI was created as a 16-bit binary DICOM file that included only whole voxels rather than contoured points. Anonymized datasets were uploaded to The Cancer Imaging Archive. QIN indicates Quantitative Imaging Network.

The DSC-MR imaging volume was co-registered to the T1+C images via the reference scan obtained with the same slice prescription as the DSC-MR imaging. The AIF locations were determined semiautomatically using IB Neuro (Imaging Biometrics) with manual adjustments when necessary. An average signal generated from 3 pixels constituted the AIF. The whole-brain mask was made available to prevent disparity in values that could result from threshold variations that each software platform might use. Using the IB Delta Suite, we determined tumor ROIs from deltaT1 maps, which are standardized difference maps²² that enable clear visualization of enhancing lesions free of bright signal from blood products or proteinaceous material. Nonenhancing lesions, apparent as areas of dark signal on T1WIs, were delineated by a neuroradiologist with >20 years of experience. Each ROI was created as a 16-bit binary DICOM file that included only whole voxels rather than contoured points. This approach prevents differences in the applied ROIs because each software platform applies different rules regarding whether a voxel is considered inside or outside an ROI. Anonymized datasets were uploaded to The Cancer Imaging Archive,^23,24 where they were further vetted to ensure the compatibility of complete datasets for the analysis platform of each site. All sites were blinded to tumor grade.

Image and Statistical Analyses

Seven sites (1–7) using 7 different software platforms provided 20 different rCBV measurements (Table 1) and 12 different CBF measurements (Table 2). Details for each software platform are listed in the tables. Several sites used >1 platform or analysis method. When applicable, the analysis for measurements of standardized rCBV was grouped separately from nRCBV due to image-scale differences. Agreement between each pair of values was assessed by computing the Lin concordance correlation coefficient (LCCC). An LCCC > 0.8 indicates good agreement, and LCCC > 0.9 indicates excellent agreement.

Table 1:

Teams and rCBV analysis methods

Team/Entry No.	Software	Scaling	Leakage Correction	Description
A1	IB Neuro	NAWM	Yes	Modified BSW21
B1	IB Neuro	NAWM	Yes	Modified BSW21
C1	IB Neuro	NAWM	Yes	Modified BSW21
B2	Matlaba	NAWM	Yes	120-point TI (BSW21)
B3	Matlab	AIF	Yes	120-point TI (BSW21)
D1	nordicICE	NAWM	Yes	120-point TI (BSW21)
D2	nordicICE	Population AIF	Yes	120-point TI (BSW21)
E1	PGUI (LC)b	NAWM	Yes	120-point TI (BSW21)
E2	PGUI (C)	NAWM	Yes	AUC of C(t) ratio for ROI/AIF
E3	PGUI (PM LC)	NAWM	Yes	Bayesian parametric FT44
D3	Slicerc	NAWM	Yes	120-point TI (BSW21)
F1	FIATd	NAWM	Yes	Modified Weisskoff45,46
G1	MCcbve	NA	Yes	Modified Weisskoff45,46
A2	IB Neuro	NA (standardized)47	Yes	Modified BSW21 2006
A3	IB Neuro	NAWM	No	Leakage correction not used
B4	IB Neuro	NAWM	No	Leakage correction not used
E4	PGUI	NAWM	No	Leakage correction not used
E5	PGUI	NAWM	No	Deconvolution of residue function (sSVD)
E6	PGUI	NAWM	No	Deconvolution of residue function (oSVD)
A4	IB Neuro	N/A (standardized)47	No	Leakage correction not used

Note:—TI indicates trapezoidal integration; BSW, Boxerman-Schmainda-Weisskoff; NA, not applicable; FT, Fourier Transform; sSVD, standard singular value decomposition; oSVD, block circulant singular value decomposition; C(t) ratio, concentration time course.

^aMathWorks, Natick, Massachusetts.

^bPerfusion Graphical User Interface; MINDlab, Center of Functionally Integrative Neuroscience, Aarhus University Hospital Norrebrogade, Denmark.44

^chttp://www.slicer.org.

^dFunctional Image Analysis Tool (University of Michigan).

^eMayo Clinic cerebral blood volume (Mayo Clinic, Rochester, Minnesota).37

Table 2:

Teams and nCBF analysis methods

Team/Entry No.	Software	Scaling	Description
A5	IB Neuro	NAWM	SVD: deconvolution of ΔR2* with AIF
B5	IB Neuro	NAWM	SVD: deconvolution of ΔR2* with AIF
C2	IB Neuro	NAWM	SVD: circular deconvolution of ΔR2* with AIF
B6	Matlab	NAWM	SVD applied to leakage-corrected tissue ΔR2* and AIF ΔR2* curves
D4	nordicICE	NAWM	No AIF
D5	nordicICE	Population AIF	Population AIF
E7	PGUI	NAWM	nCBF
E8	PGUI	NAWM	sSVD
E9	PGUI	NAWM	oSVD
E10	PGUI	NAWM	Bayesian FT deconvolution44
D6	Slicer	NAWM	No AIF
F2	FIAT	NAWM	nCBF

Note:—SVD indicates singular value decomposition; sSVD, standard singular value decomposition; oSVD, block circulant singular value decomposition; FT, Fourier Transform..

The ability of each metric to distinguish LGG from HGG was determined using the Wilcoxon rank sum test, with P < .05 indicating significance. A receiver operating characteristic analysis was performed to identify the threshold that provides the optimal sensitivity (SN) and specificity (SP) to distinguish LGG from HGG. The DeLong test for comparing ≥2 receiver operating characteristic curves was used to determine whether there were differences in the ability of each measurement to classify tumors.

To determine “consensus” cutoff points, we created boxplots of the sum of SN and SP. Optimal thresholds were identified as those with maximum SN + SP mean, according to the Youden selection criteria with small variance. The random effects model was used to assess the reliability of measurements across sites and platforms. The reliability is quantified by the following:

where σ is the SD within or between software platforms.

Finally, to assess the clinical relevance of the study observations, we determined the false-positive rate from T1+C images in comparison with each of the perfusion parameters (nRCBV, standardized rCBV, nCBF). The false-positive rate is defined as the proportion of low-grade tumors thought to be aggressive and/or high-grade, as indicated by the decision for tumor resection, relative to all tumors resected. The false-negative rate was not determined because data from all patients, including those who did not undergo an operation, were not available. By means of the T1+C images, the false-positive rate was defined as the proportion of tumors that are low-grade and had contrast agent–enhancing lesions. For the perfusion parameters, a false-positive rate is defined as a low-grade tumor with a value above the threshold determined for distinguishing low- from high-grade tumors.

Results

Sixty-three subjects met inclusion criteria for this study, with 14 excluded for the following reasons: The contrast agent bolus was delayed during DSC acquisition, preventing capture of the postbolus steady-state signal (n = 4); the contrast agent bolus was injected too slowly and in an irregular pattern (n = 4); the contrast agent bolus was not present during acquisition of images (n = 2); there were severe ghosting and motion artifacts of DSC images (n = 2); or anatomic images were not available for lesion delineation (n = 2). Forty-nine co-registered LGG (n = 13) and HGG (n = 36) DSC-MR imaging datasets were preprocessed, anonymized, and uploaded to The Cancer Imaging Archive (Fig 1). Tumor grade was confirmed with histopathology a median of 3 days (range, 0–4 days) following MR imaging. Examples of postprocessed datasets are shown in Fig 2. LCCC results are displayed in a matrix listing each nRCBV (Fig 3) or nCBF (Fig 4) entry on both the x- and y-axes. For tumor nRCBV, 75% of the entries showed excellent agreement with LCCC ≥ 0.9 and 19% with good agreement (0.80 ≤ LCCC < 0.90), leaving only 6% with poor concordance (LCCC < 0.80). The concordance was best for nRCBV values determined with leakage correction. For nCBF, only 59% had 0.90 ≤ LCCC ≤ 1.0 and 34% had 0.80 ≤ LCCC ≤ 0.89.

Fig 2.

Sample parameter maps. Shown are postcontrast T1-weighted images (A and D) and corresponding rCBV (B and E) and CBF (C and F) maps for one patient with a low-grade glioma (A–C) and another patient with a high-grade glioma (D–F).

Fig 3.

Lin concordance results for nRCBV. Listed are the Lin concordance correlation coefficients for the nRCBV values for each pair of sites. Whether leakage correction was applied is indicated by green labels, with red labels indicating no leakage correction. Pairs with excellent concordance (LCCC > 0.90) are highlighted with blue-filled squares; very good concordance (0.80 < LCCC < 0.90), with green squares; and weaker concordance (LCCC < 0.80), with white squares.

Fig 4.

Lin concordance results for nCBF. Listed are the Lin concordance correlation coefficients for the normalized CBF values for each pair of sites. Pairs with excellent concordance (LCCC > 0.90) are highlighted with blue-filled squares; very good concordance (0.80 < LCCC <0.90), with green squares; and weaker concordance (LCCC < 0.80), with white squares.

For all software platforms, both nRCBV and nCBF showed statistically significant differences between LGG and HGG (Tables 3 and 4), with a mean nRCBV = 1.4 ± 0.13 and mean nCBF = 1.57 ± 0.24. The SN/SP for nRCBV ranged from 81%–97%/77%–85% and was slightly worse for nCBF with SN/SP = 64%–97%/69%–85%. By means of the DeLong test, no significant differences were found among the 18 nRCBV (P = .72) metrics to distinguish LGG from HGG. While differences among the nCBF metrics were borderline significant (P = .05), if the entry with the lowest area under the curve (0.658) was excluded, there was no significant difference between the remaining measures (P = .49). The DeLong test for the standardized rCBV showed no significant distinction between the 2 submissions for this measure (P = .23).

Table 3:

nRCBV low- vs high-grade thresholds and ROC analysis results

Team/ Entry No.	Software Platform	Leakage Correction Used	LG vs HG (P Value)	Threshold	SN	SP	AUC
A1	IB Neuro	Yes	5.34E-06	1.605	0.861	0.846	0.889
B1	IB Neuro	Yes	5.36E-06	1.605	0.861	0.846	0.889
C1	IB Neuro	Yes	5.39E-06	1.605	0.861	0.846	0.889
B2	Matlab	Yes	2.02E-06	1.315	0.972	0.769	0.892
B3	Matlab	Yes	1.75E-06	1.330	0.972	0.769	0.896
D1	nordicICE	Yes	3.30E-06	1.305	0.972	0.769	0.895
D2	nordicICE	Yes	3.30E-06	1.305	0.972	0.769	0.895
E1	PGUI (LC)	Yes	7.32E-06	1.615	0.861	0.846	0.892
E2	PGUI (C)	Yes	1.14E-05	1.400	0.861	0.769	0.871
E3	PGUI (PM LC)	Yes	1.18E-05	1.400	0.889	0.769	0.856
D3	Slicer	Yes	1.08E-06	1.415	0.944	0.769	0.891
F1	FIAT	Yes	2.50E-05	1.235	0.944	0.769	0.853
G1	McCBV	Yes	1.26E-06	1.425	0.944	0.769	0.891
A3	IB Neuro	No	1.55E-06	1.330	0.917	0.769	0.895
B4	IB Neuro	No	1.56E-06	1.330	0.917	0.769	0.895
E4	PGUI	No	6.94E-06	1.450	0.861	0.769	0.887
E5	PGUI	No	5.67E-05	1.585	0.806	0.846	0.872
E6	PGUI	No	1.19E-05	1.395	0.861	0.769	0.873

Note:—AUC indicates area under the curve; LG, low-grade; HG, high-grade.

Table 4:

nCBF low- vs high-grade thresholds and ROC analysis results

Team/Entry No.	Software	LG vs HG (P Value)	Threshold	SN	SP	AUC
A5	IB Neuro	2.90E-05	1.815	0.806	0.846	0.861
B5	IB Neuro	2.38E-05	1.555	0.917	0.769	0.864
C2	IB Neuro	3.82E-06	1.415	0.944	0.769	0.880
B6	Matlab	2.61E-04	2.045	0.722	0.846	0.814
D4	nordicICE	9.81E-06	1.255	0.972	0.769	0.868
D5	nordicICE	3.03E-05	1.580	0.861	0.846	0.866
E7	PGUI	1.30E-04	1.575	0.806	0.846	0.846
E8	PGUI	3.85E-05	1.375	0.917	0.692	0.845
E9	PGUI	1.67E-04	1.460	0.833	0.769	0.812
E10	PGUI	7.86E-03	1.920	0.639	0.692	0.658
D6	Slicer	1.04E-06	1.405	0.944	0.769	0.887
F2	FIAT	2.65E-04	1.460	0.861	0.769	0.816

Note:—AUC indicates area under the curve; LG, low-grade; HG, high-grade.

Alternatively, the data can be pooled, as shown by the boxplots of SN + SP (Fig 5), for which median and quartile values are indicated. The maximum sums were the following: nRCBV = 1.4 (SN/SP = 90%/77%) and nCBF = 1.58 (SN/SP = 86%/77%). For these consensus thresholds, the minimum individual SN/SP was 83%/77% for nRCBV and 80%/70% for nCBF.

Fig 5.

Boxplots showing the sum of sensitivity and specificity values for nRCBV (A) and nCBF (B) for all sites and metrics. From these data, values that gave a maximum sum of specificity plus sensitivity and minimal variance were chosen as the consensus threshold values.

For the 18 nRCBV measurements, the reliability was determined to be 0.93, indicating that 93% of the variation can be attributed to differences in tumors, and 7%, to differences in analysis methods. The reliability was 95% for nRCBV determined with leakage correction and 93% for the group without leakage correction. For the nRCBV computed with one of the most common leakage-correction algorithms (Boxerman-Schmainda-Weisskoff²¹), the reliability improved to 98%. The reliability of standardized rCBV was 96%. For the 12 nCBF measurements, the reliability was 61%.

Finally, the false-positive rate using T1+C was found to be 31%. In comparison, the false-positive rate across all software platforms was 15%–23% for nRCBV, 15%–31% for nCBF, and 8%–15% for standardized rCBV.

Discussion

By means of carefully curated DSC-MR imaging datasets, obtained with a single acquisition approach, all nRCBV and nCBF metrics, processed by 7 different sites, could distinguish LGG from HGG. The optimal nRCBV and nCBF thresholds varied by only 9% and 15%, respectively. Unique to this study, consensus thresholds of nRCBV = 1.4 and nCBF = 1.58 were determined, indicating good accuracy overall and for each individual site. These results should bolster confidence in the ability of DSC-MR imaging to provide reliable and consistent cross-platform perfusion metrics for the evaluation of brain tumors and, specifically, for distinguishing low- from high-grade gliomas.

The range of nRCBV threshold values determined in this study is much tighter than the 0.7–3.0 range previously reported for distinguishing tumor grade,^25,26 predicting differences in survival,^{10,17,27–29} and distinguishing true progression from pseudoprogression³⁰ and tumor from treatment effect.^11,12,31 While this large range of threshold values has been attributed to different acquisition and postprocessing schemes,²⁰ differences in patient populations and the clinical questions addressed also contribute to the variabilities. While it is unlikely that a single threshold can be universally applied for all clinical questions, these studies suggest that with well-defined studies to address a specific outcome under carefully controlled conditions, it is possible to reach consensus.

The present study also demonstrates a greater cross-platform concordance than that previously reported. For example, in one study,³² 2 commercial software packages (nordicICE; NordicNeuroLab, Bergen, Norway and BrainSTAT; GE Healthcare) were compared. Like the present study, 1 dataset of 24 patients with de novo glioblastoma was used and ROIs of tumor and reference brain were predetermined. However, unlike the present study, vastly different mathematic algorithms were applied, resulting in very disparate definitions for nRCBV and CBF; thus, a wide range of values was reported. In the present study, most algorithms involved the integration of the concentration-time course and the application of Boxerman-Schmainda-Weisskoff leakage correction,²¹ which, in a subanalysis, also showed better reliability. In the previous study, 5 of 10 algorithms relied on the determination of the AIF.³² Using AIF to compute nRCBV resulted in coefficients of variation of 15%, but only 2% when AIF was not used. The challenges of reliably determining the AIF are well-known and may largely explain the poor repeatability.^33,34 Most software platforms in this study did not incorporate AIF for nRCBV calculation and may therefore also explain the excellent concordance across sites. Yet, the computation of CBF requires the determination of AIF and is likely a primary reason for the greater variance in comparison with nRCBV (Figs 3 and 4). Also, the individual nCBF thresholds calculated using IB Neuro varied across sites because some sites chose to use circular deconvolution of the AIF for processing while others did not.

Five of 10 analysis methods in the previous software comparison study often used γ-variate fitting.³² Several studies reported a lower SNR³⁵ as well as greater inaccuracy when γ-variate fitting was used for brain tumor DSC-MR imaging data, especially in the presence of contrast agent leakage.^19,20 Although γ-variate fitting suppresses the postbolus baseline, making it appear that leakage has been corrected, there is no physiologic basis for this correction and it does not appropriately consider leakage that can occur during the bolus.³⁶ Gamma-variate fitting was not used by any of the software platforms evaluated in the present study.

In another study,³⁷ nRCBV values were generated from 3 FDA-approved software packages including IB Neuro 1.1, FuncTool software 4.5.3 (GE Healthcare) and nordicICE 2.3.13 and 1 in-house software platform. While effort was made to use the tools in a similar way, more user interaction was required of some (FuncTool, nordicICE), and FuncTool did not have the option for leakage correction. The largest differences between the in-house and commercial software occurred with the tool that required the most user interaction (nordicICE), further motivating the development of more automated workflows with less need for user interaction. Yet another study comparing these same 3 packages also found significant differences, with the outlying package depending heavily on the type of rCBV metric used.³⁸ This finding again suggests that it is imperative that the same output metric be used when making such comparisons.

Of relevance to the current study, rCBV maps generated with IB Neuro showed superior leakage correction and stronger correlation with image-guided microvessel quantification as well as higher accuracy in distinguishing tumor recurrence from pseudoprogression/radiation necrosis compared with other software platforms.³⁹ These results are relevant, given the number of sites in the present study that chose to use IB Neuro for their processing.

A limitation of the current study is the use of a DSC-MR imaging dataset that was obtained at a single center using a single approach. Use of a range of acquisition methods would likely result in greater variation in the DSC-MR imaging perfusion results. A previous study confirmed this by comparing a range of acquisition and analysis methods, which also influenced the ability to distinguish high-grade tumor from reference brain.²⁰ However, a consensus regarding best practices for DSC-MR imaging data acquisition is being reached, as described in a recent review,⁴⁰ and includes the approach used for this study. Specifically, use of a preload of contrast agent and a flip angle <90° is proving to be one of the most accurate approaches, further confirmed by 2 recent studies,^19,41 both incorporating sophisticated simulations of DSC-MR imaging data representative of brain tumor. Use of a preload might also be an important reason for greater consistency across postprocessing methods in this study compared with previous studies (eg, Orsingher et al³²). Collecting DSC-MR imaging data after preload was shown to decrease the dependence of tumor rCBV on the chosen method of analysis.²⁰

An additional limitation of this study is the use of laboratory or proprietary commercial packages for which many of the details of the algorithmic implementation are not available and thus cannot be further evaluated as potential sources of differences. Also, the software platforms used for this study were dictated entirely by platforms being used at each participating site. Consequently, this is not a comprehensive comparison of all available software platforms with DSC-MR imaging postprocessing capabilities.

The general application of the results of this study is somewhat limited because the preprocessing steps were carefully controlled so that consistent input data were provided to all sites and software platforms. In practice, subjective manipulation of the preprocessing steps is common; therefore, consistency is less likely, as the discussion of the previous studies reveals. Yet the identification of preprocessing as a key confound should not inhibit use of DSC-MR imaging but rather motivate improving automation of the preprocessing steps. In fact, several efforts to automate tumor segmentation are well underway,^42,43 which remove this source of discrepancy entirely.

Conclusions

This study demonstrates that nRCBV and nCBF can be used to distinguish LGG from HGG in a consistent fashion and using a single consensus threshold. This result should increase confidence in using nRCBV primarily, but also nCBF, on a routine basis, potentially motivating its incorporation into the updated Response Assessment in Neuro-Oncology criteria. Finally, these results provide strong motivation for the development of more automated preprocessing workflows that are less dependent on subjective user interaction.

ABBREVIATIONS:

AIF

arterial input function

HGG

high-grade glioma

LCCC

Lin concordance correlation coefficient

LGG

low-grade glioma

NAWM

normal-appearing white matter

nCBF

normalized cerebral blood flow

nRCBV

normalized relative cerebral blood volume

rCBV

relative cerebral blood volume

SN

sensitivity (in the context of receiver operating characteristic analysis)

SP

specificity (in the context of receiver operating characteristic analysis)

T1+C

postcontrast T1WI