Assessment of Quantitative Methods for Enhancement Measurement on Vessel Wall Magnetic Resonance Imaging Evaluation of Intracranial Atherosclerosis

Abstract

INTRODUCTION:

Quantitative measures of vessel wall magnetic resonance imaging (vwMRI) for the evaluation of intracranial atherosclerotic disease (ICAD) offers standardization not available with previously used qualitative approaches that may be difficult to replicate.

METHODS:

vwMRI studies performed to evaluate ICAD that had caused a stroke were analyzed. Two blinded reviewers qualitatively rated culprit lesions for the presence of enhancement on T1 DANTE SPACE images. At least 3 months later, quantitative analysis was performed of the same images, comparing lesion enhancement to reference structures. Cohen’s kappa and intraclass correlation coefficients were calculated to assess agreement. Ratios of enhancement of lesions to references were compared to qualitative ratings.

RESULTS:

Studies from 54 patients met inclusion criteria. A mean of 49 (90.7%) lesions were qualitatively rated as enhancing, with good inter-rater agreement (κ=0.783). Among reference structure candidates, low infundibulum demonstrated the highest inter-rater agreement on pre- and post-contrast imaging. The ratio of percentage increase in plaque signal following contrast to the same measure in low infundibulum demonstrated the highest agreement with qualitative assessment, with highest agreement seen with a ratio of 0.8 set as a threshold (κ=0.675).

CONCLUSION:

Quantitative metrics can yield objective data to better standardize techniques and acceptance of vwMRI evaluation of ICAD. The low infundibulum had the highest inter-rater agreement on both pre- and post-contrast images and is best suited as a normally enhancing reference structure. Such quantitative techniques should be implemented in future research of vwMRI for the evaluation of ICAD.

INTRODUCTION

Vessel wall magnetic resonance imaging (vwMRI) techniques are earning growing acceptance for evaluation of intracranial atherosclerotic disease (ICAD), expanding understanding by allowing visualization of vessel walls themselves utilizing black blood techniques such as delay alternating with nutation for tailored excitation (DANTE).[1–6] Current variability among acquisition and interpretation techniques hinders wider applicability and acceptance.[7–15] To better validate and standardize VWI techniques, particularly with respect to ICAD analysis, quantitative assessment can help define what constitutes enhancement. This study assesses quantitative techniques for the interpretation of vwMRI studies of ICAD, which incorporate several references structures also evaluated in this study, comparing quantitative metrics against qualitative ratings more commonly employed in prior vwMRI studies. In order to better standardize techniques as VWI studies become more widespread in clinical use and undergo further investigation, such quantitative metrics should be utilized when possible.

vwMRI techniques utilize flow suppression techniques to detect vessel wall pathology. This has led to novel evaluation of a host of diseases throughout the body.[3] Specifically, vwMRI has emerged as an important tool for evaluation of ICAD.[16,2,3,1] When analyzing ICAD with vwMRI, multiple lesion traits can be assessed quantitatively, including plaque enhancement, stenosis, presence of intraplaque hemorrhage, and pattern of remodeling. Plaque enhancement is an important trait that is distinctly prone to variability in interpretation that introduce error.[7–15,17,18,3,19–23] To mitigate such variability, quantitative techniques are needed to allow standardization that can improve repeatability and applicability of findings. This study describes and evaluates such an approach to ICAD plaque enhancement that we hypothesize agrees well with currently utilized qualitative VWI interpretation techniques.

To properly quantify the degree of ICAD plaque enhancement on vwMRI, reliable reference structures that normally enhance can aid assessment.[24] Among previous investigations of vessel wall enhancement on vwMRI, some describe no methodology for comparison against reference control structures.[7,8,13] Other investigations have utilized non-enhancing tissues subject to the blood-brain barrier, most commonly gray matter, less commonly ventricles.[15,11,14] Others have compared normal vessel segments.[9] To address these previous limitations, this study also investigates normal vessels and multiple different structures that normally enhance following contrast administration to evaluate them as reference structures by virtue of their enhancement characteristics and inter-rater agreement.[25] Such information can help validate references so that future research can better standardize interpretation techniques, lending to more widely applicable results. Finally, to provide context for the methods examined in this study against the existing vwMRI literature, the various quantitative measures are compared against a previously used binary qualitative analysis. Many of the quantitative or semi-quantitative techniques reported in prior research are robust and effective. Indeed, techniques employed as described below correspond with those reported in some of this prior work. However, heterogeneity limits progress in this field, and quantitative techniques can increase consistency and move this field of study forward.

METHODS

According to an IRB-approved protocol, prospectively maintained records were queried to perform retrospective analysis of patients at a major academic medical center undergoing vwMRI during evaluation of new ischemic stroke. We included patients who had vwMRI within 14 days of their index stroke and who were determined to have ICAD-related stroke. An experienced vascular neurologist (AD) adjudicated the stroke mechanism. Patients with the following were excluded: bilateral stroke or stroke in multiple vascular distributions, >50% extracranial stenosis of an artery proximal to the stroke parent artery (e.g. right carotid stenosis >50% in a patient with a right MCA stroke), stroke DWI lesion diameter <1.5mm or lack of multiple infarctions if <1.5mm, prior diagnosis of atrial fibrillation or atrial fibrillation on standard-of-care 30 day mobile cardiac telemetry, hypercoaguable disorders, chronic anticoagulation, and uncommon causes of stroke (eg. vasculitis, CADASIL). Patients with incomplete or nondiagnostic vwMRI studies were excluded.

All images were acquired on Siemens (Erlangen, Germany) 3T MRI scanners (Prisma, Trio, or Verio) with dedicated head coils. Protocol details are provided in Table 1. Two neuroradiologists were blinded to diffusion and clinical data and instructed to analyze the arterial tree to which the new infarct was referable. The reviewing radiologists were instructed which parent arteries to assess after these were determined by a treating neurologist based on available clinical data and MRI findings. Reviewing images in the axial plane in which they were obtained, each reviewer identified the most likely culprit lesion in the stroke parent artery and qualitatively rated it as enhancing or not. Each culprit lesion, defined as the single likely source of downstream infarction, was determined by each reviewer and confirmed to be the same lesion for each patient. While consensus was established for which culprit lesion to assess, scores were blinded between reviewers. Assessment was repeated for the culprit plaques by the same reviewers at least three months later for quantitative analysis. Patients for whom multiple lesions could potentially be considered the culprit, i.e. multiple lesions upstream to an infarction, were excluded from analysis to reduce potential bias. Maximum voxel signal intensity values for the same plaques were noted on 3D T1-weighted SPACE with DANTE flow suppression pre- and post-contrast.[6] Figure 1 demonstrates a sequence diagram for these acquisitions. For each variable measured, the mean of three data points was computed after obtaining the maximum single-voxel intensity values at each site of interest. The same normal reference vessel was also evaluated by each reviewer. If a contralateral analogue with normal appearance was present, this vessel was evaluated. In cases involving contralateral disease or the basilar artery, the nearest normal arterial wall segment was interrogated.

Table 1:

T1 DANTE Imaging Parameter

Echo time (ms)	21
Repetition time (ms)	800
Flip Angle (degrees)	10
Field of view (cm)	18x18
Acquisition Matrix	384x384
Slice thickness (mm, after interpolation)	0.5
In-plane resolution (mm)	0.46 x 0.46
DANTE preparation time (ms)	150
Number of averages	1.4
TSE factor	62
Acquisition time per slice (s)	1.85
# of slices (after interpolation)	128
Bandwidth	789
Total Acquisition Time (min)	6:30

Figure 1.

Sequence diagram for 3D DANTE T1W SPACE sequence, which consists of 150 trains of DANTE module followed by 3D T1W SPACE readouts.

Structures expected to normally enhance were also measured, including the mid infundibulum, defined as the midway point between the inferior margin of the hypothalamus and the upper margin of the pituitary gland; low infundibulum, defined as the lowest segment of the infundibulum distinguishable from the pituitary gland; choroid plexus in a lateral ventricle; cavernous sinus, measured at an intravascular site seen on post-contrast images; and muscle, measured in the temporalis muscle with signal medial to the mid-belly fibrous band of the muscle.[25] Sample regions of interest for each of these structures, as well as the intensity measured at each site, are presented in Figure 2.

Figure 2.

Representative images of structures analyzed and intensities recorded before and after administration of contrast: (A, B) symptomatic left MCA ICAD plaque, (C, D) normal right MCA wall segment, (E, F) low infundibulum, (G, H) mid infundibulum, (I, J) choroid plexus, (K, L) cavernous sinus, (M, N) temporalis muscle.

Descriptive statistics were obtained to summarize patient and lesion characteristics. Cohen’s kappa and intraclass correlation coefficients (ICC) were calculated for binary and continuous variables, respectively, to assess agreement between interrogated methods, including inter-rater agreement for the initially obtained qualitative assessments, subsequently obtained quantitative methods, and comparison between quantitative and qualitative methods. Additionally, degree of stenosis measured on post-contrast 3D T1-weighted SPACE with DANTE images was calculated according the WASID technique using the following formula, [1 - (diameter at stenosis / diameter normal segment)] x 100.[26,27] Stenosis was measured at the site of the most severe stenosis, and normal segment was measured at the nearest proximal normal artery; if the proximal artery was diseased, a distal normal segment was chosen.[26,27]

Multiple methods were employed to quantitatively assess enhancement. For all interrogated structures, percentage increase in enhancement was measured according to the following formula, (post-contrast mean – pre-contrast mean)/pre-contrast mean. The ratios of plaque signal change to signal change in the various reference structures assessed were calculated. Optimal thresholds were determined by calculating all possible ratios from 0.3–1.5 for each combination interrogated. Statistics were performed using Stata version 15 (College Station, TX).

RESULTS

Studies from 54 patients met inclusion criteria. Mean time to vwMRI after diagnosis of infarct was 7 days. Patient demographics and lesion characteristics are summarized in Table 2. On qualitative analysis, a mean of 49 (90.7%) lesions were scored as demonstrating enhancement, with good inter-rater agreement (κ=0.783). Mean stenosis was 53.8% (±23.2%). 45 (83.3%) lesions had >25% stenosis, 32 (59.3%) had >50% stenosis, and 9 (16.7%) had >75% stenosis. 2 (3.7%) vessels were occluded, and 13 (24.1%) had stenosis ≥70%. ICC to measure inter-rater agreement for stenosis was (ICC=0.620).

Table 2:

Demographic and Lesion Characteristics

Mean Age (Years)		62.8 (±16.2, IQR 55.0–74.7)
Race
	Caucasian	42 (77.8%)
	Latino	5 (9.3%)
	Native American	4 (7.4%)
	Pacific Islander	3 (5.6%)
Male Gender		33 (61.1%)
Stroke Parent Artery
	Internal Carotid	6 (11.1%)
	Middle Cerebral	18 (33.3%)
	Anterior Cerebral	2 (3.7%)
	Vertebral	14 (25.9%)
	Basilar	10 (18.5%)
	Posterior Cerebral	4 (7.4%)
Tandem Lesions		32 (59.3%)
Branch Point Lesions		12 (22.2%)
Mean Stenosis		53.8% (±23.2%)
Qualitative Enhancement		49 (90.7%)

Results summarizing evaluation of reference structures (signal measurement characteristics and ICC as a measure of inter-rater agreement) are summarized in Table 3. Normal vessel segments demonstrated no substantial enhancement, with mean post-contrast values 112% of pre-contrast values. ICAD plaques demonstrated mean signal increase to 225% of pre-contrast values. Among normally enhancing structures, change in the cavernous sinus was most robust. Muscle demonstrated the lowest degree of enhancement. When comparing inter-rater correlation among the interrogated reference structures, low infundibulum demonstrated the highest values on both pre- and post-contrast imaging. Muscle resulted in high agreement on pre-contrast images but poor agreement post-contrast. Normal vessel demonstrated poor agreement on pre-contrast measurements but good agreement on post-contrast images.

Table 3:

Plaque and Reference Signal Measurement

	Pre-Cortrast Signal Measurements				Post-Contrast Signal Measurements
Structure	Mean	Std Dev	Std Error	ICC	Mear	Std Dev	Std Error	ICC
Plaque (n=54)	162.5	42.0	17.1	0.451	366.4	111.0	45.3	0.920
Normal Vessel (n=54)	112.5	27.9	11.4	0.463	126.2	33.9	13.8	0.819
Mid Infundibulum (n=49)	159.9	32.0	13.1	0.491	266.1	65.4	26.7	0.478
Low Infundibulum (n=44)	160.3	38.1	15.6	0.798	275.9	71.4	29.1	0.782
Choroid Plexus (n=54)	157.4	38.9	15.9	0.477	280.2	79.9	32.6	0.574
Cavernous Sinus (n=52)	138.4	38.2	15.6	0.492	371.9	95.1	38.8	0.370
Muscle (n=54)	99.8	45.3	18.5	0.760	134.5	29.2	11.9	0.275

Considering MRI intensity recorded by raters, agreement for mean pre-contrast plaque signal was moderate (ICC=0.451). Mean post-contrast signal for plaques demonstrated excellent inter-rater agreement (ICC=0.920). When assessing the mean difference between post- and pre-contrast signal, excellent agreement was again observed (ICC=0.820). Assessing plaque enhancement irrespective of reference structures, the highest degree of agreement with binary qualitative assessment was found with a threshold of 80% increase in signal, yielding moderate agreement (κ=0.466). Among quantitative assessment using a normally-enhancing reference structure, the ratio of plaque enhancement to low infundibulum enhancement demonstrated the highest agreement with qualitative assessment. A ratio of 0.8 for this measure resulted in κ=0.675. The next highest value was noted for a ratio of 0.3 of plaque enhancement to muscle enhancement (κ=0.547). Cohen’s kappa values were lower for agreement between qualitative assessment and quantitative analysis utilizing the remaining reference structures. Representative agreement values are summarized in Table 4.

Table 4:

Cohen’s Kappa Values for Various Plaque and Reference Enhancement Ratios

	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1.0	1.1	1.2	1.3	1.4	1.5
Plaque Intrinsic	0.151	0.291	0.438	0.438	0.438	0.466	0.328	0.255	0.256	0.209	0.148	0.086	0.065
Normal Vessel	0.312	0.312	0.312	0.312	0.312	0.312	0.312	0.312	0.312	0.312	0.245	0.245	0.193
Low Infundibulum	0.338	0.493	0.493	0.493	0.493	0.675	0.569	0.450	0.388	0.239	0.195	0.170	0.159
Mid Infundibulum	0.117	0.350	0.350	0.350	0.314	0.282	0.282	0.328	0.301	0.337	0.274	0.224	0.182
Choroid	0.245	0.193	0.338	0.438	0.391	0.466	0.425	0.357	0.277	0.235	0.293	0.293	0.224
Cavernous Sinus	0.438	0.389	0.328	0.168	0.080	0.035	0.023	0.053	0.047	0.047	0.031	0.031	0.026
Muscle	0.547	0.462	0.394	0.394	0.493	0.493	0.493	0.438	0.512	0.425	0.425	0.357	0.357

DISCUSSION

Imaging options and understanding of ICAD have expanded with the advent of vwMRI techniques.[16,2,3,1] To better facilitate widespread clinical acceptance, standardization of techniques and interpretation are needed. Lesion enhancement in particular is subject to heterogeneity among evaluators.[7–15] The current study was undertaken to compare qualitative rating of enhancement against quantitative techniques in an attempt to better bridge current VWI evaluation with more objective methods.

Among features assessed on vwMRI, plaque enhancement is particularly problematic with respect to variability among institutions using vwMRI and the raters interpreting these studies. Among previously published investigations of enhancement characteristics, a wide variety of techniques have been reported. Some use strictly qualitative methods, while others employ semi-quantitative techniques, and others offer no description of the methods for determining enhancement.[7–15] Additionally, some techniques assess enhancement against the vessel itself, while others assess enhancement with respect to non-vascular structures, including those that normally enhance and others that do not.[7–15] An additional source of heterogeneity lies in the variable characterization of vessel wall enhancement. Certain investigations characterize enhancement in a binary fashion while others qualitatively or semi-quantitatively assign degree of enhancement according to an ordinal scale.[7–15] This study does not seek to assess superiority among the previously reported interpretation methods. Rather, to better harmonize understanding of ICAD enhancement on future vwMRI studies, this study seeks to compare the qualitative binary approach previously used by our group against quantitative metrics, some of which are similar to previously reported methods.

Assessment of plaque enhancement can be augmented by comparison against normal structures, a practice that has gained acceptance in evaluation of the extracranial carotids with vwMRI.[28,29] Most brain structures do not enhance following administration of gadolinium due to exclusion of the contrast material by the blood-brain barrier.[25] The arterial vessel walls are among such normally non-enhancing structures; although normal intracranial arteries lack vasa vasorum, minimal enhancement sometimes visualized may reflect some baseline intimal permeability.[30] A small number of intracranial structures are not subject to contrast exclusion by the blood-brain barrier and enhance normally.[25] These structures can be taken advantage of to assess the degree of normal enhancement above background.

Previous investigations of vessel wall enhancement on vwMRI have mostly used qualitative comparison between pre- and post-contrast of the area of interest or comparison against various structures.[7–15] Many studies describe no methodology for comparison against reference control structures.[7,8,13] Various investigations have utilized non-enhancing tissues subject to the blood-brain barrier like gray matter, ventricles, or normal vessel segments.[9,15,11,14] In the current study of multiple enhancing structures, muscle showed the weakest enhancement. Muscle has been utilized in prior studies, at times using lesion enhancement greater than 150% of pre-contrast muscle signal as the threshold for determining enhancement.[31,13] In this study, quantitative assessment using the temporalis muscle as a reference yielded the second highest agreement with the binary qualitative assessment. Others have compared enhancement against the pituitary gland or its infundibulum, several accounting for its robust enhancement by using these structures as thresholds for “strong” enhancement.[9,10,12] This is consistent with greater enhancement seen in the infundibulum compared to muscle in the current study.

In addition to variability in methods among previously reported investigative groups, heterogeneity can also exist within studies between raters. In this study, inter-rater variability was measured to assess different reference structures. Measurements of the low infundibulum had the highest inter-rater agreement on both pre- and post-contrast images. Muscle demonstrated high agreement pre-contrast measurements but poor agreement post-contrast measurements, while normal vessel segments demonstrated poor agreement on pre-contrast measurements and high agreement on post-contrast measurements. Mid infundibulum, choroid plexus, and cavernous sinus measurements had inferior agreeability on both pre- and post-contrast imaging.

The previous method of enhancement evaluation employed by our group, qualitatively assessing a binary rating of presence or absence of enhancement, including subjective comparison against the infundibulum, demonstrated good inter-rater agreement with κ=0.783. However, agreement within our group does not necessarily lend to agreement with raters trained differently or with potentially conflicting interpretative tendencies. As such, we sought to compare our qualitative approach against several quantitative metrics. The quantitative measure showing the most agreement with this qualitative binary assessment was a ratio of lesion enhancement to enhancement of the low infundibulum. A threshold of 0.8 for a ratio of these two measures resulted in κ=0.675. This demonstrated superior agreement with the qualitative binary method compared to agreement between the qualitative method and enhancement without comparison to a reference structure. Additional metrics demonstrated much lower values for Cohen’s kappa.

To our knowledge, the current study is the first to assess quantitative evaluation of enhancement against a previously utilized qualitative metric. The methods demonstrating good agreement, specifically a ratio of 0.8 of plaque enhancement to low infundibulum enhancement, are easily performed and reproducible. If low infundibulum is excluded from view, comparison against the temporalis muscle can be reliably used as well, with a ratio of 0.3 optimum for agreement with the qualitative technique. We advocate for adoption of such techniques in future vwMRI studies so that more uniform evaluation can be conducted to promote consistency in the field. vwMRI studies of ICAD should utilize such quantitative metrics to reliably assess this disease in a way to reproducibly identify which lesions are likely to cause strokes. Better metrics, such as those described above, can in turn lead to more refined treatment algorithms to improve patient outcomes.

This study contains several limitations that warrant discussion. While it elucidates how certain quantitative techniques relate to previously used qualitative assessment, further investigation is needed to understand exactly what enhancement represents physiologically so that it can better inform clinical understanding. Retrospective studies such as this can introduce bias despite the prospective acquisition of their data. Additionally, selection bias may exist in the examined cohort, which only includes patients with confirmed culprit ICAD. Further evaluation is needed to assess ICAD lesions against normal vessels, as well as comparison between symptomatic and asymptomatic lesions, to assess what features cause certain lesions to result in strokes while other plaques remain quiescent.

Additionally, there are current limitations of vwMRI studies in general as currently constituted. Enhancement depends on T1 values and on the T1-weighting of a given sequence, which in turn can be affected by the flip angle and the refocusing angle. While such factors that can affect image readout were controlled for in the present study by utilizing the same protocol on all patients, differences among protocols across different centers can lead to heterogeneity. Such considerations related to this study’s limitations and current factors limiting vwMRI writ large must be incorporated into future research, ideally incorporating quantitative metrics such as those described above to better achieve reproducible results.

CONCLUSION

Quantitative techniques are needed to better standardize investigation of arterial enhancement with vwMRI for the evaluation of ICAD. As part of such an approach, plaque enhancement can be compared against background structures. In this study, measurements of the low infundibulum had the highest inter-rater agreement on both pre- and post-contrast images. Incorporating this standard, the quantitative approach described in these methods demonstrates high inter-rater agreement that corresponds well to previously utilized qualitative binary assessment of enhancmeent. Such reproducible quantitative techniques warrant further investigation and should be implemented in future research of vwMRI for the evaluation of ICAD.

Figure 3.

Graphical representation of Cohen’s Kappa values for ratios of signal measurements of plaque over various reference structures.