In vivo semi-automatic segmentation of multicontrast cardiovascular magnetic resonance for prospective cohort studies on plaque tissue composition: Initial experience

Abstract

Purpose

Automatic in vivo segmentation of multicontrast (multisequence) carotid magnetic resonance for plaque composition has been proposed as a substitute for manual review to save time and reduce inter-reader variability in large-scale or multicenter studies. Using serial images from a prospective longitudinal study, we sought to compare a semi-automatic approach versus expert human reading in analyzing carotid atherosclerosis progression.

Methods

Baseline and 6-month follow-up multicontrast carotid images from 59 asymptomatic subjects with 16–79% carotid stenosis were reviewed by both trained radiologists with 2–4 years of specialized experience in carotid plaque characterization with MRI and a previously reported automatic atherosclerotic plaque segmentation algorithm, referred to as Morphology-Enhanced Probabilistic Plaque Segmentation (MEPPS). Agreement on measurements from individual time points, as well as on compositional changes, was assessed using the intraclass correlation coefficient (ICC).

Results

There was good agreement between manual and MEPPS reviews on individual time points for calcification (CA) (area: ICC; 0.85–0.91; volume: ICC; 0.92–0.95) and lipid-rich necrotic core (LRNC) (area: ICC; 0.78–0.82; volume: ICC; 0.84–0.86). For compositional changes, agreement was good for CA volume change (ICC; 0.78) and moderate for LRNC volume change (ICC; 0.49). Factors associated with LRNC progression as detected by MEPPS review included intraplaque hemorrhage (positive association) and reduction in low-density lipoprotein cholesterol (negative association), which were consistent with previous findings from manual review.

Conclusions

Automatic classifier for plaque composition produced results similar to expert manual review in a prospective serial MRI study of carotid atherosclerosis progression. Such automatic classification tools may be beneficial in large-scale multicenter studies by reducing image analysis time and avoiding bias between human reviewers.

Introduction

Histopathological studies indicate that the large lipid rich necrotic core (LRNC) is one of the most important features characterizing the vulnerable plaque [1, 2]. Imaging LRNC and monitoring its progression or regression therefore may provide important in vivo evidence for optimizing clinical management. As a noninvasive tool, the utility of multicontrast (multisequence) magnetic resonance imaging (MRI) for in vivo assessment of plaque composition such as LRNC has been established in carotid arteries using histology as the gold standard [3–6]. Consequently, efforts introducing carotid MRI to prospective cohort studies have provided in vivo evidence on the effects of risk factors and atheroprotective medications on atherosclerotic plaques, in particular intraplaque hemorrhage (IPH), hypercholesterolemia, and lipid-lowering agents [7–11].

In previous cohort studies, human readers manually segmented plaque components based on two or more contrast weightings that were carefully registered [3–11]. Manual segmentation provides a means to derive various quantitative measurements on plaque composition. But it can be time-consuming, particularly in advanced lesions. Furthermore, different levels of reader experience in vascular pathology and imaging lead to inter-observer variability, which is of concern in large-scale studies involving multiple human readers. By dissecting and following the cognitive process of human reading, automatic segmentation algorithms have been proposed to address the problems associated with manual segmentation [12–16]. In their initial validation in cross-sectional studies, good agreement with manual or histological measurements has been observed on common plaque components including the LRNC. However, application of automatic segmentation in prospective or serial clinical studies is scarce. To what extent can automatic segmentation be used to monitor plaque progression or regression remains unclear.

By use of images from a prospective cohort study with baseline and follow-up carotid MRI, we sought to compare a semi-automatic approach with expert human reading (the current standard) in analyzing plaque tissue composition.

Materials and Methods

Study subjects

Subjects were from a previous longitudinal study (ClinicalTrials.gov Identifier ID: NCT00851500) in which carotid MRI was performed in asymptomatic carotid stenosis to examine changes in LRNC over a 6-month period [7]. The inclusion criteria included absence of cardiovascular events for at least 12 months, 16–79% carotid stenosis on screening duplex ultrasonography, and detectable LRNC on screening carotid MRI. If a subject was eligible, the screening MRI would also be used as baseline to compare with a follow-up scan done at 6-month. Other details regarding subject recruitment and clinical management have been reported previously [7]. Subjects provided informed consent, and study procedures were approved by the institutional review board.

Carotid MRI Protocol

Subjects were scanned using 3T whole body MRI scanners (3T Philips Achieva and 3T GE Signa) and dedicated carotid phased-array surface coils [17]. A standardized multicontrast MRI protocol was used for plaque characterization, including three-dimensional time-of-flight (TOF), T1-weighted (T1w), T2-weighted (T2w) imaging. Contrast–enhanced T1-weighted (CE-T1w) images were also acquired, approximately 5 minutes after injection of gadopentetate dimeglumine (Magnevist; Bayer Healthcare, Wayne, NJ). All images were obtained with a field-of-view of 16 × 16 cm, matrix size of 256 × 256, and slice thickness of 2 mm with no inter-slice gap (acquired spatial resolution 0.625 × 0.625 × 2 mm). Imaging longitudinal coverage was 32 mm. Total image acquisition time was 30–40 minutes.

Image analysis

1. Manual review

Manual review was performed by two trained radiologists with 2–4 years of specialized experience in carotid plaque characterization with MRI using custom-designed image analysis software (CASCADE; University of Washington, Seattle, WA) [17, 18]. Reviewers read the baseline and follow-up images side-by-side but were blinded to the temporal order of the scans and to all clinical variables. Lumen and outer wall boundaries were outlined on axial T1w images with which the remaining contrast weightings were carefully aligned to facilitate the use of multicontrast information for identifying plaque components [18]. Based on published criteria that were previously validated by multiple groups [3–6], two common plaque components were identified and outlined (Fig. 1): 1) calcification (CA): areas that appear hypointense on all contrast weightings; 2) LRNC: non- or less-enhanced areas on CE-T1w images compared to surrounding fibrous tissue.

Fig. 1.

Segmentation results from manual and MEPPS reviews

The top panel shows serial multicontrast images of a plaque in the internal carotid artery with lumen (red) and outer wall contours (light blue). The middle and lower panel show segmentation results from manual and MEPPS reviews. Light green indicates CA and yellow indicates LRNC. CA calcification, CE-T1w contrast-enhanced T1-weighted, LRNC lipid-rich necrotic core, T1w T1-weighted, TOF time-of-flight

2. Automatic review

Automatic classifiers that are currently applicable to in vivo images are training-based Bayesian probability models [12, 16]. We used a previously published algorithm known as Morphology-Enhanced Probabilistic Plaque Segmentation (MEPPS) which automatically detects and delineates plaque components based on morphological and MR imaging signal characteristics [12]. The central function of MEPPS is to estimate the probability of a given pixel being tissue type T_i given the set of observed intensities x in the multiple contrast-weighted images and morphological information in the form of the local wall thickness t and distance from the lumen d. In MEPPS, this probability is modeled as the Bayesian network:

$$Pr(T_i\mid t,d,x)=\frac{p(t,d\mid T_i)p(x\mid T_i)Pr(T_i)}{{\sum }_{j=1}^{4}p(t,d\mid T_j)p(x\mid T_j)Pr(T_j)}$$

where p(x|T_i) and p(t,d|T_i) are two conditionally independent probability density functions and Pr(T_i) is the relative frequency of each of the tissue types [12]. To test if MEPPS would be a possible substitute for manual review in understanding change in plaque composition, all cases were re-analyzed by MEPPS. The manually delineated lumen and outer wall boundaries were retained and used as input to MEPPS, as MEPPS only analyzes plaque components within a defined region. Based on these lumen and wall contours, CA and LRNC areas were segmented fully automatically by MEPPS without any manual corrections (Fig. 1). In previous studies, CE-T1w images have been shown to afford better agreement with histology and higher reproducibility than T2w images for measuring LRNC [4, 19, 20]. Based on this knowledge, manual segmentation of LRNC was performed on CE-T1w images using pre-contrast T1w as reference, irrespective of signal intensities on T2w images. A similar approach was pursued in MEPPS. Three image weightings were used as input to MEPPS: TOF, T1w and CE-T1. If CE-T1w was not available then it was replaced by T2w.

3. Image measurements

Contours generated during manual and MEPPS review were used to obtain area measurements of lumen, outer wall, CA and LRNC on each slice. Wall area was calculated as the difference between outer wall and lumen area. Volumetric measurements were calculated by summing the slice-based area measurements multiplied by the slice thickness (2 mm). In line with manual review data previously reported [7], compositional changes were calculated as annualized change in CA and LRNC in the common coverage between baseline and the follow-up MRI.

Statistical analysis

Manual and MEPPS measurements on plaque components were compared at both the slice-level and the artery-level. For the slice-level analysis, a bootstrap approach was used to account for the dependence between slices from the same subject [21]. Component areas, volumes and annualized changes were summarized as mean ± standard deviation (SD). Cohen’s kappa was used to evaluate agreement on detecting CA and LRNC. Agreement on continuous measurements between manual and MEPPS review was summarized qualitatively using scatterplots, and quantitatively using Pearson’s correlation coefficient (R) and the intra-class correlation coefficient (ICC), based on the two-way random-effects model. A value of R or ICC was interpreted as high when > 0.75, moderate when 0.4 to 0.75, and poor when < 0.4 [22].

To replicate previous analyses of manual review data, linear regression models were used to test for associations between annualized change in LRNC as measured by MEPPS (outcome variable) and a number of clinical and imaging variables. Baseline LRNC volume was included in the model as a covariate to avoid regression-to-the mean effects.

All analyses were performed using SPSS (version 19; SPSS Inc.; Chicago, Ill), JMP (version 11; SAS Institute Inc.; Cary, NC) and R (version 3.1.3; Vienna, Austria).

Results

Our cohort consisted of 59 subjects who completed the longitudinal study and had baseline and follow up carotid MRI available. There were 46 (78%) males with mean age 64.9 ± 7.7 (SD). Detailed demographics have been previously reported [7].

Slice-level agreement

Multicontrast image sets at baseline and follow-up were both included, resulting in a total of 1490 cross-sectional slices for slice-level comparison. CA was detected on 280 (18.5%) slices by manual review versus 198 (13.3%) slices by MEPPS review (kappa=0.72 [0.64, 0.78)]. Area measurement on CA demonstrated a high agreement (R=0.92, ICC=0.91) between manual and MEPPS review (Table 1). LRNC was detected in 524 (35.2 %) slices by manual review, and in 353 (23.7 %) slices by MEPPS review (kappa=0.47 [0.37, 0.57]). Agreement between manual and MEPPS review for measuring LRNC area was high (R=0.82, ICC=0.82). Component area measurements by MEPPS were generally smaller compared to manual review, particularly for CA where a significant difference was noted (Table 1).

Table 1.

Comparison of plaque compositional measurements between manual and MEPPS reviews

		Mean ± SD		Mean Difference
Variable	No.	Manual	MEPPS	Value	(95% CI)	P- value*	R	(95% CI)	ICC	(95% CI)
Area per slice, from all slices (mm2)†
CA	1490	1.1 ± 3.4	0.8 ± 2.9	0.34	(0.20–0.50)	<0.001	0.92	(0.85–0.95)	0.91	(0.81–0.94)
LRNC	1490	2.4 ± 5.4	2.1 ± 5.9	0.31	(−0.09–0.69)	0.13	0.82	(0.70–0.88)	0.82	(0.69–0.87)
TP1 Volume (mm3)
CA	59	28.0 ± 53.4	19.0 ± 42.3	9.0	(4.6, 13.4)	<0.001	0.96	(0.94–0.98)	0.92	(0.83–0.96)
LRNC	59	62.9 ± 63.9	54.4 ± 77.0	8.5	(−1.8, 18.8)	0.11	0.86	(0.77–0.91)	0.84	(0.74–0.90)
TP2 Volume (mm3)
CA	59	28.4 ± 53.7	20.2 ± 45.5	8.2	(4.4, 12.0)	<0.001	0.97	(0.95–0.98)	0.95	(0.87–0.97)
LRNC	59	60.7 ± 71.2	53.4 ± 84.2	7.2	(−3.3, 17.8)	0.18	0.88	(0.80–0.93)	0.86	(0.78–0.92)
Annualized Change in Volume (mm3/year)
CA	59	0.6 ± 17.5	2.1 ± 20.6	−1.5	(−4.8, 1.8)	0.36	0.79	(0.67–0.87)	0.78	(0.66–0.86)
LRNC	59	−5.2 ± 34.3	−1.7 ± 37.2	−3.5	(−12.9, 5.9)	0.46	0.49	(0.27–0.66)	0.49	(0.27–0.66)

CA calcification, LRNC lipid-rich necrotic core, TP1 time-point 1, TP2 time-point 2

^*Test for mean difference against 0;

^†TP1 and TP2 slices are pooled.

Artery-level agreement

Similar to the slice-based results, the mean CA and LRNC volumes by MEPPS also tended to smaller on average than by manual review (Table 1). The agreement between manual and MEPPS remained high at the artery-level with ICC between 0.84 and 0.95 (Table 1; Fig. 2), which was consistently seen for both baseline and follow-up image sets.

Fig. 2.

Correlations between manual and MEPPS reviews on compositional measurements

Solid lines indicate the regression lines. CA calcification, LRNC lipid-rich necrotic core

Agreement on compositional changes

The annualized changes in CA and LRNC volumes were 0.6±17.5 mm³/y and −5.2±34.3 mm³/y by manual review [7], and 2.1±20.6 mm³/y −1.7±37.2 mm³/y by MEPPS review, respectively (Table 1). The differences between manual and MEPPS measurements of compositional change were not statistically significant (p=0.36 for CA and 0.46 for LRNC). At the individual level, a high level of agreement was seen for change in CA volume (ICC=0.78 [0.66, 0.86]) while moderate agreement was seen for change in LRNC (ICC=0.49 [0.27, 0.66]) (Fig. 2).

Although these compositional changes were small at the group level, manual results have suggested that IPH presence (positively) and low-density lipoprotein cholesterol (LDL-C) reduction (negatively) were associated with LRNC progression in this cohort (previously published data) [7]. Similar findings were observed based on data from MEPPS composition review. Subjects with IPH were found to have an increase in LRNC volume compared to a slight decrease in those without IPH (48.4±31.0 vs −4.4±35.8 mm³/y, p=0.015). Furthermore, greater LRNC regression was observed in the group with LDL-C reduction < −10 mg/dl (median) compared with the group with LDL-C reduction ≥ −10 mg/dl (−13.0±37.6 vs. 9.2±33.9 mm³/y, p=0.02). No other clinical factors were associated with LRNC progression based on MEPPS review.

Discussion

This study represents the first head-to-head comparison of semi-automatic plaque component segmentation against expert human reading of serial multicontrast carotid MRI in the setting of a clinical cohort study. The correlation between manual and MEPPS reviews was high in measuring compositional volumes at each time point, which is consistent with previous technical reports and supports the use of automatic plaque analysis in cross-sectional studies [18, 23]. Also noted was a moderate correlation between manual and MEPPS reviews in detecting compositional changes over a 6 month period. The positive association of IPH presence and negative association of LDL-C reduction with LRNC progression, previously reported based on manual review, were also detected based on MEPPS interpretations [7]. This indicates that the fair agreement between manual and MEPPS reviews may allow MEPPS to detect the same biological signals when using moderate to large study samples. However, for individual patient assessment in clinical practice, such as follow-up imaging for monitoring treatment response, the use of automatic algorithms may be limited.

Manual vs. automatic plaque segmentation

Although the utility of multicontrast MRI for in vivo assessment of atherosclerotic carotid plaque composition is well established [3–6], quantitative image interpretation is time-consuming and accuracy and reproducibility can vary between readers [24]. In our experience, 2–3 months of systematic training on both MR plaque imaging and vascular pathology is needed for new readers to achieve similar performance as experienced readers before embarking on case review. And depending on plaque complexity, the time spent on component segmentation could vary from under 5 minutes (no or few components) to over 15 minutes (long, advanced plaques). As more and more MR centers become capable of performing plaque imaging, challenges associated with quantitative human reading have hampered its application in large-scale, multicenter clinical research for improved understanding of disease mechanisms. As a possible solution, automatic plaque segmentation algorithms have been developed [12–16], simulating the manual review process with information such as MR signal intensities and lesion morphology. Given the pivotal role of LRNC enlargement in the pathophysiology of clinical complications [2, 25], large-scale, multicenter trials examining changes in plaque tissue composition are long awaited, which can be facilitated by coupling MR plaque imaging with automatic plaque segmentation for more efficient and effective quantification of LRNC.

Studies with cross-sectional imaging data

The ability of MRI for characterizing plaque substructures add to our armamentarium of atherosclerosis imaging in population studies, providing novel insights into disease mechanisms [8]. Also made possible are prospective studies assessing the prognostic value of plaque composition in various clinical conditions [26, 27]. In these studies, only one MR scan is performed and measurements reflect cross-sectional between-subject differences on plaque composition. In our data, high correlation between manual and MEPPS reviews was observed in obtaining measurements at a single time point, so that MEPPS is expected to be a viable substitute to manual review in studies with cross-sectional imaging data.

As documented in previous validation studies, bias between manual and MEPPS reviews was seen, with smaller measures of CA and LRNC volume noted by MEPPS [18, 23]. A substantial number of small LRNC areas were drawn by human readers but not MEPPS, especially when wall thickening was only mild (wall thickness is taken into account by MEPPS in determining the probability of plaque components). This bias should be kept in mind when comparing results between studies using manual review versus those using MEPPS review. Kerwin et al [18] speculated that the difference in area measures between MEPPS and manual review may be because automatic classifiers define plaque components in the areas common to all available contrast weightings while human readers tend to outline compositional contours in a single contrast weighting. The former may be more sensitive to slight misregistrations and offsets in the z-direction that are intrinsic to multicontrast imaging. Moreover, the performance of the underlying Bayesian model is influenced by the training set. MEPPS was trained using patients with end-stage plaques rather than a population of moderate and asymptomatic stenosis as in this study [12]. Specific training of automatic classifiers according to the study population where they will be applied may improve the agreement.

Studies with serial imaging data

In contrast to the mutually corroborating evidence from preclinical and clinical investigations on how atherosclerosis initiates and develops in the early stage, there has been limited knowledge on how subclinical plaques progress to cause clinical complications in the advanced stage. Thus, one of the most intriguing applications of multicontrast MR plaque imaging is to understand clinical, biochemical, and imaging variables that signals rapid LRNC progression in studies with serial imaging data. In a previous study, MEPPS detected LRNC regression after 2-year rosuvastatin treatment [28]. Yet there have been no studies on the agreement between MEPPS and manual reviews on plaque compositional changes.

It is worth mentioning that excellent agreement in measurements at each time point does not necessarily translate into excellent agreement in subtracted changes. In our cohort, the correlation coefficient between manual and MEPPS reviews in detecting compositional changes was 0.79 and 0.49 for CA and LRNC, respectively, which are much lower than that in measuring CA or LRNC volume at baseline or at follow-up (0.96–0.97 and 0.86–0.88, respectively). Multiple reasons may have contributed to this observation. First, it may reflect the different methods of manual versus MEPPS review in handling serial images. While human readers reviewed images from multiple time points side-by-side for differences (blinded to the order of scans), MEPPS handles multiple time point scans separately. Second, the time interval between scans in this study was rather short. Compositional change was generally small, which may have affected the correlation between MEPPS and human review due to reduced between-subject variability. Finally, compositional change was obtained by subtracting baseline from follow-up measurements. The resulting total measurement error is inherently larger when compared to each individual measurement. Interestingly, the statistically significant associations of LRNC progression with IPH presence (positive association) and LDL-C reduction (negative association) originally found based on manual review could be replicated based on MEPPS measurements. Therefore, we expect that strong biological signals are detectable by MEPPS, particularly in studies with larger sample size and longer follow-up.

Relationship between MRI protocol and automatic segmentation algorithms

In this study, MRI protocol setup and image review were done in a clinical trial setting. However, there is currently no standardized carotid protocol and it is expected that MRI protocols may vary from study to study. For example, T2w images are often used for LRNC detection when gadolinium administration is not an option; some recently proposed 3-dimensional sequences have more complex contrast properties [29]. Although automatic segmentation is heavily dependent on tissue contrast information afforded by MRI protocols, training-based algorithms such as MEPPS are not bound with specific protocols. In studies involving new sequences, a small set of cases may be designated for training automatic algorithms, which can then be applied to the remaining cases, as a time-efficient solution in large-sample studies.

Limitations

As one limitation of this study, no direct histological verification of the absolute accuracy of manual and MEPPS reviews was available. The goal of this study was to see if automatic plaque segmentation could be a substitute for human readers in clinical cohort studies. As histopathological assessment is not an option in these studies, expert human reading is considered the current standard, which is the approach used in all previous studies. However, in large-scale studies, it might be helpful to run automatic analysis to further confirm the findings from manual review or identify issues if results are contradictory. Another limitation is that lumen and outer wall boundaries were manually defined (with computer assistance) even in MEPPS review due to the need to relate to adjacent slices on images with inconspicuous outer wall boundaries. Further efforts are needed to combine manual and automatic reading for minimally-supervised reading that is accurate and time-efficient. Inter- and intra-observer variability were not assessed in this study. It is therefore unclear if the variability between manual and MEPPS analyses is higher or comparable to inter-observer variability. In light of the variable inter-observer reproducibility in measuring plaque components as shown in previous studies [19, 24, 30, 31], automatic algorithms are appealing in that they are less subject to reader variability and will consistently give the same segmentation results with the same images. Side-by-side comparison, which human reviewers leverage to eliminate artifacts and discern real changes, is current not achievable by computer algorithms, could be one of the main reasons accountable for the lower agreement between manual and MEPPS results on compositional changes. However, segmentation results from automatic algorithms may still be used as a good starting point for further manual editing if necessary.

Conclusions

Applying MEPPS as an automatic classifier for plaque composition to a prospective cohort study showed good correlation between expert human review and MEPPS review in obtaining cross-sectional data on plaque tissue composition, and a fair correlation in analyzing compositional changes. Automatic plaque segmentation may be helpful for large-scale multicenter studies to reduce image analysis time and avoid bias between readers of various levels of experience and across institutions.