Validation of bedside manual versus automated measurements of brain arterial diameters from MR angiography

Abstract

Background and Purpose:

Brain arterial luminal diameters are reliably measured with automated imaging software. Non-automated imaging software alternatives such as a Picture Archiving Communication System are more common bedside tools used for manual measurement. This study is aimed at validating manual measurements against automated methods.

Methods:

We randomly selected 600 participants of the Northern Manhattan Study (NOMAS) and 260 participants of the Atahualpa project studied with 1.5 Tesla MR angiography. Using radiant ^™ measuring tool, three independent readers (general practitioner, neurology resident and vascular neurologist) measured manually the diameter of arterial brain vessels. The same vessels were also measured by LKEB Automated Vessel Analysis (LAVA)^™. We calculated the intraclass correlation coefficient (ICC) of each rater’s diameters versus those obtained with LAVA.

Results:

The ICC between diameters obtained by the general practitioner or the neurology resident compared to LAVA were excellent for both Internal Carotid Arteries (ICA) and Basilar Arteries (BA) (ICC > 0.80 in all comparisons) in NOMAS. In the Atahualpa project, ICC between diameters obtained by a vascular neurologist and LAVA was good for the ICC for both ICA and BA (ICC > 0.60 in all comparisons). The ICCs for the measurements of the remaining arteries were moderate to poor.

Conclusion:

Results suggest that manual measurements of ICA and BA diameters, but not MCA or ACA, are valid and could be used to identify dilated brain arteries at the bedside and to select. Patients with dolichoectasia may qualify for an intervention or trial.

Introduction

Brain artery lumen is an important determinant of brain vessel health. Dolichoectasia, defined as a pathological increase in arterial diameter, can present as a vascular compression syndrome or a cerebrovascular event.^1,2 However, most patients with dolichoectasia are asymptomatic. Therefore, the majority of asymptomatic dolichoectasia research comes from population-based epidemiological studies. In such studies, the most frequently used procedure for arterial diameter measurement has been a three-dimensional time of flight MR angiography. Its benefits over contrast-enhanced computed tomography or digital subtraction angiogram include that MR angiographies are non-invasive and do not require intravenous contrast administration.^3–5 To enhance scientific rigor and reproducibility, we and others have used automated vessel analysis software (such as LKEB-MR angiography vessel wall analysis [LAVA]), which can provide a precise assessment of luminal arterial diameter with little user interaction.^3–5

Specifically, the automated software that we have used,^6,7 reads an MR angiography in a Digital Imaging and Communications in Medicine (DICOM) format and creates a three-dimensional (3D) image of the arterial anatomy from available slides. Once the 3D image is created, the user determines a distal and proximal point in the vessel to be measured. Unfortunately, programs such as LKEB- MR angiography are costly and inaccessible at the bedside. On the other hand, manual measurements using a picture-archiving communication system (PACS) or similar radiology viewers, could be used at the bedside and are widely accessible. Validating the reliability of manual measurements of brain arterial diameters compared to automated software measurements might increase the translational potential regarding the role of brain arterial remodeling (defined by diameters) with risk of stroke and other neurological outcomes such as dementia. The present study aims to assess the validity of manual measurements by using two well-established population-based prospective cohorts of community-dwellers from different genetic backgrounds.

Methods

Study Participants

We included participants from the Northern Manhattan Study (NOMAS) and the Atahualpa Project. Participants from NOMAS were selected from a multi-ethnic community of participants in New York City. The subjects were recruited and enrolled between 1993 and 2001, they were considered eligible if they had no previous diagnosis of stroke, were over the age of 40, and had lived in northern Manhattan for a minimum of 3 months in a household with a telephone. From 2003 to 2008 participants that were stroke-free, were 55 years or older, and had no contraindications, were submitted to MR angiography.⁸ Later, from 2006 to 2008, 199 more members from the same cohort were introduced to the MRI sub-study. Out of 1290 participants in the MRI sub study, we randomly selected 600 for this study. The selection process involved randomly selecting participants based on their identifications to ensure an unbiased representation. The NOMAS was approved by the institutional review board at Columbia University Medical Center.⁹ Participants signed written an informed consent.

Participants from the Atahualpa Project were identified by means of annual door-to-door surveys (2012–2019) and invited to undergo a brain MRI and MR angiography of intracranial vessels. Atahualpa is a small rural village located in coastal Ecuador where more than 95% of the population identifies as Ecuadorian native/Mestizo ethnic group.¹⁰ The institutional review board of Hospital Kennedy located in Guayaquil, Ecuador, approved the protocol and the informed consent.¹¹ Participants signed a written informed consent. A total of 478 individuals aged ≥60 years have been enrolled, 403 of whom have a baseline MR angiography. Of them, 260 (65%) were randomly selected for this study. The same process of selection was used for Atahualpa as in NOMAS.

Individuals enrolled in NOMAS and the Atahualpa Project received similar interviews and procedures for assessment of demographics (age, sex, level of education) and traditional cardiovascular risk factors by the use of the Life’s simple 7 construct of the American Heart Association (AHA), which classifies each of seven cardiovascular health metrics according to the following cutoffs: 1) Poor smoking status if the subject is a current smoker or quit <1 year prior; 2) Poor body mass index if ≥30 kg/m²; 3) Poor physical activity if there is no moderate and vigorous activity; 4) Poor diet if there is none or only one of five AHA healthy dietary components; 5) Poor blood pressure if ≥140/90 mm Hg; 6) Poor fasting glucose if ≥126 mg/dl; and 7) Poor total cholesterol blood levels if ≥240mg/dl. In addition, the use of medications relevant to the control of the above-mentioned risk factors was ascertained. The enrollment process for NOMAS participants was done at the Columbia University Medical Center (CUMC) or at the participant’s home. For the Atahualpa Project, enrollment was at the participant’s home.

MR angiography acquisition

The 600 MR angiography from the NOMAS cohort were performed on a 1.5 Tesla (T) MRI (Philips Medical Systems, Best, Netherlands) at Columbia University Irving Medical Center following a standard protocol⁸. We used a three dimensional time-of-flight MR angiography with the following parameters: 1mm effective slice thickness, acquisition matrix interpolated to a 256 by 228 matrix, flip angle of 25 and repetition time/echo time of 20 and 2.7 ms respectively.¹²

MR angiographies from the Atahualpa project were performed on a Philips Intera 1.5T MR scanner (Philips Medical System. Eindhoven, the Netherlands) at Hospital-Clínica Kennedy, Guayaquil. MR angiography used a three-dimensional time-of-flight sequence (slice thickness was interpolated down at 1 mm). Pre-established parameters for this sequence included: field of view 20 cm, 1 mm effective slice thickness, acquisition matrix interpolated to 332×222 matrix, flip angle of 20 degrees, repetition time/echo time 19 and 6.9 ms, respectively.

Arterial diameter measurements

For brain arterial luminal diameter manual measurements, we used Radiant (Version 2022.1, Medixant, Poznan, Poland, https://www.radiantviewer.com/) to view MR angiography images in a DICOM format. Three readers with different levels of training measured brain vessel luminal diameter - a General Practitioner, a Neurology Resident, and a Vascular Neurologist. MR angiographies were excluded if severe motion artifacts were present. The vessels measured included the internal carotid arteries (ICA) in the ascending portion of the cavernous carotid, the middle cerebral artery (MCA) at its most proximal 5 mm segment, the anterior cerebral artery (ACA) at the most proximal 5 mm segment and the basilar artery (BA) at the most proximal measurable segment after the vertebrobasilar junction. The posterior cerebral artery and the posterior communicating artery were excluded because of their variability due to the circle of Willis collateral network. In the case of the BA and the ICA manual measurements, two measurements were done perpendicular to each other in the axial plane and then averaged. Examples of these measurements are shown in Figure 1. The general practitioner and the neurology resident were tasked with measuring the NOMAS cohort while the vascular neurologist measured the Atahualpa cohort. The General practitioner and the Neurology resident measured the same 600 participants.

Figure 1:

Axial three-dimensional Time-of-Flight MR angiography (1.5 Tesla) the brain. (A) Time-of-Flight at the level of the cavernous sinus. The internal carotid artery can be observed ascending. The vessel diameter was calculated by two measurements, perpendicular to each other then they are averaged (B) Circle of Willis, both anterior cerebral artery (ACA) can be seen. The right ACA was measured. (C) Circle of Willis, both middle cerebral artery (MCA) can be seen from their origin. The left MCA is being measured. (D) Posterior cerebral fossa at the level of the pontomedullary junction. The basilar artery can be observed, the lumen diameter was determined by two measurements perpendicular to each other and then average. In all images millimeters are represented by mm.

Automated brain arterial luminal diameter measurements were performed by an independent reader at Columbia University Irving Medical Center. The measurements of both cohorts were done from a remote workstation with LAVA (version 5.10.0, Leiden University Medical Center, Leiden, The Netherlands) in a DICOM format blinded to the manual measurements as reported elsewhere. The diameters measured in LAVA for the BA, MCA, ICA and the ACA were done in the first millimeter of its origin.

Statistical Analysis

The characteristics of the studied population are reported in mean and standard deviations, or frequencies. The primary aim of the study was to determine the validity of brain arterial luminal diameter measurements that were done manually in Radiant) and those performed automated by LAVA. We calculated inter-observer measurement reliability by determining the intraclass coefficient (ICC) (two way mixed; consistency). IBM SPSS Statistics (version 28.0.0.0,IBM, Armonk, New York, USA, https://www.ibm.com/spss) was used in all statistical analyses.

Results

This validation study included a random NOMAS subsample of 600 participants (mean age: 70.4 ±8.7 years, 39.8% males, 65.8% Hispanic), and an Atahualpa project subsample of 260 participants (mean age: 68.9 ±8.7 years, 43.1% males, 100% Indigenous Ecuadorians). Baseline characteristics of individuals enrolled in both cohorts are shown in Table 1. Some notable differences are seen in dyslipidemia (89.6% For NOMAS and 1.9% for Atahualpa) and hypertension (79.5% for NOMAS and 56.92 % for Atahualpa).

Table 1:

Baseline Characteristics of the Northern Manhattan Study and Atahualpa cohort.

Baseline Characteristics
Cohort	Northern Manhattan Study (600)	Atahualpa (260)	P value
Age, means years. ±SD	70.36 ±8.69	68.9 ±8.48	0.038
Males, n (%)	239 (39.8%)	112 (43.07%)	0.650
Height, mean centimeters ±SD	163.60 ±9.70	148.39 ±8.89	<0.001
Weight, mean kilograms ±SD	74.43 ±13.98	59.07 ±11.34	<0.001
Race/ethnicity, n (%)
Indigenous Ecuadorians	0 (0)	260 (100%)
Hispanic	395 (65.8%)	0 (0)
Black	114 (19%)	0 (0)
White	91 (15.2%)	0 (0)
Comorbidities, n (%)
Hypertension	477 (79.5%)	148 (56.92%)	<0.001
Diabetes mellitus	148 (24.7%)	88 (33.8%)	0.003
Dyslipidemia	537 (89.6%)	5 (1.9%)	<0.001
Current Smoking	81 (13.5%)	3 (1.2%)	<0.001

SD: standard deviation; n: number

Compared with automated measurements, the ICC of manual measurement obtained by the readers in NOMAS images were > 0.8 for the BA and the ICA. The results for MCA and the ACA were <0.6 for both readers in NOMAS except for the left MCA of the NR (0.68). The vessels which presented the highest ICC in Atahualpa were the ICA and the BA, but their values were between 0.6–0.8. The MCA and ACA had values <0.6 (Table 2).

Table 2:

Intraclass Correlation Coefficient between manual measurements and LKEB Automated Vessel Analysis measurements

Intraclass Correlation Coefficient c
Cohort	Northern Manhattan Study a				Atahualpa b
Rater	General Practitioner		Neurology Resident		Vascular Neurologist
Basilar Artery	0.84	(0.81–8.70)	0.89	(0.84–0.87)	0.77	(0.70–0.82)
Left ICA	0.90	(0.88–0.91)	0.89	(0.88–0.91)	0.66	(0.57–0.73)
Right ICA	0.89	(0.88–0.91)	0.90	(0.88–0.91)	0.70	(0.58–0.71)
Left MCA	0.58	(0.51–0.65)	0.69	(0.63–0.73)	0.44	(0.29–0.56)
Right MCA	0.57	(0.49–0.63)	0.60	(0.52–0.66)	0.40	(0.23–0.53)
Left ACA	0.30	(0.17–0.40)	0.42	(0.32–0.51)	0.45	(0.29–0.57)
Right ACA	0.42	(0.32–0.51)	0.47	(0.37–0.54)	0.58	(0.46–0.67)

^aGeneral Practitioner and Neurology Resident measured participants from the Northern Manhattan Study cohort.

^bThe Vascular Neurologist measured participants from the Atahualpa Cohort.

^cValues above 0.8 are considered excellent, between 0.8 and 0.6 are good, between 0.4 and 0.6 are fair and below 0.4 are poor.

ICA: Internal carotid artery; MCA: Middle cerebral artery; ACA: Anterior cerebral artery

Comparing measurements for each artery in a Bland-Altman plot, we can see that mean of the difference is positive for the BA. For ICA the mean of the difference is positive, except for the neurology resident. In the case of the MCA and the ACA the results were mixed between each rater. (Figure 2).

Figure 2:

Bland-Altman plot for each artery measured by the raters. Each column represents the results of the rater. Each point represents a measured artery, the Y-axis is the difference between the measurements of LKEB MR angiography vessel analysis software (LAVA) and the rater. Positive results represent overestimation. The X axis represents the Mean between the result of the rater and LAVA. The larger the vessel the greater the mean will mean. The red line represents the mean of the difference, and the dotted lines will represent ± 1.96 standard deviation.

Discussion

Brain arterial remodeling phenotypes, such as dolichoectasia or intracranial stenosis, are imaging biomarkers of vascular and cognitive risk. Although historically these phenotypes have been defined by visual assessment, over the last two decades there has been greater emphasis on using various arterial diameter thresholds to improve the consistency of the terms and replicability of research results.^13–15 The evaluation of intracranial stenosis has been traditionally done manually and it is current practice to select participants for trials using manual diameter measurements at the levels of stenosis and in a non-diseased referent level.¹⁶ The measurements of diameters in patients with dolichoectasia or large, dilated arteries is not well validated. Automated software such as LAVA have been validated in intracranial time of flight MR angiography by our group and others.^4,17 For this reason, validation of manual measurements would be important to provide practical standards for the diagnosis of dolichoectasia and recognition of increased vascular and cognitive decline risk, as well as eventual enrollment into clinical trials.

The results presented here validate manual bedside measurements of arterial diameters for the intracranial portion of the ICA and BA. The results for the ACA and MCA were suboptimal, thus showing their limitations for clinical and research applicability. Possible explanations of our findings include the use of axial images to measure ICA and BA diameters. The axis of these arteries is perpendicular to the axial plane, which facilitates the visualization of the carotid and basilar artery cross-sectional diameters whereas the ACA or MCA run in a parallel plane and have more variable course. A parallel plane measurement of the anterior or middle cerebral arteries lumen could randomly reflect the bottom, middle or top part of the artery, therefore increasing the variability of results compared to true cross-sectional diameters. It is possible that if MR angiographies were acquired using isometric voxels, the coronal and sagittal plane projections could improve the reliability of the anterior and middle cerebral arteries, respectively. For example, some studies used orthogonally reconstructed axial time-of-flight MR angiography to measure the MCA with some success.¹⁸ In addition, previous studies suggested that the spatial resolution of 1.5T MR angiographies is inaccurate at lower diameter levels (i.e., < 2.5 mm).¹⁹ The average size for the MCA in our studies were 2.7mm, 2.4mm, and 2.9mm for the general practitioner, neurology resident, and vascular neurologist, respectively. For the ACAs, diameters were 2.1mm, 1.7mm, and 2.2mm for the same raters, respectively. Whether using 3T or 7T imaging might improve the reliability of manual measurement for smaller caliber brain arteries remains to be determined. Since the signal to noise ratio has been shown to improve with higher resolution for larger arteries.²⁰

The results presented here for the ICA and BA are also likely to apply to CT angiograms. While programs like LAVA allow us to obtain accurate measurements of vessels with little user interaction using MR angiographies, a drawback of this and other automated software used in research is that they are less accessible to clinicians. It is not routine for radiologists to comment on brain arterial luminal measurements and whether a given patient might have diameter-based dolichoectasia. The absence of systematic reporting of the diagnosis of dolichoectasia is perhaps related to the lack of major clinical trials in this condition, the consensus of its diagnosis or its clinical implication.²¹

Our study has several strengths. For example, the studied samples represent two large well-established population-based prospective cohorts of community-dwellers from different geographic ancestry. The combined sample is large and testing the reliability of the measure separately supported the generalizability of the results. It is worth also discussing limitation to this study. The images used were acquired in an axial plane and they were not isometric. Consequently, sagittal or coronal planes were not possible. This influences the measurements of vessels such as the middle cerebral and anterior cerebral artery that are quantified by one measurement. Additionally, the arteries can only be measured within a limited section, typically 2 to 4 slides. This leads to an increase of variability in the measurement, especially if the slide does not intersect the widest portion of the vessel. Therefore, depending on a single measurement for a vessel does not fully capture its true dimension. It is possible that the ICC of the ACA and MCA could be improved in a sagittal or coronal plane that captures the artery cross-sectionally in the same way that axial planes show the ICA and the BA. We cannot comment on whether the validity of manual measurement could vary by method of imaging such as CT angiography (CTA) or digital subtraction angiography (DSA). It’s worth mentioning that DSA is still considered the most accurate method for quantification of arterial diameter. Additionally, the MRI acquisition parameter varied by cohort, and it may partially relate to the variability in the ICC values against LAVA. It is also important to mention the limitations of time-of-flight MR angiography. This technique depends on the direction and flow of blood. This can show an overestimation of stenosis in vessels due to a flow void defect, which in turn leads to incorrect results. Lastly, is has been shown that a negative signal intensity in middle cerebral artery with a positive delineation in CTA has been associated with altered cerebral vascular reactivity.²²

In summary, our results suggest that manual measurements of the diameter of the ICA and BA can be used to identify dilated carotid or basilar arteries reliably. Manual measurements are widely accessible to clinicians using PACS and could facilitate the identification of individuals with asymptomatic dolichoectasia in the BA and ICA that might be eligible for possible interventions and/or studies. The association with the ACA or MCA were too variable to support the same conclusion. In addition, more validation data would be useful to compare results of the present study with other populations and imaging methods.