Abstract
Purpose
To determine the intra- and interrater agreement of mean flow velocity (MFV) and pulsatility index (PI) measurement in middle cerebral arteries, assessed by transcranial Doppler (TCD) with M-mode.
Methods
Masked experienced neurosonologists performed TCD with M-mode using handheld probe in healthy adult volunteers. The Bland–Altman method for concordance and intraclass correlation coefficient were used.
Results
Seventy-seven healthy volunteers and seven raters participated (3 on regular TCD shift and 4 off-shift). The intrarater absolute mean difference between measurements was 5.5 cm/s [95% confidence interval (CI), 4.7–6.3] for MVF and 0.073 (95% CI, 0.063–0.083) for PI. The difference between MFV measurements was significantly higher in off-shift raters (p = 0.015). The interrater absolute mean difference between measurements was 6.5 cm/s (95% CI, 5.5–7.5) for MVF and 0.065 (95% CI, 0.059–0.071) for PI. No influence was found for the middle cerebral artery side, volunteer’s sex, or age, and there was no significant difference between raters. The intraclass correlation coefficient was 82.2% (95% CI 77.8–85.6) and 72.9% (95% CI 67.4–77.6) for MFV and PI, respectively.
Conclusions
There exists good intra- and interrater agreement in MFV and PI measurements using M-mode TCD. These results support the use of this noninvasive tool and are important for clinical and investigational purposes.
KEYWORDS: agreement, intraclass correlation coefficient, mean flow velocity, reproducibility, transcranial Doppler
Introduction
Transcranial Doppler ultrasound (TCD) is a noninvasive, safe, and real-time method for assessing intracranial blood hemodynamics. Since the first description of the technique by Rune Aaslid in early 20th century, it has gained increasing acceptance as an accurate diagnostic and therapeutic tool in both cerebrovascular disease and neurocritical care [1,2,3].
TCD has similar diagnostic performance as computed tomography angiography in detecting acute arterial obstructions in patients admitted with an acute ischemic stroke, particularly in the anterior circulation arteries [4]. Pulsatility index (PI) is a representation of flow resistance of the cerebral circulation and helps differentiate a velocity reduction due to diminished cardiac output from increased distal resistance in intracranial atherosclerotic disease [5]. Its use to monitor intracranial pressure remains controversial [6,7,8]. Few studies have analyzed TCD agreement in determination of cerebral blood flow velocities and have focused mainly in cerebral vasoreactivity [9] or flow detection in the ophthalmic artery [10]. Scarce information is available about TCD reliability in middle cerebral artery (MCA) mean flow velocity (MFV) measurements, between and among different operators, even though it is the most often used hemodynamic parameter in cerebrovascular disease [11,12,13]. Moreover, these studies have not used Power Motion mode (M-mode) and included a reduced number of examiners and patients. Good agreement of TCD has been described [12,13], with operator’s experience and regular practice being key for good reliability [11,13].
In this study, we aimed to describe inter- and intrarater agreement examining MCA MFV and PI by hand-held TCD with M-mode in healthy volunteers from a single center.
Participants and methods
Study participants and raters
Staff neurosonologists at Clínica Alemana de Santiago, Chile, with TCD training (defined as having received a formal training in TCD ultrasonography) and experience (defined as performing TCD on a regular basis during the year prior to study commencement and/or having more than a 3-year experience performing TCD [11]) were invited to participate as raters in the study. They were classified as “on-shift” or “off-shift” according to their regular participation in TCD shifts during the year prior to study commencement, involving weekly performance of TCD [11].
The examined group consisted of healthy volunteers—defined as not having either a cerebrovascular disease or any acute illness—who gave consent to participate.
Study variables
Masked repeated MFVs (cm/s) and PI (value provided by the TCD machine) in the M1 segments of MCAs through both temporal bones sonographic windows were obtained in all volunteers. The automatic measurement given by the envelope tool was registered. If this was not possible, a manual measurement was performed. Demographic characteristics and arterial blood pressure (BP) of the examined volunteers before every TCD were also registered.
Assessments
Each rater performed TCD examinations of both MCAs through temporal sonographic windows. An independent investigator was present during all assessments, to register MFV and PI, depth of the measurement, date, time, and BP, and to ensure MFV masking by placing a cover over the results display screen in the TCD machine. This cover only masked the right side of the curve screen, leaving M-mode and deepness information available to the rater on the left. The volume was turned on for the rater to listen. We instructed the raters to record the best sonographic curve displayed on the screen. All assessments were performed with Spencer Technologies TCD (Redmond, WA, USA) with Power M-Mode 150. Measurements were performed with a sample of 6–9 mm, a 100-Hz filter, 100 mW/cm2 power, and a 2-MHz handheld probe. TCD recordings started at 55 mm of depth. Once detected, the MCA was followed to the proximal M1 segment with slight changes in angulation of the probe, until the inverse and weaker Doppler signal was detected, corresponding to anterior cerebral artery (60 to 65 mm). The MCA was also followed distally to 30–35 mm depth. Once the optimal M1 MCA flow signal was detected (optimal at 55 mm deepness, but 50–60 mm range was allowed), the image was captured. We assumed absence of temporal acoustic window when the neurosonologist was not able to detect the flow signal in MCA after a period of 5 minutes (Figure 1).
Figure 1.
Transcranial Doppler curve and M-mode.
Volunteers were in a supine (lying flat) position throughout the examination, and had three sets of MFV and PI measurements bilaterally by two raters. The first rater assessed both MCAs. The independent investigator registered the results, cleaned the Doppler gel, and opened a new registry in the Doppler machine. The first rater then went out of the room, and the second rater came in to perform a second bilateral MCA assessment. The same registry was performed. Once the second rater was finished, the first rater came back to repeat the measurements, within a 20-minute time frame.
Statistical analysis
Agreement of repeated measurements for MFV and PI was performed by calculating the interrater intraclass correlation coefficient (ICC). ICC was calculated with a two-way mixed model, and absolute agreement was used. The Bland–Altman method for concordance was performed, and plots were obtained, showing limits of agreement (LoA) [14].
Multivariate regression was used to determine the association between individual raters, volunteer’s demographic information, and difference between systolic and diastolic BP with measurements of MFV and PI. Analysis of variance or t test was used to compare continuous by categorical variables. A two-sided p value <0.05 was set as the level for statistical significance. The analysis was performed with STATA (StataCorp. 2013, Stata Statistical Software: Release 13; StataCorp LP, College Station, TX, USA).
Ethics
The institutional Ethics Committee approved the study protocol. Written informed consent was obtained from the participants.
Results
Seventy-seven healthy volunteers were included in this study. Their mean age was 35 years [standard deviation (SD), 11.7], and 66% were women. Seven experienced raters participated—three on-shift and four off-shift. Measurements were performed at a mean depth of 54.9 mm (SD 0.48). Interrater and intrarater mean time between assessments was 9 (SD 5.9) and 17 minutes (SD 10.8), respectively. The absolute mean difference between systolic BP prior to TCD was 6.6 mmHg (SD 8.5) for interrater measurements and 7.3 mmHg (SD 6.6) for intrarater measurement, and the absolute mean difference between diastolic BP prior to TCD was 4.8 mmHg (SD 4.3) and 5.3 mmHg (SD 5.0), respectively. There was consensus between raters in bilateral presence of transtemporal acoustic windows in 74 volunteers and in its absence in one case. In two cases, one of the raters did not find the left acoustic transtemporal bone window but the other did.
Intrarater agreement
MFV intrarater absolute mean difference between measurements was 5.5 cm/s (95% CI, 4.7–6.3), whereas PI absolute mean difference was 0.073 (95% CI, 0.063–0.083). The mean difference between measurements and 95% LoA are shown in Table 1.
Table 1.
Intra-and interrater mean difference between measurements and 95% limits of agreement (LoA).
| Variable | Mean difference (SD) | 95% LoA | Absolute Mean difference (SD) | 95% CI | ICC (95% CI) | |
|---|---|---|---|---|---|---|
| Intrarater | MFVa | –0.9 (7.5) | –15.5 to 13.8 | 5.5 (5.3) | 4.7–6.3 | |
| PI | 0.016 (0.095) | –0.171 to 0.203 | 0.073 (0.064) | 0.063–0.08 | ||
| Interrater | MFVa | 2.0 (9.0) | –15.6 to 19.6 | 6.5 (6.53) | 5.5–7.5 | 82.2% (77.8e85.6) |
| PI | –0.005 (0.088) | –0.178 to 0.168 | 0.065 (0.059) | 0.059–0.071 | 72.9% (67.4–77.6) |
CI = confidence interval; ICC = intraclass correlation coefficient; MVF = mean flow velocity; PI = pulsatility index; SD = standard deviation.
a MVF is expressed in cm/s.
Bland–Altman plots are depicted in Figure 2, showing no systematic bias in measurements.
Figure 2.
Intrarater agreement. (A) Bland–Altman plot of MFV difference between measurements versus average of paired measurements (cm/s). (B) Bland–Altman plot of PI difference between measurements versus average of paired measurements. MFV = mean flow velocity; PI = pulsatility index.
The mean difference in ratios was 1.0 (SD 0.14) and 95% LoA in ratios were 0.73–1.26 for MFV. The mean difference in ratios was 1.0 (SD 0.12) and 95% LoA in ratios were 0.78–1.27 for PI. No influence of the MCA side, volunteer sex, or age was found. There was no significant difference between raters; however, the intrarater difference between MFV measurements was significantly higher in off-shift raters (p = 0.015). Conversely, no difference in PI measurement between on- and off-shift groups was detected. Table 2 shows mean measurements by on- and off-shift groups.
Table 2.
Intrarater mean difference and 95% limits of agreement (LoA), by raters on-and off-shift.
| Variable | Mean difference (SD) | Absolute mean difference (SD) | p | Mean difference | 95% CI | |
|---|---|---|---|---|---|---|
| MFVa | On-shift | –0.9 (6.4) | 4.9 (4.2) | 0.015 | 1.13 | 0.55–5.03 |
| Off-shift | 1.9 (8.2) | 5.9 (5.9) | ||||
| PI | On-shift | –0.005 (0.092) | 0.066 (0.064) | NS | ||
| Off-shift | –0.025 (0.097) | 0.077 (0.064) |
CI = confidence interval; MVF = mean flow velocity; PI = pulsatility index; SD = standard deviation.
a MVF is expressed in cm/s.
Interrater agreement
The MFV interrater absolute mean difference between measurements was 6.5 cm/s (95% CI, 5.5–7.5), with a PI absolute mean difference between measurements of 0.065 (95% CI, 0.059–0.071). Measurements mean difference and 95% LoA are shown in Table 1. No systematic bias in measurements was detected in Bland–Altman plots (Figure 3).
Figure 3.
Interrater agreement. (A) Bland–Altman plot of MFV difference between measurements versus average of paired measurements (cm/s). (B) Bland–Altman plot of PI difference between measurements versus average of paired measurements. MFV = mean flow velocity; PI = pulsatility index.
The mean difference in ratios was 1.1 (SD 0.19) and the 95% LoA in ratios were 0.67–1.43 for MFV. The mean difference in ratios was 1.0 (SD 0.11) and the 95% LoA in ratios were 0.78–1.22 for PI. No influence of the MCA side, volunteer sex, or age was found. There was no significant difference between raters. Contrary to the different intrarater agreement, no significant differences were found in mean differences in MFV and PI measurements between the on- and off-shift raters (Table 3). The ICC was 82.2% (95% CI, 77.8–85.6) for MFV and 72.9% (95% CI, 67.4–77.6) for PI (Table 1).
Table 3.
Interrater mean difference in measurements and 95% limits of agreement (LoA), by raters on-and off-shift.
| Variable | Mean difference (SD) | 95% LoA | Absolute mean difference (SD) | 95% CI | |
|---|---|---|---|---|---|
| On–On | MFVa | 0.18 (12.3) | –11.8 to 36.5 | 7.8 (9.5) | 5.1–10.5 |
| PI | –0.008 (0.081) | –0.078 to 0.239 | 0.050 (0.063) | 0.032–0.070 | |
| On–Off | MFVa | –0.2 (8.7) | –8.4 to 25.8 | 6.4 (5.9) | 5.5–7.3 |
| PI | –0.002 (0.082) | –0.079 to 0.244 | 0.062 (0.054) | 0.054–0.070 | |
| Off–Off | MFVa | 1.6 (8.2) | –7.9 to 24.3 | 6.0 (8.2) | 4.3–7.7 |
| PI | –0.020 (0.094) | –0.090 to 0.279 | 0.075 (0.056) | 0.065e0.085 |
Group on–on: both raters on-shift.
Group on–off: one rater on-shift and the other off-shift.
Group off–off: both raters off-shift. CI = confidence interval; MVF = mean flow velocity; PI = pulsatility index; SD = standard deviation,
a MVF is expressed in cm/s.
Discussion
In this experienced group of neurosonologists, we found good agreement between measurements of MFV and PI when healthy volunteers were assessed using hand-held TCD with M-mode. Intrarater agreement was better within on-shift raters compared to off-shift raters; however, there was no influence in interrater agreement when the data were stratified by on- or off-shift. The absolute mean differences found are small and unlikely to be clinically relevant.
This cohort showed higher intra- and interrater agreement compared with previous series. McMahon et al [11] reported intra- and interrater agreement using a DWL pulsed Doppler ultrasound device with 2-MHz probe and similar technique. The intrarater mean difference between MFV measurements in experienced users, one research fellow, and one research nurse, was –1.6 (95% LoA ± 19.3) cm/s, compared to –0.9 (95% LoA ± 14.7) in our center. The interrater mean difference in MFV between experienced users was 1.8 (95% LoA ± 22.1) cm/s, compared to 2.0 (95% LoA± 17.6) in our study. Regarding TCD device and technique, use of a TCD with M-mode might have influenced positively the agreement results as M-mode improves the detection of acoustic window and assessment, allowing for a more reliable measurement [15,16].
Another technology, color-coded duplex ultrasound, has been associated with considerable higher inter- and intrarater variability, especially in subarachnoid hemorrhage (SAH) patients. Staalso et al [17] reported MFV measurement LoA in ratios performed by experienced users; when the healthy control group was examined, the intrarater LoA in ratios were 0.67–1.50, compared to 0.73–1.26 in our center, and the interrater LoA in ratios were 0.65–1.55, compared to 0.67–1.43 in our cohort. Staalso et al [17] also reported larger LoA values in ratios when patients with angiographic vasospasm were assessed.
There are three sources of interrater concordance variance: participants (healthy volunteers), trials (raters), and residuals. Up to one-third of the variance has been described to correspond to intraobserver variance, and most depend on the examined participants [17]. Moment-to-moment variation of cerebral blood flow velocity has been detected with continuous TCD [18]. To account for this variance, we reduced the time between measurements and asked participants to maintain the same resting position during the sessions to avoid changes in physiological parameters. This translated to a negligible BP difference between assessments.
A lower ICC for PI compared to MFV was detected, and several possible explanations can account for this. A large number of factors affect the PI, including arterial pressure, vascular compliance, and PaCO2. In this study, it is unlikely that these factors have influenced the PI values, because the measurements were performed within a short time difference and the conditions were not modified. Weakness of the signal in peak systolic and diastolic velocities and any artifact may influence the PI value, particularly when the automatic measurement is used (envelope). This effect is less evident in MFV measurements, as its measure considers the whole area under the curve of flow velocity. Another possible explanation is that the range of normal values in the PI is narrower than that of the MFV; therefore, small differences (i.e., ±0.1) have more impact in PI that in MFV.
To our knowledge, this marks the first time that handheld TCD measurement agreement has been described in a large selection of raters and using TCD with M-mode. Moreover, we could not find any previous reports on PI interrater agreement measured with TCD in adult patients with which to compare our results. The other strength of this prospective study is that raters were masked to their own and other raters’ results, and there were only short delays between examinations. A limitation of the study was the inclusion of healthy young volunteers instead of patients with acute ischemic stroke (AIS). Nevertheless, the assessment of healthy individuals is similar to the AIS clinical scenario than in previous reports that have used other clinical scenarios, such as SAH patients. In AIS patients, TCD is not aimed at recognizing high ranges of MFV, as is the case for SAH. Therefore, in SAH patients, a wider measurement error occurs as compared with volunteers, probably related to moment-to-moment variability in patients with vasospasm, treatment effects, and difficulty of recognition of short constricted segments [17].
In conclusion, we found good intra- and interrater reliability of MFV and PI measurements using hand-held TCD with M-mode, performed in healthy volunteers by experienced neurosonologists. These results support the use this noninvasive tool for clinical and investigational purposes.
Author contribution
PMV conceived and designed the study, made acquisition, analysis, and interpretation of data for the work and wrote the first draft of the manuscript. AB, VVO, and PML contributed to the study design and acquisition of data, made critical review, and edited the manuscript. JG, SI, JL, AR, PB, and FG contributed to the acquisition of data, made critical review, and edited the manuscript. GC contributed to the data analysis and made critical review of the manuscript. All authors approved the final version of the manuscript.
Footnotes
Conflicts of interest: PMV, AMB, JG, SI, JL, AR, and PML performed TCD in their regular clinical practice, and it is included in their salary. VVO, PB, FG, and GC declare no conflicts of interest.
References
- [1].Demchuk AM, Saqqur M, Alexandrov AV. Transcranial Doppler in acute stroke. Neuroimag Clin N Am. 2005;15:473–480. doi: 10.1016/j.nic.2005.08.008. ix. [DOI] [PubMed] [Google Scholar]
- [2].Kalanuria A, Nyquist PA, Armonda RA, et al. Use of transcranial Doppler (TCD) ultrasound in the neurocritical care unit. Neurosurg Clin N Am. 2013;24:441–56. doi: 10.1016/j.nec.2013.02.005. [DOI] [PubMed] [Google Scholar]
- [3].Balucani C, Alexandrov AV. Ultrasound- and microspheresenhanced thrombolysis for stroke treatment: state of the art. Curr Cardiol Rep. 2010;12:34–41. doi: 10.1007/s11886-009-0082-0. [DOI] [PubMed] [Google Scholar]
- [4].Brunser AM, Lavados PM, Hoppe A, et al. Accuracy of transcranial Doppler compared with CT angiography in diagnosing arterial obstructions in acute ischemic strokes. Stroke. 2009;40:2037–41. doi: 10.1161/STROKEAHA.108.542704. [DOI] [PubMed] [Google Scholar]
- [5].Bude RO, Rubin JM. Stenosis of the main artery supplying an organ: effect of end-organ vascular compliance on the poststenotic peak systolic velocity. J Ultrasound Med. 1999;18:603–13. doi: 10.7863/jum.1999.18.9.603. [DOI] [PubMed] [Google Scholar]
- [6].Behrens A, Lenfeldt N, Ambarki K, et al. Transcranial Doppler pulsatility index: not an accurate method to assess intracranial pressure. Neurosurgery. 2010;66:1050–7. doi: 10.1227/01.NEU.0000369519.35932.F2. [DOI] [PubMed] [Google Scholar]
- [7].Schmidt B, Czosnyka M, Klingelhofer J. Clinical applications of a non-invasive ICP monitoring method. Eur J Ultrasound. 2002;16:37–45. doi: 10.1016/s0929-8266(02)00044-7. [DOI] [PubMed] [Google Scholar]
- [8].Bellner J, Romner B, Reinstrup P, et al. Transcranial Doppler sonography pulsatility index (PI) reflects intracranial pressure (ICP) Surg Neurol. 2004;62:45–51. doi: 10.1016/j.surneu.2003.12.007. discussion 51. [DOI] [PubMed] [Google Scholar]
- [9].McDonnell MN, Berry NM, Cutting MA, et al. Transcranial Doppler ultrasound to assess cerebrovascular reactivity: reliability, reproducibility and effect of posture. PeerJ. 2013;1:e65. doi: 10.7717/peerj.65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].Rojanapongpun P, Morrison B, Drance SM. Reproducibility of transcranial Doppler ultrasound examinations of the ophthalmic artery flow velocity. Br J Ophthalmol. 1993;77:22–4. doi: 10.1136/bjo.77.1.22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].McMahon CJ, McDermott P, Horsfall D, et al. The reproducibility of transcranial Doppler middle cerebral artery velocity measurements: implications for clinical practice. Br J Neurosurg. 2007;21:21–7. doi: 10.1080/02688690701210539. [DOI] [PubMed] [Google Scholar]
- [12].Maeda H, Etani H, Handa N, et al. A validation study on the reproducibility of transcranial Doppler velocimetry. Ultrasound Med Biol. 1990;16:9–14. doi: 10.1016/0301-5629(90)90080-v. [DOI] [PubMed] [Google Scholar]
- [13].Shen Q, Stuart J, Venkatesh B, et al. Inter observer variability of the transcranial Doppler ultrasound technique: impact of lack of practice on the accuracy of measurement. J Clin Monit Comput. 1999;15:179–84. doi: 10.1023/a:1009925811965. [DOI] [PubMed] [Google Scholar]
- [14].Bland JM, Altman DG. Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet. 1995;346:1085–7. doi: 10.1016/s0140-6736(95)91748-9. [DOI] [PubMed] [Google Scholar]
- [15].Moehring MA, Spencer MP. Power M-mode Doppler (PMD) for observing cerebral blood flow and tracking emboli. Ultrasound Med Biol. 2002;28:49–57. doi: 10.1016/s0301-5629(01)00486-0. [DOI] [PubMed] [Google Scholar]
- [16].Saqqur M, Dean N, Schebel M, et al. Improved detection of microbubble signals using power M-mode Doppler. Stroke. 2004;35:e14–7. doi: 10.1161/01.STR.0000106771.62928.66. [DOI] [PubMed] [Google Scholar]
- [17].Staalso JM, Edsen T, Romner B, et al. Transcranial Doppler velocimetry in aneurysmal subarachnoid haemorrhage: intraand interobserver agreement and relation to angiographic vasospasm and mortality. Br J Anaesth. 2013;110:577–85. doi: 10.1093/bja/aes458. [DOI] [PubMed] [Google Scholar]
- [18].Venkatesh B, Shen Q, Lipman J. Continuous measurement of cerebral blood flow velocity using transcranial Doppler reveals significant moment-to-moment variability of data in healthy volunteers and in patients with subarachnoid hemorrhage. Crit Care Med. 2002;30:563–9. doi: 10.1097/00003246-200203000-00011. [DOI] [PubMed] [Google Scholar]