Abstract
Introduction
The purpose of this study is to compare the variability of PCT results obtained by automatic selection of the arterial input function (AIF), venous output function (VOF) and symmetry axis versus manual selection.
Methods
Imaging data from 30 PCT studies obtained as part of standard clinical stroke care at our institution in patients with suspected acute hemispheric ischemic stroke were retrospectively reviewed. Two observers performed the post-processing of 30 CTP datasets. Each observer processed the data twice, the first time employing manual selection of AIF, VOF and symmetry axis, and a second time using automated selection of these same parameters, with the user being allowed to adjust them whenever deemed appropriate. The volumes of infarct core and of total perfusion defect were recorded. The cerebral blood volume (CBV), cerebral blood flow (CBF), mean transit time (MTT) and blood–brain barrier permeability (BBBP) values in standardized regions of interest were recorded. Interobserver variability was quantified using the Bland and Altman's approach.
Results
Automated post-processing yielded lower coefficients of variation for the volume of the infarct core and the volume of the total perfusion defect (15.7% and 5.8%, respectively) compared to manual post-processing (31.0% and 12.2%, respectively). Automated post-processing yielded lower coefficients of variation for PCT values (11.3% for CBV, 9.7% for CBF, and 9.5% for MTT) compared to manual post-processing (23.7% for CBV, 32.8% for CBF, and 16.7% for MTT).
Conclusion
Automated post-processing of PCT data improves interobserver agreement in measurements of CBV, CBF and MTT, as well as volume of infarct core and penumbra.
Keywords: Perfusion CT, Post-processing, Reproducibility, Acute stroke
Introduction
In recent years, perfusion-CT (PCT) imaging has emerged as a useful tool to evaluate patients with suspected stroke [1, 2]. PCT has been advocated to detect and characterize the ischemic penumbra [3, 4], but also to assess blood–brain barrier permeability in an effort to predict the risk of hemorrhagic conversion after stroke [5–7]. As stated in the Acute Stroke Imaging Research Roadmap [8], one of the major obstacles to the widespread use of PCT technique to select acute stroke patients for reperfusion therapy is the lack of interobserver reliability, which led to a consensus recommendation for a more automated method of PCT data post-processing [8]
More specifically, the parameters to be selected during the post-processing of PCT data and, therefore, prone to influence the final PCT results are: the arterial input function (AIF), used to calculate the mean transit time (MTT) and the blood–brain barrier permeability (BBBP) maps; the venous outflow function (VOF), used to calculate the cerebral blood volume (CBV); the symmetry axis, used to calculate infarct and penumbra maps. These maps are generated by comparing MTT and CBV values in the ischemic and contralateral hemispheres and applying specific thresholds [9–11]. Several commercially available PCT software packages allow for automatic selection of the AIF, VOF and symmetry axis, and for automatic post-processing of PCT maps, with the possibility for the user to correct or adjust the aforementioned parameters in case the automatic selection is not adequate and/or not completely satisfactory.
The purpose of this study was to evaluate whether the reproducibility of quantitative PCT results can be improved by automated post-processing with automatic selection of the AIF, VOF and symmetry axis, corrected by the user if deemed necessary. In order to do so, we measured and compared the degree of interobserver reproducibility of PCT-derived quantitative measurements obtained using manual and automated post-processing methods in patients with suspected acute ischemic stroke. We selected one specific software implementation for our study, based on its ability to display maps of infarct core and penumbra based on CBV and MTT thresholds.
Materials and methods
Patients
Imaging data from 30 PCT studies obtained as part of standard clinical stroke care at our institution in patients with suspected acute hemispheric ischemic stroke were retrospectively reviewed with the approval of our institutional review board. At our institution, patients suspected of acute stroke and with no history of renal insufficiency or iodinated contrast allergy routinely undergo a stroke CT survey including noncontrast CT (NCT) of the brain, PCT at two cross-sectional positions, CT-angiogram (CTA) of the cervical and intracranial vessels, and post-contrast cerebral CT.
Perfusion-CT protocol
PCT studies were obtained on 16-slice (15 patients) and 64-slice (15 patients) CT scanners. Each PCT study involved successive gantry rotations performed in cine mode during intravenous administration of iodinated contrast material. Images were acquired and reconstructed at a temporal sampling rate of 1 image/s for the first 45 s. Additional images were acquired at 60, 90, 120, 150, and 180 s. Acquisition parameters were 80 kVp and 100 mAs. Two successive PCT series at two different levels were performed following the noncontrast CT and before the CTA. At each PCT level, two 10-mm-thick sections (16-section CT scanners) or eight 5-mm-thick sections (64-section CT scanners) were assessed. The first PCT series was performed at the level of the third ventricle and the basal ganglia, and the second PCT series was performed above the lateral ventricles. For each PCT series, a 40-mL bolus of iohexol (300 mg/mL of iodine, Omnipaque; Amersham Health, Princeton, NJ) was administered into an antecubital vein by using a power injector at an injection rate of 5 mL/s for all patients. CT scanning was initiated 7 s after the start of the injection of the contrast bolus.
PCT raw data post-processing
PCT data were analyzed using a research version of a PCT software package (Philips Medical Systems, Cleveland, OH) to calculate MTT, CBV, CBF, and BBBP maps. This software relies on the central volume principle, which is the most accurate for low injection rates of iodinated contrast material [12]. After motion correction and noise reduction by an anisotropic, edge-preserving spatial filter, the software applies curve fitting by least mean squares to obtain mathematical descriptions of the time-enhancement curves for each pixel, describing the wash-in and the wash-out of contrast in these pixels. A closed-form (noniterative) deconvolution is then applied to calculate the MTT map. The deconvolution operation requires a reference arterial input function and venous outflow function. For this purpose, the user draws regions of interest, typically within the anterior cerebral artery (ACA) and a vertical segment of the superior sagittal sinus (SSS). The software then identifies the appropriate reference voxels within these regions of interest. Alternatively, in the research version used in this study, the user may let the software automatically select the reference voxels for AIF and VOF, and adjust them if needed. Similarly, the user can manually define a symmetry axis, or let the software define one and adjust it if needed.
The CBV map is calculated from the area under the time-enhancement curves for each pixel. The central volume principle equation allows the calculation of cerebral blood flow (CBF = CBV/MTT). From the parametric maps the software generates summary maps of penumbra and infarct core. In our study, absolute CBV and relative MTT thresholds (comparing the voxels on the stroke side to their contralateral counterparts around the symmetry axis) were used as reported in literature (total perfusion defect: MTT > 145% of the contralateral side values; infarct core, CBV < 2.0 ml × 100 g−1) [9]. BBBP maps were calculated by applying the Patlak model to the delayed PCT acquisition [13]
Post-processing of the 30 CTP datasets was performed by two observers with more than 6 months of experience with PCT post-processing each. Both individuals were trained by a neuroradiologist who has 10 years of experience in PCT post-processing software application. Each observer processed the data twice, a first time employing manual selection of AIF, VOF, and symmetry axis, and a second time using automated selection of these same parameters, with the user being allowed to adjust them whenever deemed appropriate. The two processing sessions for each reviewer were performed more than 2 months apart, in order to reduce recall bias.
Data analysis
The volume of perfusion abnormality corresponding to infarct core and tissue at risk were recorded. The sum of the volumes of infarct core and tissue at risk was calculated and recorded under the label of “total perfusion defect”.
To obtain mean values of CBV, CBF, MTT, and BBBP, regions of interest (ROIs) were drawn in a single representative slice in the MCA territory of the ischemic and non-ischemic hemispheres at the level of the lateral ventricles in the slice just above the choroid plexus. This region of interest included white matter of the corona radiata and the superficial cortical gray matter (Fig. 1). These ROIs were systematically drawn in the exact same location for all post-processing rounds.
Fig. 1.
Axial PCT map depicting the area of infarct core (red) and tissue at risk (green) in the right MCA territory. Also demonstrated are the regions of interest (ROIs) that were systematically drawn for all post-processing rounds in the MCA territory of the ischemic (yellow) and contralateral hemisphere (blue). Mean values of CBV, CBF and MTT measured in these ROIs were recorded
Statistical analysis
Interobserver agreement was quantified for each of the recorded PCT parameters using the approach proposed by Bland and Altman [14]. Two descriptive statistics of the interobserver agreement were calculated: the repeatability coefficient and the coefficient of variation. The repeatability coefficient is related to the absolute difference between the measurements performed by the two reviewers for the different patients and the different variables: in 95% of the cases, the absolute difference between the measurements performed by the two reviewers will be less than the repeatability coefficient. Lower repeatability values mean better agreement between two measurements. The coefficient of variation is calculated as a ratio of the repeatability coefficient over the mean value of the considered variables, and expressed in percentages. A lower coefficient of variation means better agreement between two different measurements is achieved.
Also, the measurements performed by the two reviewers for the different patients and the different variables were evaluated for linear regression. The slopes, intercepts and coefficients of correlation of the linear regressions were recorded.
Results
Patients
Our study population included 30 consecutive patients (14 males and 16 females; median age, 61; age range, 47–89) presenting between January and March 2008 with symptoms suspicious for acute hemispheric ischemic stroke. The average National Institute of Health Stroke Scale (NIHSS) score on admission was 13 (range, 3–21). In 26 patients, a diagnosis of ischemic stroke involving the MCA territory was made, while two patients had an ACA stroke and two were considered to have normal perfusion-CT studies.
Manual versus automated post-processing
The coefficients of variation for the volume of the infarct core and the volume of the total perfusion defect (Table 1) ranged between 5.8% and 31.0%. They were higher for the volume of the infarct core (15.7–31.0%) compared to the volume of the total perfusion defect (5.8–12.2%). Automated post-processing yielded improved interobserver agreement (15.7% for volume of the infarct core and 5.8% for the volume of the total perfusion defect) compared to manual post-processing (31.0% for volume of the infarct core and 12.2% for the volume of the total perfusion defect). Linear regression showed slopes and coefficients of correlation close to 1, both for the automated and the manual post-processing.
Table 1.
Interobserver variability for the manual and automated post-processing methods for the volumes of the infarct core and of the total perfusion defect.
| Interobserver variability | |||
|---|---|---|---|
| Manual post-processing | Automated post-processing | p value | |
| Volume of infarct core (mL) | |||
| Mean ± SD | 16.89 ± 21.01 | 17.42 ± 20.84 | 0.739 |
| Repeatability coefficient | 5.23 | 2.73 | |
| Coefficient of variation | 31.0% | 15.7% | |
| Slope | 1.069 | 0.987 | |
| Intercept | −1.133 | 0.631 | |
| Correlation | 0.992 | 0.997 | |
| Volume of total perfusion defect (mL) | |||
| Mean ± SD | 68.11 ± 49.60 | 68.36 ± 49.87 | 0.998 |
| Repeatability coefficient | 8.33 | 3.99 | |
| Coefficient of variation | 12.2% | 5.8% | |
| Slope | 1.015 | 0.995 | |
| Intercept | −2.239 | 0.698 | |
| Correlation | 0.995 | 0.999 | |
The mean volume of the regions of interest drawn in the MCA territory was 24.3 cc2 (range: 21.9–28.1 cc2). The coefficients of variation for the ROI-generated values of CBV, CBF and MTT in the ischemic hemisphere (Table 2) ranged between 9.5% and 32.8%. They were higher for CBV (11.3%–23.7%) and CBF (9.7%–32.8%) compared to MTT (9.5%–16.7%). Automated post-processing yielded improved interobserver agreement (11.3% for CBV, 9.7% for CBF and 9.5% for MTT) compared to manual post-processing (23.7% for CBV, 32.8% for CBF and 16.7% for MTT).
Table 2.
Interobserver variability for the manual and automated post-processing methods for PCT values of cerebral blood volume (CBV), cerebral blood flow (CBF), mean transit time (MTT) and blood–brain barrier permeability (BBBP) in the ischemic brain parenchyma.
| ROI on ischemic hemisphere | Interobserver variability | ||
|---|---|---|---|
| Manual post-processing | Automated post-processing | p value | |
| CBV (mL/100 g) | |||
| Mean ± SD | 3.00 ± 0.95 | 2.89 ± 0.93 | 0.650 |
| Repeatability coefficient | 0.71 | 0.32 | |
| Coefficient of variation | 23.7% | 11.3% | |
| Slope | 0.724 | 0.849 | |
| Intercept | 0.766 | 0.376 | |
| Correlation | 0.921 | 0.930 | |
| CBF (mL/100 g/s) | |||
| Mean ± SD | 18.94 ± 11.45 | 17.67 ± 9.84 | 0.808 |
| Repeatability coefficient | 6.21 | 1.71 | |
| Coefficient of variation | 32.8% | 9.7% | |
| Slope | 0.960 | 1.004 | |
| Intercept | 1.030 | −0.070 | |
| Correlation | 0.948 | 0.994 | |
| MTT (s) | |||
| Mean ± SD | 15.23 ± 21.02 | 15.67 ± 21.93 | 0.952 |
| Repeatability coefficient | 2.55 | 1.49 | |
| Coefficient of variation | 16.7% | 9.5% | |
| Slope | 0.966 | 1.035 | |
| Intercept | −0.300 | 0.006 | |
| Correlation | 0.997 | 0.995 | |
| BBBP (mL/100 g/s) | |||
| Mean ± SD | 1.10 ± 0.61 | 0.98 ± 0.61 | 0.514 |
| Repeatability coefficient | 0.40 | 0.33 | |
| Coefficient of variation | 35.9% | 34.2% | |
| Slope | 1.180 | 0.812 | |
| Intercept | −0.184 | 0.166 | |
| Correlation | 0.945 | 0.953 | |
The coefficients of variation for the ROI-generated values of CBV, CBF, and MTT were similar between the ischemic and non-ischemic hemispheres (Table 3).
Table 3.
Interobserver variability for the manual and automated post-processing methods for PCT values of cerebral blood volume (CBV), cerebral blood flow (CBF), mean transit time (MTT) and blood–brain barrier permeability (BBBP) in the non-ischemic brain parenchyma.
| ROI on contralateral, non-ischemic hemisphere | Interobserver variability | ||
|---|---|---|---|
| Manual post-processing | Automated post-processing | p value | |
| CBV (mL/100 g) | |||
| Mean ± SD | 3.04 ± 0.67 | 2.94 ± 0.50 | 0.117 |
| Repeatability coefficient | 0.74 | 0.29 | |
| Coefficient of variation | 24.3% | 9.7% | |
| Slope | 0.734 | 0.921 | |
| Intercept | 0.696 | 0.238 | |
| Correlation | 0.805 | 0.942 | |
| CBF (mL/100 g/s) | |||
| Mean ± SD | 34.40 ± 10.86 | 32.17 ± 8.31 | 0.082 |
| Repeatability coefficient | 12.19 | 4.69 | |
| Coefficient of variation | 35.4% | 14.6% | |
| Slope | 0.811 | 0.951 | |
| Intercept | 6.863 | 1.632 | |
| Correlation | 0.779 | 0.942 | |
| MTT (s) | |||
| Mean ± SD | 5.59 ± 1.23 | 5.70 ± 1.16 | 0.307 |
| Repeatability coefficient | 1.67 | 0.45 | |
| Coefficient of variation | 29.8% | 8.0% | |
| Slope | 0.790 | 0.877 | |
| Intercept | 0.985 | 0.680 | |
| Correlation | 0.696 | 0.979 | |
| BBBP (mL/100 g/s) | |||
| Mean ± SD | 1.10 ± 0.38 | 0.96 ± 0.38 | 0.424 |
| Repeatability coefficient | 0.40 | 0.25 | |
| Coefficient of variation | 36.7% | 25.6% | |
| Slope | 0.923 | 0.927 | |
| Intercept | 0.092 | 0.083 | |
| Correlation | 0.811 | 0.929 | |
Interobserver agreement for BBBP measurements
The coefficients of variation for BBBP measurements ranged from 25.6% to 36.7%. Automated post-processing yielded improved interobserver agreement (34.2% in ischemic hemisphere and 25.6% in non-ischemic hemisphere) compared to manual post-processing (35.9% in ischemic hemisphere and 36.7% in non-ischemic hemisphere).
Corrections by the observers during automated post-processing of PCT raw data
During automated post-processing, both observers corrected the software selection of AIF in 3 cases each (10%). In these cases, which were the same for both observers, the automatically selected AIF voxels were on the anterior portion of the SSS, being manually repositioned to the ACA. Automatically selected VOF voxels were considered inadequate by both users in five cases (16.7%). These studies had head motion that was not completely corrected by the registration algorithm, with VOF voxels originally placed on the skull by the automatic algorithm. These were manually repositioned to the superior sagittal sinus. The symmetry axis was manually adjusted by each observer in one patient, whose head was severely tilted at the time of the scan.
Discussion
This study demonstrated that the reproducibility of PCT measurements, including CBV, CBF, MTT, and BBBP values, as well as of PCT abnormality volumes based on CBV and MTT thresholds, was improved when an automated post-processing algorithm minimizing user interaction was used. The average interobserver variability for CBV, CBF and MTT measurements using an automated post-processing approach was 10.4%, almost three times less than the interobserver variability of 27.1% observed for a manual post-processing approach. This improved interobserver reproducibility when using an automated approach was observed both in the ischemic parenchyma (10.1% with automated post-processing, 24.4% with manual post-processing) and in the non-ischemic parenchyma (10.8% with automated post-processing, 29.8% with manual post-processing). In agreement with previous studies [15, 16], our results showed that MTT was the most reproducible PCT parameter.
In this study, we reported not just the variability of measurements of CBV, CBF, MTT, and BBBP, but also the variability of measurements of infarct core and total perfusion defect volumes based on CBV and MTT thresholds. To our knowledge, the interobserver variability for measurements of infarct core and total perfusion defect volumes has not been reported before. The interobserver variability we obtained for the measurements of CBV, CBF and MTT using a manual processing approach is similar to what was reported in a prior published study comparing manual post-processing of PCT datasets by different CT technologists [16]. In another study, lower interobserver variability in case of manual PCT post-processing was achieved by the use of a uniform and standardized approach [17]. In the same study, paradoxically, greater interobserver variability was introduced with the use of an automated approach [17]. This reflects that not all automated PCT post-processing algorithms, as implemented in commercially available packages, are equivalent, and that each of them needs to be carefully evaluated, and possibly improved.
Since a completely automated approach to post-processing PCT data would yield a perfect degree of agreement (0% of interobserver variability), it is clear that the interobserver variability around 10.4% observed when using the automated approach was introduced by the user-defined manual adjustments. The manual adjustments performed by the observers in our study were mainly justified by an incorrect selection of the AIF and VOF by the automatic algorithm in studies with severe head motion artifact. Although these manual corrections were necessary to obtain accurate PCT results, they introduced interobserver variability for the “automated” PCT results. This again emphasizes the need for further improvement of automated PCT post-processing algorithms, with the ultimate goal being to completely eliminate user interaction.
Two main limitations should be noted in our study. The first one is that the results we obtained apply only to a specific software implementation. Therefore, the variability between automated and manual post-processing observed in our study should not be assumed to be similar to other automated PCT post-processing algorithms. The second limitation is the absence of a well-defined criterion of acceptability in the variability of PCT-derived measurements. However, prior studies with similar results [16, 17] have shown that an overall variability around 10% does not change the qualitative visual assessment of PCT maps, thus being very unlikely to alter clinical management decisions.
In conclusion, automated post-processing of PCT data improves interobserver variability in measurements of CBV, CBF, MTT and BBBP, as well as volume of infarct core and penumbra based on CBV and MTT thresholds. PCT post-processing automation is a way to ensure reliability of PCT technique when widely used in the community, which is one of the requirements for it to be used in the selection of stroke patients for acute reperfusion therapies.
Acknowledgments
Max Wintermark receives funding from the NIH National Center for Research Resources, Grant KL2 RR024130, GE Healthcare and Philips Healthcare. The content of the article is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Research Resources, the National Institutes of Health or the other sponsors.
Joerg Bredno is an employee of Philips Research North America.
Conflict of interest statement We declare that we have no conflict of interest.
Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
References
- 1.Koenig M, Klotz E, Luka B, Venderink DJ, Spittler JF, Heuser L (1998) Perfusion CT of the brain: diagnostic approach for early detection of ischemic stroke. Radiology 209:85–93 [DOI] [PubMed]
- 2.Mayer TE, Hamann GF, Baranczyk J et al (2000) Dynamic CT perfusion imaging of acute stroke. AJNR Am J Neuroradiol 21:1441–1449 [PMC free article] [PubMed]
- 3.Nabavi DG, Cenic A, Henderson S, Gelb AW, Lee TY (2001) Perfusion mapping using computed tomography allows accurate prediction of cerebral infarction in experimental brain ischemia. Stroke 32:175–183 [DOI] [PubMed]
- 4.Eastwood JD, Lev MH, Azhari T et al (2002) CT perfusion scanning with deconvolution analysis: pilot study in patients with acute middle cerebral artery stroke. Radiology 222:227–236. doi:10.1148/radiol.2221010471 [DOI] [PubMed]
- 5.Lin K, Kazmi KS, Law M, Babb J, Peccerelli N, Pramanik BK (2007) Measuring elevated microvascular permeability and predicting hemorrhagic transformation in acute ischemic stroke using first-pass dynamic perfusion CT imaging. AJNR Am J Neuroradiol 28:1292–1298. doi:10.3174/ajnr.A0539 [DOI] [PMC free article] [PubMed]
- 6.Bisdas S, Hartel M, Cheong LH, Koh TS, Vogl TJ (2007) Prediction of subsequent hemorrhage in acute ischemic stroke using permeability CT imaging and a distributed parameter tracer kinetic model. J Neuroradiol 34:101–108 [DOI] [PubMed]
- 7.Dankbaar JW, Hom J, Schneider T et al (2008) Dynamic perfusion CT assessment of the blood–brain barrier permeability: first pass versus delayed acquisition. AJNR Am J Neuroradiol 29:1671–1676. doi:10.3174/ajnr.A1203 [DOI] [PMC free article] [PubMed]
- 8.Wintermark M, Albers GW, Alexandrov AV et al (2008) Acute stroke imaging research roadmap. Stroke 39:1621–1628. doi:10.1161/STROKEAHA.107.512319 [DOI] [PubMed]
- 9.Wintermark M, Flanders AE, Velthuis B et al (2006) Perfusion-CT assessment of infarct core and penumbra: receiver operating characteristic curve analysis in 130 patients suspected of acute hemispheric stroke. Stroke 37:979–985. doi:10.1161/01.STR.0000209238.61459.39 [DOI] [PubMed]
- 10.Murphy BD, Fox AJ, Lee DH et al (2006) Identification of penumbra and infarct in acute ischemic stroke using computed tomography perfusion-derived blood flow and blood volume measurements. Stroke 37:1771–1777. doi:10.1161/01.STR.0000227243.96808.53 [DOI] [PubMed]
- 11.Lee TY, Murphy BD, Aviv RI et al (2006) Cerebral blood flow threshold of ischemic penumbra and infarct core in acute ischemic stroke: a systematic review. Stroke 37:2201 author reply 2203. doi:10.1161/01.STR.0000237068.25105.aa [DOI] [PubMed]
- 12.Wintermark M, Maeder P, Thiran JP, Schnyder P, Meuli R (2001) Quantitative assessment of regional cerebral blood flows by perfusion CT studies at low injection rates: a critical review of the underlying theoretical models. Eur Radiol 11:1220–1230. doi:10.1007/s003300000707 [DOI] [PubMed]
- 13.Patlak CS, Blasberg RG, Fenstermacher JD (1983) Graphical evaluation of blood-to-brain transfer constants from multiple-time uptake data. J Cereb Blood Flow Metab 3:1–7 [DOI] [PubMed]
- 14.Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1:307–310 [PubMed]
- 15.Waaijer A, van der Schaaf IC, Velthuis BK et al (2007) Reproducibility of quantitative CT brain perfusion measurements in patients with symptomatic unilateral carotid artery stenosis. AJNR Am J Neuroradiol 28:927–932 [PMC free article] [PubMed]
- 16.Fiorella D, Heiserman J, Prenger E, Partovi S (2004) Assessment of the reproducibility of postprocessing dynamic CT perfusion data. AJNR Am J Neuroradiol 25:97–107 [PMC free article] [PubMed]
- 17.Sanelli PC, Nicola G, Tsiouris AJ et al (2007) Reproducibility of postprocessing of quantitative CT perfusion maps. AJR Am J Roentgenol 188:213–218. doi:10.2214/AJR.05.2188 [DOI] [PubMed]