Intra- and Interobserver Agreement and Impact of Arterial Input Selection in Perfusion CT Measurements Performed in Squamous Cell Carcinoma of the Upper Aerodigestive Tract

Abstract

BACKGROUND AND PURPOSE: CT Perfusion (CTP) has shown potential for assessing head and neck tumors. Our purposes were to assess the inter- and intraobserver agreement of CTP measurements and to investigate whether the selection of arterial input, ipsilateral versus contralateral to the tumor or left-versus-right external carotid artery (ECA), may affect CTP measurements in patients with squamous cell carcinoma (SCCA) of the upper aerodigestive tract.

MATERIALS AND METHODS: Twenty-six patients with SCCA were enrolled in this prospective study and underwent CTP. Data were analyzed by 2 expert readers and by an inexperienced reader for interobserver agreement and by the 2 expert readers for intraobserver agreement assessment, by using the ECA ipsilateral to tumor site as arterial input. All 3 readers repeated their analysis by using the ECA contralateral to tumor site as arterial input. Inter- and intraobserver agreement was assessed by using the Bland-Altman approach; CTP measurements by using ipsilateral-versus-contralateral or left-versus-right ECA were compared by using the Wilcoxon signed rank test.

RESULTS: The geometric mean of the ratios (95% limits of agreement) for inter- and intraobserver agreement ranged from 0.96 (0.75–1.23) to 1.00 (0.92–1.10) for blood flow (BF), from 0.88 (0.63–1.21) to 1.00 (0.88–1.14) for blood volume (BV), from 0.96 (0.64–1.44) to 0.98 (0.76–1.27) for mean transit time (MTT), and from 0.85 (0.41–1.76) to 1.14 (0.70–1.86) for permeability surface area product (PS). Significantly higher tumor PS and MTT for 2 readers and lower tumor BF for 1 of 3 readers were observed when the arterial input was placed in the left ECA.

CONCLUSIONS: BF, BV, and MTT demonstrated higher inter- and intraobserver agreement than PS. The selection of arterial input, right-versus-left ECA, may determine changes in CTP measurements in patients with SCCA of the upper aerodigestive tract.

CT perfusion (CTP) of squamous cell carcinoma (SCCA) of the upper aerodigestive tract has demonstrated potential for tumor diagnosis, with significantly different perfusion parameters compared with normal structures,^1,2 and therapy monitoring, correlating with endoscopic response following induction chemotherapy.³ In addition, CTP seems to predict local outcome in patients with SCCA of the upper aerodigestive tract undergoing radiation therapy⁴ and induction chemotherapy.⁵

There is, then, a growing interest among researchers in the integration of this technique in the clinical trials and in the drug-development process. If CTP is to be increasingly applied in a wide range of research and clinical settings, the technique must be demonstrated to be both reproducible and reliable. There are studies that have investigated the intra- and interobserver agreement of CTP measurements in body tumors, such as colorectal,^6,7 lung,⁸ and head and neck⁹ tumors, and have addressed some technical issues of CTP imaging in body tumors, such as arterial input function selection⁹ and tumor volume coverage in the z-axis.¹⁰

To the best of our knowledge, no previous study has adequately evaluated the inter- and intraobserver agreement of CTP measurements in a homogeneous and quite large cohort of patients with SCCA of the upper aerodigestive tract; and no one has addressed, in all the relevant aspects, the issue of arterial input selection.

The first purpose of this prospective study was to assess the inter- and intraobserver agreement of CTP measurements in patients with SCCA of the upper aerodigestive tract. Our second purpose was to investigate whether the selection as arterial input of the ipsilateral-versus-contralateral external carotid artery (ECA) with respect to the tumor and of the left-versus-right ECA influences CTP measurements in patients with SCCA of the upper aerodigestive tract.

Materials and Methods

Patients

After approval by our institutional ethics committee, untreated patients with biopsy-confirmed SCCA of the upper aerodigestive tract (the oral cavity, oropharynx, larynx, and hypopharynx) were prospectively enrolled into a study investigating the potential of CTP for induction-chemotherapy monitoring. Written informed consent was obtained from all the participants before the study.

Patients were included if they met the following criteria: stage III or IVA-B tumor according to tumor node metastasis system,¹¹ ≥18 and ≤75 years of age, Eastern Cooperative Oncology Group performance status from zero to 2, white blood cell count ≥3000 mm³, neutrophil count ≥1500 mm³, platelet count ≥100,000 mm³, serum creatinine level ≤1 mg/dL, and serum bilirubin level ≤1.25 mg/dL. Patients with previous neoplasms in other sites not treatable for recovery, pregnancy, or allergy to iodinated contrast media were excluded from the study.

Local tumor staging was based on clinical evaluation by endoscopy and contrast-enhanced CT of the head and neck; contrast-enhanced MR imaging was used instead of CT for tumors of the oral cavity and oropharynx. Distant metastases were researched in all patients by using positron-emission tomography-CT and CT or MR imaging. Patients with carotid stenosis >50% were excluded by pretherapy Doppler sonography because such vascular disease may introduce a source of variability in CTP measurements.^12,13

From November 2004 to December 2007, a total of 30 consecutive patients (27 men and 3 women; median age, 60 years; range, 47–73 years) met the selection criteria and were enrolled in the study. Tumors of 6 patients were stage III, 20 were stage IVA, and 4 were stage IVB (Table 1) at the time of presentation. Before initiation of induction chemotherapy (baseline), all patients underwent clinical evaluation by endoscopy and, within 1 week, baseline CTP and diagnostic CT of the head and neck.

Table 1:

Enrolled patients and tumor characteristics

	No.	%
Total	30	100
Site
Oral cavity	1	3.3
Oropharynx	17	56.7
Hypopharynx-larynx	12	40
Stage
III	6	20
IVA	20	66.7
IVB	4	13.3
TNM
T1N3M0	1	3.3
T1N2M0	1	3.3
T2N1M0	3	10
T2N2M0	11	36.7
T3N0M0	3	10
T3N2M0	3	10
T4N0M0	2	6.7
T4N1M0	3	10
T4N2M0	2	6.7
T4N3M0	1	3.3

Imaging Technique

CT examinations were performed with a 16-section multidetector CT (MDCT) scanner (LightSpeed 16; GE Healthcare, Milwaukee, Wis). Preliminary noncontrast CT of the head and neck (2.5-mm thickness) was performed to localize the tumor. An expert radiologist (expert reader A), who had 2 years of experience in CTP imaging at the start of the study, selected a 20-mm scanning range for the CTP; the range was chosen to include the largest tumor area identified on the noncontrast localizing images. A total of 40 mL of nonionic iodinated contrast material was injected (iomeprol, Iomeron [400 mg I/mL]; Bracco, Milan, Italy), followed by 40 mL of saline solution at a rate of 5 mL/s via an 18-ga cannula, which was placed in the right antecubital vein in all patients to exclude any source of variability. CTP scanning started with a 7-second delay after contrast material administration, with the following parameters: 4 contiguous 5-mm reconstructed sections at a constant table position, 1-second gantry rotation time, 120 kilovolt (peak), and 100 mA. Images were acquired every second for 45 seconds for first-pass enhancement assessment (first-pass scanning) and every 10 seconds up to 122 seconds (prolonged scanning) for late-enhancement assessment.

Immediately after completion of CTP scanning, diagnostic CT of the head and neck was performed by using 16 × 0.625 (10 mm) collimation; section thickness and increment, 1.25 mm; standard reconstruction filter; 180 mA; 120 kV(p); rotation time, 0.6 seconds; speed, 9.38 mm/rotation; FOV, 18 cm; intravenous contrast material (Iomeron [400 mg I/mL]); injection rate, 2 mL/s; volume, 90 mL; acquisition delay, 70 seconds). The average total examination dose-length product for the combined CTP and diagnostic CT of the head and neck was 1196.13 mGy cm.

Preliminary noncontrast CT of the head and neck and CTP and diagnostic CT of the head and neck were performed by using quiet respiration. The patients were warned to avoid swallowing and taking a deep breath when experiencing the “hot flush” resulting from the rapid bolus of iodinated contrast material.

Image and Data Analysis

The images and data obtained were anonymized and transferred to an image-processing workstation (Advantage Windows 4.2, GE Healthcare). CTP data were analyzed by 3 independent readers, 2 expert and the other inexperienced in CTP imaging. Before their CTP analyses for this study, the first expert reader (expert reader A) had 5 years’ experience in CTP imaging, whereas the second expert reader (expert reader B) had 13 years’ experience in head and neck CT imaging and 3 years’ experience in CTP imaging. Each performed a first CTP analysis en masse after all the data were collected and a repeated CTP analysis on the same anonymous dataset during a period ranging from 4 to 8 weeks (median, 6.8 and 6.5 weeks for expert readers A and B respectively), to allow evaluation of intraobserver agreement.

The inexperienced reader was a resident in radiology, with no previous experience in CTP imaging, in training for head and neck imaging. At the time of his CTP analysis, which was performed en masse after all the data were collected, he had undergone 8 weeks of training in CTP imaging of head and neck tumors (excluding patients in the present study cohort) led by a radiology expert in CTP and head and neck CT imaging, who was not among the readers.

Commercially available software (CT Perfusion 3, GE Healthcare) was used for perfusion parameter calculation, based on a deconvolution-based technique. The arterial input was obtained from a standardized 4-pixel region of interest placed in the ECA ipsilateral to the tumor site, with selection of the section that allowed the best visualization. A time-attenuation curve, expressed in Hounsfield units per second, was automatically generated by the software for the arterial input; its geometric evaluation allowed readers to assess the timing of the CTP scans in each patient, to exclude any early enhancement, and to identify correctly the end of the first pass of contrast agent, to exclude any recirculation effect in the CTP measurements.

By viewing the unprocessed CTP images in the cine-loop mode, attention was paid to drawing the regions of interest correctly to exclude peritumoral fat and large vessels in all the images of the CTP scanning. Regions of interest were chosen so that on all images they were drawn over regions of tumor throughout the image series irrespective of motion; if excessive motion artifacts precluded drawing a region of interest that stayed within tumor margins in all the images of the CTP scan, the patient was excluded from the study.

Functional maps of blood flow (BF), blood volume (BV), mean transit time (MTT), and permeability surface area product (PS) were generated according to the central volume principle, which relates BF, BV, and MTT by the equation: BF = BV / MTT. Given a linear relationship between contrast agent concentration and attenuation in CT images, deconvolution of arterial- and tissue-enhancement curves allows one to calculate the MTT (average time taken by the blood to traverse the capillary bed from the arterial end to the venous end) and the BV (the volume of circulating blood within the vasculature in a tissue region) as the area under the tissue enhancement curve divided by the area under the arterial enhancement curve. The central volume principle equation is then solved for the BF (the volume flow rate of blood through the vasculature in a tissue region).¹⁴

The PS is the product of the permeability and total surface area of capillary endothelium in a unit mass of tissue and hence is the total diffusional flux across all capillaries. It is calculated according to the following equation:

where E is the extraction fraction (the fraction of contrast medium that leaks into the extravascular space from the intravascular space).¹⁴

Tumor perfusion parameters were averaged over all sections of the regions of interest drawn for each tumor. For display purposes, the functional maps were presented by using a color scale with pixel values of BF measured in milliliters per 100 g wet tissue per minute; BV in milliliters per 100 g of wet tissue; MTT in seconds; and PS in milliliters per 100 g of wet tissue. Regions of interest were manually drawn along the visible margins of the tumor by electronic cursor in all the sections in which tumor was visible (area range, 109–479 mm²) and saved for each patient.

All 3 readers repeated CTP analysis on the same anonymized datasets with the previously saved tissue regions of interest, defining at each reading the arterial input by drawing a standardized 4-pixel region of interest in the ECA contralateral to the tumor site, selecting the section that they determined allowed the best visualization. This process was repeated for arterial input regions of interest drawn in the ipsilateral ECA. Differences between CTP measurements of the tumor obtained from arterial input region of interest placed in the ipsilateral or contralateral and in the left or right ECA were evaluated. Figure 1 shows examples of the placement of the arterial input region of interest in 2 patients.

Fig 1.

Functional maps of BF from 2 patients with oropharynx SCCA showing the BF calculated in each pixel of the image in a color scale; in the same figure are also shown the arterial inputs obtained from standardized 4-pixel regions of interest placed in the right and in the left ECA. A, Right base of tongue tumor with extension into floor of the mouth. B, Tumor involving the left tonsillar region with focal extension into the tongue base.

Statistical Analysis

The median, mean, and SD of BF, BV, MTT, and PS measurements in the study subjects were determined. Inter- and intraobserver agreement was assessed by using both the interclass correlation coefficient, with 95% confidence intervals (CI), and the Bland-Altman approach.^15,16 To obtain comparable plots for the 4 perfusion parameters, we avoided using the original data with different units of measure for each parameter and instead applied a logarithmic transformation of the data and presented the plots of the ratios between measurements against their geometric mean, as suggested by Bland and Altman.^15,16

Paired CTP measurements obtained from an arterial input region of interest placed in the ipsilateral or contralateral and in the left or right ECA were compared by using the Wilcoxon signed rank test. P values < .05 were considered significant. The analyses were performed by using SAS software (SAS Institute, Cary, NC), Version 8.2.

Results

All patients tolerated the CTP examinations well, with no side effects due to high-flow injection of the contrast agent. CTP examinations from 26 patients were considered adequate for inclusion in the study by expert reader A at the time of first analysis. Of the 4 remaining patients, 2 were excluded for excessive motion artifacts, preventing regions of interest from being drawn that remained over regions of tumor for all time points of the scan, and 2 patients with lower larynx tumors were excluded because 1 or both ECAs were not visible within the scan coverage. The end of the first pass of contrast agent (range, 32–38 seconds; average, 35.1 seconds from contrast agent administration) could be identified by all 3 readers in all 26 processed patients.

For an indication of the perfusion heterogeneity of tumors, SDs for each of the 4 CTP parameters in each patient as read by the expert reader B are presented in On-line Table 1. Examples of perfusion curves showing (as error bars) the range of values obtained in 2 representative patients with differing BF/BV number are illustrated in Fig 2. The average SD was 63.74 (20.38–140.75) for BF, 3.29 (1.96–4.98) for BV, 4.50 (1.39–11.61) for MTT, and 18.58 (6.45–34.83) for PS.

Fig 2.

Example of perfusion curves for 2 patients: with high (A−C) and low (D−F) BF and BV, for each of the observers. A and D, Expert radiologist A. B and E, Expert radiologist B. C and D, Nonexpert radiologist. The range of values within the region of interest is indicated by the error bars.

Inter- and Intraobserver Agreement

Interobserver agreement was assessed by comparing, for each pair of readers, the 2 sets of perfusion parameters for SCCA of the upper aerodigestive tract calculated from the 2 readers, for a total of 208 paired measurements per comparison and 312 total measurements taken by the 3 readers. The geometric mean of the ratios and the corresponding 95% limits of agreement ranged, respectively, from 0.96 (0.75–1.23) to 0.98 (0.87–1.11) for BF, from 0.88 (0.63–1.21) to 0.95 (0.72–1.24) for BV, from 0.96 (0.64–1.44) to 0.98 (0.76–1.27) for MTT, and from 0.85 (0.41–1.76) to 1.00 (0.77–1.31) for PS (Fig 3). The interclass correlation coefficients (ICC) were very high for BF and BV (0.98–0.99 and 0.88–0.95, respectively), good for MTT (0.77–0.94), and lower for PS (0.65–0.94) (Table 2).

Fig 3.

Bland-Altman plots of ratios between measures taken by each pair of observers versus the mean of both measurements. Top and bottom dashed lines show the 95% limits of agreement; middle line shows the geometric mean of the ratios. A, BF values (in milliliters per minute per 100 g). B, BV values (in milliliters per 100 g). C, MTT values (in seconds). D, PS values (in milliliters per minute per 100 g)

Table 2:

ICC and 95% CIs for inter- and intraobserver measurements of BF, BV, MTT, and PS

Perfusion Parameter	ICC	95% CI
Interobserver: expert reader A vs nonexperienced reader
BF	0.98	0.95–0.99
BV	0.88	0.76–0.95
MTT	0.78	0.57–0.90
PS	0.67	0.39–0.84
Interobserver: expert reader B vs nonexperienced reader
BF	0.99	0.99–1.00
BV	0.91	0.80–0.96
MTT	0.77	0.55–0.89
PS	0.94	0.87–0.97
Interobserver: expert reader A vs expert reader B
BF	0.98	0.96–0.99
BV	0.95	0.88–0.98
MTT	0.94	0.87–0.97
PS	0.65	0.36–0.82
Intraobserver: expert reader A
BF	0.99	0.98–1.00
BV	0.95	0.93–0.98
MTT	0.97	0.95–0.99
PS	0.70	0.55–0.83
Intraobserver: expert reader B
BF	1.00	0.99–1.00
BV	0.99	0.98–0.99
MTT	0.94	0.90–0.97
PS	0.87	0.79–0.93

Note:—ICC indicates interclass correlation coefficient; CI, confidence interval; BF, blood flow; BV, blood volume; MTT, mean transit time; PS, permeability surface area product.

Intraobserver agreement was evaluated from the 2 different readings of the perfusion parameters made by both expert readers A and B for a total of 208 paired measurements per reader and 416 total measurements taken by the 2 readers. The geometric mean of the ratios and the corresponding 95% limits of agreement for expert readers A and B were, respectively, 0.99 (0.88–1.12) and 1.00 (0.92–1.10) for BF, 0.97 (0.79–1.20) and 1.00 (0.88–1.14) for BV, 0.97 (0.82–1.15) and 0.98 (0.71–1.34) for MTT, and 0.85 (0.44–1.65) and 1.14 (0.70–1.86) for PS (Fig 4). High intraobserver ICC was found for BF, BV, and MTT (0.99 and 1.00, 0.95 and 0.99, 0.97 and 0.94 for experienced readers A and B, respectively), whereas lower correlation was observed for PS (0.70 and 0.87) (Table 2). The ICC was higher for intraobserver agreement than for interobserver agreement for all 4 perfusion parameters investigated, with only the exception of PS for reader B, for which interobserver agreement with the nonexperienced reader was higher than intraobserver agreement. The generally higher intra- than interobserver agreement was also confirmed by tighter 95% limits of intraobserver agreement compared with those obtained for the interobserver measurements.

Fig 4.

Bland-Altman plots of ratios between the 2 measures taken by expert readers A and B versus the mean of both measurements. Top and bottom dashed lines show the 95% limits of agreement; middle line shows the geometric mean of the ratios. A, BF values (in milliliters per minute per 100 g). B, BV values (in milliliters per 100 g). C, MTT values (in seconds). D, PS values (in milliliters per minute per 100 g).

Arterial Input Region of Interest: Ipsilateral-versus-Contralateral and Left-versus-Right ECA

No significant differences were observed in any of the 4 perfusion parameters (BF, BV, MTT, and PS) calculated for the tumor, whether the arterial input was placed in the ipsilateral- or contralateral-to-tumor-site ECA by the expert reader A or B and by the inexperienced reader (Table 3).

Table 3:

Median values and range of perfusion parameters in 26 patients, according to arterial input region of interest^*

Perfusion Parameter	Reader	Ipsilateral ECA	Contralateral ECA	P Value
BF	Expert A	84.0 (40.8–344.3)	81.5 (43.3–356.3)	.97
	Expert B	87.8 (38.4–301.8)	82.7 (33.0–246.3)	.09
	Nonexperienced	82.8 (35.2–289.3)	81.6 (39.4–284.3)	.82
BV	Expert A	6.1 (4.2–16.6)	6.1 (3.6–14.2)	.10
	Expert B	6.0 (4.2–12.9)	5.7 (3.8–12.9)	.09
	Nonexperienced	6.1 (4.4–13.7)	5.8 (4.3–14.6)	.58
MTT	Expert A	6.0 (2.9–12.4)	6.0 (2.7–12.8)	.40
	Expert B	5.8 (2.6–11.9)	6.1 (2.6–13.4)	.97
	Nonexperienced	5.4 (3.0–12.0)	5.2 (3.0–12.4)	.23
PS	Expert A	15.9 (3.7–28.9)	14.6 (2.2–46.4)	.27
	Expert B	13.4 (1.2–29.4)	13.0 (1.7–32.7)	.62
	Nonexperienced	13.9 (1.3–26.7)	12.7 (0.5–31.4)	.53

^*Wilcoxon signed rank test for the difference between paired perfusion values in right-versus-left and in ipsilateral-versus-contralateral ECAs.

No significant differences were observed for tumor BV, whether the arterial input was placed in the left or in the right ECA. Significantly higher tumor PS and MTT for 2 readers and lower tumor BF for 1 of 3 readers were observed when the arterial input was placed in the left ECA (Table 4).

Table 4:

Median values and range of perfusion parameters in 26 patients, according to arterial input region of interest^*

Perfusion Parameter	Reader	Right ECA	Left ECA	P Value
BF	Expert A	86.9 (43.3–344.3)	79.0 (40.8–356.3)	.28
	Expert B	83.9 (38.4–301.8)	83.7 (33.0–246.3)	.22
	Nonexperienced	84.0 (40.1–289.3)	81.2 (35.2–284.3)	.02
BV	Expert A	6.1 (3.6–16.6)	6.1 (4.2–14.2)	.37
	Expert B	5.9 (3.8–11.9)	5.7 (4.2–12.9)	.07
	Nonexperienced	5.9 (4.3–13.7)	6.0 (4.4–14.6)	.22
MTT	Expert A	5.7 (2.9–12.4)	6.4 (2.7–12.8)	.08
	Expert B	5.9 (2.6–11.9)	6.1 (2.6–13.4)	<.01
	Nonexperienced	5.2 (3.0–12.0)	5.4 (3.0–12.4)	.02
PS	Expert A	14.6 (2.2–28.9)	16.4 (3.7–46.4)	.03
	Expert B	13.0 (1.7–29.4)	13.4 (1.2–32.7)	<.01
	Nonexperienced	12.4 (0.5–25.5)	14.2 (1.3–31.4)	.65

Note:—ECA indicates external carotid artery.

^*Wilcoxon signed rank test for the difference between paired perfusion values in right-versus-left and in ipsilateral-versus-contralateral ECAs.

Discussion

CTP imaging has been increasingly performed for solid tumors in the body, as reflected in the increasing number of published articles on this topic. In part, this can be attributed to the developments in multisection CT technology, which has improved CTP techniques, making it possible to combine both high temporal resolution (subsecond scanning) and wide volume coverage (>40 mm in the z-axis for the latest multisection CT scanners) so that whole tumor data collection is feasible in clinical applications and allowing data analysis to be optimized.

A second factor could be the increased clinical use of the antiangiogenic drugs, which are more likely to induce cytostatic rather than cytotoxic effects,¹⁷ in which the morphologic criteria of conventional imaging for response assessment, relying on monodimensional Response Evaluation Criteria in Solid Tumors or bidimensional measurements (WHO), may not be suitable. Vascular imaging, including CTP and dynamic contrast-enhanced MR imaging, shows promise as a surrogate measure of objective vascular response, though further evidence is required to determine whether such response is informative in terms of patient survival or other clinically relevant outcomes.¹⁸ This is true, for example, for the head and neck tumors, because survival rates and clinical outcome still depend on the accurate staging of the disease, size of disease burden on the host, involvement of the eloquent areas of the head and neck, and the accurate histopathologic grading of the tumor. However, a multivariate analysis of different covariable factors, including the tumor volumetry and CTP measurements, may be needed to determine which covariable factors influence clinical outcome and survival. There is, therefore, an urgent need to characterize the reliability of CTP of head and neck tumors.

We investigated variations in CTP arising through intra- and interobserver differences and choice of arterial input in patients with SCCA of the upper aerodigestive tract because these are likely sources of random and systematic errors that may significantly affect perfusion parameter calculation, thus limiting multicentric application and wide use of CTP in the clinical routine. To the best of our knowledge, no previous study adequately assessed the inter- and intraobserver variability of CTP in SCCA of the upper aerodigestive tract. A study by Rumboldt et al⁹ examined interobserver variability in CTP measurements of SCCA, showing no statistical difference between the 2 observers; however, the hypothesis was examined by analysis of variance and no further results were provided.

In our study, we assessed inter- and intraobserver agreement by Bland-Altman plots and interclass correlation coefficients for CTP measurements in SCCA of the upper aerodigestive tract. As expected, we found that interobserver agreement was lower than intraobserver agreement for BF, BV, and MTT: the 95% limits of agreement of the Bland-Altman plots were wider for interobserver than for intraobserver measurements, whereas the highest agreement for PS was reached when comparing readings by expert reader B and the inexperienced reader. Similar to previous studies using CTP in other body tumor sites,^6–9 good inter- and intraobserver agreement was found for BF, BV, and MTT, with correlation coefficients ranging from 0.77 to 0.99 for interobserver and from 0.94 to 1.00 for intraobserver.

Due to the high inter- and intraobserver agreement, BF, BV, and MTT show potential for the assessment of SCCA of the upper aerodigestive tract. The limits of agreement within which 95% of measurements can be expected to lie were identified for the 4 perfusion parameters in our cohort. BF was the parameter that showed the lowest variability both inter- and intraobserver, with the limits of agreement covering ranges of 0.75–1.23 for interobserver and from 0.88 to 1.12 for intraobserver. For BV, the intraobserver limits of agreement were 0.79–1.20 and the interobserver limits of agreement were 0.63–1.24, whereas those for MTT were 0.71–1.34 and 0.64–1.44, respectively.

We observed lower and more variable agreement for PS, consistent with Goh et al,⁶ with correlation coefficients ranging from 0.65 to 0.94 for interobserver and from 0.70 to 0.87 for intraobserver agreement, compared with the remaining perfusion parameters. The 95% limits of agreement for the Bland-Altman plot were wider for PS than for the other perfusion parameters both for intraobserver analysis and for 2 of 3 comparisons in the interobserver analysis, showing that the variability of this latter parameter was the highest one. According to the plots, we expected that 95% of PS measures would lie between −59% and +76% for the second compared with the first reader and between −56% and +86% for the second compared with the first measurement taken by the same observer. Due to the lower and more variable inter- and intraobserver agreement, PS seems less suitable than the other parameters for tumor assessment and for therapy monitoring in our cohort.

The higher inter- and intraobserver agreement of BF, BV, and MTT than PS could relate to the different CT scanning intervals and algorithms used for their calculation: BF, BV, and MTT were calculated by using the first-pass phase, according to the central volume principle,¹⁴ whereas the PS used a prolonged CT scanning period (extending to 2 minutes in this study) and was calculated, according to the distributed parameter model.¹⁴ The PS calculation, being reliant on prolonged scanning, it likely more susceptible to the cumulative effects of small motions than the parameters derived from first-pass scanning due to the greater scope for movement of the tumor boundaries relative to the manually drawn region of interest and resulting in an imperfect tumor registration and lower inter- and intra-observer agreement. Software programs for CTP analysis with the ability to reliably correct motion artifacts are highly desirable because they could limit the impact of motion and improve the reliability in CTP measurements.

We also assessed the difference in CTP parameters of upper aerodigestive tract SCCA, calculated by using the ipsilateral-versus-contralateral ECA and the left-versus-right ECA as arterial input. Similar to the previous study by Rumboldt et al,⁹ no significant differences were found in any of the 4 perfusion parameters (BF, BV, MTT, and PS), whether the arterial input was placed in the ipsilateral or in the contralateral ECA. To the best of our knowledge, no previous study has adequately investigated the issue of left-versus-right ECA selection as arterial input. Whereas no significant differences were observed in tumor BV for all the 3 readers, significantly higher tumor PS and MTT were seen for 2 readers and lower tumor BF, for 1 out of 3 readers when the arterial input was placed in the left ECA relative to placement in the right ECA. As our cohort almost symmetrically included left and right-sided tumors (respectively 12 and 14 patients), it may be that differences between using left and right ECA as arterial input were diluted in the combined group. However, differences in the arterial inputs were not significant when ipsilateral or contralateral ECA arterial input were compared.

The significant differences in CTP parameters could be related to the specific software program used for CTP analysis in the present study. Of course, the possibility exists for additional variance to be introduced into the measurements if additional software programs were compared, even for the same model. A fundamental consideration is the choice of the kinetic model underlying the software, in our case, a distributed parameter model. It would be interesting to know whether a compartmental model applied to the same regions would be more or less sensitive to the choice of arterial input function.

Those differences could be also related to the scanning duration and temporal sampling in the present study: particularly for PS, the duration of the scanning (≤2 minutes) may greatly increase the amount of motion present during the period of the curve most relevant to determination of this parameter. Additionally, the PS is sensitive to the estimates of the curve fit during the early enhancement phase and depends on the relatively small number of time points sampled during the washout period, which adds to its variability.

Finally, the differences could, in part, be related to the different anatomy for the right ECA (branch of right common carotid artery arising from the innominate artery) and the left ECA (branch of left common carotid artery arising from the distal aortic arch), which may result in a slight arrival-time difference and, due to mixing within the arch, possibly a more blurred input function for contrast agent arriving in the left ECA compared with the right. These properties of the input function may explain the observed tendency for higher PS and MTT values and lower BF when measured using the left ECA for the input function.

The specific software used for CTP analysis, the choice of acquisition time, and the selection of arterial input are but some of the confounding factors limiting the accurate reproducibility of CTP. Our results, then, need to be verified in further studies and may not apply for other software packages and CTP protocols or for the choice of an input artery region of interest other than the ECA.

There are limitations to the study. The perilesional edema, perilesional inflammatory response, and perilesional fibrotic host response may interfere with defining a region of interest within tumor margins, giving rise to partial volume errors in the tumor perfusion measurements. It is unlikely that our readers entirely avoided these perilesional confounds, so these factors are likely to be represented in the inter- and intraobserver variability for CTP measurements in the study. Software for CTP analysis with the ability to break a region of interest into smaller ones could, in theory, reduce the impact of tumor margins on overall tumor assessment to address this issue and provide distinct assessment of the marginal zone. In the present study however, there was no chance to control noninvasively for contamination of the tumor region of interest by perilesional responses and thus evaluate the impact of those factors on CTP measurements in the study patients, all of whom were undergoing induction chemotherapy. Further studies including comparative analysis of the results of pathologic examination of surgical specimens and the CTP measurements would be the preferred approach to address this issue.

We have included a predominantly male population (male-female, 9:1) and a larger number of patients with stage IV tumors than with stage III tumors (stage IV-III, 4:1). The predominantly male patient population of the study nearly reflected the higher incidence of upper aerodigestive tract SCCA for men than women that is estimated by the American Cancer Society²⁰ and is reported in the literature.²¹ Patients were consecutively enrolled as they presented if they met inclusion criteria (clinical stages III-IVA-B SCCA) of the clinical study, of which the present investigation is a part. The lack of patients with stages I-II tumor did not allow comparison of perfusion parameters between different clinical stages. The inclusion of patients exclusively with advanced-stage tumor, which may present with necrotic areas and, then, with probably higher perfusion heterogeneity than tumors in early stages, may have potentially led into an overestimation of both the inter- and intraobserver variability for CTP measurements. Study results, then, may only apply for a similar patient population.

The small statistical sample size can lead to a recognition pattern of the cases and introduce further bias in the study. However, to our knowledge, this is the first study that adequately evaluated inter- and intraobserver variability of CTP measurements in patients with SCCA of the upper aerodigestive tract, and we had to rely on previously published experiences with different tumors that evaluated CTP measurements for interobserver variability with a similar patient sample size⁶ and for intraobserver variability after a similar⁶ or shorter time interval.²² We think that a larger patient cohort would have provided more precise limits of agreement; however, ethical constrains restrict us from enrolling more patients for this study because, at our institution, CTP is not routinely performed in all patients with SCCA of upper aerodigestive tract due to the high dose but is limited only to ones enrolled in specific protocols who meet specific inclusion criteria and have provided a written informed consent.

Our protocol for CTP with prolonged scanning ≤2 minutes may have caused excessive motion artifacts for PS calculation. The scanning and analysis protocol considered adequate for assessing PS was defined at the start of the study on the basis of a previously published work.¹⁹ In the interim, reports have been published wherein 65 seconds have been found to be adequate for PS assessment.^23,24 Such a protocol should reduce motion artifacts, and software for motion correction would likely further improve agreement in PS measurements.

As with previous studies,^1–3 regions of interest were manually drawn along tumor margins for CTP analysis in the present study, providing an average of CTP parameter value for the whole region of interest volume, because we believe that the definition of region of interest is more consistent with whole-tumor drawing and that a single large region of interest is, due to its volume, less sensitive to small variations in definition. However, we believe that, particularly in tumors with necrotic or desquamative components (which locally advanced SCCA included in the study is likely to present), the measurements of CTP parameters over a whole region-of-interest volume could under-represent the variability of perfusion characteristics that may be present in such tumors. As indicated in our results by the large SDs of the CTP measurements, the heterogeneity of the tumors is an issue that needs to be addressed better in the future. Clinically, one may currently rely on visual interpretation of the maps or perform excessively long manual segmentations, but both are difficult to propagate into reliable quantitative descriptions suitable for trials. To provide a robust assessment of tumor perfusion, software with the ability to understandably summarize a pixel-by-pixel tumor analysis and deal with segmentation of a whole-tumor region of interest into smaller ones in ways that are both reliable and clinically meaningful remains highly desirable.

Finally, although our analysis let us identify limits of agreement below which changes can be attributable to inter- and intraobserver variability, we could not find limits above which changes can reliably account for response to therapy because repeatability of CTP measurements could not be investigated. We cannot overcome this limitation because repeat CT studies would have been unacceptable to our institutional ethics committee due to the excessive dose exposure; however, we can rely on previous experiences with CTP repeatability in body tumors.^10,25,26

Conclusions

BF, BV, and MTT demonstrated higher inter- and intraobserver agreement than PS for the assessment of SCCA of the upper aerodigestive tract in our cohort, showing potential for therapy monitoring; lower inter- and intraobserver agreement of PS measurements has to be addressed for reliable clinical application in therapy monitoring. The selection of arterial input, right-versus-left ECA, may determine changes on CTP measurements in patients with SCCA of the upper aerodigestive tract because study results may suggest that further studies are highly desirable to verify this preliminary evidence.