Intraobserver Variability in the MR Determination of Tumor Volume in Squamous Cell Carcinoma of the Pharynx

Abstract

BACKGROUND AND PURPOSE: If tumor volumes are to be used for evaluating responses to treatment and long-term outcomes of patients with primary pharyngeal carcinomas, the reproducibility of these measurements must be established. We determined the intraobserver variability of MR imaging–based volume measurements of these cancers and their regional metastases.

METHODS: We used an interactive computer program (IDL) that enables the extraction of tumor volumes from 3D MR data to obtain 202 volume measurements in 17 patients with pharyngeal carcinoma (two to five time points each). The primary cancer and largest nodal mass were manually outlined on every T2-weighted image of each MR study. The same neuroradiologist reanalyzed this MR dataset 2–41 weeks later. Measurement error and percentage measurement error (intraobserver variability) were determined. Differences in intraobserver variability between primary lesions and nodes, as well as between stages of treatment were tested with a Wilcoxon rank sum test.

RESULTS: The mean and median percentage measurement errors, respectively, were 13% and 12% (range, 0–53%; 95% CI: 10%, 16%) for primary tumors and 9% and 7% (range, 0–37%; 95% CI: 7%, 12%) for nodal metastases. The difference in the percentage measurement error between primary lesions and cervical nodes approached statistical significance (P = .07). Differences in the variation of volume measurements based on the stage of therapy were significant (P = .01).

CONCLUSION: Our results suggest that MR imaging–based tumor volumes are reliably reproducible. Such measurements may be important in predicting patient outcome, determining appropriate therapy, and conducting patient follow-up.

According to the Cancer FactBook of the National Cancer Institute, squamous cell carcinoma of the head and neck was responsible for almost 12,000 deaths in 2001 (1). Patients with head and neck cancer have an estimated overall 5-year survival rate of 40–60% (1). Therapeutic options usually include surgery or irradiation or both. However, concurrent chemotherapy and radiation therapy is increasingly used to treat patients with this cancer. Specific management depends on the stage of the tumor, the patient’s preference, the patient’s general underlying medical condition, as well as the institution where the patient will be treated. To tailor therapy and maximize the potential impact of treatment options, it is necessary to have prognostic factors and a reliable means by which to accurately stage the disease before treatment. Also needed is a quantitative measure to evaluate the response to therapy.

The literature suggests that pretreatment tumor volumes may be reliable predictors of both local control and overall survival in patients with head and neck cancer treated with irradiation (2–12). In general, larger lesions have poorer control rates, although the critical volumes necessary for local control are variable and depend on the location of the primary tumor (2–8). In addition, investigators have suggested that tumor volumes may also be useful in predicting local control in laryngeal carcinoma treated with surgery (13, 14).

Tumor volumes may serve as not only prognostic variables but also quantitative measures of the response to therapy. Currently, both clinical assessment and cross-sectional imaging are used to monitor responses to treatment. Clinical determination of tumor size alone is more subjective and requires sophisticated equipment combined with a high level of expertise, which may not be available away from specialized medical centers (8, 15–17). While mucosal lesions are well visualized endoscopically, clinical examination does not permit adequate assessment of many head and neck cancers because of the submucosal extension of disease.

CT and MR imaging are currently the most accurate and reproducible methods available to measure the response of cancers to therapy (17). Tumor size is usually determined by measurements made in one or two dimensions (17, 18). More recently, tumor volumes determined by using CT measurements have been used to assess tumor burden. Although CT is effective in identifying submucosal disease, inaccuracies in defining tumor boundaries are common because edema and tumor can be difficult to distinguish and because the attenuation of tumor and that of adjacent normal structures (eg, muscle) frequently overlap (19–26). MR imaging is reportedly more accurate than CT in assessing extension of the primary tumor (15, 19–21, 27). Like CT, MR imaging may not be reliable in differentiating edema from tumor (19, 28); however, it is more sensitive in distinguishing abnormal tissue (edema and neoplasm) from normal adjacent structures (15, 20, 21, 27).

If tumor volumes are to be used in clinical practice and in prospective studies as a standard for determining responses to treatment and, potentially, long-term outcomes, it is necessary to establish the reproducibility of these measurements (29, 30). The purpose of this study was to examine the intraobserver variability of MR imaging–based tumor volumes in squamous cell cancers of the pharynx. We studied volume measurements of primary tumors and cervical nodal metastases before and during treatment (irradiation or chemotherapy or both).

Methods

We studied serial MR images of 17 consecutive male patients aged 34–72 years (mean, 54 years), with newly diagnosed, pathologically proved, squamous cell carcinoma of the pharynx (Table 1). Sixteen patients had cervical nodal metastases (Table 2). The patients were taken from a cohort enrolled in a prospective National Institutes of Healthy study analyzing MR spectroscopy to predict the responses of head and neck cancer to therapy. All but two patients underwent MR study before treatment followed by one to four additional MR studies during treatment.

TABLE 1:

Characteristics of primary cancers

Patient (No.)	Tumor Location	Stage	MR Imaging Time Points	Percentage Measurement Error, %
1	Base of tongue	T4	2	23*, 19
2	Pyriform sinus	T4	3	18, 20, 16
3	Base of tongue	T3	5	5, 7, 12, 17, 2
4	Base of tongue	T4	1	0
5	Tonsil	T4	3	1, 2, 10
6	Tonsil	T3	4	1, 3, 22†, 29†
7	Base of tongue	T3	0‡	None
8	Tonsil	T4	3	6, 15, 53†
9	Base of tongue	T2	4	13, 8, 37*, 11
10	Base of tongue	T3	2	16, 11
11	Base of tongue	T2	3	1, 3, 12
12	Base of tongue	T2	3§	29*, 22*, 2
13	Base of tongue	T3	3	4, 8, 19
14	Base of tongue	T4	3	9, 3, 22†
15	Pyriform sinus	T4	3	7, 2, 23†
16	Tonsil	T2	3	0, 13, 26†
17	Base of tongue	T2	3	4, 4, 2

^*Percentage measurement error was greater than 20% because of difficulty in differentiating the neoplasm from adjacent lymphoid tissue at the base of the tongue.

^†Percentage measurement error was greater than 20% because of a small absolute differences in the volume measurements of small lesions.

^‡The primary tumor was too large to measure.

^§T1-weighted images were used because of artifact on T2-weighted images.

TABLE 2:

Characteristics of regional nodal metastases

Patient (No.)	Stage	Nodal Level	MR Imaging Time Points	Percentage Measurement Error, %
1	N0	None	0*	None
2	N2c	L II/III	4	14, 13, 11, 11
3	N2b	R II	5	2, 1, 3, 9, 31†
4	N2c	R II/III	3	2, 7, 3
5	N2c	R II/III	3	1, 2, 10
6	N2b	L II-IV	4	1, 1, 3, 8
7	N2c	R II/III	3	14, 26‡, 4
8	N2a	R II	3	5, 1, 4
9	N2b	R II	4	4, 12, 0, 11
10	N2b	R II/III	2	3, 30†
11	N2a	L II	3	4, 1, 1
12	N3	R II	3	17, 16, 27*
13	N2b	R II	3	3, 17, 17
14	N2c	L II	4	4, 16, 13, 19
15	N3	R I-V	3	3, 7, 20
16	N2b	R II	3	4, 9, 37†
17	N2c	L II/III	3	9, 6, 4

^*No nodal metastases.

^†Percentage measurement error was greater than 20% because of small absolute differences in the volume measurements of small lesions.

^‡Percentage measurement error was greater than 20% because of inaccuracies in the volume measurements due to patient motion.

MR images were obtained on the same 1.5-T system (Signa; GE Medical Systems, Milwaukee, WI). Imaging consisted of conventional spin-echo imaging with an anteroposterior neck coil. Acquisition consisted of 5-mm-thick interleaved sagittal T1-weighted images (TR/TE/NEX, 600/10/2), axial T1-weighted images (600/10/2), and axial fat-suppressed fast spin-echo T2-weighted images (4000/100/2). Other imaging parameters included a 24-cm field of view, a 256 × 128 matrix for sagittal and axial T1-weighted images, and a 256 × 192 matrix for axial T2-weighted images.

MR images were transferred to an interactive computer program (IDL) developed at our institution that enables the extraction of tumor volumes from the 3D MR data (31). Calculations with this program were previously described (31) and were used to estimate the volume of spherical phantoms to within 1% of their true volume. A neuroradiologist (L.A.L.) reviewed the images obtained with all sequences and then used the sequential, axial T2-weighted images to outline the primary tumor by using a mouse-controlled cursor (Fig 1). The entire area of abnormal T2 signal intensity in the region of interest was included in the volumetric analysis. At a separate reading, the same method was used to assess the largest nodal mass (Fig 2). The computer program calculated the volume by only counting the voxels included within the outlined regions, with a further correction algorithm used for partial volume effects (31). The voxel volumes generated (in the x, y, and z dimensions) were then converted to physical units in cubic centimeters by using the following equation: (xFOV/xRES) × (yFOV/yRES) × thickness = (24 cm/256) × (24 cm/256) × 0.5 cm = 4.3945 × 10⁻³ cm³ = one pixel, where FOV is the field of view and RES is the resolution.

Fig 1.

MR imaging–based volumetric analysis of a carcinoma on the left side of the base of the tongue in a 54-year-old man.

A and B, Two sequential axial T2-weighted MR images (4000/100/2) were used to outline the primary tumor on each sequential axial image by using a mouse-controlled cursor. In this case, the neoplasm (outlined) is hypointense relative to the lymphoid tissue (L) of the lingual tonsil.

C, Serial axial T2-weighted MR images (4000/100/2) show all images where a primary tumor is present, and the outlines of the tumor on each section. A computer program calculated the volume by counting voxels included within the outlined regions.

Fig 2.

MR imaging–based volumetric analysis of a regional nodal metastasis in a patient with a primary cancer at the base of the tongue.

A and B, Two sequential axial T2-weighted images show how the nodal mass was outlined within a mouse-controlled cursor.

C, Serial axial T2-weighted MR images (4000/100/2) show all images on which nodal metastases are outlined and the outlines on each image.

Images from each MR study were reviewed twice by the same neuroradiologist subspecialized in head and neck imaging who was responsible for all such studies at our institution. To minimize potential learning effects, the primary tumors and largest nodal mass in each patient were measured at two independent readings separated by 2–41 weeks (mean, 20 weeks). During a reading for a single patient, images from every MR study (before and during therapy) were reviewed so that measurements were consistent. (All potential errors were included at all time points.) The results of the first review were not available at the second reading. A total of 202 volume measurements were obtained for 16 primary tumors and 16 nodal masses; each was measured at two to five time points and analyzed at two separate times (Tables 1 and 2).

For statistical analysis, measurement error was calculated as the absolute value of the difference between the measured volumes of the same lesions at the two readings. The percentage measurement error (intraobserver variability) was determined by dividing the measurement error by the average of the two measurements for each lesion (Tables 1 and 2). The measurement error is reported as a percentage, rather than an absolute difference, to incorporate our results into lesions of all sizes. The mean, median, standard deviation, and 95% confidence interval (CI) were calculated for the primary cancers and the nodal masses. Differences in measurement error between primary lesions versus nodes and between different stages of treatment were tested with a Wilcoxon rank sum test. For primary lesions, the effect of lesion subsite in the pharynx on measurement error was tested with a Kruskal-Wallis rank sum test. Nonparametric tests were used because the data were not normally distributed. For all statistical tests, significance was defined as P < .05.

Results

The measured volume of the primary cancers were 0.6–65.4 cm³ (average of the two readings) with a mean and median of 14 and 6.9 cm³, respectively. The measured volume of the nodal metastases was 0.24–127.4 cm³, with a mean and median of 12.9 and 6.2 cm³, respectively.

The percentage measurement error for primary tumors was 0–53%, with a mean of 13% and a median of 12% (95% CI: 10%, 16%). The percentage measurement error of 53% corresponded to an absolute measurement error of 3.02 cm³. When this measurement was categorized as an outlier, the next largest percentage measurement error for primary cancers was 37%. The percentage measurement error for the cervical nodes was 0–37%, with a mean of 9% and a median of 7% (95% CI: 7%, 12%) (Table 3).

TABLE 3:

Percentage measurement error for primary pharyngeal cancer and regional nodal metastases

Percent Measurement Error	Primary Tumor	Nodal Mass
Mean, %	13	9
Standard deviation, %	11	9
Median, %	12	7
95% CI, %	10,16	7,12
Range, %	0,53	0,37

No statistically significant difference was found for the variation in primary tumor volume measurements based on the subsite in the pharynx (base of the tongue, tonsil, and pyriform sinus) (P = .35). The variability of volume measurements based on stage of therapy was statistically significant (P = .01). Pretreatment volume measurements had significantly less variation than measurements made during therapy (mean, 7% vs 13%). The difference in the percentage measurement error between the primary lesions and cervical nodes approached statistical significance (P = .07).

Discussion

Tumor volumes provide quantitative measures that can potentially be used to select treatment for patients with cancer of the head and neck, as well as to predict patient outcomes. Specifically, investigations suggest that pretreatment tumor volumes may help predict what therapy offers the best likelihood of cure for the patient (2–4, 7–14, 23, 32–34). Furthermore, responses reflected by objective changes in tumor volume during the course of treatment may determine if a change in therapy is indicated (17).

Some investigators are using tumor volumes to predict local control and survival rates in patients with head and neck cancer treated with irradiation (2–12, 33). An association between tumor volume and the number of clonogenic cells that must be sterilized by radiation has been reported (9, 11, 35). Currently, CT is the imaging technique predominantly used in the volumetric analysis of primary tumors in cancer patients (2–4, 6, 8, 9, 11–14, 23, 33, 36, 37). Some have suggested threshold volumes in head and neck cancer based on subsite; above these thresholds, local control rates are poor (3, 4, 6, 8, 23). Tumor volumes may also help predict the local control of laryngeal carcinoma treated with surgery (13, 14). The few studies that have used MR imaging–derived tumor volumes have also shown an association between tumor volume and regional control (5, 7).

The role of cervical nodal volumes as a prognostic factor for head and neck cancer treated with radiation therapy has been more variable (8–12, 33). Most studies combined the volumes of the cervical nodes with those of the primary lesion, and the nodal mass was not assessed independently. The volume of cervical nodal metastases may prove to be an independent predictor of local control or patient survival or both (8, 10, 12, 33). This issue is clinically important because many patients with cervical nodes larger than 2–3 cm who undergo neck dissection after radiation therapy (because of uncertainty of control) may be able to avoid unnecessary surgery. To our knowledge, studies of MR imaging–based volume measurements to specifically examine regional metastases have not been performed.

Many methods have been used to calculate tumor volumes from cross-sectional images. Most of these systems involve manually tracing a tumor on hard-copy images, digitizing the outlines, and then transferring the images to a computer where the summation-of-areas technique or a cuboid formula are used to calculate the volume (2–6, 8, 9, 11–14, 33, 37). Volumes generated from CT are generally within 10–20% of the true tumor volumes (3, 4, 6, 13, 23). We used a computer program that generates accurate measurements of volume directly from the 3D MR data (31). Manually outlining the region of interest on the computer allowed us to include every contour of an irregularly shaped mass in the volumetric analysis (Figs 1 and 2).

Imaging-based tumor volumes must be reproducible to define the critical volumes that affect therapeutic decision making and to enable their use as variables in predicting outcomes or in monitoring responses to treatment. The reproducibility is as important as the absolute accuracy of individual tumor volumes in detecting changes over time (36, 38). Few studies have addressed the reproducibility of volumetric measurements in head and neck cancer. Hermans et al (37) found that the pooled coefficient of variation (for all observers and sessions) in CT-based volume measurements of laryngeal tumors was 16.5–114%. Rasch et al (39) found that MR imaging–derived volumes of head and neck tumors had less interobserver variation than the volumes derived from CT (intraobserver variation was not examined). To our knowledge, no studies have expressly examined the reproducibility of volumes for regional metastases.

We specifically examined the intraobserver variability in tumor volume measurements in pharyngeal carcinoma. Interobserver variability was not a focus of this study. Our mean and median percentage measurement errors for primary tumors were 13% and 12%, respectively (95% CI: 10%, 16%) (Table 3). The degree of measurement error is important in determining if changes in tumor volume occur over the course of therapy or if the changes in measured volumes are due to intraobserver variability. At our institution, tumor response is determined both clinically and radiologically by using MR imaging or CT. Patients treated with chemotherapy and irradiation undergo CT or MR studies both during as well as 6–8 weeks after the completion of all therapy. The goal of treatment is a complete response, which is defined as a 100% reduction in the size of the primary lesion (18). In cases in which residual disease are hard to distinguish from treatment-induced changes, short-term follow-up imaging, biopsy, and/or positron emission tomography imaging may be necessary (17). Patients who are initially treated with only chemotherapy undergo CT or MR study after two cycles of chemotherapy. If the primary tumor is reduced by 50% (ie, partial response) (18), irradiation is added to the chemotherapy. When the response is less than 50% (stable or progressive disease) (18), patients go on to salvage surgery or intensive chemotherapy and hyperfractionated irradiation. On the basis of our percentage measurement error (95% CI: 10%, 16%), MR imaging–based tumor volume reductions of greater than 66% can reliably be classified as a partial clinical response at least, and standard irradiation can be added to the chemotherapy regimen. Volume reductions of less than 34% can reliably be classified as no response or progressive disease, and salvage surgery or intensive chemotherapy and hyperfractionated irradiation may be indicated. Volume reductions of 34–66% may be indeterminate, and further clinical or radiologic evaluation may be necessary.

In our study, the measurement error for primary tumors was not significantly affected by subsite in the pharynx. However, our 17 patients had 11 cancers at the base of the tongue but only four tonsillar and two pyriform sinus cancers. Because the base of the tongue often has lymphoid tissue that can be indistinguishable from neoplasm, we speculated that variations in volume measurements at this subsite were greater than those at other locations. In fact, we found that some of the tongue-base lesions that we examined had percentage measurement errors greater than 20% because of the difficulty in differentiating neoplasm from adjacent lymphoid tissue (Table 1). Therefore, a larger sample may reveal the true differences on the basis of subsite.

The mean and median percentage measurement errors for nodal volumes were 9% and 7%, respectively (95% CI: 7%, 12%) (Table 3). The difference in the variability of volume measurements between primary tumors and cervical nodal metastases approached statistical significance (P = .07). Head and neck cancers frequently have poorly defined margins and irregular contours, which can make them difficult to accurately trace. In contrast, nodal metastases are often spherical and better demarcated; therefore, their tracings may be more reliable. At our institution, patients initially treated with irradiation or chemotherapy or both subsequently undergo neck dissection when pretreatment nodes are larger than 3 cm in their maximal dimension. When pretreatment nodes are 3 cm or smaller, patients undergo neck dissection if residual nodal disease is present.

During irradiation, inflammatory tissue and edema can be difficult to differentiate from residual tumor (15, 40). This is also problematic in the early posttreatment period, when immature scars and recurrent tumors can be indistinguishable radiographically (20). Although the entire area of abnormal T2 signal intensity (including tumor and edema) in the region of interest was outlined at all time points in our study, we found that pretreatment volume measurements had significantly less variability than measurements obtained during therapy (P = .01). Differences in reproducibility based on the stage of therapy were well demonstrated in one patient who had a percentage measurement error for a primary cancer of 6% before therapy and 53% during therapy. In our retrospective review of the images, we determined that the patient had such extensive treatment-induced changes that a completely different hyperintense region might have been measured on the follow-up study; this region could have contributed to the increased variability and introduced an outlier into our results. In addition, toward the end of successful treatment, a residual primary tumor or nodal mass may be small. In these situations, the absolute-difference volume measurements between readings can be small, and the percentage measurement error is potentially large (>20%) (Table 1).

Conclusions

We assessed intraobserver variability in MR imaging–based measurements of tumor volume by using an interactive computer program and found that these measurements are reproducible. Next, it would be important to look at the interobserver variability of volume measurements with more than one radiologist experienced in head and neck cancer. Tumor volumes may provide objective measures of the extent of neoplastic disease, which can then be used for assessing the prognosis or the response to therapy. Volumes may also be factored into therapeutic decision making, including the determination of when changes in therapy are necessary.