Retroperitoneal Sarcoma Target Volume and Organ at Risk Contour Delineation Agreement Among NRG Sarcoma Radiation Oncologists

Abstract

Purpose

The purpose of this study was to evaluate the variability in target volume and organ at risk (OAR) contour delineation for retroperitoneal sarcoma (RPS) among 12 sarcoma radiation oncologists.

Methods and Materials

Radiation planning computed tomography (CT) scans for 2 cases of RPS were distributed among 12 sarcoma radiation oncologists with instructions for contouring gross tumor volume (GTV), clinical target volume (CTV), high-risk CTV (HR CTV: area judged to be at high risk of resulting in positive margins after resection), and OARs: bowel bag, small bowel, colon, stomach, and duodenum. Analysis of contour agreement was performed using the simultaneous truth and performance level estimation (STAPLE) algorithm and kappa statistics.

Results

Ten radiation oncologists contoured both RPS cases, 1 contoured only RPS1, and 1 contoured only RPS2 such that each case was contoured by 11 radiation oncologists. The first case (RPS 1) was a patient with a de-differentiated (DD) liposarcoma (LPS) with a predominant well-differentiated (WD) component, and the second case (RPS 2) was a patient with DD LPS made up almost entirely of a DD component. Contouring agreement for GTV and CTV contours was high. However, the agreement for HR CTVs was only moderate. For OARs, agreement for stomach, bowel bag, small bowel, and colon was high, but agreement for duodenum (distorted by tumor in one of these cases) was fair to moderate.

Conclusions

For preoperative treatment of RPS, sarcoma radiation oncologists contoured GTV, CTV, and most OARs with a high level of agreement. HR CTV contours were more variable. Further clarification of this volume with the help of sarcoma surgical oncologists is necessary to reach consensus. More attention to delineation of the duodenum is also needed.

Introduction

Retroperitoneal sarcoma (RPS) is rare; it comprises approximately 15% of the annual estimated incidence of approximately 12,000 soft tissue sarcoma cases in the United States (1). The only curative treatment modality for RPS is surgery. Following surgery, however, locoregional recurrence rates are high, ranging from 50% to 80% (2). Randomized trials have shown that preoperative (or postoperative) radiation therapy (RT) improves local control compared to surgery alone in the treatment of extremity soft tissue sarcoma (3-6), but similar studies have not yet been completed for RPS. Several retrospective series have reported potential local control benefits for preoperative or postoperative RT for RPS (7-14), but due to the nature of retrospective reports, these data are prone to selection bias and not conclusive. Thus, robust data do not exist regarding the role of RT for RPS. To help clarify the role of preoperative RT for RPS, the European Organization for Research and Treatment of Cancer (EORTC) is conducting a randomized trial of surgery alone versus preoperative RT (50.4 Gy) followed by surgery (EORTC study 62092-22092) (15). A similar attempt by the American College of Surgeons Oncology Group unfortunately closed early due to poor accrual (ACOSOG Z9031; NCT00091351) (16).

While we await results of this randomized trial, many sarcoma treatment teams use preoperative RT as part of initial treatment for RPS or at the time of local recurrence. Given that preoperative RT for RPS is not standard treatment, it is important to make decisions regarding such treatment in the setting of multidisciplinary consultations or tumor boards. Expert consensus guidelines have been developed for the delineation of preoperative RT target volumes for RPS including gross tumor volume (GTV), clinical target volume (CTV), planning target volume (PTV), and planning volumes incorporating internal motion (internal target volume [ITV]) (17, 18). Following development of these guidelines, we performed this study to assess target volume and OAR contouring variability among radiation oncologists provided with the consensus guideline instructions to determine whether further education and clarity might be indicated.

Methods and Materials

Participants and contouring instructions

Project participants included 12 radiation oncologists with experience treating sarcoma. Most of the participants were members of the NRG Oncology clinical research organization Sarcoma Working Group. Anonymized radiation planning computed tomography (CT) image data sets for 2 patients with RPS were distributed. The first case (RPS1) was a patient with a right upper quadrant de-differentiated (DD) liposarcoma (LPS) with a predominant well-differentiated (WD) component, and the second case (RPS2) was a patient with a left upper quadrant DD LPS with a minimal WD component. The images were obtained with patients in the supine position with their arms supported over their heads. Oral contrast was used, but intravenous contrast was not. Neither accompanying diagnostic scans nor formal reports thereof were provided. Participants were asked to download these data sets from the Image-Guided Therapy QA Center web server and import them into their contouring workstation or treatment planning system.

The following instructions were provided: (1) Contour the GTV as you feel appropriate; (2) Contour the entire “bowel bag” or peritoneal cavity using guidelines from Radiation Therapy Oncology Group (RTOG) atlas on the RTOG website (19). Contour the peritoneal space superiorly to the level of the diaphragm and include the stomach and duodenum; (3) Contour the CTV using the following guidelines (developed by the expert consensus panel): CTV reflects uniform GTV expansion of 1.5 cm with edited reduction at bone (0 mm), bowel bag, and air cavity (5 mm), renal and hepatic interfaces (2 mm), and skin surface (3 mm) (17). CTV expands fully into retroperitoneal muscles. (Please note that subsequently, the expert consensus guidelines were amended to specify CTV expansion into renal and hepatic interfaces of 0 mm and ITV terminology was substituted for CTV for cases where 4-dimensional (4D) CT planning CT images were obtained to account for internal motion) (18); (4) Contour the “high-risk CTV boost” using the following guidelines: the high-risk boost is the area considered to be at high risk for positive margins following resection. It generally includes areas of tumor located along posterior abdominal wall, ipsilateral para- and prevertebral space, major vessels, or organs that the surgeon would leave in situ. You may contour the HR CTV based on input from a surgeon. An example of a high-risk CTV boost volume was provided, (Fig. 1); and (5) Contour small bowel, colon, stomach, and duodenum.

Fig. 1.

This is the image provided to study participants illustrating an example of HR CTV boost volume, the area considered to be at high risk for positive margins following resection. It generally includes areas of tumor located along posterior retroperitoneal musculature, ipsilateral para- and prevertebral space, major vessels, or organs that the surgeon would leave in situ. The GTV is the large left upper quadrant mass outlined in red; the HR CTV boost volume is the vermiform volume outlined in blue. CTV = clinical target volume; GTV = gross tumor volume; HR = high risk.

Data analysis and statistical methods

Each participant used his or her institution's planning system to contour GTV, CTV, HR CTV, bowel bag, small bowel, colon, stomach, and duodenum. Contours were then sent back to the Image-Guided Therapy QA Center as Digital Imaging Communications in Medicine (DICOM) data sets. Individual contours were imported to the Computerized Environment for Radiation Research for analysis (20).

Statistical analysis was performed using several methods. The apparent volume overlap was assessed by calculating the intersection and mean of volumes contoured by the participating radiation oncologists. Contour data were also analyzed by the simultaneous truth and performance level estimation (STAPLE) algorithm, which uses an iterative approach to simultaneously estimate “true” contour structure (21). The STAPLE 95 contour, also called the consensus contour, is a probabilistic estimate of the true contour at a 95% confidence level. Contour agreement sensitivity is an estimate of the true positive rate, (the proportion of voxels within the true structure that are included in the consensus); similarly, specificity is the true negative rate, (the proportion of voxels outside the true structure that are excluded from the consensus) (22). The kappa statistic was used to correct for agreements that could be derived by chance. Kappa values range from – 1 to + 1, with –1 representing complete disagreement, 0 representing no agreement above chance, and +1 representing complete agreement. Descriptive categories for the strength of kappa agreement were defined by Landis and Koch as <0 is poor, 0 to 0.20 is slight, 0.21 to 0.40 is fair, 0.41 to 0.60 is moderate, 0.61 to 0.80 is substantial, and 0.81 to 1.00 is almost perfect (23).

Results

Twelve radiation oncologists with experience treating sarcoma provided target volume and organ at risk (OAR) contours. Ten radiation oncologists contoured both RPS cases, 1 contoured only RPS1, and 1 contoured only RPS2. Thus, 11 contour sets were available for each of the 2 cases. All contour sets were complete and included the 8 required contours.

The contour agreement kappa statistic values for each of the 8 structures for RPS1 and RPS2 are shown in Table 1. GTV agreement for both cases was almost perfect; CTV agreement was substantial for RPS1 and almost perfect for RPS2. However the HR CTV volume agreement was only moderate for both cases, (kappa 0.50 and 0.57, respectively.) The agreement for OARs was almost perfect or substantial for bowel bag, small bowel, colon, and stomach but only moderate or fair for the duodenum (0.41 and 0.36, respectively.) Of note, the duodenum was distorted by tumor in 1 of the cases. Figure 2 shows selected axial and coronal slices of the STAPLE 95 consensus contours for RPS1 and RPS2.

Table 1. Summary of kappa statistic agreement for RPS target and OAR volumes.

Contoured structure	Kappa agreement
	RPS1	RPS2
GTV	0.84 Almost perfect	0.92 Almost perfect
CTV	0.79 Substantial	0.86 Almost perfect
HR CTV	0.50 Moderate	0.57 Moderate
Bowel bag	0.82 Almost perfect	0.79 Substantial
Small bowel	0.73 Substantial	0.78 Substantial
Colon	0.73 Substantial	0.82 Almost perfect
Stomach	0.77 Substantial	0.83 Almost perfect
Duodenum	0.41 Moderate	0.36 Fair

Abbreviations: bowel bag = contour encompassing the contents of the peritoneal cavity to include small bowel and colon; CTV = clinical target volume; GTV = gross tumor volume; HR CTV = high-risk clinical target volume; OAR = organ at risk; RPS = retroperitoneal sarcoma.

RPS1 is a patient with a right upper-quadrant de-differentiated (DD) liposarcoma (LPS) with a predominant, well-differentiated (WD) component. RPS2 is a patient with a left upper quadrant DD LPS with a minimal WD component.

Fig. 2.

Computed tomography images show the consensus contours for RPS1 axial view (a), RPS1, coronal view (b); RPS2, axial view (c); and RPS2, coronal view (d). Note that RPS1 is a de-differentiated liposarcoma with a predominantly well-differentiated component (appearing similar to normal adipose tissue). RPS2 is a de-differentiated liposarcoma predominantly composed of a de-differentiated component (appears as solid tumor). Colors correspond to the contours: red = gross tumor volume; blue = clinical target volume; yellow = high-risk clinical target volume boost; pink = stomach; light blue = duodenum; mustard yellow = small bowel; brown = colon; and green = bowel bag.

Quantitative measurements of the contour agreement for GTV are shown in Table 2. Of note, the contour variability as demonstrated by the standard deviations was greater for all RPS1 target volumes than for RPS2 target volumes. The percentage of contour volume common to all observers is demonstrated by the intersection volume as a percentage of the mean volume and shows median volume overlap for GTV of 66.2% (RPS1) and 83.5% (RPS2) for CTV of 58.5% (RPS1) and 68.8% (RPS2), and for HR CTV of 0% (RPS1) and 9.6% (RPS2). Agreement sensitivity and specificity values are provided in Tables 2-4 and show that in all cases, the specificity values were greater than sensitivity values, and sensitivity was higher for all RPS1 target volumes compared with RPS2 target volumes.

Table 2. Summary of gross tumor volume agreement statistics for RPS1 and RPS2.

Agreement parameter	RPS1	RPS2
Maximum volume (cc)	1136.1	966.6
Minimum volume (cc)	713.1	870.8
Mean volume (cc)	936.1	919.7
Standard deviation (SD)	148.9	34.3
SD as a % of mean volume	15.9	3.7
STAPLE 95 consensus volume (cc)	1092.7	924.5
Intersection volume IV (cc)	620.1	767.8
IV as a % of mean volume	66.2	83.5
Union volume (cc)*	1222.7	1081.5
Mean agreement sensitivity (±SD)	0.83 ± 0.12	0.95 + 0.02
Mean agreement specificity (±SD)	1.0 ± 0.003	0.99 + 0.006
Overall kappa statistic (P value)	0.84 (<.0001)“Almost perfect”	0.92 (<.0001)“Almost perfect”

Abbreviation: STAPLE = simultaneous truth and performance level estimation. Other abbreviations as in Table 1.

^*Union volume is the largest volume assuming outermost contours.

Table 4. Summary of high-risk clinical target volume agreement statistics for RPS1 and RPS2.

Agreement parameter	RPS1	RPS2
Maximum volume (cc)	1333.6	974.6
Minimum volume (cc)	52.8	406.5
Mean volume (cc)	640.2	666.3
Standard deviation (SD)	316.3	216.0
SD as a % of mean volume	49.4	32.4
STAPLE 95 consensus volume (cc)	821.1	811.7
Intersection volume IV (cc)	0.0	63.7
IV as a % of mean volume	0	9.6
Union volume (cc)*	1734.5	1499.2
Mean agreement sensitivity (±SD)	0.60 ± 0.22	0.68 ± 0.19
Mean agreement specificity (±SD)	0.98 ± 0.03	0.98 + 0.03
Overall kappa statistic (P value)	0.50 (<.0001)“Moderate”	0.57 (<.0001)“Moderate”

Abbreviation: STAPLE = simultaneous truth and performance level estimation. Other abbreviations as in Table 1.

^*Union volume is the largest volume assuming outermost contours.

Discussion

GTV and CTV contours

Accurate delineation of radiation target volumes is a prerequisite for the precise delivery of radiation dose to regions of interest especially in the current era of image-guided RT (IGRT), intensity modulated RT (IMRT), proton therapy techniques and the use of small margins (often ≤5 mm) to account for patient setup error. Moreover, consistent and reproducible contouring is important to the data integrity of multi-institutional clinical trials. In this project, the GTV contour agreements among 12 sarcoma radiation oncologists for each of 2 RPS cases were “almost perfect” (kappa values of 0.84 and 0.92, respectively). This high level of GTV agreement is consistent with that found in other reports which assessed contour agreement for soft tissue sarcoma of the extremity (24, 25).

The CTV contour agreements for the 2 cases were also very good (kappa 0.79, which is substantial, and kappa 0.86, which is almost perfect) and similar to CTV agreement levels attained for sarcoma of the extremity (24, 26). Of note, instructions for CTV delineation were provided to the participants based on the first version of consensus treatment guidelines for preoperative RT for RPS as reached by an international expert panel and published in abstract form (7). These instructions specified variable CTV editing margins (2, 3, 5, and 15 mm) tailored to specific OAR. In practice, incorporation of these variable margins was time consuming and tedious. As a result of this project, the expert panel modified the consensus guidelines to make them simpler. Specifically, the 2-mm expansions into renal and hepatic interfaces were amended to 0 mm provided internal motion was taken into account at the time of simulation (eg 4DCT acquisition) (18). Because these CTV consensus guidelines were newly developed and untested, it is important to demonstrate that implementation of these complex, nonuniform expansions is achievable with a high level of agreement. It is reassuring that the CTV guidelines were understandable and that the resulting contours showed very good agreement.

As stated above, the GTV and CTV contour agreement for both RPS1 and RPS2 were very good, yet there were some consistent differences between the 2 cases. The degree of contour variability as represented by standard deviations of the volumes were larger for RPS1 than for RPS2 for both the GTV and the CTV structures (15.9% vs 3.7% and 17.3% vs 10.2%, respectively.) This is not surprising given the specifics of the cases. Histology for both cases was DD LPS, but RPS1 contained a large WD component, whereas RPS2 consisted almost entirely of a DD component. WD LPS has a radiologic (and gross) appearance similar to normal adipose tissue which makes it very difficult to accurately distinguish WD LPS from normal fat (27). Clinical judgment is required to determine how much of the normal-appearing tissue in fact represents disease. One would thus expect a higher degree of variability for RPS1 contours due to the large WD component. Oncologists experienced in the treatment of sarcoma appreciate this subtlety which explains why the contour agreement for RPS1 was still quite good. Similarly, the contour sensitivity values (true positive rates) for RPS1 were lower than those for RPS2 for both the GTV and CTV structures. These differences can be accounted for by the same reasons cited above.

HR CTV contour

The contour agreements for the HR CTV were only moderate in both cases (kappa 0.50 and 0.57, respectively.) This finding is not surprising, and the explanation is likely multifactorial. First, the concept of selective dose escalation to an HR CTV defined as the portion of tumor judged to be at risk for positive margins following resection has not been robustly defined and is, by definition, subjective. Second, although there are a few reports of this approach (28, 29), the efficacy and toxicity thereof have not been adequately tested with sufficient patient numbers and follow-up duration. Consequently, selective dose escalation for RPS is not recommended as routine treatment outside of a clinical trial (18), and therefore, several coauthors do not have familiarity defining and contouring these high-risk targets. Third, experienced sarcoma surgical oncologists were not included in this study. They would have provided additional critical input regarding particular areas at risk for positive resection margins. Nonetheless, our results showing only moderate contour agreement highlight the need for more accurate HR CTV delineation instructions and teaching tools.

OAR contours

Correct delineation of normal tissues is important for accurate toxicity assessment as well as for optimal normal tissue avoidance during the radiation planning process. The RPS OAR contour agreements were very good (almost perfect or substantial with kappa values of 0.73-0.83) for bowel bag, small bowel, colon, and stomach. These favorable results are not unexpected and are reassuring. However, the contour agreement for the duodenum was only moderate or fair (kappa values of 0.41 and 0.36, respectively.) There are several potential reasons for this. First, for 1 case, the duodenum was distorted by tumor. Second, reference diagnostic scans were not provided, so participants had only the planning CT scans to use, which might have rendered identification of the origin of the duodenum at the gastric outlet as well as the termination point at the ligament of Treitz difficult. Third, the duodenum is a small organ in which relatively small contouring differences would represent larger percentages of discrepancies than contouring differences for larger volumes. At a prescribed preoperative dose of 50 Gy, delineation of the duodenum as a structure separate from the small intestine or bowel bag may not be necessary. However, in the setting of potential dose escalation to high-risk areas of tumor, delivery of doses greater than 50 Gy to portions of duodenum may well be associated with increased toxicity (30). With this approach, precise delineation of duodenum would be critical and further clarification and attention to this structure advised.

There are several possible methods for bowel contouring, including a single-structure bowel bag, individual bowel loops, or individual loops plus a 1-cm margin, and the optimal method is not clear (15, 18, 31-33). For example, the single-structure bowel bag approach is preferred by NRG Oncology gynecologic and genitourinary cancer radiation oncologists, whereas the NRG Oncology gastrointestinal cancer radiation oncologists prefer delineation of small bowel and colon separately as depicted in the respective atlases (33). We found very good contour agreement for both approaches which demonstrates that both methods are feasible. The participating radiation oncologists agreed that the bowel bag structure is easier and faster to contour which has appeal from a work-flow perspective. For the preoperative RT RPS guidelines, the expert panel determined that either contouring approach is acceptable, but if dose escalation will be employed, contouring individual bowel loops may be preferable (18).

Limitations

A potential limitation of this project is that the participants were not provided with case histories, diagnostic imaging scans, radiology reports or consultation with collaborating surgeons. Furthermore, intravenous contrast was not used for the planning CT scans, nor were coregistered images, such as magnetic resonance images, provided. Most of these would have been available in an actual patient case and might have helped the contouring process.

Conclusions

In summary, currently, preoperative RT for RPS is not standard treatment and is the subject of an ongoing randomized trial (15). These patients are best evaluated in a multidisciplinary setting. If preoperative RT is recommended, the radiation oncologist and surgeon should work together to discuss details of the proposed surgery and to arrange for resection 4 to 8 weeks after completion of RT. This report showed that sarcoma radiation oncologists contoured RPS GTV, CTV, and most OARs with a high level of agreement. HR CTV contours were more variable. Further clarification of this volume with the help of sarcoma surgical oncologists is ongoing and necessary to reach consensus and deploy in a multi-institutional trial (34). More attention to delineation of the duodenum is also needed and would be most relevant for cases of planned selective dose escalation.

Table 3. Summary of clinical target volume agreement statistics for RPS1 and RPS2.

Agreement parameter	RPS1	RPS2
Maximum volume (cc)	2386.7	1889.0
Minimum volume (cc)	1278.5	1383.5
Mean volume (cc)	1783.8	1663.5
Standard deviation (SD)	308.0	169.6
SD as a % of mean volume	17.3%	10.2%
STAPLE 95 consensus volume (cc)	2011.9	1751.2
Intersection volume IV (cc)	1044.0	1145.1
IV as a % of mean volume	58.5%	68.8%
Union volume (cc)*	2622.6	2139.1
Mean agreement sensitivity (±SD)	0.82 ± 0.10	0.90 ± 0.07
Mean agreement specificity (±SD)	0.99 ± 0.02	0.98 ± 0.01
Overall kappa statistic (P value)	0.79 (<.0001)“Substantial”	0.86 (<.0001)“Almost perfect”

Abbreviation: STAPLE = simultaneous truth and performance level estimation. Other abbreviations as in Table 1.

^*Union volume is the largest volume assuming outermost contours.

Summary.

Twelve sarcoma radiation oncologists were given instructions for contouring target volumes and organs at risk (OARs) for 2 cases of retroperitoneal sarcoma. Gross tumor volume, clinical target volume (CTV), and most OARs were contoured with high levels of agreement. High-risk CTV (the area judged to be at risk for positive margins after resection) contours were more variable. Further clarification of this volume with the help of sarcoma surgical oncologists is necessary to improve consensus.