Interobserver Variability in Target Definition for Hepatocellular Carcinoma With and Without Portal Vein Thrombus: Radiation Therapy Oncology Group Consensus Guidelines

Abstract

Purpose

Defining hepatocellular carcinoma (HCC) gross tumor volume (GTV) requires multimodal imaging, acquired in different perfusion phases. The purposes of this study were to evaluate the variability in contouring and to establish guidelines and educational recommendations for reproducible HCC contouring for treatment planning.

Methods and Materials

Anonymous, multiphasic planning computed tomography scans obtained from 3 patients with HCC were identified and distributed to a panel of 11 gastrointestinal radiation oncologists. Panelists were asked the number of HCC cases they treated in the past year. Case 1 had no vascular involvement, case 2 had extensive portal vein involvement, and case 3 had minor branched portal vein involvement. The agreement between the contoured total GTVs (primary + vascular GTV) was assessed using the generalized kappa statistic. Agreement interpretation was evaluated using Landis and Koch’s interpretation of strength of agreement. The S95 contour, defined using the simultaneous truth and performance level estimation (STAPLE) algorithm consensus at the 95% confidence level, was created for each case.

Results

Of the 11 panelists, 3 had treated >25 cases in the past year, 2 had treated 10 to 25 cases, 2 had treated 5 to 10 cases, 2 had treated 1 to 5 cases, 1 had treated 0 cases, and 1 did not respond. Near perfect agreement was seen for case 1, and substantial agreement was seen for cases 2 and 3. For case 2, there was significant heterogeneity in the volume identified as tumor thrombus (range 0.58–40.45 cc). For case 3, 2 panelists did not include the branched portal vein thrombus, and 7 panelists contoured thrombus separately from the primary tumor, also showing significant heterogeneity in volume of tumor thrombus (range 4.52–34.27 cc).

Conclusions

In a group of experts, excellent agreement was seen in contouring total GTV. Heterogeneity exists in the definition of portal vein thrombus that may impact treatment planning, especially if differential dosing is contemplated. Guidelines for HCC GTV contouring are recommended.

Introduction

Advanced radiation procedures, such as stereotactic body radiation therapy (SBRT) or hypofractionated proton beam therapy, are effective methods of achieving local control of primary hepatocellular carcinoma (HCC) (1–5). However, the efficacy of radiation therapy is highly dependent on accurately delineating the gross tumor volume (GTV) to avoid the risk of geographic miss. Conversely, over-contouring can lead to either a lower prescription dose and, thus, lower probability of local control or a higher risk of radiation-induced liver disease. Thus, accurate target definition is highly desirable.

The radiographic definition of the GTV in HCC requires accurate identification of areas of abnormality in all phases of a multiphasic computed tomography (CT) scan and/or magnetic resonance (MR) imaging. The radiographic appearance of HCC is characterized by arterial enhancement with subsequent washout in delayed venous phases (6). The definition of GTV typically represents a union of these findings. However, when tumor vascular thrombus (TVT) is included, delineation of the combined GTV can be more complicated. TVT is often better visualized in venous or delayed phase imaging, and registration of multiple imaging datasets is routinely required for optimal delineation. Additionally, the use of radiation therapy in HCC is relatively new, and most radiation oncologists did not receive training in the treatment of HCC in residency. Accordingly, in a multi-institutional study of SBRT for HCC with TVT, the variability in contour definition is expected to be high. For this reason, we performed this study to evaluate contouring variability among a group of gastrointestinal radiation oncologists and to develop a teaching tool to facilitate reproducible contouring of HCC to aid Radiation Therapy Oncology Group (RTOG) protocol 1112, a phase 3 randomized trial evaluating sorafenib with or without SBRT for locally advanced HCC.

Methods and Materials

Panelists

Eleven radiation oncology panelists were identified based on their expertise in treating gastrointestinal cancers. Each panelist was asked the number of HCC cases she or he treated per year.

Contouring

Anonymous, multiphasic, breath hold CT scans for 3 patients were identified. Images were acquired using a large-bore 16-slice CT scanner (Philips, Amsterdam, Netherlands). Arterial phase imaging was performed at 25 seconds and delayed phase was acquired at 55 seconds using 5 cc/s to a maximum dose of 200 cc) in exhale breath hold. Patient histories, as well as the official radiographic interpretation of the scans, were provided to the panelists to aid decision making in identification of the GTV (see below). The full 3-dimensional (3D) imaging datasets were provided in Digital Imaging and Communications in Medicine (DICOM) format to each participating institution and were contoured within each institution’s planning system. Panelists were instructed as to which CT dataset represented the primary contouring dataset. For case 2, an MRI was also provided, but panelists were given specific instructions not to contour directly on the MRI due to imperfect alignment of the imaging datasets and deformation of the liver and adjacent organs. Normal structures were provided to the panelists. Panelists were encouraged to contour the primary tumor and vascular structures as separate structures. Two panelists (LAD, HJM) were aided by an institutional abdominal radiologist (TKK, PS). Final contours were transferred as DICOM structure sets and were evaluated clinically using MIM software (MIM Software, Inc., Cleveland, OH) prior to transfer into the Computerized Environment for Radiation Research (CERR; a MATLAB-based radiation therapy planning analysis tool (7) (Mathworks, Natick, MA) for statistical evaluation.

Case histories

Case 1

The patient is a 70-year-old male with recurrent HCC, 7 years after receiving a living donor liver transplant for HCC, with a background of nonalcoholic steatohepatitis-related cirrhosis. The liver function was scored as Child-Pugh class A5. The multiphasic abdominal CT scan revealed a lobulated mass in the periphery of segments 7/8, with arterial enhancement. There is no evidence of portal venous or hepatic venous tumor thrombosis. Two phases of a triphasic liver CT (arterial and venous) were provided for contouring of the primary tumor.

Case 2

The patient is a 51-year-old female with HCC and left portal vein tumor thrombus, with a background of alcoholic and hepatitis C liver cirrhosis. Cirrhosis was scored as Child-Pugh class A6. Abdominal CT scan revealed a large segment 4A lesion with washout in the venous phase, with evidence of left portal vein invasion. Two phases of a triphasic liver CT (arterial and venous) and 1 arterial phase MRI dataset were provided for contouring of the primary tumor and vascular thrombosis.

Case 3

The patient is a 54-year-old male with HCC, with a background of hepatitis B and liver cirrhosis. Cirrhosis was scored as Child-Pugh class A6. The alpha-fetoprotein concentration was 150 ng/mL. Diagnostic imaging revealed a necrotic segment 8 lesion with peripheral enhancement consistent with HCC in arterial phase, with a smaller, similar adjacent satellite lesion posterior to it. In addition, there was a right portal vein branch thrombosis consistent with tumor thrombus extending to segment 7. Two phases of a triphasic liver CT (arterial and venous) were provided for contouring of the primary tumor, and the venous phase, where the HCC was hypodense relative to the adjacent enhancing liver and vasculature, was chosen as the primary dataset for radiation planning.

Contour evaluation

Contours from the panelists were imported into CERR (7). Contours were then compared for agreement. An expectation maximization algorithm for simultaneous truth and performance level estimation (STAPLE) was used, computing a proposed consensus segmentation from the observed contours (8). The S95 contour, which was defined as the STAPLE algorithm consensus at the 95% confidence level was created for the total GTV (primary plus TVT) for each case.

The absolute volume (in cm³) for each group of contours for each case was evaluated by calculating the minimum and maximum volumes created, as well as the group’s average and standard deviation. The agreement among contours was assessed using the intersection and union of the contour volumes. Additionally a generalized kappa statistic was used, yielding a value range of −1 to 1, with −1 representing complete disagreement, 1 representing perfect agreement, and 0 representing no agreement beyond chance. Landis and Koch’s interpretation of strength was used, in which a kappa value of <0.00 is poor, 0.00 to 0.20 is slight, 0.21 to 0.40 is fair, 0.41 to 0.60 is moderate, 0.61 to 0.80 is substantial, and 0.81 to 1.00 is near perfect agreement (9). The contours created by the radiation oncologist with the highest volume of cases (L.A.D.) in conjunction with a diagnostic radiologist (T.K.K.) were defined as the reference contours.

Consensus guidelines and recommendations

Individual contours, as well as the S95, were evaluated by all panelists. Discussion included the rationale for outlier contours, as well as potential solutions. The S95 was compared to the reference contours and scored using Dice’s coefficient, defined as the intersection of volumes/average volume as a similarity metric (volumes in cubic centimeters) with higher values indicating greater agreement. The S95 was then finalized as the consensus contour.

Results

Panelists’ experience

In response to the question regarding experience with radiation therapy for HCC, 3 panelists treated >25 HCC cases per year, 2 panelists treated 11 to 25 cases per year, 2 panelists treated 6 to 10 cases per year, 2 panelists treated 1 to 5 cases, 1 panelist treated no cases, and 1 panelist did not respond. Seven panelists were from the United States, 2 were from Canada, and 2 were from Asia (1 from Taiwan and 1 from South Korea).

Contour agreement

The agreement in contours for the total GTV (primary GTV plus TVT) of each case is shown in Table 1. For case 1, which had no TVT, there was near perfect agreement, with a kappa value of 0.826. For case 2, which had large portal vein TVT, there was substantial agreement, with a kappa value of 0.804. For case 3, which had branched TVT, there was again substantial agreement, although less than case 2, with a kappa value of 0.711. The variability, as measured by standard deviation of the calculated volumes, was largest for case 3. The S95 volume for case 1 is shown in Figure 1, for case 2 in Figure 2, and for case 3 in Figure 3.

Table 1.

Agreement in contours for the total GTV of each case^*

Parameter	HCC1 GTV	HCC2 GTV	HCC3 GTV
No. of experts	11	10	11
Volume maximum (cm3)	83.71	116.38	211.63
Volume minimum (cm3)	54.55	87.94	88.78
Volume average (cm3)	66.47	101.61	157.86
Volume SD (cm3)	±9.93	±9.79	±43.12
Volume intersection	45.21	52.74	51.22
Volume union	100.34	164.70	311.03
STAPLE volume	66.27	121.23	210.01
Kappa agreement	0.826 Near perfect	0.804 Substantial	0.711 Substantial

Abbreviations: GTV = gross tumor volume; HCC1, -2, -3 = hepatocellular carcinoma case 1, 2, 3; STAPLE = simultaneous truth and performance level estimation.

^*Total GTV is the primary gross tumor volume (GTV) plus tumor venous thrombus (TVT).

Fig. 1.

Case 1 shows hepatocellular carcinoma with no tumor venous thrombus (TVT). The consensus contour is represented by the thicker red line. Although the kappa statistic suggests outstanding agreement, there were still areas that were over- or undercontoured (arrows). A color version of this figure is available at www.redjournal.org.

Fig. 2.

Case 2 shows hepatocellular carcinoma with major tumor venous thrombus (TVT). (A) The consensus contour is represented by the thicker red line. The arrows highlight a panelist’s contour whose variability was related to the poor alignment of the planning computed tomograph (CT) with the magnetic resonance image (MRI; see Fig. 5). (B) A comparison of the TVT on CT in arterial phase on left, venous phase in the middle, and MRI. Note that the TVT is better visualized in the venous phase than in the arterial phase. A color version of this figure is available at www.redjournal.org.

Fig. 3.

Case 3. (A) Hepatocellular carcinoma with small tumor venous thrombus (TVT). The consensus contour is represented by the thicker red line. Note that on the sagittal view an outlier contour is readily apparent (arrow, top left). (B) Outlier contour is shown in greater detail. In the middle panel, there is a suggestion of a blush in the arterial phase. In reviewing the portal venous phase, it is apparent that there is no washout in this area, in contrast to the primary tumor. A color version of this figure is available at www.redjournal.org.

When we compared the S95 volume to the reference contour, the S95 volume was always larger than the reference contour (case 1: 66.27 vs 66.16 cm³; case 2: 121.23 vs 116.37 cm³; case 3: 210.01 vs 147.76 cm³). The similarity was greatest for case 1 and least for case 3, consistent with the variability seen in the calculated volumes (Dice co-efficients case 1 = 0.971, case 2 = 0.918, case 3 = 0.826).

TVT contoured separately or combined with primary

For case 2, with large TVT, 5 of 11 panelists contoured the thrombus separately from the primary GTV. Among the 5 panelists who contoured the thrombus separately, there was substantial variability in contour definition, with the volume identified as tumor thrombus, varying from 0.58 cm³ to 40.45 cm³. For case 3, with branched TVT, 7 of 11 panelists contoured the thrombus separately from the primary GTV, whereas 2 did not identify the TVT at all. Again, among those who contoured the thrombus separately, there was substantial variation in what was identified as thrombus, with a range of 4.52 to 34.27 cm³. However, for both case 2 and case 3, the total GTV had substantial agreement among the panelists despite the variable definition of primary GTV versus TVT.

Notable variations identified in consensus discussion

Variation 1: Lack of washout in venous phase

In case 1, 1 panelist misidentified an area of vascular arterial enhancement as HCC. However, in the portal venous phase, this area did not demonstrate washout but rather persistent vascular enhancement (Fig. 4). The panel felt this did not constitute tumor and was deemed a variation.

Fig. 4.

(A) Variation 1. A panelist misidentified a region of arterial enhancement as gross tumor volume (GTV) (A, indicated by the arrow), but there is no subsequent washout in the delayed portal venous phase (B). In the portal venous phase, the arrow shows that the contoured area is a vein. (B) Variation 2. Poor magnetic resonance image (MRI)/computed tomograph (CT) fusion. The hepatocellular carcinoma (HCC) tumor venous thrombus (TVT) contour from the planning CT aligns poorly with the MRI (A) and is difficult to visualize on the planning CT (B). The arrow shows the shift on the MRI from the planning CT. (C) Variation 3. The yellow contour represents the consensus GTV. The arrow indicates a wedge-shaped blue contour that represents a region of perfusion abnormality, incorrectly contoured as a GTV. (D) Another example of perfusion changes. Note the enhancement in the arterial phase at left and the lack of washout in the portal venous phase (middle) and delayed phase (right). A color version of this figure is available at www.redjournal.org.

Variation 2: Difficult CT/MRI fusion

In case 2, the MRI used to aid contouring did not align well with the planning CT scan due to an image acquisition in a different position, with substantial liver deformation. This poor alignment likely contributed to the variability noted in this case (Fig. 5). Some panelists “over-contoured” the HCC to account for uncertainties in image registration and liver deformation.

Fig. 5.

Consensus guidelines for workflow in the gross tumor volume (GTV) definition of hepatocellular carcinoma (HCC). Specifically, attention should be directed to the technique of imaging and identification of the appropriate dataset on which to perform contours. Other datasets should be carefully registered, focusing on anatomy and fiducials near the tumor. After completing contours, a review with diagnostic radiology is highly recommend.

Variation 3: Identification of an area of perfusion change as tumor

In case 3, 1 panelist identified a triangular zone of arterial enhancement that was narrow medially and widened laterally (Figure 4C). This area did not demonstrate venous washout and was felt by the panel to represent a perfusion defect and not a tumor.

Discussion

The use of radiation therapy in primary HCC has increased with the advent of improved radiation treatment technology. Safety and efficacy of treatment require accurate localization of the target lesion and conformal delivery that adequately covers the lesion but spares the uninvolved liver and other organs at risk. However, although a premium has been placed on technologies emphasizing high conformality of therapy and image guidance, guidelines regarding identification of the target itself are lacking. The utility of the technological advances is limited by the accuracy of target definition.

Hepatocellular carcinoma is characterized by neo-angiogenesis as the key step in carcinogenesis (10). This feature gives rise to the common characteristic of early arterial enhancement, typically heterogeneous in appearance, with washout in the delayed portal venous phase on both CT and MRI (6). MRI can add further information as, in unenhanced sequences, the lesions typically appear T1 hypointense and T2 hyperintense (10–12). However, there is variability in the radiographic enhancement pattern and general appearance as a function of both vascularity and endothelial cell leakiness (13). Tumor necrosis can add to the heterogeneity of enhancement within the tumor mass. Furthermore, a minority of HCC tumors may be hypovascular and do not display arterial enhancement but remain hypointense compared to the liver parenchyma in portal venous phase. Large HCC can have a fibrous capsule, which may enhance more slowly than the bulk of the mass but retain contrast in later portal venous phases. Diffuse type HCC can be even more difficult to evaluate in early arterial phase due to its infiltrative growth pattern, patchy appearance, and heterogeneous enhancement, which can be difficult to distinguish from the background cirrhotic liver (14). These tumors are often easiest to define in the delayed portal venous phase as they will still display delayed washout as the surrounding liver continues to retain contrast. TVT can manifest as an expansion of a portion of the main or branched portal vein, with washout (hypointense relative to adjacent vessels and liver parenchyma) in venous and delayed phase imaging. TVT can be distinguished from bland or non-enhancing (non-tumor) thrombus by diffuse or streaky arterial enhancement within the thrombus itself (15). These features and variations can thus confound the GTV definition.

In this study, there was good agreement for total HCC GTV definition, but the addition of vascular thrombus clearly increased contouring variability. The kappa statistic, as well as the evaluation of Dice’s coefficient comparing S95 to the reference contours, indicated that the introduction of vascular thrombus decreased agreement seen among the panelists. Interestingly, the variability was greatest in case 3, where the TVT was smaller, involving a peripheral branch of the portal venous system, suggesting that defining TVT was more difficult when it was subtle. Furthermore, there was substantial variability in the decision to separately contour the TVT from the primary GTV. This may have implications if differential dosing is contemplated to protect the mucosal surfaces near the portal vein.

Other sources of variation were identified in this study. Two of the notable variations stemmed from identification of areas of arterial enhancement that lacked washout. In case 1, a branch of the hepatic artery was identified as tumor. However in delayed phases, this region lacked the characteristic washout seen in the rest of the tumor, consistent with a perfusion abnormality. In case 2, a common arterially enhancing wedge-shaped abnormality was identified as tumor. Features that suggest this is a benign radiographic finding are the lack of washout in delayed phases and the triangle-shaped distribution consistent with a perfusion zone. This is a common radiographic finding and can be distinguished from tumor by the lack of washout in portal venous and delayed liver phases (Fig. 4D). In case 2, panelists had an easier time identifying the primary tumor and TVT on the MRI. However, due to differences in the liver shape and difference in patient position between acquisition of the MRI and the planning CT, the alignment of the liver in the region of the entire HCC was poor. GTV definition on the planning CT scan, therefore, required the panelists to rely on similar anatomic landmarks visualized on both the CT and MR. The challenges with MRI alignment are an important issue, as it is often easier to identify some tumors on MRI due to the complementary information it provides. It is recommended that the patient should be as close to the treatment position as possible when MR images are acquired, to improve accuracy of image registration in the region of the liver.

Some uncertainty in the precise location of the GTV is unavoidable. Some panelists suggested using a clinical target volume (CTV) to address this uncertainty as to where the microscopic extent of the tumor was on the static planning image. The magnitude of expansion to create the CTV could vary based on the treating physician’s uncertainty in identifying the GTV. It should be noted that RTOG 1112 does not recommend the routine use of a CTV expansion. The use of CTV is highly controversial, as some data suggest the risk of extensive microscopic extension beyond the GTV can be well covered with a 5-mm margin (16). However, expanding the target volume may lead to lower doses delivered to the known GTV and resultant lower probability of local control. For this reason, it may be reasonable to consider a CTV expansion on a case-by-case basis. Other panelists suggested being more generously inclusive, rather than exclusive, in areas of ambiguity. Another suggestion to adequately treat areas of ambiguity was to rely on any microscopic disease being included in the region of dose fall-off. Last, the panel suggested the use of fiducials that were both CT- and MRI-compatible could be used to aid image registration. However, one complicating circumstance of this approach is that a diagnostic MRI is frequently obtained prior to consultation with radiation oncology. Therefore, no fiducials are present, and the scan is often acquired in a substantially different anatomic position, rendering a high accuracy fusion impossible. In the future, clinical availability of deformable image registration algorithms may also help in improving the alignment of the liver from different imaging datasets when liver deformation occurs. In the meantime, efforts should be made to fuse images as close to the target region as possible, based on both the liver contour and, potentially, vascular anatomy.

One key limitation of this study is the use of scans acquired in a radiation oncology department rather than in a diagnostic radiology department. Because the timing of contrast may not be as precisely triggered in radiation oncology departments, the images do not uniformly demonstrate classic enhancement patterns and may have contributed to the variability seen in this study. However, we feel that this reflects the quality of scans that many radiation oncologist may be using for treatment planning.

The panel agreed that multiphasic imaging (and often multimodal imaging) was necessary to accurately identify both the GTV and TVT. Image registration of the liver in the region of the tumor is a key step required prior to HCC contouring. Although small numbers precluded an analysis regarding the role of aid from a diagnostic radiologist, radiation oncologists are encouraged to consult with a diagnostic hepatobiliary radiologist to aid in GTV definition given the variability in interpretation of imaging. In particular, attention should be directed to areas of questionable TVT, especially in peripheral branches of the portal venous system. TVT typically has the same imaging characteristics as the primary, with arterial enhancement and venous washout, best seen in the venous phase. Additionally, the radiographic criteria described in this study should be applied only to HCC rather than hepatic metastases as each solid tumor may have its own unique imaging characteristics. In summary, our consensus recommendations (Fig. 5) for workflow of defining the GTV incorporate these elements.

Conclusions

Among a group of experts, excellent agreement was seen in contouring the primary HCC GTV. Heterogeneity exists in the definition of portal vein tumor thrombus that may impact treatment planning, especially if differential dosing is contemplated. Furthermore the addition of TVT increases variability, which may have implications for RTOG 1112, where a large number of patients are expected to have vascular thrombus. Appropriate timing of intravenous contrast and multiphase acquisition is critical to accurately define the GTV (17). Consensus guidelines (Fig. 5) for HCC GTV contouring based on this study will be posted as an online resource. Images of the tumor in arterial and venous phases from all 3 cases, with and without contours, are included in the online supplement (see Supplementary Material Application 1). The full atlas will be posted online at the NRG Oncology website.

Summary.

A panel of 11 gastrointestinal radiation oncologists defined hepatocellular carcinoma (HCC) gross tumor volumes (GTV), using 3 anonymous datasets with various degrees of tumor venous thrombus (TVT). Variations are noted in cases with TVT and recommendations are made to reduce variations in GTV definition. These guidelines will be used in an ongoing phase 3 randomized trial evaluating stereotactic body radiation therapy in HCC.