Influence of the type of imaging on the delineation process during the treatment planning

Abstract

Aim

The aim of this study was to compare the intra- and interobserver contouring variability for structures with density of organ at risk in two types of tomography: kilovoltage computed tomography (KVCT) versus megavoltage computed tomography (MVCT). The intra- and interobserver differences were examined on both types of tomography for structures which simulate human tissue or organs.

Materials and methods

Six structures with density of the liver, bone, trachea, lung, soft tissue and muscle were created and used. For the measurements, the special water phantom with all structures was designed. To evaluate interobserver variability, five observers delineated the structures in both types of computed tomography (CT).

Results

Intraobserver variability was in the range of 1–14% and was the largest for the liver. The observers segmented larger volumes on MVCT compared with KVCT for the trachea (79.56 ccm vs.74.91 ccm), lung (87.61 vs. 82.50), soft tissue (154.24 vs. 145.47) and muscle (164.01 vs. 157.89). For the liver (98.13 vs. 99.38) and bone (51.86 vs. 67.97), the volume on MVCT was smaller than KVCT. The statistically significant differences between observers were observed for structures with density of the liver, bone and soft tissue on KVCT and for the liver, lung and soft tissue on MVCT. For the structures with density of the trachea and muscles, there were no significant differences for both types of tomography.

Conclusions

During the contouring process the interobserver and intraobserver contouring uncertainty was larger on MVCT, especially for structures with HU near 80, compared with KVCT.

1. Background

Precise delineation of target volume and organs at risk is dictated by a high quality of imaging and experience of the observer. Generally, all contours for radiotherapy planning are delineated on kilovoltage computed tomography (KVCT) scans without contrast. Improvements in KVCT technology allow accurate contouring, planning of radiation therapy (RT) and individualization of therapy for each patient. Although in some cases only simple two-dimensional (2D) images are required, the three-dimensional (3D) kilovoltage scan allows to visualize structural details of tissues without any invasion and has become the gold standard in radiation therapy. For patients with dental fillings or metal prosthesis, megavoltage computed tomography (MVCT) is used.¹ However, the alternative to MVCT can be dual energy CT (DECT) which allows to reduce metal artifacts and may significantly enhance the diagnostic value in the evaluation of metallic implants and the region around the implants. In DECT, the simultaneous adaptation of two different energies allows the differentiation of materials according to density. Familiarity with the capabilities of DECT may improve diagnostic performance.² New technologies are surely an immense technological leap forward, although not without certain risks.^3,4 The second type of imaging used in this study is MVCT acquired by the TomoTherapy machine. Helical TomoTherapy (HT) is dedicated for the intensity modulated radiation therapy (IMRT) followed by the image guidance (IG) procedures. From the technical point of view, HT is a combination of a computed tomography scanner and a linear accelerator which allows to work in two independent modes – treatment and imaging.⁵ The 6 MV X-ray beam is used in the treatment mode, while imaging mode uses the energy of the X-ray beam decreased to 3.5 MV.^6,7 At the Greater Poland Cancer Centre, for accurate positioning of patients, the registration procedures of megavoltage (MV) CT scans on HT are performed every day and adjusted to the pre-treatment kilovoltage (KV) CT scans from the diagnostic CT.

One of them is manual delineation which is time-consuming and varies between the operators.⁸ It is reasonable to expect the contouring variability in MVCT scans to be larger than in conventional KVCT scans because of increased noise and reduction of soft-tissue contrast associated with megavoltage beams used for image reconstruction.⁹

In this study, the uncertainty of delineation process was evaluated and compared between kilovoltage and megavoltage scans. Authors compared the inter- and intraobserver variability in contouring on KVCT used for radiotherapy planning with MVCT acquired with helical TomoTherapy.

A major uncertainty, which could potentially limit the benefit of radiotherapy accuracy in radiation delivery to the prescribed volume and thereby the treatment outcome, however, is the interobserver and intraobserver variability in contouring the target and organs at risk. The reduction of soft-tissue contrast while using energy in megavoltage imaging could obviously affect the observer's ability to recognize the precise borders of the analyzed structure. That is why the variability of contours in MVCT is expected to be greater than in KVCT.⁸ Additionally, the increased interest in the use of MVCT data for treatment planning requires knowledge of the electron density of the tissues and checking how the structures of different densities are delineated by different observers.¹⁰

2. Materials and methods

In the first step, a special water phantom was designed and created. Inside a water phantom of 60 cm × 30 cm × 35 cm six structures were immersed. Each of these structures had different volume, shape and electron density. All structures were placed inside the phantom according to the protocol specially developed for this study. For this protocol, the authors designed three configurations of structure location. In each configuration, the same structures were located in different places of the phantom. This represented an additional task for the observers who every time identified the structure on their own. These structures were set on the special fence, some of the fences remained empty (without any structure) just to distract the observer or check out his attention.

Fig. 1 presents the location of structures in the phantom and scans from KVCT.

Fig. 1.

Homemade phantom with immersed structures. (A) The first configuration of the structures. (B) The view of the phantom from computer tomography with different density of the structures.

The structures correspond to the Hounsfield Units (HU) of the liver, bone, trachea, lung, soft tissue and muscle. The phantom was scanned using the Somatom Sensation Open 20 CT scanner (Siemens AG, Erlangen, Germany) in the following conditions: 120 kV and 60 mAs with 2 mm slice thickness. Then, the images were exported to the treatment planning system. In order to determine the HU of the homemade structures in the treatment planning system, a calibration curve was used. Calibration curve defines the relationship between the electron density and Hounsfield Units. KVCT calibration curve was determined using Cheese Phantom (Quality Assurance Kit for the TomoTherapy^® System, Accuray). The phantom with different density plugs was scanned using 120 kV, with mAs adjusted automatically. Next, in the computer system, the regions of interest were chosen and Hounsfield Units for each plug were automatically calculated. Based on the readings, the KVCT calibration curve was marked.

To determine HU values of the structures in the homemade phantom, tools available in Eclipse software (ver. 10.0, Varian Medical Systems, Palo Alto, CA, USA) were used. HU units were read in 10 different places of the structure, then, the mean value and standard deviation were calculated. Table 1 shows Hounsfield Units for individual structures.

Table 1.

Mean Hounsfield Units values for structures included in the homemade phantom.

Structure	Mean HU	SD (HU)
Liver	56	5
Bone	1070	29
Trachea	−725	32
Lung	−645	18
Soft tissue	79	2
Muscle	73	9

Structures inside the phantom were distributed in three different configurations and scanned using KVCT and MVCT modes. Then the scans were transferred to the 3D treatment planning system (Eclipse ver. 10, Varian Medical Systems), where the structures were delineated. Five independent observers participated in the study and contoured all structures independently and separately on KVCT and MVCT scans. To establish interobserver variability, the data obtained from the delineation performed by four radiation therapists and one medical physicist were analyzed.

In order to obtain KVCT, Somatom Sensation Open 20 CT scanner (Siemens AG, Erlangen, Germany) was acquired while the MVCT scans were performed on the helical TomoTherapy machine (HT, Accuray Inc., Sunnyvale, CA, USA) in the “fine” mode, which corresponds to the slice thickness of 2 mm. During the MVCT study, a very low dose of 1–2 cGy was delivered.^11,12 The scans were reconstructed with the pixel matrix of 512 × 512 for both modalities.¹³

The next step included manual delineations of the structures on KVCT and MVCT scans performed in Eclipse Treatment Planning System (ver. 10.0, Varian Medical Systems) in the “Contouring” mode. All structures were contoured three times on KVCT and three times on MVCT by observers in each configuration (three configurations for both modalities). All observers were instructed to delineate the structures on their own set of scans under the condition that the break between two contouring sessions should be at least 1 week.^14,15 Observers did not have the access to the scans of other participants of the study. The support tools were HU scale and zooming. The variability between observers was established using in-house software by measuring the differences obtained from the volumes, shapes and locations of the structures.

To present variations in the delineation between observers, the coefficient of variation (CV) was calculated, according to the formula below:

$$C_v=\frac{\sigma }{\mu }\times 100\%$$

C_v – coefficient of variation; σ – standard deviation; μ – mean volume for all observers.

Additionally, the mean volume of structures and structure ratio, defined as the volume of the structure delineated by each observer divided by the mean volume for all observers, were calculated for each configuration for both sets of images. The mean volume delineated by observer was compared with the actual volume of each structure. Because of the irregular shape of the structures and no information about the electron density of each structures, the actual volume was measured using a cylinder with water. The objects were fully submerged in the cylinder and, then, the values on the scale were read. Next, the second volume was subtracted from the first one. The measurements were repeated three times and the mean actual volume was calculated. The variations were compared using the individual volume of the structure for observer, common volume and standard deviation. Phantom data were imported to the special software created at the Greater Poland Cancer Centre using MATLAB program to perform detailed analysis. The volume of each delineated structure was calculated using interpolation on 1 mm³ grid in the developed program.¹⁶

The comparison of all structures between observers was made using the analysis of variance for variables related. Where there were any statistically significant differences, the analysis was subjected to POST-HOC Tukey tests.

3. Results

Table 2 shows the mean volumes obtained from three configurations of the structures for each observer and for delineations performed on the KVCT and MVCT scans of the structures. The average observer was calculated as mean volume of structure delineated by all observers. Additionally, standard deviation of the mean volume and coefficient of variance for all observers for each structure are presented.

Table 2.

Mean volume of the structure for average observer, standard deviation of the mean volume (SD) and coefficient of variance (CV) for all observers for each structure.

	Mean volume	SD	CV		Mean volume	SD	CV
	(ccm)	(ccm)	(%)		(ccm)	(ccm)	(%)
KV	Liver			MV	Liver
99.38	8.02	8.07	98.13	4.49	4.58
Bone			Bone
67.97	2.51	3.69	51.86	7.16	13.81
Trachea			Trachea
74.91	2.87	3.83	79.56	3.38	4.25
Lung			Lung
82.5	2.98	3.61	87.61	6.32	7.21
Soft tissue			Soft tissue
145.47	6.00	4.12	154.24	8.08	5.21
Muscle			Muscle
157.89	4.82	3.05	164.01	6.50	3.96

The volumes on the KVCT were smaller for the trachea, lung, soft tissue and muscles compared with the MVCT result. For the structure with electron density corresponding to the liver, the volume of the structure was higher for KVCT than for MVCT. Delineated volumes of the bone density were higher for KVCT, but the CV was higher for MVCT. The highest discrepancy for the immersed structure was found in the cranio-caudal directions. The coefficient of variation was the largest for the structure with density of the liver (KVCT) and bone (MVCT) and the lowest for that with density of the muscles for both modalities. On KVCT, SD value was in the range from 2.51 (bone) to 8.02 ccm (liver). The maximum variance (CV) between observers was found for the liver, the minimum for the muscle. On MVCT, standard deviation (SD) was minimum for the trachea and maximum for the soft tissue. Surprisingly, the highest variability was found for the bone and the lowest for the liver, opposite to the KVCT.

Table 3 shows the structure ratio per observer and structure. Structure ratio was defined as the volume of the structure determined by each observer divided by the mean volume of all five observers. The volume defined by each observer was normalized to the KVCT/MVCT average observer volume. The value of the structure ratio indicates how close the delineated volume is to the average observer volume.

Table 3.

Structure ratio per observer and structure.

Structure	KVCT					MVCT
	Obs. 1	Obs. 2	Obs. 3	Obs. 4	Obs. 5	Obs. 1	Obs. 2	Obs. 3	Obs. 4	Obs. 5
Liver	0.87	1	1.09	1.02	1.02	0.95	0.96	1.05	1.03	1.01
Bone	1.04	1.03	1.01	0.99	0.94	0.97	1.07	0.78	1.14	1.04
Trachea	0.94	1.02	1.03	1.02	0.99	0.94	1.05	0.98	1.01	1.02
Lung	0.94	1.02	0.99	1.03	1.02	0.92	1.12	0.98	1	0.98
Soft tissue	0.95	0.98	1.06	1.01	1	1	0.92	1.05	0.99	1.04
Muscle	1	0.97	1.05	1	0.98	0.99	0.95	1.06	0.99	1

Obs. 1 – observer 1.

Fig. 2 shows the relation between the volume of structures on KVCT (blue), MVCT (red) for each observer and the actual volume (green). In most cases, the volume on KVCT is very similar to the actual volume. For the trachea, lung, soft tissue and muscle, the volume was different for each observer, but was very close to the actual volume and fell within the SD confidence interval. Because each structure was contoured three times by each observer for both types of imaging, the authors could calculate the individual standard deviations present in Fig. 2.

Fig. 2.

Mean volume of the structures (with SD) for each observer on KVCT and MVCT and actual volume.

The statistically significant differences between observers were observed for structures with density of the liver, bone and soft tissue on KVCT and for the liver, lung and soft tissue on MVCT. For the structures with density of the trachea and muscles, there were no significant differences for both types of tomography. Those with density of the liver demonstrated a statistically significant difference on KVCT and MVCT. Post hoc analysis showed the greatest difference between observers 1 and 3 on KVCT. In the case of the bone, the differences were significant only for KVCT versus MVCT. Observer 5 differed significantly from observers 1 and 2. For the structure with the trachea density, there were no statistical differences for both types of imaging. The contours of the lung were statistically significant on MVCT and the difference was the largest between observers 1 and 2. The statistical differences were significant also for KVCT and MVCT for the soft tissue. On KVCT, significant differences were found between observer: 1 vs. 3, 2 vs. 3; however on MVCT, observer 2 differed significantly from 1, 3 and 5. For the muscle, there were no significant differences either in the case where KVCT or MVCT was used.

The interobserver and inter-structure differences are presented below (Table 4).

Table 4.

Inter-structure variability between the volumes for delineation performed by all observer and for both modalities of imaging (KVCT and MVCT). p-Value resulted from analysis of variance for variables related. Test performed at α = 0.05.

Structure	p-Value
	KVCT	MVCT
Liver	0.027	0.040
Bone	0.012	0.056
Trachea	0.113	0.419
Lung	0.127	0.013
Soft tissue	0.007	0.004
Muscle	0.128	0.525

The mean coefficient of variation (CV) within the same examiner (intraobserver CV) was in the range of 1–14% and the results are shown in Table 5.

Table 5.

Intraobserver variability (CV) on KVCT and MVCT for each structure (%).

Structure	Obs. 1		Obs. 2		Obs. 3		Obs. 4		Obs. 5
	KVCT	MVCT	KVCT	MVCT	KVCT	MVCT	KVCT	MVCT	KVCT	MVCT
Liver	2.02	3.75	13.75	4.55	3.74	3.79	1.48	1.69	1.91	3.72
Bone	1.20	3.59	3.04	0.91	4.13	7.09	4.09	12.69	5.40	3.00
Trachea	2.24	0.28	3.12	7.32	4.89	3.13	3.90	2.64	5.76	10.89
Lung	5.20	1.65	4.13	6.13	5.13	2.86	3.33	8.24	3.20	4.63
Soft tissue	3.30	7.12	3.33	8.46	2.38	7.21	0.83	11.23	1.80	4.29
Muscle	5.61	4.34	1.49	10.24	3.14	8.15	5.48	1.10	1.72	7.91

Obs. 1 – observer 1.

The intraobserver variability was dependent on the individual observer and delineated structure. Very low variability characterized observers 1 and 3. Intraobserver variability was higher on MVCT compared with KVCT.

4. Discussion

The analysis for every single observer confirmed that the delineation process is dependent on the observer. According to observer's knowledge of radiological anatomy, the level of the delineation uncertainty can be different. The authors observed that interobserver and intraobserver delineation uncertainty was higher in MVCT compared with standard KVCT. It is the shape of the structure that had a significant impact on the difference between the observers and the volumes of the structures. For the bone and soft tissue, which have a complex shape, differences between observers were the highest. For the bone, a complicated shape of the structure with many thin parts could be a significant hurdle, requiring high accuracy during contouring. In contrast to the above case, the volume on KVCT for the trachea was smaller than on MVCT. This is surprising because the electron density of the structure is close to that of air and the structure is well demarcated also for KVCT and MVCT computed tomography. In the case of the liver, it was very difficult to determine the main trend. Observers 4 and 5 delineated the volume very close to the actual volume for both types of imaging. However, observers 1 and 2 delineated the volume as much smaller on KVCT and MVCT compared with the actual volume. The volume of the lung only for observer 2 on MVCT was significantly higher than KVCT. Lungs are the structure with a very good contrast to the soft tissue (water in the phantom), and therefore they were easy to delineate. In spite of its density being close to that of the soft tissue, the volume of the muscle was delineated higher on KVCT and MVCT compared with the actual volume. However, the differences between the observers were very small and the mean volumes were very similar. Observer 2 was characterized by a relatively high variability – he had the least experience in the contouring.

The difference of delineation should be controlled and observed, because the errors during the contouring process have a negative influence on radiotherapy success and estimation of doses to organs at risk.

This is very important because nowadays advanced technologies in radiation therapy can lead to improvements in treatment results. The use of new methods requires precision of contours, which is the basis for the calculation of dose distribution. Errors during contouring and significant differences between observers may have an impact on the course of radiation therapy.¹⁷ The target area is one of the unstable regions in the body during RT. The interobserver contouring uncertainties of organs with electron density similar to organs at risk in human body were measured using KVCT and MVCT.^18,19 Additionally, the subjective image quality for perception and delineation of structures was assessed. This paper provides an opportunity to explore differences of contouring structures with different densities in a water phantom. The authors intended in this study to demonstrate the changes in position, shape and volumes of the structures between the MVCT and KVCT tomography and between the observers. This knowledge will potentially contribute to the development of standards for contouring and responding to any discrepancies. Based on the literature, it can be concluded that the differences between the observers will affect the PTV margin and IGRT.²⁰ But there is no consensus in the literature regarding the evaluation of interobserver variability.²¹

According to other reports (Geets and Loo), the cranial and caudal directions of the structures presented the largest variations.^22,23

The intra- and interobserver variability is well known in literature.^{8,14,15,18,22,23} Similar results were obtained in the publication by Song et al.¹⁵ Song et al. compared the delineation process for the prostate on KVCT and MVCT. The uncertainty in prostate contouring (prostate has the HU value near the “soft tissue” and “muscles” in this study) was higher on MVCT vs. KVCT. The prostate volume was higher on MVCT scans by nearly 10% than KVCT. In our study, the results (volumes) for structures with similar HU were also higher on MVCT: for soft tissue the volume was higher by 6%, for muscles by 4%. The study confirms the limitation of using MVCT in the contouring process during the treatment planning.

Loo et al.²³ confirmed that the review session and standardization contouring procedures might in the future reduce the intra- and interobserver variability. However, the delineation of structures is always related to subjective perception.

There are many aspects which could reduce the variability between observers. In the standardization delineation procedure, automatic delineation tools can be helpful. However, these tools require careful validation. The variability between observers should be constantly monitored to ensure and improve the accuracy of treatment.¹⁵

In conclusion, due to the limitations mentioned above, authors do not recommend the use of MVCT in a routine treatment planning process. KVCT imaging allows anatomical structures to be visualized in more detail in the target region. Differences in delineation among the observers were caused by difficulties in distinguishing organ boundaries on the KVCT and MVCT.

Authors observed that interobserver contouring uncertainty was higher in MVCT compared with conventional KVCT scans. It should be noted that the level of noise in the MVCT was 4% while in the KVCT only 2%.^24,25 This parameter has a major impact on image quality.

As expected, the structures’ volumes defined on MVCT were larger than volumes on KVCT. The main reason was the reduction of soft tissue contrast and the fact that the variability depended on the density of structures. The structures with extreme HU density (i.e.: lung, trachea) and regular shape were contoured by each observer as very similar (the differences were very small). Data acquired by TomoTherapy (MVCT) into the treatment process require further investigation and larger number of studies. Still, the most important factor for optimal contouring of structures is the experience in systems integrated with computed treatment planning systems.

Advantages and validation of the developed program on MVCT scans make it possible to verify the position of the patient during treatment session and allow to check the dose in delineated volumes. The lack of significant differences in volumes contoured on the MVCT scans in comparison to KVCT makes them a useful tool for adaptive radiotherapy and dosimeters outside the therapeutic field.^26,27

This study gives promising results for adaptive radiotherapy (ART) and image guided radiotherapy IGRT procedures performed on TomoTherapy. Its main advantage is to escalate the dose in the target and to minimize its margin for better protection of normal tissues surrounding the planning target volume (PTV).^28,29

5. Conclusions

The inter- and intraobserver delineation variability was larger for the MVCT vs. KVCT, because of the reduction of soft tissue contrast. Although MVCT was inferior to KVCT, the application of MVCT in radiotherapy treatment planning remains useful. Type of imaging is particularly important because it affects the ability to visualize the structures. This study shows that only in some cases the MVCT images can be useful in the treatment planning process for structure delineation (i.e. dental fillings or metal prosthesis). For some electron density (similar to the soft tissue) and for complex shape of a structure, we should use the MVCT with caution during delineation and verification procedures. The manual delineation of structure with HU of about 80 is particularly difficult using MVCT.

Conflict of interest

None declared.

Financial disclosure

The publication was written under a grant “Analysis of the anatomic variations of the salivary gland (the position, size, shape, volume) between the kV and MV images in patients with head and neck cancer treated with helical tomotherapy”.