Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jul 6.
Published in final edited form as: J Neurosurg Pediatr. 2016 Jul 8;18(5):565–572. doi: 10.3171/2016.4.PEDS16132

A novel magnetic resonance imaging segmentation technique for determining diffuse intrinsic pontine glioma tumor volume

Ranjodh Singh 1, Zhiping Zhou 2, Jamie Tisnado 5, Sofia Haque 5, Kyung K Peck 5,6, Robert J Young 5, Apostolos John Tsiouris 3, Sunitha B Thakur 5,6, Mark M Souweidane 2,4,7
PMCID: PMC5498312  NIHMSID: NIHMS868905  PMID: 27391980

Abstract

OBJECTIVE

Accurately determining diffuse intrinsic pontine glioma (DIPG) tumor volume is clinically important. The aims of the current study were to 1) measure DIPG volumes using methods that require different degrees of subjective judgment; and 2) evaluate interobserver agreement of measurements made using these methods.

METHODS

Eight patients from a Phase I clinical trial testing convection-enhanced delivery (CED) of a therapeutic antibody were included in the study. Pre-CED, post–radiation therapy axial T2-weighted images were analyzed using 2 methods requiring high degrees of subjective judgment (picture archiving and communication system [PACS] polygon and Volume Viewer auto-contour methods) and 1 method requiring a low degree of subjective judgment (k-means clustering segmentation) to determine tumor volumes. Lin’s concordance correlation coefficients (CCCs) were calculated to assess interobserver agreement.

RESULTS

The CCCs of measurements made by 2 observers with the PACS polygon and the Volume Viewer auto-contour methods were 0.9465 (lower 1-sided 95% confidence limit 0.8472) and 0.7514 (lower 1-sided 95% confidence limit 0.3143), respectively. Both were considered poor agreement. The CCC of measurements made using k-means clustering segmentation was 0.9938 (lower 1-sided 95% confidence limit 0.9772), which was considered substantial strength of agreement.

CONCLUSIONS

The poor interobserver agreement of PACS polygon and Volume Viewer auto-contour methods high-lighted the difficulty in consistently measuring DIPG tumor volumes using methods requiring high degrees of subjective judgment. k-means clustering segmentation, which requires a low degree of subjective judgment, showed better interob-server agreement and produced tumor volumes with delineated borders.

Keywords: diffuse intrinsic pontine glioma, k-means clustering, magnetic resonance imaging, tumor volume determination, oncology

Diffuse intrinsic pontine gliomas (DIPGs) are inoperable and lethal brainstem gliomas that are found in children, with a median age of diagnosis of 6–7 years.6 DIPGs account for approximately 80% of pediatric brainstem tumors. Despite numerous efforts to improve treatment, the median survival of patients with these tumors is less than 1 year.6,10

Accurately determining DIPG tumor volume is important for therapeutic planning, monitoring disease progression, and measuring tumor response to therapy.5,18 Previous studies using MRI and semiautomated image segmentation methods to determine DIPG tumor volume showed poor interobserver agreement.5,8,17 The image segmentation methods used in these studies depend highly on subjective judgment of the observers. We hypothesized that a method requiring a low degree of an observer’s subjective judgment would produce better interobserver agreement and more consistent measurements. The aims of the current study were to 1) measure DIPG tumor volumes using methods that require different degrees of subjective judgment, and 2) evaluate interobserver agreement of measurements made using these methods.

Methods

Patients and image selection

Eight patients with radiographically confirmed diagnoses of DIPG were the subjects of this analysis. Patients participated in a Phase I clinical trial testing convection-enhanced delivery (CED) of therapeutic murine monoclonal antibody 124I-8H9 (clinical trial registration no. NCT01502917; http://clinicaltrials.gov), approved by the Memorial Sloan Kettering Cancer Center (MSKCC) Institutional Review Board. Informed consent was obtained from all patients or their parents/legal guardians. All patients were enrolled in this trial within 4–14 weeks after standard conventional external-beam radiation therapy had been administered, and in the absence of clinical and radiographic disease progression. Pre-CED, post–radiation therapy axial T2-weighted images were acquired at 3T (GE Discovery MR750; TE 99–104 msec, TR > 3000 msec, acquisition matrix size 256 × 256 [2 scans] or 512 × 512 [6 scans], voxel size 0.555–2.651 mm3).

Axial T2-weighted images were analyzed by 2 independent observers trained to identify DIPG on T2-weighted images using 3 methods, including 2 image segmentation methods that require high degrees of subjective judgment (picture archiving and communication system [PACS] polygon method and Volume Viewer auto-contour method) and a parameter-dependent method that requires a low degree of subjective judgment (k-means clustering segmentation). One independent observer (R.S.) had 2 years of experience in measuring DIPG volumes and 2 years of experience in image segmentation software. The other observer (J.T.) had 1 year of experience in measuring DIPG volumes, 2 years of experience in neuroradiology, and 1 year of experience in image segmentation software. Tumor volume assessment was overseen by a board-certified radiologist (S.H.) holding a certificate added qualification in neuroradiology (11 years of neuroradiology experience).

Image Segmentation Methods Requiring High Degrees of subjective Judgment

Tumor volumes were measured using the PACS polygon method (Centricity PACS 4.0 SP3, GE Healthcare) and the Volume Viewer auto-contour method (Advantage Workstation V11.3–4.2, GE Healthcare). For the PACS polygon method, observers manually outlined the tumor and the software determined the tumor area per slice. Observers then added the areas from consecutive slices and multiplied the sum by slice spacing to derive total tumor volume (Fig. 1).

Fig. 1.

Fig. 1

Open in a new tab

PACS polygon method. Observers manually outline the tumor in each tumor-containing slice and the software determines the tumor area per slice. The areas from consecutive slices are then summed and multiplied by slice spacing to produce the total tumor volume ([Area 1 + Area 2 + Area 3 + Area 4 + Area 5] × slice spacing = tumor volume). Figure is available in color online only.

The Volume Viewer auto-contour method is based on a region-growing method. Observers selected a seed region in the tumor on an axial T2-weighted slice and allowed it to grow to the border of the tumor, after which the software automatically highlighted the tumor in other slices and represented the tumor in reconstructed sagittal and coronal planes. A 3D representation of the tumor was constructed and tumor volume was calculated by the software (Fig. 2). The observers were allowed to select and unselect highlighted areas to improve tumor approximation.

Fig. 2.

Fig. 2

Open in a new tab

Volume Viewer auto-contour method. Green lines represent the tumor border that the software identifies. a: Observers select a seed region in the tumor in an axial T2-weighted MRI slice and allow it to grow to the border of the tumor. b and C:The software automatically highlights the tumor in other slices and represents the tumor in reconstructed sagittal and coronal planes. D: A 3D representation of the tumor is constructed and tumor volume is calculated. Figure is available in color online only.

Parameter-Dependent image segmentation Method

Tumor volumes were also determined using k-means clustering segmentation in ImageJ (National Institutes of Health). k-means clustering segmentation is a parameter-dependent segmentation technique in which the parameters are the number of clusters assigned by the observers, where clusters are generated based on the signal intensity and location of the voxels of the images. For each patient, to apply k-means clustering segmentation to the site of tumor, the observers selected a region of interest (ROI) that included the entire tumor with a safe margin on the slice containing the largest cross-sectional area of the tumor. The ROI was thereafter extended orthogonally to the remaining tumor-containing slices. The observers then assigned the number of clusters (k), which remained unchanged for all slices, and applied k-means clustering segmentation (plugin developed by Jaros aw Sacha, Procter & Gamble Company, Miami Valley Innovation Center; http://sourceforge.net/projects/ij-plugins/) to the slices. The algorithm partitions the voxels into the designated number of clusters by iteratively calculating and minimizing intracluster dissimilarity until further reduction is not possible, i.e., each cluster contains voxels that are similar to each other in signal intensity and location and dissimilar to voxels in other clusters (Fig. 3). In this study, all images were successfully clustered using 4–6 clusters.

Fig. 3.

Fig. 3

Open in a new tab

k-means clustering segmentation. a: Flowchart illustrating the k-means clustering segmentation algorithm. b: A simplistic schematic of how k-means clustering generates 4 distinct clusters (black circles) from a given dataset. C:k-means clustering segmentation as used in the current study. To apply k-means clustering segmentation to the site of tumor, an ROI (yellow outline) is selected that includes the entire tumor with a safe margin on the slice containing the largest cross-sectional area of the tumor and extended orthogonally to the remaining tumor-containing slices. The observer then specifies the number of clusters (“4” in this case) and lets the algorithm cluster the images. For each individual slice, a histogram of the clustered image (the thin yellow line in the center figure is used to select the region that a histogram is generated from) is generated and a list is produced containing the exact number of voxels for the clusters of choice (“3” in this case). The total number of voxels from the tumor-containing clusters is multiplied by voxel size to produce tumor volume per slice: (voxel size) × (no. of voxels in slice) = tumor volume in slice. In the example in this figure, voxel size = 0.556 mm3; therefore 0.556 mm3× 2146 = 1191.3 mm3 = 1.1913 ml. After all slices are analyzed, tumor volumes from individual slices are summed to produce total tumor volume. Figure is available in color online only.

Statistical analysis

Interobserver agreement was evaluated using Lin’s concordance correlation coefficient (CCC). 11 Modified Bland-Altman plots2 were used to visually demonstrate the presence or absence of agreement between measurements made by the 2 observers.

Results

The median age of patients was 6 years (range 3–8 years), and the mean (± SD) time between completion of radiation therapy and the analyzed MR images was 65 ± 16 days. The average tumor volumes measured by the 2 observers using the PACS polygon method were 11.48 ± 7.71 cm3 and 13.61 ± 7.76 cm3 (Table 1). The CCC of the 2 observers’ measurements was 0.9465 with a lower 1-sided 95% confidence limit of 0.8472, which was considered poor agreement.15,16 Precision (0.9872, lower 1-sided 95% confidence limit 0.9453) and accuracy (0.9588, lower 1-sided 95% confidence limit 0.8756) suggested that the poor agreement resulted mainly from differences between the judgments of the 2 observers. One observer consistently measured greater tumor volumes than the other (Table 1). The modified Bland-Altman plot showed that the difference between the observers’ measurements was more than 25% in 3 of the 8 tumors (Fig. 4A).

Table 1.

Tumor volume measurements*

Case No. PACS Polygon (cm3)†
Volume Viewer Auto-Contour (cm3)‡ †
k-means Clustering Segmentation (cm3)§ †
Observer 1 Observer 2 Difference (%) Observer 1 Observer 2 Difference (%) Observer 1 Observer 2 Difference (%)
1 6.95 10.51 41 9.53 10.15 6.2 7.25 6.60 9.4
2 23.86 25.18 5.4 22.85 22.55 1.3 16.93 16.86 0.4
3 9.64 13.65 34 10.31 19.42 61 8.71 7.89 9.9
4 13.97 14.85 6.1 12.89 18.44 35 8.79 8.12 7.9
5 21.23 24.25 13 21.90 24.04 9.3 17.06 17.77 4.1
6 8.99 10.40 15 15.03 11.35 28 7.49 7.62 1.7
7 5.64 6.50 14 13.11 7.50 55 4.29 5.09 17
8 1.59 3.50 75 3.49 4.71 30 1.23 1.07 14
Mean ± SD 11.48 ± 7.71 13.61 ± 7.76 13.64 ± 6.40 14.77 ± 7.26 8.97 ± 5.55 8.88 ± 5.69
Open in a new tab
*

Difference was calculated as the absolute difference between measurements made by the 2 observers divided by the average of the measurements made by the 2 observers.

†

Poor agreement between observers, CCC = 0.9465 (lower 1-sided 95% confidence limit 0.8472).

‡

Poor agreement between observers, CCC = 0.7514 (lower 1-sided 95% confidence limit 0.3143).

§

Substantial agreement between observers, CCC = 0.9938 (lower 1-sided 95% confidence limit 0.9772).

Fig. 4.

Fig. 4

Open in a new tab

Modified Bland-Altman plots of measurements made by the 2 observers. In these plots, the percentage difference between measurements made by the 2 observers is plotted against the average of the measurements made by the 2 observers. The 2 dashed lines denote 25% difference (positive or negative). a: When using the PACS polygon method, the measurements of the observers differed by more than 25% in 3 of 8 tumors. One observer consistently obtained larger tumor volumes than the other observer. b: When using the Volume Viewer auto-contour method, the measurements of the observers differed by more than 25% in 5 of 8 tumors. C: When using k-means clustering segmentation, none of the measurements of the 2 observers differed by more than 25%.

The average tumor volumes measured by the 2 observers using the Volume Viewer auto-contour method were 13.64 ± 6.40 cm3 and 14.77 ± 7.26 cm3. The CCC of the 2 observers’ measurements was 0.7514 with a lower 1-sided 95% confidence limit of 0.3143, which was considered poor agreement.15,16 Precision (0.7691, lower 1-sided 95% confidence limit 0.2753) and accuracy (0.9769, lower 1-sided 95% confidence limit 0.3873) suggested that the poor agreement was a result of both poor consistency in the judgments of each individual observer and differences between the judgments of the 2 observers. The modified Bland-Altman plot highlighted that the difference between the 2 observers’ measurements was more than 25% (positive or negative) in 5 of the 8 tumors (Fig. 4B).

The average tumor volumes measured by the 2 observers using k-means clustering segmentation were 8.97 ± 5.55 cm3 and 8.88 ± 5.69 cm3 (Table 1). The CCC of the 2 observers was 0.9938 with a lower 1-sided 95% confidence limit of 0.9772, which was considered substantial strength of agreement.15,16 Precision (0.9942, lower 1-sided 95% confidence limit 0.9752) and accuracy (0.9996, lower 1-sided 95% confidence limit 0.9432) suggested that both judgments between the 2 observers and consistency within each individual observer were good. The modified Bland-Altman plot showed that none of the measurements of the 2 observers differed by more than 25% (Fig. 4C).

k-means clustering segmentation delineated the tumor in all images analyzed (Fig. 5 upper), as verified by the board-certified radiologist with a certificate of added qualification in neuroradiology (S.H.). k-means clustering segmentation also delineated components of the tumor with different signal intensities (Fig. 5 lower).

Fig. 5.

Fig. 5

Open in a new tab

k-means clustered images. Upper: k-means clustering segmentation is able to delineate the tumor in all images analyzed. The left column represents original axial MR images of DIPGs and the right column the clustered images. lower: k-means clustering segmentation is also able to delineate components of the tumor that have significantly different signal intensities.

Discussion

Accurate and reproducible measurements of DIPG tumor volumes are clinically important. CED, one of the major available techniques to overcome the blood-brain barrier in treating DIPG, requires an accurate determination of DIPG tumor volume to determine optimal dosage. Furthermore, tumor volume serves as a marker for tumor response and progression. In our study we assessed the utility of using MR images and image segmentation techniques requiring different degrees of an observer’s subjective judgment to determine DIPG tumor volume.

In both PACS polygon and Volume Viewer auto-contour methods, the 2 observers showed poor agreement, a reflection of the high degrees of subjective judgment involved. Poor interobserver agreement undermines the believability of the methods. In the clinical setting, poor interobserver agreement also creates difficulty in continuous care from multiple providers. Previous studies using a method similar to the PACS polygon method obtained tumor area per slice by using the 2 longest diameters rather than outlining the tumor perimeter.3,21,23 Outlining the tumor perimeter, as performed in the current study, produces less variability between observers compared with using the 2 longest diameters,20 especially for irregular, nonspherical tumors. Despite this, the interob-server agreement was still poor using the PACS polygon method in our study. These data underscore the difficulty of achieving interobserver agreement in measuring DIPG tumor volumes using methods that require high degrees of subjective judgment, a difficulty that has also been demonstrated in other studies.3,9,19,20,23 Furthermore, many of the tumor volumes determined using PACS polygon and Volume Viewer auto-contour methods differed by more than 25%, which could potentially lead to significantly under- or overdosing patients receiving CED therapy for DIPG treatment.

Due to the poor interobserver agreement of methods requiring high degrees of subjective judgment, we investigated the use of k-means clustering segmentation, a parameter-dependent segmentation technique that requires a low degree of subjective judgment. k-means clustering segmentation demonstrated a higher interobserver agreement than the PACS Polygon or Volume Viewer auto-contour methods. None of the measurements of the 2 observers using k-means clustering differed by more than 25%, which is another demonstration that the data showed better interobserver agreement. Average tumor volumes measured using k-means clustering segmentation were lower than those from the other 2 methods. The likely reason for these outcomes is that k-means clustering segmentation delineated the tumor, which allowed the observer to select only clusters making up the tumor. k-means clustering segmentation also delineated components of the tumor with different signal intensities.

While k-means clustering segmentation delineated the tumor and showed substantial agreement between observers, this methodology needs improvement before it could be used routinely by clinicians. k-means clustering segmentation was the most time-consuming of the 3 methods because observers had to individually cluster each slice containing the tumor. The ImageJ k-means clustering plugin has the capability to perform segmentation on 3D images. However, it was difficult to clearly visualize the clusters and to identify clusters containing tumor on the segmented 3D images. Therefore we elected to perform k-means clustering segmentation on individual slices. Segmentation of 3D images would be ideal if the segmented clusters could be easily inspected. However, to our knowledge, there is currently no user-friendly medical image processing software for performing and visualizing k-means clustering segmentation on 3D images.

By design, k-means clustering segmentation is parameter dependent, and requires the observer to assign the value of k (number of clusters), which could introduce subjectivity into tumor volume determination. To identify an adequate k value in the current study, as in all applications of k-means clustering, the observers had to experiment with a few values until a clustered image that delineated the tumor was generated. All images were successfully clustered with 4–6 clusters. Choosing the optimal value of k is important in generating the most medically relevant results, which also requires further study. One approach to choosing an optimal k value is to generate a histogram of voxel signal intensity values of the images. The k value should be equal to or greater than the number of peaks in the histogram. If a peak consists of groups of voxels at more than 1 spatial location, then this peak needs to be represented by more than 1 cluster. The other question in producing the most medically relevant clusters is which clustering algorithms (both the initialization method and the iterative refinement method) to use. The ImageJ k-means clustering segmentation plugin uses the k-means++1 initialization and MacQueen13 iterative refinement (called the standard k-means method by many), which are the more robust ones among available algorithms. While the most commonly used algorithm in recent years is the Hartigan-Wong7 method, the Forgy,4 Lloyd,12 and MacQueen13 methods all have their advantages and disadvantages. Even though there has been considerable discussion in the literature, there does not seem to be a universally advantageous algorithm.

While a few other sequences—such as T1-weighted imaging with and without Gd contrast and T2-FLAIR—are also included in MRI studies of patients with DIPG, T2-weighted images were used in this study for tumor volume determination because T2-FLAIR imaging generates more artifacts in the posterior fossa. Furthermore, T2-FLAIR imaging may miss white matter lesions in the posterior fossa, which is noteworthy because DIPGs classically grow along white matter tracts.22 We did not use T1-weighted imaging in the analysis because in DIPGs, T1-weighted imaging without contrast does not show tumor well, and contrast enhancement with Gd is variable, with no enhancement being common.14

One limitation of this study was that the actual DIPG tumor volume was not known, as resection is not a treatment option for these tumors. However, because PACS polygon and Volume Viewer auto-contour methods produced poor interobserver agreement, they may be less likely to represent the actual tumor volume compared with k-means, which produced substantial interobserver agreement. It was also not possible to accurately differentiate the tumor boundary from peritumoral edema using standard MRI techniques. This is an inherent shortcoming of all studies attempting to determine preoperative brain tumor volumes. We attempted to minimize the extent of peritumoral edema by using scans from approximately 2 months after radiation therapy. Another limitation was that the current study used only 2 of the numerous available image segmentation methods that require a significant degree of subjective judgment. The PACS polygon and Volume Viewer auto-contour methods were chosen because they were readily available and widely used by clinicians.

Conclusions

Image segmentation methods that require high degrees of subjective judgment show poor interobserver agreement in determining DIPG tumor volumes. k-means clustering segmentation, which is parameter-dependent and requires a low degree of subjective judgment, shows good interob-server agreement and delineates DIPG tumors. To make k-means clustering segmentation easily usable in clinical settings, further studies should be conducted to optimize this image segmentation method, including developing algorithms for optimal parameter setting, and 3D image visualization for easy inspection of clusters on clustered images. The aim would be to streamline the processing steps into a more automated method.

Acknowledgments

We would like to acknowledge the following organizations and programs for supporting the study: The Alex Lemonade Stand Foundation Pediatric Oncology Student Training Program, The Matthew Larson Foundation, The Cure Starts Now Foundation, The Dana Foundation, The Solving Kids’ Cancer Foundation, The Lyla Nsouli Foundation, The Perry’s Promise Fund, Witmer Fund-Pediatric Brain Tumor, Cookies For Kids’ Cancer, MSKCC internal funding, The St. Baldrick’s Foundation, The Pediatric Brain Tumor Foundation, The McKenna Claire Foundation, The Cristian Rivera Foundation, and The Ian’s Friends Foundation. The authors are thankful to Ms. Maria Donzelli, Pediatric Neurosurgery Nurse Practitioner, MSKCC, for providing the necessary patient information. The authors are also thankful for the expert editorial advice provided by Ms. Ada Muellner, Editor, Department of Radiology, MSKCC.

ABBREVIATIONS

CCC

concordance correlation coefficient

CED

convection-enhanced delivery

DIPG

diffuse intrinsic pontine glioma

MSKCC

Memorial Sloan Kettering Cancer Center

PACS

picture archiving and communication system

ROI

region of interest

Footnotes

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Zhou, Singh, Haque, Peck, Young, Tsiouris, Thakur, Souweidane. Acquisition of data: Singh, Tisnado, Haque. Analysis and interpretation of data: all authors. Drafting the article: Zhou, Singh, Tisnado, Haque, Peck, Young, Souweidane. Critically revising the article: Zhou, Singh, Tisnado, Haque, Tsiouris, Souweidane. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Zhou. Statistical analysis: Zhou. Study supervision: Zhou, Haque, Souweidane.

References

  • 1.Arthur D, Vassilvitskii S. k-means++: the advantages of careful seeding. Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms; Philadelphia: Society for Industrial and Applied Mathematics; 2007. pp. 1027–1035. [Google Scholar]
  • 2.Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–310. [PubMed] [Google Scholar]
  • 3.Dempsey MF, Condon BR, Hadley DM. Measurement of tumor “size” in recurrent malignant glioma: 1D, 2D, or 3D? AJNR Am J Neuroradiol. 2005;26:770–776. [PMC free article] [PubMed] [Google Scholar]
  • 4.Forgy EW. Cluster analysis of multivariate data: efficiency versus interpretability of classification. Biom. 1965;21:768–769. [Google Scholar]
  • 5.Haas-Kogan DA, Banerjee A, Poussaint TY, Kocak M, Prados MD, Geyer JR, et al. Phase II trial of tipifarnib and radiation in children with newly diagnosed diffuse intrinsic pontine gliomas. Neuro Oncol. 2011;13:298–306. doi: 10.1093/neuonc/noq202. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Hargrave D, Bartels U, Bouffet E. Diffuse brainstem glioma in children: critical review of clinical trials. Lancet Oncol. 2006;7:241–248. doi: 10.1016/S1470-2045(06)70615-5. [DOI] [PubMed] [Google Scholar]
  • 7.Hartigan JA, Wong MA. A K-means clustering algorithm. Appl Stat. 1979;28:100–108. [Google Scholar]
  • 8.Hayward RM, Patronas N, Baker EH, Vézina G, Albert PS, Warren KE. Inter-observer variability in the measurement of diffuse intrinsic pontine gliomas. J Neurooncol. 2008;90:57–61. doi: 10.1007/s11060-008-9631-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Henson JW, Ulmer S, Harris GJ. Brain tumor imaging in clinical trials. AJNR Am J Neuroradiol. 2008;29:419–424. doi: 10.3174/ajnr.A0963. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Jansen MH, van Vuurden DG, Vandertop WP, Kaspers GJ. Diffuse intrinsic pontine gliomas: a systematic update on clinical trials and biology. Cancer Treat Rev. 2012;38:27–35. doi: 10.1016/j.ctrv.2011.06.007. [DOI] [PubMed] [Google Scholar]
  • 11.Lin L, Hedayat AS, Sinha B, Yang M. Statistical methods in assessing agreement: models, issues, and tools. J Am Stat Assoc. 2002;97:257–270. [Google Scholar]
  • 12.Lloyd SP. Least squares quantization in PCM. IEEE Trans Inf Theory. 1982;28:128–137. [Google Scholar]
  • 13.MacQueen J. In: Le Cam LM, Neyman J, editors. Some methods for classification and analysis of multivariate observations; Fifth Berkeley Symposium on Mathematical Statistics and Probability; Berkeley, CA: University of California Press; 1967. pp. 281–297. [Google Scholar]
  • 14.Mauffrey C. Paediatric brainstem gliomas: prognostic factors and management. J Clin Neurosci. 2006;13:431–437. doi: 10.1016/j.jocn.2005.05.015. [DOI] [PubMed] [Google Scholar]
  • 15.McBride GB. Equivalence Measures for Comparing the Performance of Alternative Methods for the Analysis of Water Quality Variables. [Accessed May 17, 2016];NIWA Client Report: HAM2007-017. ( https://www.health.govt.nz/system/files/documents/publications/equivalence-measures-2007.pdf)
  • 16.McBride GB. NIWA Client Report: HAM2005-062. Hamilton, New Zealand: National Institute of Water & Atmospheric Research; 2005. [Accessed May 17, 2016]. A Proposal for Strength-of-Agreement Criteria for Lin’s Concordance Correlation Coefficient. ( https://www.medcalc.org/download/pdf/McBride2005.pdf) [Google Scholar]
  • 17.Poussaint TY, Kocak M, Vajapeyam S, Packer RI, Robertson RL, Geyer R, et al. MRI as a central component of clinical trials analysis in brainstem glioma: a report from the Pediatric Brain Tumor Consortium (PBTC) Neuro Oncol. 2011;13:417–427. doi: 10.1093/neuonc/noq200. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Sedlacik J, Winchell A, Kocak M, Loeffler RB, Hillenbrand CM. MR imaging assessment of tumor perfusion and 3D segmented volume at baseline, during treatment, and at tumor progression in children with newly diagnosed diffuse intrinsic pontine glioma. AJNR Am J Neuroradiol. 2013;34:1450–1455. doi: 10.3174/ajnr.A3421. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Shah GD, Kesari S, Xu R, Batchelor TT, O’Neill AM, Hochberg FH, et al. Comparison of linear and volumetric criteria in assessing tumor response in adult high-grade gliomas. Neuro Oncol. 2006;8:38–46. doi: 10.1215/S1522851705000529. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Sorensen AG, Patel S, Harmath C, Bridges S, Synnott J, Sievers A, et al. Comparison of diameter and perimeter methods for tumor volume calculation. J Clin Oncol. 2001;19:551–557. doi: 10.1200/JCO.2001.19.2.551. [DOI] [PubMed] [Google Scholar]
  • 21.Steffen-Smith EA, Baker EH, Venzon D, Shandilya S, Bent RS, Warren KE. Measurements of the pons as a biomarker of progression for pediatric DIPG. J Neurooncol. 2014;116:127–133. doi: 10.1007/s11060-013-1266-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Stevenson VL, Parker GJ, Barker GJ, Birnie K, Tofts PS, Miller DH, et al. Variations in T1 and T2 relaxation times of normal appearing white matter and lesions in multiple sclerosis. J Neurol Sci. 2000;178:81–87. doi: 10.1016/s0022-510x(00)00339-7. [DOI] [PubMed] [Google Scholar]
  • 23.Warren KE, Patronas N, Aikin AA, Albert PS, Balis FM. Comparison of one-, two-, and three-dimensional measurements of childhood brain tumors. J Natl Cancer Inst. 2001;93:1401–1405. doi: 10.1093/jnci/93.18.1401. [DOI] [PubMed] [Google Scholar]

RESOURCES