Skip to main content
PLOS One logoLink to PLOS One
. 2024 Oct 10;19(10):e0311805. doi: 10.1371/journal.pone.0311805

An exploratory study of the impact of CT slice thickness and inter-rater variability on anatomical accuracy of malunited distal radius models and surgical guides for corrective osteotomy

Emilia Gryska 1,2,*, Per Fredrikson 1,2, Katleen Libberecht 1,2, Charlotte Stor Swinkels 1,2,3, Peter Axelsson 1,2, Anders Björkman 1,2
Editor: Xiaohui Zhang4
PMCID: PMC11476685  PMID: 39388405

Abstract

High-resolution CT images are essential in clinical practice to accurately replicate patient anatomy for 3D virtual surgical planning and designing patient-specific surgical guides. These technologies are commonly used in corrective osteotomy of the distal radius. This study evaluated how the virtual radius models and the surgical guides’ surface that is in contact with the bone vary between experienced raters. Further, the discrepancies from the reference radius of surgical guides and radius models created from CT images with slice thicknesses larger than the reference standard of 0.625mm were assessed. Maximum overlap with radius model was measured for guides, and absolute average distance error was measured for radius models. The agreement between the lower-resolution guides surface and the raters’ guide surface was evaluated. The average inter-rater guide surface overlap was -0.11mm [95% CI: -0.13–0.09]. The surface of surgical guides designed on CT images with a 1mm slice thickness deviated from the reference radius within the inter-rater range (0.03mm). For slice thicknesses of 1.25mm and 1.5mm, the average guide surface overlap was 0.12mm and 0.15mm, respectively. The average inter-rater radius surface variability was 0.03mm [95% CI: 0.025–0.035]. The discrepancy from the reference of all radius models created from CT images with a slice thickness larger than the reference slice thickness was notably larger than the inter-rater variability but, excluding one case, did not exceed 0.2mm. The results suggest that 1mm CT images are suitable for surgical guide design. While 1.25mm slices are commonly used for virtual planning in hand and forearm surgery, slices larger than 1mm may approach the limit of clinical acceptability. Discrepancies in radius models were below 1mm, likely below clinical relevance.

Introduction

3D virtual surgical planning (VSP) and patient-specific surgical guides (PSSGs) for the reconstruction of anatomy in complex skeletal deformities are becoming more common in hand and forearm surgery [13]. PSSGs are designed to fit a patient’s bone perfectly in the guide’s designed position. Therefore, PSSG design assumes that the virtual bone models used to design the guides exactly represent the patient’s anatomy [46]. High-quality and high-resolution CT images are used in 3D VSP and PSSG design to fulfil this assumption. In hand and forearm surgery, companies offering 3D VSP and PSSG design request CT images with slice thicknesses between 0.5 and 1.25mm [7, 8]. The preference for high-resolution images often leads to additional dedicated CT exams, despite the availability of lower-resolution scans from routine clinical work-ups. This results in extra radiation exposure for patients and increased resource utilization.

Virtual bone models created from semiautomatic, expert segmentations are currently the gold standard in research and clinical practice. Although the patient-specific approach assumes that the virtual bone models exactly represent a patient’s anatomy, discrepancies from the ground truth occur in practice. Even with high-resolution images, variations in image segmentation are expected between raters. Additionally, the accuracy of 3D-printed surgical guides is affected by printing errors. Despite these deviations, clinical outcomes of the patient-specific approach are still superior to the conventional method [13].

As 3D VSP and PSSG design hold the potential to become the gold standard for complex corrections in skeletal surgery, it is crucial to analyse the discrepancies in the virtual models created from the same images by various raters (inter-rater variability) and the CT image slice thickness.

Previous studies have assessed how increasing the reconstructed CT slice thickness influences the accuracy of virtual or printed anatomical models for mandible [9, 10], bovine vertebra [11], and skull models [12]. The results of these studies varied, suggesting the maximum slice thickness to be between 0.3mm [10] and 3mm [9]. No previous study, however, assessed the impact of increasing CT slice thickness on the accuracy of virtual radius models; whether these radius models can be used to design well-fitting PSSGs; and the inter-rater variability in models created from the standard CTs with low slice thickness.

The aim of this exploratory study was three-fold. First, we aimed to analyse the inter-rater surface variability of the typical contact area for surgical guides on the volar distal radius. Second, we aimed to analyse the surface discrepancy of surgical guides designed for radius models created from lower-resolution CT images (slice thickness 1.6–4 times larger than the reference 0.625mm) when fitted to the high-resolution reference radius model. Our final aim was to analyse the inter-rater surface variability of the entire radius and radius models created from the lower-resolution CT images. This exploration will lay the groundwork for ensuring that the chosen CT acquisition protocols guarantee adequate image quality while considering patients’ safety and the efficient use of hospital resources.

Materials and methods

CT images and VSP of twelve consecutive participants (11 women and one man, median age 61 years, range 21 to 73 years) scheduled for corrective osteotomy of the distal radius were available for this study. The data were collected with the approval of the Swedish Ethics Review Board (2021–01974). The patients’ data were accessed between September 15th, 2023, and June 26th, 2024. The authors had access to information that could identify individual participants during and after data collection.

Radiological images

All CT images used for the surgical planning and guide design were acquired according to the Materialise (NV Leuven, Belgium) protocol [7] with 0.39mm x 0.39mm pixel size and 0.625mm slice thickness on a GE Discovery CT750 HD scanner (GE Healthcare, Milwaukee, Wisconsin, USA). The original CT of all twelve participants were resliced to slice thickness equal to 1mm, 1.25mm, 1.5mm, 1.875mm, and 2.5mm: the images were converted from DICOM (Digital Imaging and Communications in Medicine, http://medical.nema.org/) to NIfTI (Neuroimaging Informatics Technology Initiative, https://nifti.nimh.nih.gov/) format using the dcm2niix tool [13] and resliced in Python (v3.10.13) using the resample_img() function with bicubic interpolation (interpolation =“continuous”) from nilearn library (v0.10.2). The pixel size remained unchanged for all images. The resliced images were converted back to DICOM format in 3D Slicer (v5.2.2, https://www.slicer.org/) so they could be opened by the Mimics (Materialise NV, Leuven, Belgium) software we use for segmentation and for generating virtual models.

Virtual models

Virtual radius models were created in Mimics (Materialise NV, Leuven, Belgium) from CT images. First, each image was segmented: a radius mask was created using semiautomatic segmentation functions and manual editing. Then, the segmentation mask was converted to a 3D Part using the Optimal quality level option in Mimics. Further local manual smoothing of the 3D Part edges was done as necessary to ensure that the radius model outline followed the shape of the bone in the image. Each rendered bone model was saved as a Stereolithography (STL) file representing a virtual model’s surface with a triangular mesh.

A surgical guide for a corrective osteotomy typically consists of a base that is shaped so that the guide is stable once positioned on the bone. The surface of the guide base that touches the bone perfectly matches the surface of the virtual radius model. The guide has openings for cutting the bone and pre-drilling the bone for screws to fix the plate. In this study, we were only interested in whether and how the guide surfaces designed based on lower-resolution radius models deviate from the reference radius model. Therefore, only a guide base for each radius model was created in 3-Matic and saved as an STL file. The workflow of creating the models is shown in Fig 1.

Fig 1. The workflow shows how the models used in this study were created for one participant.

Fig 1

In short, from each CT image (top row of the diagram), a radius model (middle row) and a perfectly fitting surgical guide base model (bottom row of the diagram) were created. Three raters created radius and guide base models from the original CT images (left part of the diagram). One rater created radiuses and guide bases from resliced CT images (right part of the diagram). A sample of the CT, radius, and guide base models is shown on the left. This procedure was repeated for all twelve participants.

Reference models: three experienced raters (one surgeon and two clinical engineers) created reference radius models and surgical guide bases of the same shape, fitting these reference models for all 12 patients using the original CT images (slice thickness of 0.625mm).

“Lower-resolution” models: One rater created five radius- and five guide models—one pair for each resliced image—for all twelve participants. Fig 2 shows an example of a radius model at each resolution.

Fig 2.

Fig 2

Two views (a and b) of one participant’s radius models at each CT image slice thickness (c); the models are arranged left to right according to the slice thickness of the CT image used to generate the model. The leftmost model is the reference created on the original CT slice thickness of 0.625mm.

Analysis

Our analysis was based on a part-to-part comparison of guide models to the reference radius models (guide-to-radius analysis). An additional analysis of the radius model (radius-to-radius) was also conducted for the whole radius and individually for the distal and proximal parts. Each radius model was split by a plane perpendicular to the long axis of the bone positioned directly below the proximal edge of the guide base. The Visualization Toolkit library (v9.2.6) [14] implemented in Python was used to import the STL models as vtkPolyData. The distances between the corresponding guide and reference radius mesh nodes were calculated using the vtkDistancePolyDataFilter function.

The guide-to-radius analysis measured the discrepancies between the guide surface and the radius surface in the guide’s designed position. Specifically, we calculated the negative Hausdorff distance (HDF) for guide-to-radius comparison, which indicates the largest overlap between the reference radius model and the guide base surface. Such overlap could potentially lead to a worsened fit of the guide in real life. The absolute average distance error (ADE) was calculated for radius-to-radius comparisons. Both the negative HDF and absolute ADE were selected as the most relevant measures of surface discrepancy.

To examine the inter-rater guide (and radius) surface variability, we calculated the average negative HDF across the raters: each rater’s guide models were compared to the other two rater’s radius models (Fig 1., left part of the diagram). For the twelve participants and two comparisons for each of the three raters, we calculated the average and the 95% confidence interval (CI) of 72 (12x3x2) HDF values (and absolute ADE values for radius-to-radius comparison).

To analyse the impact of increasing the CT slice thickness on the surface discrepancy of guide models, each lower-resolution guide was compared to all three reference radius models and the average negative HDF of the three values was taken. (Fig 1, right part of the diagram).

Bland-Altman plots were constructed to assess the agreement in the average surface discrepancy from the reference (negative HDF) between the raters’ guides and guides created for each lower-resolution radius model. The mean difference between the raters’ and each lower-resolution model was compared to the average inter-rater variability. The normality of the differences was assessed with Q-Q plots (S1, S2 Tables in S1 File).

Finally, the models were visually analysed to determine whether any anatomical regions were particularly prone to deviations. The distance between models was visualised in FreeCAD (v0.21.1, https://www.freecad.org/).

Results

The average negative HDF across all raters’ guides was -0.11mm [95% CI: -0.13–0.09]. The Bland-Altman analysis (Fig 3) showed an increasing mean difference between the raters and each consecutive CT slice thickness model. Even though one outlier outside of the 95% CI was present for the differences between the raters and the models from CT with 1.875mm slice thickness, we assumed a normal distribution of the data based on the Q-Q plots (S1 Table in S1 File).

Fig 3.

Fig 3

Bland-Altman plots assessing agreement of guide surface discrepancy from reference radius models between guides created by three raters on the original CT image with 0.625mm slice thickness and guides created from CT images with a) 1mm, b) 1.25mm, c) 1.5mm, d) 1.875mm, and e) 2.5mm slice thickness. The y-axis shows the difference between the average inter-rater surface discrepancy and the average discrepancy of a guide’s surface created from lower-resolution CT from each reference model for each participant. The x-axis shows the mean of the average inter-rater discrepancy and the average discrepancy of the lower-resolution models. The grey area shows confidence intervals for the bias; the blue areas show the confidence intervals for the limits of agreement; the purple line indicates the ±average inter-rater discrepancy.

For CT slice thickness = 1mm, the mean difference from the raters was 0.03mm. Although the CI of the mean difference does not include the equality line (Fig 3A), the limits of agreement are within the average inter-rater variability of ±0.11mm. For CT slice thickness of 1.25mm and 1.5mm (Fig 3B and 3C), the mean difference was 0.12mm and 0.15, respectively. The lower CI bounds of both mean differences fell within the average inter-rater variability, indicating agreement on the clinically acceptable limit. For CT slice thickness of 1.785mm and 2.5mm, the mean difference and its CI interval exceeded the inter-rater discrepancy, indicating poor agreement (Fig 3D and 3E).

The average inter-rater absolute ADE for radius-to-radius comparison was 0.03mm [95% CI: 0.025–0.035]. For the distal radius, it was 0.025mm [95% CI: 0.023–0.027] and for the proximal part: 0.03mm [95% CI: 0.22–0.037]. Since we could not assume a normal distribution for the Bland-Altman analysis in two out of five groups (S2 Table in S1 File), we show the distribution of the absolute ADE for every pair of higher-resolution models compared to every rater’s reference model, separately for the distal and proximal parts (Fig 4). The median absolute ADE is generally larger than the average inter-rater variability for all CT slice thicknesses and both the distal and proximal parts. Overall, we observed larger ADE values for the distal part than the proximal part. The median absolute ADE created from CT with a slice thickness of 1.25m and 1.5mm are comparable. Much larger absolute ADE and data spread are observed for models created from CT images with a slice thickness of 1.875mm and 2.5mm, especially for the distal radius.

Fig 4. Distribution of absolute ADE for radius models separated into the distal (dist) and proximal (prox) parts, created from CT images with a slice thickness of 1mm, 1.25mm, 1.5mm, 1.875mm, and 2.5mm compared to each rater’s (EG, KL, CS) reference.

Fig 4

The dashed lines indicate the average inter-rater absolute ADE. One outlier of ADE > 0.65 for CSv2.5mm not shown.

Visual analysis showed that the increment in slice thickness affected the accuracy of parts of the bone protruding along the long axis of the radius. This was also the direction along which the image resolution decreased (the CT images were also acquired axially). The deviation, however, did not seem to follow a consistent pattern. For example, Fig 5 shows how the radius models deviated from the reference.

Fig 5. Two views of the distance (shown in the colour bar) between the reference radius and lower-resolution radius models generated from CT images with slice thickness equal to 1mm, 1.25mm, 1.5mm, 1.875mm and 2.5mm for participant 1.

Fig 5

Discrepancies from the reference model can be noted in the head of the radius and in the most distal parts of the radius.

The visual analysis of the guide-to-radius fit shows deviations in bone protrusions. Again, the pattern is not consistent as the slice thickness increases. In Fig 6, a deviation of -0.5mm for the guide model created on 1.825mm slice thickness can be observed around the volar ulnar corner of the radius. The negative distance indicates that the guide overlapped with the bone. A deviation in a similar spot is visible for a slice thickness of 2.5mm but in a positive direction. An inconsistent deviation was also observed on the wings of the guide that oscillates between positive and negative deviation for consecutive slice thickness within 0.1mm discrepancy (Fig 7).

Fig 6. The distance (indicated in the colour bar) between the reference radius and guide bases fitting radius models generated from CT images with slice thickness equal to 1mm, 1.25mm, 1.5mm, 1.875mm and 2.5mm for participant 3.

Fig 6

A deviation of 0.5mm is observed in opposing directions for the 1.875mm and 2.5mm models.

Fig 7. Two views that show the distance within the 0.1mm range (indicated in the colour bar) between the reference radius model and guide bases fitting radius models generated from CT images with slice thickness equal to 1mm, 1.25mm, 1.5mm, 1.875mm and 2.5mm for participant 4.

Fig 7

Discussion

In this exploratory study, we evaluated the inter-rater surface variability of distally malunited radii and PSSG designed for their corresponding radius models. Further, the impact of increasing the CT image slice thickness on the surface discrepancies of radius models and surgical guides designed for these models from gold-standard references was examined.

The surgical guides’ average inter-rater surface variability (negative HDF) was -0.11mm [95% CI: -0.13–0.09]. We propose to use this average inter-rater variability as the clinically acceptable surface discrepancy. Based on this clinically acceptable limit, we can conclude that CT images with a 1mm slice thickness can be used for creating virtual radius models and designing surgical guides. For guides created from CT images with slice thicknesses of 1.25mm and 1.5mm, the surface discrepancies were found to be on the limit of being clinically acceptable. For radius models, the inter-rater absolute ADE was 0.03mm (0.03 for the proximal and 0.025 for the distal part).

To our knowledge, this is the first study to assess the variability of gold-standard virtual radius models and PSSG surfaces across raters experienced in 3D VSP and PSSG design. The models created by any of the three raters can be considered the gold standard and have been used in clinical cases. Therefore, we decided to use all three as references. Since the gold standard is the best approximation of the ground truth given currently available imaging techniques, we assume that the variability in the gold standard is a clinically acceptable level of surface discrepancy. In this study, we used this variability as the limits of clinically acceptable surface discrepancy of PSSG designed for radius models created from CT images with varying slice thickness. Our results suggest that CT images with a slice thickness of 1mm can be used for PSSG design. Although CT images with a slice thickness of 1.25mm are used for 3D VSP and PSSG in hand and forearm surgery [8], our results suggest that the accuracy of PSSG created with these CTs may be at the limit of clinical acceptability.

Corroborated by larger-scale studies, our results could be used to adjust CT acquisition and reconstruction parameters for 3D VSP and PSSG design. During acquisition, the parameter that can be adjusted to decrease the slice thickness is the helical pitch. Increasing the pitch will result in a lower resolution but also decreased radiation exposure and time of the examination. Once the data is acquired, the final image resolution can be adjusted during reconstruction. Decreasing image resolution during reconstruction will result in less noisy image. Lower-resolution images also take up less space. Even though this is rarely considered when optimising imaging protocols, virtual storage has a cost. Our and future results should guide medical physicists in designing new protocols to optimise resource use and patient safety. Previous studies that investigated the “docking robustness” [4, 5] of PSSG and developed a method to assess a guide’s fit to the bone [6] assumed that the guide surface matches exactly that of the bone in its designed position. These studies also focused on modifying the shape of the guide base rather than assessing the accuracy of the models for which the guides are designed. Therefore, we could not use the previously proposed approach to assess how the guides designed for lower-resolution radius models fit the reference model, as the surface match assumption could not be met. While clinically acceptable discrepancy levels are known, we presume that measuring surface discrepancy is an alternative way to assess how well a guide will fit the bone. We used the negative HDF to measure guides’ surface discrepancy as we needed a metric sensitive to localised overlaps with the bone. We suspect that the fit of the guide could worsen if such overlaps in virtual models are present.

Printing errors will also impact the fit of the actual guide to the bone. At our institute, the surgical guides are printed using selective laser sintering (SLS). The printing error reported in the literature for SLS varies significantly: Salmi et al. [15] reported an SLS print error of 0.79mm, while Msallem et al. [16]– 0.07mm. The impact of the printing errors on the accuracy of the guides designed with CT images with lower resolution must be tested on physical models or in a cadaver study. Such a study will also validate the surface discrepancy as a reliable measure of PSSG fit.

The secondary aim of our study was to compare radius models. The absolute ADE from the reference was discernibly larger than the inter-rater absolute ADE at every CT slice thickness level. Previous studies that assessed the accuracy of 3D (virtual and printed) anatomical models created at varying CT slice thicknesses focused on models of skulls [12] and mandibles [9, 10]. Ford and Decker [12] concluded that a CT slice thickness of 1.25mm should not be exceeded for accurate anatomical reconstruction of a skull. A similar threshold was suggested by Whyms et al. [10] in a study on mandible specimens. From our analysis, we can notice that the absolute ADE from the reference was relatively similar between models from CTs with 1.25mm and 1.5mm slice thickness. Visibly larger absolute ADE vas noted for models created from CTs with 1.875mm and 2.5mm slice thickness, especially for the distal radius. We chose the absolute ADE as the most relevant metric to assess radius model discrepancies, and we can see the expected influence of increasing the CT slice on the discrepancy; however, we suspect that the deviations in absolute ADE within 1mm are not clinically relevant [17, 18]. This indicates that absolute ADE and our study design may not be adequate to establish the largest CT slice thickness suitable for 3D VSP.

The study has several other limitations. First, we only had images, 3D VSPs, and PSSG designs available from twelve participants. Equivalence studies on a larger sample are necessary to conclude the largest CT slice thickness that can be safely used for 3D VSP and PSSG design. Furthermore, our cohort included only one man, and the median age of the participants was above 60 years old. The size variation of the assessed bones was also low in the cohort and therefore we could not assess whether the results depend on the bone size. Based on the limited, demographically homogeneous cohort, we also could not assess the impact of age or sex on the results. In our study, the lower-resolution images were created by reslicing existing images in only one direction. Therefore, our study could not account for the potential influence of noise on the images for varying slice thickness [19] nor the impact of changing the pixel size. The appearance of the images in our study could also be influenced by the interpolation method; however, a visual comparison between images interpolated with the bicubic and linear functions (results not included) did not show any perceivable differences that could alter the results.

A larger study is necessary to perform equivalence tests of models for 3D VSP and PSSG design created from CT images with larger slice thickness to those currently used. Ideally, images and VSPs from multiple centres would be acquired to further study the impact of a CT machine on the images, and various raters on the segmentation. A more varied and larger cohort would further allow to investigate other factors that could influence the optimal imaging protocol, like age and age-related conditions, like arthritis, ethnicity, or sex, mentioned in the limitations section. We also suggest using a different metric than ADE to assess the suitability to the models for 3D VSP and normalise the results with respect to the bone volume, so the results are comparable across various cohorts. Intra-articular and diaphyseal malunions should also be considered. The results may also vary for other anatomies.

Supporting information

S1 File

(DOCX)

pone.0311805.s001.docx (294.5KB, docx)
S1 Raw data. Data used for this analysis are available as supporting information in data_raw.

(ZIP)

pone.0311805.s002.zip (140.5MB, zip)

Data Availability

All relevant data are within the paper and its Supporting Information files.

Funding Statement

The study was financed by grants from the Swedish state under the agreement between the Swedish government and the county councils, the ALF-agreement (ALFGBG-966260). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.de Muinck Keizer RJO, Lechner KM, Mulders MAM, Schep NWL, Eygendaal D, Goslings JC. Three-dimensional virtual planning of corrective osteotomies of distal radius malunions: a systematic review and meta-analysis. Strat Traum Limb Recon. 2017. Aug 1;12(2):77–89. doi: 10.1007/s11751-017-0284-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Roner S, Carrillo F, Vlachopoulos L, Schweizer A, Nagy L, Fuernstahl P. Improving accuracy of opening-wedge osteotomies of distal radius using a patient-specific ramp-guide technique. BMC Musculoskeletal Disorders. 2018. Oct 15;19(1):374. doi: 10.1186/s12891-018-2279-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Schindele S, Oyewale M, Marks M, Brodbeck M, Herren DB. Three-Dimensionally Planned and Printed Patient-Tailored Plates for Corrective Osteotomies of the Distal Radius and Forearm. The Journal of Hand Surgery [Internet]. 2022. Aug 17 [cited 2023 Mar 29]; Available from: https://www.sciencedirect.com/science/article/pii/S0363502322003884 doi: 10.1016/j.jhsa.2022.06.021 [DOI] [PubMed] [Google Scholar]
  • 4.Mattheijer J, Herder JL, Tuijthof GJM, Nelissen RGHH, Dankelman J, Valstar ER. Shaping Patient Specific Surgical Guides for Arthroplasty to Obtain High Docking Robustness. Journal of Mechanical Design [Internet]. 2013. May 10 [cited 2024 Apr 23];135(071001). Available from: 10.1115/1.4024231 [DOI] [Google Scholar]
  • 5.Mattheijer J, Herder JL, Tuijthof GJM, Valstar ER. Docking Robustness of Patient Specific Surgical Guides for Joint Replacement Surgery. Journal of Mechanical Design [Internet]. 2015. Jun 1 [cited 2024 Apr 23];137(062301). Available from: 10.1115/1.4029665 [DOI] [Google Scholar]
  • 6.Van den Broeck J, Wirix-Speetjens R, Vander Sloten J. Preoperative analysis of the stability of fit of a patient-specific surgical guide. Computer Methods in Biomechanics and Biomedical Engineering. 2015. Jan 2;18(1):38–47. doi: 10.1080/10255842.2013.774383 [DOI] [PubMed] [Google Scholar]
  • 7.CT Scan Protocol—Osteotomies—Upper Extremity—English [Internet]. [cited 2023 May 25]. Available from: https://assets-eu-01.kc-usercontent.com/8ff24b0e-57a3-0157-62d1-fa4ac9734eb5/df9e5f44-7ec7-4f0b-b964-ecd50ee45de5/CT%20Scan%20Protocol%20-%20Osteotomies%20-%20Upper%20Extremity%20-%20English%20-%20L-102000-01.pdf [Google Scholar]
  • 8.Scan-Protocol Radius Forearm Reconstruction [Internet]. [cited 2024 Apr 29]. Available from: https://www.klsmartin.com/mediathek/90-381-02-04_IPS_Implants_Scan-Protocol_Radius_Forearm_Reconstruction.pdf [Google Scholar]
  • 9.Ahmed M, Melaragno LE, Nyirjesy SC, von Windheim N, Fenberg R, Herster R, et al. Higher Computed Tomography (CT) Scan Resolution Improves Accuracy of Patient-specific Mandibular Models When Compared to Cadaveric Gold Standard. J Oral Maxillofac Surg. 2023. Sep;81(9):1176–85. doi: 10.1016/j.joms.2023.05.014 [DOI] [PubMed] [Google Scholar]
  • 10.Whyms BJ, Vorperian HK, Gentry LR, Schimek EM, Bersu ET, Chung MK. The effect of computed tomographic scanner parameters and 3-dimensional volume rendering techniques on the accuracy of linear, angular, and volumetric measurements of the mandible. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology. 2013. May 1;115(5):682–91. doi: 10.1016/j.oooo.2013.02.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Searle B, Starkey D. An investigation into the effect of changing the computed tomography slice reconstruction interval on the spatial replication accuracy of three‐dimensional printed anatomical models constructed by fused deposition modelling. J Med Radiat Sci. 2020. Mar;67(1):43–53. doi: 10.1002/jmrs.382 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Ford JM, Decker SJ. Computed tomography slice thickness and its effects on three-dimensional reconstruction of anatomical structures. Journal of Forensic Radiology and Imaging. 2016. Mar 1;4:43–6. [Google Scholar]
  • 13.Li X, Morgan PS, Ashburner J, Smith J, Rorden C. The first step for neuroimaging data analysis: DICOM to NIfTI conversion. J Neurosci Methods. 2016. May 1;264:47–56. doi: 10.1016/j.jneumeth.2016.03.001 [DOI] [PubMed] [Google Scholar]
  • 14.Schroeder W, Martin K, Lorensen B. Visualization Toolkit: An Object-Oriented Approach to 3D Graphics, 4th Edition. 4th edition. Clifton Park, NY: Kitware; 2006. 528 p. [Google Scholar]
  • 15.Salmi M, Paloheimo KS, Tuomi J, Wolff J, Mäkitie A. Accuracy of medical models made by additive manufacturing (rapid manufacturing). J Craniomaxillofac Surg. 2013. Oct;41(7):603–9. doi: 10.1016/j.jcms.2012.11.041 [DOI] [PubMed] [Google Scholar]
  • 16.Msallem B, Sharma N, Cao S, Halbeisen FS, Zeilhofer HF, Thieringer FM. Evaluation of the Dimensional Accuracy of 3D-Printed Anatomical Mandibular Models Using FFF, SLA, SLS, MJ, and BJ Printing Technology. Journal of Clinical Medicine. 2020. Mar;9(3):817. doi: 10.3390/jcm9030817 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.McQueen M, Caspers J. Colles fracture: does the anatomical result affect the final function? J Bone Joint Surg Br. 1988. Aug;70(4):649–51. doi: 10.1302/0301-620X.70B4.3403617 [DOI] [PubMed] [Google Scholar]
  • 18.PROMMERSBERGER KJ, VAN SCHOONHOVEN J, LANZ UB. Outcome after Corrective Osteotomy for Malunited Fractures of the Distal End of the Radius. Journal of Hand Surgery. 2002. Feb 1;27(1):55–60. doi: 10.1054/jhsb.2001.0693 [DOI] [PubMed] [Google Scholar]
  • 19.Alshipli M, Kabir NA. Effect of slice thickness on image noise and diagnostic content of single-source-dual energy computed tomography. J Phys: Conf Ser. 2017. May;851(1):012005. [Google Scholar]

Decision Letter 0

Xiaohui Zhang

15 Sep 2024

PONE-D-24-32153An exploratory study of the impact of CT slice thickness and inter-rater variability on anatomical accuracy of malunited distal radius models and surgical guides for corrective osteotomy.PLOS ONE

Dear Dr. Gryska,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Oct 30 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Xiaohui Zhang

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following financial disclosure:

“The study was financed by grants from the Swedish state under the agreement between the Swedish government and the county councils, the ALF-agreement (ALFGBG-966260).”

Please state what role the funders took in the study.  If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

3. We are unable to open your Supporting Information file [data_raw.zip]. Please kindly revise as necessary and re-upload.

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The paper An exploratory study of the impact of CT slice thickness and inter-rater variability on

anatomical accuracy of malunited distal radius models and surgical guides for

corrective osteotomy is well written and documented. The aim was three-folded. Images are of high quality. Results and conclusions are clear. References are up to date.

Reviewer #2: The paper is very well-written, and the studies are well-designed. I have the following comments:

1. The Hausdorff distance and the ADE are absolute measures. It is true that absolute error is important in determining guide fit, however, it is possible that physically larger guides in larger patients will contribute more to the overall error in a study. That is, the composition of the patient cohort might affect the inference of the study. Hence, it might be useful to also report relative errors, i.e., absolute errors normalized by total volume, so that results are comparable across patient cohorts in future.

2. It would be useful to have a comment on the effects of image acquisition parameters that might affect the value of the optimal CT slice thickness discovered in this study.

3. Do the authors expect that the optimal CT slice thickness would be affected by demographic parameters, e.g., age, sex, given the anatomical differences associated with the same?

4. In the radius-radius comparison, it would be interesting to split the error into guide-contact region (distal radius and partial shaft) vs non-contact region (proximal radius and shaft). It is possible that given the high variations in small-scale structures in the radial head and neck, the clinically acceptable value discovered for the guide-contact region might be even lower than presented.

5. Please add a comment on factors that might contribute to different results in a large-scale study as compared to the exploratory study.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2024 Oct 10;19(10):e0311805. doi: 10.1371/journal.pone.0311805.r002

Author response to Decision Letter 0


17 Sep 2024

Dear Editor and Reviewers,

We want to thank the reviewers for taking the time to review our manuscript; we appreciate the insightful and constructive comments. We believe that the manuscript was improved after we supplied the manuscript with the reviewer’s suggestions. In the following, we address all comments made by the editor and the reviewers. Changes to the manuscript are tracked, and comments are added in the manuscript to indicate which reviewer’s comment a given change refers to.

Editor’s comments:

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming.

Authors’ response: We revised our submission and the manuscript to ensure that it meets PLOS ONE’s requirements.

2. Thank you for stating the following financial disclosure:

“The study was financed by grants from the Swedish state under the agreement between the Swedish government and the county councils, the ALF-agreement (ALFGBG-966260).”

Please state what role the funders took in the study. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

Authors’ response: The funders had no role in the study, and the above statement should be added to the financial disclosure.

3. We are unable to open your Supporting Information file [data_raw.zip]. Please kindly revise as necessary and re-upload.

Authors’ response: We apologise for that. The original data are quite large and were provided in a .pkl format that is easy to open as a data frame using pandas library in Python. We now supplied the supporting information with the raw data available in text format so that it can be opened in other programs.

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Authors’ response: We reviewed the reference list and formatted the citing style, so it meets PLOS ONE’s requirements

5. Review Comments to the Author

Reviewer #1: The paper An exploratory study of the impact of CT slice thickness and inter-rater variability on anatomical accuracy of malunited distal radius models and surgical guides for corrective osteotomy is well written and documented. The aim was three-folded. Images are of high quality. Results and conclusions are clear. References are up to date.

Authors’ response: We thank the reviewer for the positive feedback on our manuscript.

Reviewer #2: The paper is very well-written, and the studies are well-designed. I have the following comments:

1. The Hausdorff distance and the ADE are absolute measures. It is true that absolute error is important in determining guide fit, however, it is possible that physically larger guides in larger patients will contribute more to the overall error in a study. That is, the composition of the patient cohort might affect the inference of the study. Hence, it might be useful to also report relative errors, i.e., absolute errors normalized by total volume, so that results are comparable across patient cohorts in future.

Authors’ response: Although this is an interesting thought, in case of the guide fit, we doubt that the size of the bone will influence the fit of the guide as measured by Hausdorff distance. We chose this metric as one localised overlap between the guide and the bone will have the same consequence for the guide fit, regardless of the size of the bone. Normalising the Hausdorff distance with bone volume could, however, incorrectly indicate a more serious negative effect on the guide fit in a smaller bone.

Regarding the ADE, there could be a difference between larger and smaller bones, since relatively more detail is lost if the details are relatively smaller. However, as we stated in the discussion, we suspect that this metric may not be sensitive enough metric to establish the largest CT slice thickness suitable for 3D VSP. We agree, though, that the impact of bone size, e.g. in children or non-Caucasian populations should be accounted for in future studies. We mention the homogeneous bone size in our cohort as a limitation of this study:

“The size variation of the assessed bones was also low in the cohort and therefore we could not assess whether the results depend on the bone size.”

And in the last paragraph in response to Comment #5:

“We also suggest using a different metric than ADE to assess the suitability to the models for 3D VSP and normalise the results with respect to the bone volume, so the results are comparable across various cohorts.”

2. It would be useful to have a comment on the effects of image acquisition parameters that might affect the value of the optimal CT slice thickness discovered in this study.

Authors’ response: Thank you for this suggestion. We added a relevant paragraph in the Discussion:

“Corroborated by larger-scale studies, our results could be used to adjust CT acquisition and reconstruction parameters for 3D VSP and PSSG design. During acquisition, the parameter that can be adjusted to decrease the slice thickness is the helical pitch. Increasing the pitch will result in a lower resolution but also decreased radiation exposure and time of the examination. Once the data is acquired, the final image resolution can be adjusted during reconstruction. Decreasing image resolution during reconstruction will result in less noisy image. Lower-resolution images also take up less space. Even though this is rarely considered when optimising imaging protocols, virtual storage has a cost. Our and future results should guide medical physicists in designing new protocols to optimise resource use and patient safety.

3. Do the authors expect that the optimal CT slice thickness would be affected by demographic parameters, e.g., age, sex, given the anatomical differences associated with the same?

Authors’ response: We thank the reviewer for again bringing up a relevant point. We know from experience that image segmentation of arthritic and osteoporotic bone is difficult, and we suspect that the difficulty would increase with increased CT slice thickness. We know also that arthritis incidence increases with age and in older cohorts, women are more at risk. Given that our cohort mostly consisted of older women, we don’t have enough data to investigate the impact of the demographic parameters on the results. We added this to the limitations section in the manuscript:

“Furthermore, our cohort included only one man, and the median age of the participants was above 60 years old. The size variation of the assessed bones was also low in the cohort and therefore we could not assess whether the results depend on the bone size. Based on the limited, demographically homogeneous cohort, we also could not assess the impact of age or sex on the results. “

4. In the radius-radius comparison, it would be interesting to split the error into guide-contact region (distal radius and partial shaft) vs non-contact region (proximal radius and shaft). It is possible that given the high variations in small-scale structures in the radial head and neck, the clinically acceptable value discovered for the guide-contact region might be even lower than presented.

Authors’ response: We agree that this information would make our findings more interesting. We now added the results of radius models discrepancies split between the distal and the proximal parts in figure 4, as well as in the manuscript:

“An additional analysis of the radius model (radius-to-radius) was also conducted for the whole radius and individually for the distal and proximal parts. Each radius model was split by a plane perpendicular to the long axis of the bone positioned directly below the proximal edge of the guide base. “

“For the distal radius, it (ADE) was 0.025mm [95% CI: 0.023 -0.027] and for the proximal part: 0.03mm [95% CI: 0.22 -0.037].Since we could not assume a normal distribution for the Bland-Altman analysis in two out of five groups (S2 Table), we show the distribution of the absolute ADE for every pair of higher-resolution models compared to every rater’s reference model, separately for the distal and proximal parts (Fig 4). The median absolute ADE is generally larger than the average inter-rater variability for all CT slice thicknesses and both the distal and proximal parts. Overall, we observed larger ADE values for the distal part than the proximal part. The median absolute ADE created from CT with a slice thickness of 1.25m and 1.5mm are comparable. Much larger absolute ADE and data spread are observed for models created from CT images with a slice thickness of 1.875mm and 2.5mm, especially for the distal radius.”

Fig 4. Distribution of absolute ADE for radius models separated into the distal (dist) and proximal (prox) parts, created from CT images with a slice thickness of 1mm, 1.25mm, 1.5mm, 1.875mm, and 2.5mm compared to each rater's (EG, KL, CS) reference. The dashed lines indicate the average inter-rater absolute ADE. One outlier of ADE > 0.65 for CSv2.5mm not shown.

5. Please add a comment on factors that might contribute to different results in a large-scale study as compared to the exploratory study.

Author’s response: We added the following paragraph in the Discussion:

“A larger study is necessary to perform equivalence tests of models for 3D VSP and PSSG design created from CT images with larger slice thickness to those currently used. Ideally, images and VSPs from multiple centres would be acquired to further study the impact of a CT machine on the images, and various raters on the segmentation. A more varied and larger cohort would further allow to investigate other factors that could influence the optimal imaging protocol, like age and age-related conditions, like arthritis, ethnicity, or sex, mentioned in the limitations section. We also suggest using a different metric than ADE to assess the suitability to the models for 3D VSP and normalise the results with respect to the bone volume, so the results are comparable across various cohorts. Intra-articular and diaphyseal malunions should also be considered. The results may also vary for other anatomies.”

Attachment

Submitted filename: Response to reviewers.docx

pone.0311805.s003.docx (63.5KB, docx)

Decision Letter 1

Xiaohui Zhang

26 Sep 2024

An exploratory study of the impact of CT slice thickness and inter-rater variability on anatomical accuracy of malunited distal radius models and surgical guides for corrective osteotomy.

PONE-D-24-32153R1

Dear Dr. Gryska,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Xiaohui Zhang

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

**********

Acceptance letter

Xiaohui Zhang

1 Oct 2024

PONE-D-24-32153R1

PLOS ONE

Dear Dr. Gryska,

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team.

At this stage, our production department will prepare your paper for publication. This includes ensuring the following:

* All references, tables, and figures are properly cited

* All relevant supporting information is included in the manuscript submission,

* There are no issues that prevent the paper from being properly typeset

If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps.

Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Xiaohui Zhang

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 File

    (DOCX)

    pone.0311805.s001.docx (294.5KB, docx)
    S1 Raw data. Data used for this analysis are available as supporting information in data_raw.

    (ZIP)

    pone.0311805.s002.zip (140.5MB, zip)
    Attachment

    Submitted filename: Response to reviewers.docx

    pone.0311805.s003.docx (63.5KB, docx)

    Data Availability Statement

    All relevant data are within the paper and its Supporting Information files.


    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES