Skip to main content
PeerJ logoLink to PeerJ
. 2020 Feb 4;8:e8397. doi: 10.7717/peerj.8397

Development and validation of statistical shape models of the primary functional bone segments of the foot

Tamara M Grant 1,2, Laura E Diamond 1,2, Claudio Pizzolato 1,2, Bryce A Killen 3, Daniel Devaprakash 1,2, Luke Kelly 4, Jayishni N Maharaj 4, David J Saxby 1,2,
Editor: Tim Doyle
PMCID: PMC7006516  PMID: 32117607

Abstract

Introduction

Musculoskeletal models are important tools for studying movement patterns, tissue loading, and neuromechanics. Personalising bone anatomy within models improves analysis accuracy. Few studies have focused on personalising foot bone anatomy, potentially incorrectly estimating the foot’s contribution to locomotion. Statistical shape models have been created for a subset of foot-ankle bones, but have not been validated. This study aimed to develop and validate statistical shape models of the functional segments in the foot: first metatarsal, midfoot (second-to-fifth metatarsals, cuneiforms, cuboid, and navicular), calcaneus, and talus; then, to assess reconstruction accuracy of these shape models using sparse anatomical data.

Methods

Magnetic resonance images of 24 individuals feet (age = 28 ± 6 years, 52% female, height = 1.73 ± 0.8 m, mass = 66.6 ± 13.8 kg) were manually segmented to generate three-dimensional point clouds. Point clouds were registered and analysed using principal component analysis. For each bone segment, a statistical shape model and principal components were created, describing population shape variation. Statistical shape models were validated by assessing reconstruction accuracy in a leave-one-out cross validation. Statistical shape models were created by excluding a participant’s bone segment and used to reconstruct that same excluded bone using full segmentations and sparse anatomical data (i.e. three discrete points on each segment), for all combinations in the dataset. Tali were not reconstructed using sparse anatomical data due to a lack of externally accessible landmarks. Reconstruction accuracy was assessed using Jaccard index, root mean square error (mm), and Hausdorff distance (mm).

Results

Reconstructions generated using full segmentations had mean Jaccard indices between 0.77 ± 0.04 and 0.89 ± 0.02, mean root mean square errors between 0.88 ± 0.19 and 1.17 ± 0.18 mm, and mean Hausdorff distances between 2.99 ± 0.98 mm and 6.63 ± 3.68 mm. Reconstructions generated using sparse anatomical data had mean Jaccard indices between 0.67 ± 0.06 and 0.83 ± 0.05, mean root mean square error between 1.21 ± 0.54 mm and 1.66 ± 0.41 mm, and mean Hausdorff distances between 3.21 ± 0.94 mm and 7.19 ± 3.54 mm. Jaccard index was higher (P < 0.01) and root mean square error was lower (P < 0.01) in reconstructions from full segmentations compared to sparse anatomical data. Hausdorff distance was lower (P < 0.01) for midfoot and calcaneus reconstructions using full segmentations compared to sparse anatomical data.

Conclusion

For the first time, statistical shape models of the primary functional segments of the foot were developed and validated. Foot segments can be reconstructed with minimal error using full segmentations and sparse anatomical landmarks. In future, larger training datasets could increase statistical shape model robustness, extending use to paediatric or pathological populations.

Keywords: Musculoskeletal modelling, Statistical shape modelling, Subject-specific modelling, Foot bones

Introduction

Musculoskeletal models are tools to aid investigation of movement patterns, tissue loading, and neuromechanics (Bahl et al., 2019; Rajagopal et al., 2016; Sommer, Miller & Pijanowski, 1982). Bone anatomy plays an important role in muscle-tendon and articular force estimates through its influence on muscle attachment sites (Suwarganda et al., 2019) and articular geometry (Gerus et al., 2013). The foot is typically modelled as either a single rigid body or multiple rigid segments that represent the rear-, mid-, and fore-foot (Bruening, Cooney & Buczek, 2012; Leardini et al., 2007; Malaquias et al., 2017). Multi-segment foot modelling has improved our understanding of foot function, including foot contribution to power generation and absorption during locomotion (Bruening, Cooney & Buczek, 2012; Kelly, Lichtwark & Cresswell, 2015; Zelik & Honert, 2018), which is not well represented by single rigid body models (Zelik & Honert, 2018). However, studies commonly do not personalise bone geometry (Delp et al., 1990; Prinold et al., 2016), potentially effecting biomechanical analyses (Prinold et al., 2016).

Linear scaling of generic bone templates and creating models from medical imaging are two methods used to personalise bone geometry for musculoskeletal models. Linear scaling orthonormally scales a bone template to match an individual’s anthropometry. Generic bone templates are typically created from one individual or are averaged from a limited dataset (Sommer, Miller & Pijanowski, 1982). Generic models have been found to poorly characterise an individual’s bone geometry of the pelvis and femur. Indeed, reconstruction error for linearly scaled generic models has been found to be more than double that of personalised bone geometries (~14 mm compared to ~6 mm for the femur) (Suwarganda et al., 2019). This lack of fidelity to the individual being studied can potentially reduce the accuracy of subsequent biomechanical analyses, including muscle moment arms and joint contact forces (Bahl et al., 2019; Correa et al., 2011; Gerus et al., 2013; Lerner et al., 2015; Zelik & Honert, 2018; Zhang et al., 2016). Accurate bone geometry can be generated by segmenting medical images (Modenese et al., 2018). However, due to acquisition cost and processing time related to medical imaging (Correa et al., 2011), these methods are impractical for large sample sizes.

Statistical shape modelling is an alternative to medical imaging segmentation that can be used to reconstruct personalised bone geometries (Barratt et al., 2008; Suwarganda et al., 2019). Statistical shape modelling is a mathematical process that determines the shape variation in a training dataset using a statistical procedure (e.g. principal component analysis) that is computationally efficient and robust to artefacts (Zheng, Li & Szekely, 2017). Once established, a statistical shape model can be morphed to fit a known shape (e.g. a bone) (Barratt et al., 2008). Statistical shape models are well-established tools for image analysis (Cootes et al., 1995) and have previously been used to characterise and reconstruct the pelvis, femur, tibia, and patella (Bahl et al., 2019; Baka et al., 2012; Bryan, Nair & Taylor, 2009; Rao et al., 2013; Suwarganda et al., 2019; Zhang & Besier, 2017). On the other hand, few studies have applied statistical shape modelling to the foot bones (Melinska et al., 2017; Melinska et al., 2015). Two previous studies reported a statistical shape model of the cuboid, navicular, calcaneus, and talus, but did not assess model performance (i.e. reconstruction accuracy) (Melinska et al., 2017; Melinska et al., 2015). Statistical shape models of the primary functional segments of the foot have the potential to inform clinical decisions (by well monitoring growth & development and degeneration), prosthesis design, and preoperative planning (Melinska et al., 2017; Melinska et al., 2015). However, before statistical shape models of the foot segments can be used in musculoskeletal models, their ability to accurately reconstruct personalised bone geometries must be established.

The primary aim of this study was to develop and validate statistical shape models of the first metatarsal, midfoot (second-to-fifth metatarsals, medial, intermediate and lateral cuneiforms, cuboid, and navicular), calcaneus, and talus. The secondary aim of this study was to assess the performance of these statistical shape models when supplied with sparse anatomical data, commonly acquired in clinical gait laboratories, to evaluate potential for future clinical use. We used segmented bone geometries from magnetic resonance imaging (MRI) as our gold standard, and compared reconstruction accuracy of statistical shape models using both full segmentations from MRI and sparse anatomical data from commonly used anatomical landmarks (Leardini et al., 2007; Leardini et al., 1999). We hypothesised that the statistical shape models would accurately reconstruct (i.e. high shape similarity compared to gold standard) foot bone geometries using both full segmentations and sparse anatomical data, but reconstructions from full segmentations would be more accurate than those from sparse anatomical data.

Materials and Methods

Participants

Twenty-four participants (age = 28.2 ± 5.8 years, 46% female, height = 1.73 ± 0.08 m, mass = 67.1 ± 13.4 kg) underwent bilateral lower limb MRI at one of two imaging clinics (QSCAN, Parkwood Village, QLD, Australia, and Universal Medical Imaging, Calvary Hospital, Bruce, ACT, Australia). Scans at both locations were undertaken with a 3 T Philips Ingenia scanner (Philips Medical Systems, Best, Netherlands). Axial T1-weighted three-dimensional fast field echo sequences were acquired from the superior iliac crests to below the toes with participants extended and supine on the imaging gantry. Images were acquired in five stations, consisting of ~252 slices per station with 10 mm overlap between stations. Slice thickness and interslice gap were both 1 mm, and pixel resolution was 0.59 mm × 0.59 mm × 0.59 mm. Participants were enrolled in one of two studies with institutional ethical approval (Griffith Human Research Ethics Committee references 2017/521 and 2017/020). All participants provided written informed consent prior to participating.

Magnetic resonance image processing

Magnetic resonance imaging data were processed in two steps (Fig. 1): (1) medical image segmentation, and (2) statistical shape model generation. First, MRI data were segmented in Mimics v20 (Materialise, Leuven, Belgium) to create three-dimensional reconstructions of the four foot bone segments: first metatarsal, midfoot (second-to-fifth metatarsals, medial, intermediate and lateral cuneiforms, cuboid, and navicular), calcaneus, and talus. The talus and calcaneus were considered independently, because the subtalar joint and mid-tarsal joint represent two primary foot articulations. Segmentations were created by manually identifying outer contours of bone segments of interest on each image slice using greyscale thresholding to create masks (Bryan, Nair & Taylor, 2009). The second-to-fifth metatarsals, medial, intermediate and lateral cuneiforms, cuboid, and navicular were segmented as individual bones, but within the same mask (i.e. as bone body: the midfoot). The first metatarsal, calcaneus, and talus were segmented into individual masks. Masks were then wrapped and smoothed following the manufacturer’s guidelines to reduce stair-step artefacts (DeVries et al., 2008), and point clouds exported to stereolithography (STL) format.

Figure 1. Outline of the process of creating and validating a statistical shape model.

Figure 1

(A) Magnetic resonance image processing; (B) statistical shape model generation; and (C) statistical shape model validation.

Statistical shape model generation

Second, three-dimensional point clouds were used to create statistical shape models. Point clouds of the left foot bones were mirrored using the mirror function in 3-Matic (Materialise, Leuven, Belgium). Volumetric shape similarity was compared for left and mirrored right point clouds. For each point cloud, a three-dimensional coordinate system was defined consistent with recommendations from the International Society of Biomechanics for ankle joint coordinate systems (Wu et al., 2002). The coordinate system was used to align segmentations, where it was applied to each segmentation digitally using 3-Matic (Materialise, Leuven, Belgium), and extracted as three-dimensional spatial coordinates. A transformation matrix containing a rotation matrix, matrix padding, and three-dimensional spatial coordinates of the native and new positions of the point cloud was created. Point clouds were then transformed about the transformation matrix (Craig, 2005). The GIAS2 toolbox (https://pypi.org/project/gias2/) was then used to register the point clouds from each participant for a specific bone segment (e.g. calcaneus) in two ways. First, a non-rigid registration was performed using radial basis functions. Second, a rigid iterative closest-point registration (Besl & McKay, 1992) was performed. Once registered, a principal component analysis was performed. The principal components represent the variation across the data. A mean point cloud was then created using the first three principal components from the principal component analysis. The mean point cloud and associated principal components could then be used to reconstruct personalised bone geometries.

Statistical shape model validation

Each of the four statistical shape models (i.e. first metatarsal, midfoot, calcaneus, and talus) were validated by testing reconstruction accuracy in a leave-one-out cross validation. The GIAS2 toolbox was used to reconstruct bone geometries. Mean point clouds were fit to target data (i.e. full segmentations or sparse anatomical landmarks) using rigid body translation, rotation, and deformation along a specified number of principal components (seven or eight for full segmentation reconstructions, or three for the sparse anatomical landmarks), minimising the distance between the target data and mean point cloud. The objective of the optimisation was to minimise distance between target data (either the full segmentations or sparse anatomical landmarks) and the deformed shape model (Zhang, Hislop-Jambrich & Besier, 2016). A penalty weight (i.e. Mahalanobis distance) was used to constrain statistical shape model deformation by quantifying the similarity of the deformed and mean statistical shape models (Suwarganda et al., 2019; Zhang et al., 2016). Mahalanobis distance ranges from 0 to 1, where a value closer to one result in less similarity between the deformed and mean statistical shape models. Mahalanobis distance was set to 0.1, consistent with previous studies (Suwarganda et al., 2019), and was implemented to avoid spurious shape model deformation.

A leave-one-out cross validation was performed to assess reconstruction accuracy. In this validation, a statistical shape model is created by excluding one participant’s bone geometry, then used to reconstruct the bone excluded geometry, and then reconstruction accuracy assessed against excluded geometry. This process is repeated for all possible combinations in the entire dataset (29 first metatarsals, 33 mid-feet, 26 calcanei, 33 tali). Previous studies have used varying numbers of principal components in reconstructions, accounting for a range of population shape variation (Zhang et al., 2016; Zhang et al., 2014). One study selected a threshold of 75% population variation to select principal components for reconstructions (Zhang et al., 2016). To be conservative, the minimum number of principal components that accounted for 80% of shape variance was used for reconstructions from full segmentations in this study (Fig. 2). The performance of the statistical shape model was assessed using the accuracy of reconstructions generated from the X, Y, and Z coordinates of point clouds of full bone segmentations and the X, Y, and Z coordinates of sparse anatomical landmarks. Sparse anatomical landmarks were located on the full segmentations and their coordinates extracted to an Extensible Markup Language (XML) file. The sparse anatomical landmarks included in our method corresponded to the motion capture marker sets commonly used in gait analysis (Fig. 3) (Leardini et al., 2007; Leardini et al., 1999). Three landmarks were selected per bone segment (Leardini et al., 2007; Leardini et al., 1999). Each landmark was identified from manual segmentations for each individual and digitally placed using 3-matic (Materialise, Leuven, Belgium). The talus was excluded from the sparse analysis as it has no externally accessible anatomical landmarks. The sparse anatomical data reconstructions were performed along the first three principal components (Fig. 2). The first three principal components were used because they represented the maximum number that can be used to create unique weightings, given that we have included only three unique anatomical landmarks for each functional segment in the model (Wold, Esbensen & Geladi, 1987).

Figure 2. Percentage of shape variation explained by the first nine principal components for each functional segment.

Figure 2

(A) First metatarsal; (B) calcaneus; (C) midfoot; and (D) talus. Individual bars represent the percentage of shape variation explained by each principal component. A threshold of 80% total shape variation explained was used to determine the number of principal components used for reconstructions from complete segmentation (dashed vertical line). Three principal components were used for reconstructions generated from sparse anatomical landmarks (solid vertical line). The midfoot segment includes the second-to-fifth metatarsal, cuneiforms (medial, intermediate, and lateral), cuboid, and navicular.

Figure 3. Anatomical landmarks used for sparse reconstructions.

Figure 3

(A) First metatarsal; (B) midfoot; and (C) calcaneus. TL = most lateral projection of the head of the first metatarsal; TM = most medial projection of the head of the first metatarsal; TB = most medial projection of the base of the first metatarsal; VMH = head of the fifth metatarsal; SMH = head of the second metatarsal; VMB = base of the fifth metatarsal; SMB = base of the second metatarsal; ID = mid-point between the apex of the tuberosity of the navicular and the base of the fifth metatarsal; TN = most medial apex of the tuberosity of the navicular; PT = lateral apex of the peroneal tubercle of the calcaneus; ST = most medial apex of the sustenaculum tali; and CA = upper central ridge of the posterior surface of the calcaneus. The midfoot segment includes the second-to-fifth metatarsal, cuneiforms (medial, intermediate, and lateral), cuboid, and navicular. Note: (A) based on Leardini et al. (1999), (B) and (C) based on Leardini et al. (2007).

Statistical analysis

Volumetric shape similarity between left and mirrored right foot bones for each participant was assessed using Jaccard index (0–1). Accuracy of the reconstructed bone segments was assessed using a nearest neighbour algorithm to calculate Jaccard index, root mean square error (RMSE; mm), and Hausdorff distance (mm). Jaccard index assesses volumetric similarity by quantifying the intersection of two registered volumes (i.e. manual segmentation and statistical shape model reconstruction) relative to the union of two volumes (Taha & Hanbury, 2015), where an index of 1 indicates 100% overlapping volume. The RMSE is a measure of mean error between the reconstructed data (i.e. from full segmentations or sparse anatomical data) and the gold standard (i.e. full segmentations). Hausdorff distance is the maximum distance between two similar points on two registered point clouds (i.e. segmentation and reconstruction) through many-to-many correspondence (Taha & Hanbury, 2015). Shapiro–Wilk tests were used to assess data normality. Jaccard index, RMSE, and Hausdorff distance did not meet the assumptions of normality for any bone segments. Consequently, Wilcoxon Signed-Ranks tests were used to compare accuracy from reconstructions generated from full segmentations and sparse anatomical landmarks. Statistical Package for Social Sciences version 25 (SPSS Inc., Chicago, IL, USA) was used for all statistical analyses. Significance was set at 0.05.

Results

The fitting errors (RMSE) for the non-rigid registration (i.e. the first registration step) were 0.494 ± 0.05 mm, 0.642 ± 0.08 mm, 0.643 ± 1.17 mm, and 0.55 ± 0.12 mm for the first metatarsal, midfoot, calcaneus, and talus, respectively. The fitting errors approach the dimensions of the MRI resolution, and thus can be considered acceptable. The mean Jaccard indices between the left and mirrored right point clouds foot segment point clouds were 0.49 ± 0.14, 0.48 ± 0.1, 0.68 ± 0.12, and 0.76 ± 0.09 for the first metatarsal, midfoot, calcaneus, and talus bone segments, respectively. Principal component analysis revealed a typical decaying slope from low to high numbers of principal components (Fig. 2). Greater than 80% of sample variance was explained by seven principal components in the first metatarsal and talus, and eight principal components in the midfoot and calcaneus. As such, seven and eight principal components were used to reconstruct relevant bones from full segmentations (Fig. 2). Reconstructions generated using the statistical shape models and full segmentations for the first metatarsal, midfoot, calcaneus, and talus had mean Jaccard indices between 0.77 ± 0.04 (midfoot) and 0.88 ± 0.03 (talus), mean RMSE between 1.02 ± 0.26 mm (first metatarsal) and 1.28 ± 0.25 mm (calcaneus), and Hausdorff distances between 3.08 ± 1.04 mm (first metatarsal) and 6.63 ± 3.68 mm (midfoot) (Fig. 4).

Figure 4. Shape similarity for reconstructions generated from complete magnetic resonance imaging segmentations.

Figure 4

(A) First metatarsal Jaccard indices; (B) midfoot Jaccard indices; (C) calcaneus Jaccard indices; (D) talus Jaccard indices; (E) first metatarsal root mean square error; (F) midfoot root mean square error; (G) calcaneus metatarsal root mean square error; (H) talus metatarsal root mean square error; (I) first metatarsal Hausdorff distances; (J) midfoot Hausdorff distances; (K) calcaneus Hausdorff distances; and (L) talus Hausdorff distances. Horizontal dashed lines represent mean values. The midfoot segment includes the second-to-fifth metatarsals, cuneiforms (medial, intermediate, and lateral), cuboid, and navicular. RMSE—root mean square error.

For the reconstructions using sparse anatomical landmarks, the first three principal components from the relevant statistical shape models were used to reconstruct the first metatarsal, midfoot, and calcaneus (Fig. 5). The sum of the shape variation explained from the first three principal components was 71.2%, 57.9%, and 63.8% for the first metatarsal, midfoot, and calcaneus segments, respectively. The reconstructions had mean Jaccard indices between 0.67 ± 0.06 (midfoot) and 0.83 ± 0.05 (calcaneus), mean RMSE between 1.21 ± 0.55 mm (first metatarsal) and 1.66 ± 0.41 mm (calcaneus), and mean Hausdorff distances between 3.21 ± 0.94 mm (first metatarsal) and 7.19 ± 3.54 mm (midfoot) across all bone segments (Table 1).

Figure 5. Shape similarity for reconstructions generated from sparse anatomical landmarks.

Figure 5

(A) First metatarsal Jaccard indices; (B) midfoot Jaccard indices; (C) calcaneus Jaccard indices; (D) first metatarsal root mean square error; (E) midfoot metatarsal root mean square error; (F) calcaneus root mean square error; (G) first metatarsal Hausdorff distances; (H) midfoot Hausdorff distances; and (I) calcaneus Hausdorff distances. Horizontal dashed lines represent mean values. The midfoot segment includes the second-to-fifth metatarsals, cuneiforms (medial, intermediate, and lateral), cuboid, and navicular. RMSE—root mean square error.

Table 1. Comparison of shape similarity between reconstructions generated from complete magnetic resonance imaging segmentations and sparse anatomical landmarks.

Jaccard index P-value RMSE (mm) P-value Hausdorff distance (mm) P-value
Complete segmentation Sparse landmarks Complete segmentation Sparse landmarks Complete segmentation Sparse landmarks
First metatarsal 0.82 (0.05) 0.79 (0.06) <0.01* 1.02 (0.26) 1.21 (0.55) <0.01* 3.08 (1.04) 3.21 (0.94) 0.55
Midfoot 0.77 (0.04) 0.67 (0.06) <0.01* 1.07 (0.14) 1.45 (0.30) <0.01* 6.63 (3.68) 7.19 (3.54) 0.03*
Calcaneus 0.87 (0.03) 0.83 (0.05) <0.01* 1.28 (0.25) 1.66 (0.41) <0.01* 4.89 (1.91) 5.55 (1.73) 0.03*

Notes:

All values are mean (standard deviation).

RMSE—root mean square error; the midfoot segment includes the second-to-fifth metatarsals, cuneiforms (medial, intermediate, and lateral), cuboid, and navicular.

*

Denotes significant differences in shape similarity between reconstructions generated from complete segmentations and sparse anatomical landmarks using a Wilcoxon Signed-Ranks test.

The statistical shape models using full segmentations generated significantly more accurate bone reconstructions (Fig. 4) for the first metatarsal, midfoot, and calcaneus compared to those generated from sparse anatomical landmarks in terms of Jaccard indices (P < 0.01) and RMSE (P < 0.01) (Table 1). The statistical shape models using full segmentations generated significantly more accurate bone reconstructions of the calcaneus (P = 0.03) and midfoot (P = 0.03) in terms of Hausdorff distance. The best- and worst-case reconstruction examples for the reconstructions generated from full segmentation in terms of point-to-point (Euclidean) distance are presented visually in Fig. 6. The best- and worst-case reconstruction examples for the reconstructions generated from sparse anatomical landmarks in terms of Euclidean distance are presented visually in Fig. 7.

Figure 6. Best- and worst-case reconstructions generated from full magnetic resonance imaging (MRI) segmentations in terms of absolute Euclidean distance (mm).

Figure 6

(A) Plantar view of best first metatarsal reconstruction; (B) plantar view of worst first metatarsal reconstruction; (C) medial view of best first metatarsal reconstruction; (D) medial view of worst first metatarsal reconstruction; (E) plantar view of best calcaneus reconstruction; (F) plantar view of worst calcaneus reconstruction; (G) lateral view of best calcaneus reconstruction; (H) lateral view of worst calcaneus reconstruction; (I) plantar view of best midfoot reconstruction; (J) plantar view of worst midfoot reconstruction; (K) proximal view of best midfoot reconstruction; (L) proximal view of worst midfoot reconstruction; (M) plantar view of best talus reconstruction; (N) plantar view of worst talus reconstruction; (O) lateral view of best talus reconstruction; and (P) lateral view of worst talus reconstruction. Non-shaded background: best-case reconstruction from the leave-one-out cross validation; and shaded background: worst-case reconstruction from the leave-one-out cross validation. Euclidean distance is represented by the colour map which ranges from 0 mm (cream) to 8 mm (brown). The midfoot segment includes the second-to-fifth metatarsals, cuneiforms (medial, intermediate, and lateral), cuboid, and navicular bones.

Figure 7. Best- and worst-case reconstructions generated from sparse anatomical data in terms of absolute Euclidean distance (mm).

Figure 7

(A) Plantar view of best first metatarsal reconstruction; (B) plantar view of worst first metatarsal reconstruction; (C) medial view of best first metatarsal reconstruction; (D) medial view of worst first metatarsal reconstruction; (E) plantar view of best midfoot reconstruction; (F) plantar view of worst midfoot reconstruction; (G) proximal view of best midfoot reconstruction; (H) proximal view of worst midfoot reconstruction (I) plantar view of best calcaneus reconstruction; (J) plantar view of worst calcaneus reconstruction; (K) lateral view of best calcaneus reconstruction; and (L) lateral view of worst calcaneus reconstruction. Non-shaded background: best-case reconstruction from the leave-one-out cross validation; and shaded background: worst-case reconstruction from the leave-one-out cross validation. Euclidean distance is represented by the colour map which ranges from 0 mm (cream) to 8 mm (brown). The midfoot segment includes the second-to-fifth metatarsals, cuneiforms (medial, intermediate, and lateral), cuboid, and navicular bones.

Discussion

This study is the first to develop and validate statistical shape models of the primary functional segments of the foot. For statistical shape models to be useful, they must accurately reconstruct bone geometries without a requirement for time-consuming and costly medical imaging. Our statistical shape models were able to reconstruct bone geometries from sparse anatomical landmarks, albeit with lower accuracy than those generated from full segmentations. Statistical shape models, such as those presented in this study, may be an effective way to personalise representations of the functional segments of the foot in a musculoskeletal model. Future studies may consider comparing personalised bone geometries from these statistical shape models to generic scaled models establishing their suitability for biomechanical analyses.

First metatarsal, calcaneus, and talus geometries generated using statistical shape models and full MRI segmentations had excellent accuracy (>80% volumetric similarity), consistent with previous reports focused on other lower limb bones (Suwarganda et al., 2019). Reconstructions of the midfoot segment had lower volumetric similarity (Jaccard index = 0.77) compared to the other analysed bone segments, which is unsurprising given the midfoot segment is markedly more complex (nine bones) compared to segments consisting of a single bone. Previous studies (Zhang & Besier, 2017) have reported femur reconstructions from statistical shape models with lower error than we found for the primary foot bones. This discrepancy may relate to their comparatively large training datasets (n = 204) (Zhang & Besier, 2017) and the lesser geometric complexity of the femur compared to the foot bones. Statistical shape models of the navicular, cuboid, calcaneus, and talus have been created (Melinska et al., 2017; Melinska et al., 2015) using spherical harmonics (a set of orthonormal basis functions that can be used to characterise three-dimensional objects) (Heimann & Meinzer, 2009; Schönefeld, 2005) rather than principal component analysis. These prior foot-focused studies presented results for a subset of the foot-ankle complex (navicular, cuboid, calcaneus, and talus) with the aim of a future finite-element modelling application. These studies did not assess the reconstruction accuracy of their statistical shape model, thus preventing direct comparison with our results. Other approaches have been used to develop statistical shape models of the foot bones (Melinska et al., 2017; Melinska et al., 2015); however, to the authors’ knowledge, this is the first study to validate the reconstruction accuracy of statistical shape models of the primary functional segments of the foot.

Foot bone geometries reconstructed using sparse anatomical landmarks yielded shape similarities consistent with previous studies of lower limb bones, where motion capture landmarks reconstructed femur and pelvis geometries with Jaccard indices of ~0.68 and 0.41, respectively (Suwarganda et al., 2019). Pelvis and femur bone geometries generated using a statistical shape model and sparse anatomical data acquired from skin-surface markers remain superior to linearly-scaled generic models in terms of RMSE and Jaccard index (Suwarganda et al., 2019), in addition to accuracy of muscle moment arms and joint contact force estimates (Correa et al., 2011; Gerus et al., 2013; Lerner et al., 2015).

Previous studies have used incomplete segmentations of medical imaging to reconstruct lower limb bone geometries (Suwarganda et al., 2019) and sparse anatomical landmarks to estimate lower limb position and muscle attachment sites (Zhang et al., 2016). Our statistical shape models were able to reconstruct the first metatarsal, midfoot, and calcaneus bone segments using sparse anatomical landmarks, producing Jaccard indices between 0.67 and 0.83. Previous statistical shape models of the foot (Melinska et al., 2017; Melinska et al., 2015) have not been used in conjunction with sparse anatomical landmarks, thus this paper presents the first foray into this area. The lower reconstruction accuracy observed when using sparse anatomical landmarks compared to full segmentation may relate to less information provided to the model for reconstruction (fewer number of principal components used—three vs. seven or eight) and could potentially be improved if additional information was provided to the model, for example additional anatomical landmarks or anthropometric measurements. We excluded the talus from our sparse landmark analysis as it had no externally accessible anatomical landmarks. In future, scaling a statistical shape model of the talus using anthropometric data (e.g. foot length) may be investigated as an alternative to using costly medical imaging such as MRI. Alternatively, three-dimensional ultrasound may provide a fast, affordable, and accessible alternative to acquire critical sparse data of an individual’s musculoskeletal anatomy, including data that is otherwise inaccessible (e.g. regarding the talus) without expensive MRI or ionising radiation from X-ray (Cong et al., 2017; Raum et al., 2014).

Statistical shape models can be used to characterise natural shape variation in anatomy not well represented by commonly used generic bone templates (Bryan, Nair & Taylor, 2009; Zhang & Besier, 2017), but without the cost, time, and potential risk associated with medical imaging. The statistical shape models presented in this study can be used in conjunction with sparse anatomical landmarks to reconstruct personalised models of the primary functional segments of the foot with minimal error. Using sparse anatomical landmarks to create personalised geometric bone models can improve biomechanical analyses by increasing model precision of the individual being studied without requirement for laborious manual image segmentation (Krähenbühl et al., 2017; Suwarganda et al., 2019). Statistical shape models of the primary functional segments of the foot also have a range of applications in the medical field, including improving preoperative planning and postoperative assessment, monitoring growth in children and degeneration of bone in individuals with conditions such as osteoarthritis, and assessing implant placement (Melinska et al., 2015).

This study has limitations that should be considered. First, the phalanges were not included, although the relevance of personalising these structures for subsequent biomechanical analysis of locomotion is unclear. Second, the talus segment was not reconstructed using sparse anatomical landmarks due to a lack of externally accessible landmarks. As mentioned above, future investigations should develop a method to reconstruct talus geometry using less costly medical imaging (e.g. three-dimensional ultrasound) (Devaprakash et al., 2019; Treece et al., 2003). Ultrasound can also provide a much higher image resolution than routine MRI scans (Kositsky et al., 2020). Third, in a preliminary analysis (data not reported) we observed RMSE similar to previous reports when using the number of principal components corresponding to 75% of sample variance, thus we did not explore any effects of varying the number of principal components (Zhang et al., 2016). Fourth, anatomical landmarks used in sparse reconstructions were digitally placed directly to the bone surface of three-dimensional reconstructions. In future, if skin-surface markers are used (e.g. from a clinical gait laboratory), protocols to limit soft tissue sliding and ensure quality anatomical landmark identification should be developed, and their effects on bone reconstruction accuracy assessed. Fifth, there are some regional anatomical errors in reconstructions, particularly in those created from sparse anatomical data (e.g. the posterior talar articular surface of the calcaneus, and the dorsal portion of the calcaneus) (Fig. 7). These regional errors are not present in all reconstructions, however they are a consideration for future applications and subsequent analyses. Sixth, the statistical shape models in the present study were created using data from healthy adults. It is unclear whether these shape models can be extended to children or people with bone pathologies that have abnormal bone geometry. Seventh, these statistical shape models were created using the feet of 24 individuals (29 first metatarsals, 33 mid-feet, 26 calcanei, 33 tali). Including more data in the training set will increase the robustness of the models.

Conclusions

Results show the primary functional segments of the foot can be reconstructed with minimal error using segmentations from MRI. Importantly, a subset of the foot bone segments can be reconstructed with minimal error using sparse anatomical landmarks that are consistent with skin-surface marker locations from clinical motion capture methods. Statistical shape models could reduce the need for expensive medical imaging and subsequent tedious manual image segmentation.

Supplemental Information

Supplemental Information 1. Individual participant information.
DOI: 10.7717/peerj.8397/supp-1

Funding Statement

This work was supported by the Australian Research Council (LP150100905). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Additional Information and Declarations

Competing Interests

The authors declare that they have no competing interests.

Author Contributions

Tamara M. Grant conceived and designed the experiments, performed the experiments, analysed the data, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

Laura E. Diamond conceived and designed the experiments, authored or reviewed drafts of the paper, and approved the final draft.

Claudio Pizzolato conceived and designed the experiments, authored or reviewed drafts of the paper, and approved the final draft.

Bryce A. Killen conceived and designed the experiments, authored or reviewed drafts of the paper, and approved the final draft.

Daniel Devaprakash conceived and designed the experiments, authored or reviewed drafts of the paper, and approved the final draft.

Luke Kelly conceived and designed the experiments, authored or reviewed drafts of the paper, and approved the final draft.

Jayishni N. Maharaj conceived and designed the experiments, authored or reviewed drafts of the paper, and approved the final draft.

David J. Saxby conceived and designed the experiments, authored or reviewed drafts of the paper, and approved the final draft.

Human Ethics

The following information was supplied relating to ethical approvals (i.e. approving body and any reference numbers):

Griffith Human Research Ethics Committee (Ethical application reference numbers: 2017/521 and 2017/020) granted ethical approval to carry out this study.

Data Availability

The following information was supplied regarding data availability:

The data is available at Zenodo: Grant, Tamara; Diamond, Laura; Pizzolato, Claudio; Killen, Bryce; Devaprakash, Daniel; Kelly, Luke; Maharaj, Jayishni; Saxby, David. (2019). Development and validation of statistical shape models of the primary functional bone segments of the foot. (Data set). Zenodo. DOI 10.5281/zenodo.3464747.

References

  • Bahl et al. (2019).Bahl JS, Zhang J, Killen BA, Taylor M, Solomon LB, Arnold JB, Lloyd DG, Besier TF, Thewlis D. Statistical shape modelling versus linear scaling: effects on predictions of hip joint centre location and muscle moment arms in people with hip osteoarthritis. Journal of Biomechanics. 2019;85:164–172. doi: 10.1016/j.jbiomech.2019.01.031. [DOI] [PubMed] [Google Scholar]
  • Baka et al. (2012).Baka N, De Bruijne M, Van Walsum T, Kaptein BL, Giphart JE, Schaap M, Niessen WJ, Lelieveldt BPF. Statistical shape model-based femur kinematics from biplane fluoroscopy. IEEE Transactions on Medical Imaging. 2012;31(8):1573–1583. doi: 10.1109/TMI.2012.2195783. [DOI] [PubMed] [Google Scholar]
  • Barratt et al. (2008).Barratt DC, Chan CSK, Edwards PJ, Penney GP, Slomczykowski M, Carter TJ, Hawkes DJ. Instantiation and registration of statistical shape models of the femur and pelvis using 3D ultrasound imaging. Medical Image Analysis. 2008;12(3):358–374. doi: 10.1016/j.media.2007.12.006. [DOI] [PubMed] [Google Scholar]
  • Besl & McKay (1992).Besl PJ, McKay ND. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1992;14(2):239–256. doi: 10.1109/34.121791. [DOI] [Google Scholar]
  • Bruening, Cooney & Buczek (2012).Bruening DA, Cooney KM, Buczek FL. Analysis of a kinetic multi-segment foot model part II: kinetics and clinical implications. Gait & Posture. 2012;35(4):535–540. doi: 10.1016/j.gaitpost.2011.11.012. [DOI] [PubMed] [Google Scholar]
  • Bryan, Nair & Taylor (2009).Bryan R, Nair PB, Taylor M. Use of a statistical model of the whole femur in a large scale, multi-model study of femoral neck fracture risk. Journal of Biomechanics. 2009;42(13):2171–2176. doi: 10.1016/j.jbiomech.2009.05.038. [DOI] [PubMed] [Google Scholar]
  • Cong et al. (2017).Cong W, Yang J, Ai D, Song H, Chen G, Liang X, Liang P, Wang Y. Global patch matching (GPM) for freehand 3D ultrasound reconstruction. BioMedical Engineering Online. 2017;16(1):124. doi: 10.1186/s12938-017-0411-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Cootes et al. (1995).Cootes TF, Taylor CJ, Cooper DH, Graham J. Active shape models-their training and application. Computer Vision and Image Understanding. 1995;61(1):38–59. doi: 10.1006/cviu.1995.1004. [DOI] [Google Scholar]
  • Correa et al. (2011).Correa TA, Baker R, Kerr Graham H, Pandy MG. Accuracy of generic musculoskeletal models in predicting the functional roles of muscles in human gait. Journal of Biomechanics. 2011;44(11):2096–2105. doi: 10.1016/j.jbiomech.2011.05.023. [DOI] [PubMed] [Google Scholar]
  • Craig (2005).Craig JJ. Introduction to robotics: mechanics and control. Third Edition. Upper Saddle River, New Jersey: Pearson Education; 2005. [Google Scholar]
  • Delp et al. (1990).Delp SL, Loan JP, Hoy MG, Zajac FE, Topp EL, Rosen JM. An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures. IEEE Transactions on Biomedical Engineering. 1990;37(8):757–767. doi: 10.1109/10.102791. [DOI] [PubMed] [Google Scholar]
  • Devaprakash et al. (2019).Devaprakash D, Lloyd DG, Barrett RS, Obst SJ, Kennedy B, Adams KL, Hunter A, Vlahovich N, Pease DL, Pizzolato C. Magnetic resonance imaging and freehand 3-D ultrasound provide similar estimates of free achilles tendon shape and 3-D geometry. Ultrasound in Medicine & Biology. 2019;45(11):2898–2905. doi: 10.1016/j.ultrasmedbio.2019.07.679. [DOI] [PubMed] [Google Scholar]
  • DeVries et al. (2008).DeVries NA, Gassman EE, Kallemeyn NA, Shivanna KH, Magnotta VA, Grosland NM. Validation of phalanx bone three-dimensional surface segmentation from computed tomography images using laser scanning. Skeletal Radiology. 2008;37(1):35–42. doi: 10.1007/s00256-007-0386-3. [DOI] [PubMed] [Google Scholar]
  • Gerus et al. (2013).Gerus P, Sartori M, Besier TF, Fregly BJ, Delp SL, Banks SA, Pandy MG, D’Lima DD, Lloyd DG. Subject-specific knee joint geometry improves predictions of medial tibiofemoral contact forces. Journal of Biomechanics. 2013;46(16):2778–2786. doi: 10.1016/j.jbiomech.2013.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Heimann & Meinzer (2009).Heimann T, Meinzer HP. Statistical shape models for 3D medical image segmentation: a review. Medical Image Analysis. 2009;13(4):543–563. doi: 10.1016/j.media.2009.05.004. [DOI] [PubMed] [Google Scholar]
  • Kelly, Lichtwark & Cresswell (2015).Kelly LA, Lichtwark G, Cresswell AG. Active regulation of longitudinal arch compression and recoil during walking and running. Journal of the Royal Society Interface. 2015;12(102):20141076. doi: 10.1098/rsif.2014.1076. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Kositsky et al. (2020).Kositsky A, Gonçalves BAM, Stenroth L, Barrett RS, Diamond LE, Saxby DJ. Reliability and validity of ultrasonography for measurement of hamstring muscle and tendon cross-sectional area. Ultrasound in Medicine & Biology. 2020;46(1):55–63. doi: 10.1016/j.ultrasmedbio.2019.09.013. [DOI] [PubMed] [Google Scholar]
  • Krähenbühl et al. (2017).Krähenbühl N, Horn-Lang T, Hintermann B, Knupp M. The subtalar joint: a complex mechanism. EFORT Open Reviews. 2017;2(7):309–316. doi: 10.1302/2058-5241.2.160050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Leardini et al. (2007).Leardini A, Benedetti MG, Berti L, Bettinelli D, Nativo R, Giannini S. Rear-foot, mid-foot and fore-foot motion during the stance phase of gait. Gait & Posture. 2007;25(3):453–462. doi: 10.1016/j.gaitpost.2006.05.017. [DOI] [PubMed] [Google Scholar]
  • Leardini et al. (1999).Leardini A, Benedetti MG, Catani F, Simoncini L, Giannini S. An anatomically based protocol for the description of foot segment kinematics during gait. Clinical Biomechanics. 1999;14(8):528–536. doi: 10.1016/S0268-0033(99)00008-X. [DOI] [PubMed] [Google Scholar]
  • Lerner et al. (2015).Lerner ZF, DeMers MS, Delp SL, Browning RC. How tibiofemoral alignment and contact locations affect predictions of medial and lateral tibiofemoral contact forces. Journal of Biomechanics. 2015;48(4):644–650. doi: 10.1016/j.jbiomech.2014.12.049. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Malaquias et al. (2017).Malaquias TM, Silveira C, Aerts W, De Groote F, Dereymaeker G, Vander Sloten J, Jonkers I. Extended foot-ankle musculoskeletal models for application in movement analysis. Computer Methods in Biomechanics and Biomedical Engineering. 2017;20(2):153–159. doi: 10.1080/10255842.2016.1206533. [DOI] [PubMed] [Google Scholar]
  • Melinska et al. (2017).Melinska AU, Romaszkiewicz P, Wagel J, Antosik B, Sasiadek M, Iskander DR. Statistical shape models of cuboid, navicular and talus bones. Journal of Foot and Ankle Research. 2017;10(1):6. doi: 10.1186/s13047-016-0178-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Melinska et al. (2015).Melinska AU, Romaszkiewicz P, Wagel J, Sasiadek M, Iskander DR. Statistical, morphometric, anatomical shape model (atlas) of calcaneus. PLOS ONE. 2015;10(8):e0134603. doi: 10.1371/journal.pone.0134603. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Modenese et al. (2018).Modenese L, Montefiori E, Wang A, Wesarg S, Viceconti M, Mazzà C. Investigation of the dependence of joint contact forces on musculotendon parameters using a codified workflow for image-based modelling. Journal of Biomechanics. 2018;73:108–118. doi: 10.1016/j.jbiomech.2018.03.039. [DOI] [PubMed] [Google Scholar]
  • Prinold et al. (2016).Prinold JAI, Mazzà C, Di Marco R, Hannah I, Malattia C, Magni-Manzoni S, Petrarca M, Ronchetti AB, Tanturri de Horatio L, Van Dijkhuizen EHP, Wesarg S, Viceconti M, MD-PAEDIGREE Consortium A patient-specific foot model for the estimate of ankle joint forces in patients with juvenile idiopathic arthritis. Annals of Biomedical Engineering. 2016;44(1):247–257. doi: 10.1007/s10439-015-1451-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Rajagopal et al. (2016).Rajagopal A, Dembia CL, DeMers MS, Delp DD, Hicks JL, Delp SL. Full-body musculoskeletal model for muscle-driven simulation of human gait. IEEE Transactions on Biomedical Engineering. 2016;63(10):2068–2079. doi: 10.1109/TBME.2016.2586891. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Rao et al. (2013).Rao C, Fitzpatrick CK, Rullkoetter PJ, Maletsky LP, Kim RH, Laz PJ. A statistical finite element model of the knee accounting for shape and alignment variability. Medical Engineering and Physics. 2013;35(10):1450–1456. doi: 10.1016/j.medengphy.2013.03.021. [DOI] [PubMed] [Google Scholar]
  • Raum et al. (2014).Raum K, Grimal Q, Varga P, Barkmann R, Glüer CC, Laugier P. Ultrasound to assess bone quality. Current Osteoporosis Reports. 2014;12(2):154–162. doi: 10.1007/s11914-014-0205-4. [DOI] [PubMed] [Google Scholar]
  • Schönefeld (2005).Schönefeld V. Spherical harmonics. 2005. http://limbicsoft.com/volker/prosem_paper.pdf. [1 July 2005]. http://limbicsoft.com/volker/prosem_paper.pdf
  • Sommer, Miller & Pijanowski (1982).Sommer HJ, III, Miller NR, Pijanowski GJ. Three-dimensional osteometric scaling and normative modelling of skeletal segments. Journal of Biomechanics. 1982;15(3):171–180. doi: 10.1016/0021-9290(82)90249-4. [DOI] [PubMed] [Google Scholar]
  • Suwarganda et al. (2019).Suwarganda EK, Diamond LE, Lloyd DG, Besier TF, Zhang J, Killen BA, Savage TN, Saxby DJ. Minimal medical imaging can accurately reconstruct geometric bone models for musculoskeletal models. PLOS ONE. 2019;14(2):e0205628. doi: 10.1371/journal.pone.0205628. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Taha & Hanbury (2015).Taha AA, Hanbury A. Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Medical Imaging. 2015;15(1):29. doi: 10.1186/s12880-015-0068-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Treece et al. (2003).Treece G, Gee AH, Prager RW, Cash CJC, Berman LH. High-definition freehand 3-D ultrasound. Ultrasound in Medicine & Biology. 2003;29(4):529–546. doi: 10.1016/S0301-5629(02)00735-4. [DOI] [PubMed] [Google Scholar]
  • Wold, Esbensen & Geladi (1987).Wold S, Esbensen K, Geladi P. Principal component analysis. Chemometrics and Intelligent Laboratory Systems. 1987;2(1–3):37–52. doi: 10.1016/0169-7439(87)80084-9. [DOI] [Google Scholar]
  • Wu et al. (2002).Wu G, Siegler S, Allard P, Kirtley C, Leardini A, Rosenbaum D, Whittle M, D’Lima DD, Cristofolini L, Witte H, Schmid O, Stokes I. ISB recommendation on definitions of joint coordinate system of various joints for the reporting of human joint motion—part I: ankle, hip, and spine. Journal of Biomechanics. 2002;35(4):543–548. doi: 10.1016/S0021-9290(01)00222-6. [DOI] [PubMed] [Google Scholar]
  • Zelik & Honert (2018).Zelik KE, Honert EC. Ankle and foot power in gait analysis: Implications for science, technology and clinical assessment. Journal of Biomechanics. 2018;75:1–12. doi: 10.1016/j.jbiomech.2018.04.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • Zhang & Besier (2017).Zhang J, Besier TF. Accuracy of femur reconstruction from sparse geometric data using a statistical shape model. Computer Methods in Biomechanics and Biomedical Engineering. 2017;20(5):566–576. doi: 10.1080/10255842.2016.1263301. [DOI] [PubMed] [Google Scholar]
  • Zhang et al. (2016).Zhang J, Fernandez J, Hislop-Jambrich J, Besier TF. Lower limb estimation from sparse landmarks using an articulated shape model. Journal of Biomechanics. 2016;49(16):3875–3881. doi: 10.1016/j.jbiomech.2016.10.021. [DOI] [PubMed] [Google Scholar]
  • Zhang, Hislop-Jambrich & Besier (2016).Zhang J, Hislop-Jambrich J, Besier TF. Predictive statistical models of baseline variations in 3-D femoral cortex morphology. Medical Engineering and Physics. 2016;38(5):450–457. doi: 10.1016/j.medengphy.2016.02.003. [DOI] [PubMed] [Google Scholar]
  • Zhang et al. (2014).Zhang J, Malcolm D, Hislop-Jambrich J, Thomas CDL, Nielsen PMF. An anatomical region-based statistical shape model of the human femur. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization. 2014;2(3):176–185. doi: 10.1080/21681163.2013.878668. [DOI] [Google Scholar]
  • Zheng, Li & Szekely (2017).Zheng G, Li S, Szekely G. Statistical shape and deformation analysis: methods, implementation and applications. First Edition. Cambridge: Academic Press; 2017. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Information 1. Individual participant information.
DOI: 10.7717/peerj.8397/supp-1

Data Availability Statement

The following information was supplied regarding data availability:

The data is available at Zenodo: Grant, Tamara; Diamond, Laura; Pizzolato, Claudio; Killen, Bryce; Devaprakash, Daniel; Kelly, Luke; Maharaj, Jayishni; Saxby, David. (2019). Development and validation of statistical shape models of the primary functional bone segments of the foot. (Data set). Zenodo. DOI 10.5281/zenodo.3464747.


Articles from PeerJ are provided here courtesy of PeerJ, Inc

RESOURCES