Neutral evolution of human enamel–dentine junction morphology

Significance

Of the skeleton, teeth preserve well in the fossil record and have the strongest genetic signature and thus great potential for reconstructing evolutionary processes involved in human origins. The enamel–dentine junction (EDJ) is especially important, because it is more frequently preserved and somewhat decoupled from dietary function. Researchers have used the EDJ to test hypotheses of human evolution, but this trait has not yet been comprehensively studied across human populations. This study is the largest investigation of EDJ morphology to date, spanning 161 archaeological individuals from six continents. These data demonstrate that the EDJ evolves neutrally, and biomedical imaging of EDJ morphology can therefore be used as a noninvasive method to investigate ancient population movement and human genetic structure.

Abstract

Teeth have been studied for decades and continue to reveal information relevant to human evolution. Studies have shown that many traits of the outer enamel surface evolve neutrally and can be used to infer human population structure. However, many of these traits are unavailable in archaeological and fossil individuals due to processes of wear and taphonomy. Enamel–dentine junction (EDJ) morphology, the shape of the junction between the enamel and the dentine within a tooth, captures important information about tooth development and vertebrate evolution and is informative because it is subject to less wear and thus preserves more anatomy in worn or damaged specimens, particularly in mammals with relatively thick enamel like hominids. This study looks at the molar EDJ across a large sample of human populations. We assessed EDJ morphological variation in a sample of late Holocene modern humans (n = 161) from archaeological populations using μ-CT biomedical imaging and geometric morphometric analyses. Global variation in human EDJ morphology was compared to the statistical expectations of neutral evolution and “Out of Africa” dispersal modeling of trait evolution. Significant correlations between phenetic variation and neutral genetic variation indicate that EDJ morphology has evolved neutrally in humans. While EDJ morphology reflects population history, its global distribution does not follow expectations of the Out of Africa dispersal model. This study increases our knowledge of human dental variation and contributes to our understanding of dental development more broadly, with important applications to the investigation of population history and human genetic structure.

Teeth have been studied for decades and continue to reveal information relevant to human evolution. Many parts of the primate dentition are heritable and vary significantly inter- and intraspecifically, with variation resulting from both neutral (e.g., drift and migration) and adaptive processes (1–6). A large body of work has also demonstrated that neutral genetic variation in modern humans is patterned geographically with significant correlations between genetic variation and phenotypic variation of neutrally evolving traits (7–16).

Neutrally evolving phenotypic traits, or traits that are free from adaptive pressures, provide considerable insight into ancient human population structure and population movement in archaic and fossil communities (8, 11–14, 16). There is increasing interest in which traits of the human skeleton capture neutral genetic evolution and can be used in place of, or in combination with, more destructive DNA analyses, thereby allowing for increased sampling that preserves and protects fragile and rare specimens including holotypes and irreplaceable archaeological samples (7). The dentition is an ideal functional anatomical unit in which to investigate adaptive and neutral genetic processes, as teeth develop before eruption and function and are thus subject to fewer environmental perturbations, and dental morphology is significantly heritable (3, 17–20). To date, several studies have investigated neutral evolution of external dental morphologies (11, 21–24), but global variation in the internal morphology of the teeth remains understudied. Assessing whether dental morphology evolves neutrally in humans and other primates is of particular interest to paleoanthropologists, as neutral traits offer a way to characterize population movement in the absence of readily available genetic data, as is the case in the fossil record. This offers the opportunity to test hypotheses of range expansions, such as those proposed by the “Out of Africa” model. On the other hand, if dental traits are not evolving neutrally, they are instead likely the result of adaptive processes and therefore provide crucial insight into human origins and the evolution of human biological variation.

The outer enamel surface (OES) of the dentition has been well-studied, with research contributing to our understanding of dental development [through studies using a phenotype–genotype back approach (3)], diet, attrition, ontogeny, and morphological evolution (25–33). However, the external morphology of the dentition is subject to extensive wear processes during an individual’s lifespan. This is particularly notable in archaic and fossil populations, as well as contemporary human populations with a nonindustrialized, processed diet, limiting the utility of these morphologies in many studies (34–37). With advances in scanning technologies alleviating the need for destructive analyses (38, 39), greater attention has now been placed on the internal morphology of teeth (34–36, 40–42).

The enamel–dentine junction (EDJ) is the boundary between the enamel crown of the tooth and the underlying supportive tissues, and it captures important information about tooth development and evolution (36, 39, 42–45). The morphology of the EDJ (SI Appendix, Fig. S1) is informative, because it is not subject to the same extent of wear as the OES and thus preserves more morphology in worn specimens, particularly in mammals with relatively thick enamel like hominids (34–37). Additionally, the EDJ is formed early in development, marking the shape of the tooth at the beginning of mineralization prior to eruption, and is thus less subject to environmental and developmental perturbation (37, 39). The early development of the EDJ suggests that its topography is largely genetically determined, making it an excellent trait for tests of variation and population movement in the archaeological and fossil records.

Paleoanthropologists are particularly interested in the EDJ, because it is often the most well-preserved tooth anatomy in the fossil record (34–36). Morphological variation of the EDJ has been used extensively to investigate taxonomic relationships in the hominoid fossil record (34, 35, 42, 44, 46–48) but has not yet been comprehensively explored in modern humans. Many studies suggest that variation is taxonomically patterned (34, 39, 43). However, other studies report that considerable variation in traits of the EDJ precludes their use in taxonomic assessment (36, 37). Investigations of morphological concordance between the EDJ and OES have reported a strong phenotypic relationship between discrete traits of these anatomical layers in primates (48–50), especially for Pan (43), but others argue that anthropoids have relatively low levels of concordance compared to other euarchontoglirans (51). Overall, research suggests that traits of the EDJ and OES can be differentially expressed, and homologous accessory cusps at the OES may have different developmental origins and underlying EDJ morphology (36, 44, 45). Additionally, enamel deposition and thickness vary across the tooth surface and interspecifically (52–57), and quantitative genetic analyses of enamel thickness in primates indicate a complex genetic archetype that is additive and independent of sex and tooth size (58). There is also considerable metameric variation in EDJ morphology, and humans frequently express greater variation along the tooth row of a single individual than between individuals (39, 44). Metameric variation in EDJ morphology increases along the molar row, consistent with studies on other dental traits in primates (25, 40, 41, 47, 59). Given this reported variation, it has become essential to understand the extent of disparity in EDJ morphology before this well-preserved phenotype can be utilized in archaeological and paleontological investigations of taxonomy and evolution.

This study is of particular importance to investigation of human origins using the fossil record because it: 1) Uses a methodology that takes principles and theories from evolutionary genetics and applies them to morphological data in a manner that is directly applicable to the study of fossils; and, 2) the EDJ, the morphological trait assessed here, is widely preserved in the fossil record. Using a combination of traditional and modern geometric morphometric methods, this study undertakes an investigation of global variation in the internal dental anatomy (EDJ) of 161 late Holocene modern humans spanning six continents (Fig. 1).

Fig. 1.

Global variation in EDJ surface anatomy as represented by exemplary individuals from each continental sample. See Table 1 for continental sample sizes.

We used Holocene individuals to limit the variation introduced from more recent and wide-spread interregional and intercontinental movement (60). This dataset constitutes one of the most comprehensive investigations of human EDJ morphology to date. Our data capture the three-dimensional (3D) surface anatomy of the EDJ. We focused on the left permanent first mandibular molar and tested two hypotheses: 1) Global patterns of EDJ morphology vary according to expectations of neutral evolution, and 2) global patterns of human EDJ morphology vary according to the expectations of the Out of Africa dispersal hypothesis.

Global Patterns of EDJ Morphology Vary According to Expectations of Neutral Evolution.

We first investigate the mode of evolution of human dental morphology using EDJ topography. Our hypothesis is that phenetic distance calculated from EDJ variation of the first molar is a proxy of genetic distance. This invokes the classic model of phenetic variance being the result of additive genetic variance—that is, linear genotype to phenotype (G:P) mapping (15)—and tests whether the EDJ has evolved neutrally (where genetic drift and migration are the causative mechanisms of variation) by determining whether global phenotypic variation has a linear relationship to global differentiation in neutral genetic markers.

Furthermore, under “neutral-like” trait evolution and human dispersal, an isolation-by-distance pattern can be expected, in which phenotypic distance increases with increasing geographic distance between populations (16, 61–64). Using μ-CT biomedical imaging to characterize internal anatomies, particularly those that had previously been disregarded or assumed to be useful only in destructive DNA analyses, is a powerful method for investigating neutral genetic variation and population movement, as was recently shown using the inner ear bony labyrinth (7). Research focused on global variation of external dental morphology has reported neutral evolution of those traits (21–24). This study builds from this previous work and predicts a similar result for the underlying EDJ morphology.

Global Patterns of Human EDJ Morphology Vary According to the Expectations of the Out of Africa Dispersal Hypothesis.

The second hypothesis is that under the recent African origin (RAO) Out of Africa dispersal model, phenetic distance should increase with increasing geographic distance from the African source population, while within-population variation should decrease (15, 65). The RAO dispersal, or serial founder, model makes predictions about gene flow following recent expansion of Homo sapiens out of Africa since the last glacial period (∼100,000 yBP) that occurred independently of earlier expansions by Homo erectus and Neanderthals (15). One of the main predictions of this dispersal model is that the original populations should retain higher genetic (or phenotypic) variance than populations on the expansion front, because the expanding populations are the product of a series of founder effects (also known as “genetic surfing”) (66–68). Our study tests this hypothesis through regression of both within-population and between-population phenetic distances against geographic and genetic distance from Africa.

Results

Test of Neutral Evolution.

A Procrustes superimposition of 7 fixed and 95 sliding landmarks of the EDJ of left permanent mandibular first molars demonstrates that populations that are genetically more similar have more similar EDJ morphology, as captured by 3D EDJ surface anatomy (Fig. 2 and SI Appendix, Figs. S1 and S2).

Fig. 2.

Linear regression of pairwise kinship coefficients (n = 15) calculated from neutral genetic data (genetic R_ij) and continental mean shape of Procrustes coordinates (phenetic R_ij; r = 0.56, P = 0.03). The 95% confidence interval is plotted in gray. See SI Appendix, Table S2 for values.

Using the R_ij method, Rathmann et al. (21) previously demonstrated that phenetic variation of external dental morphology is correlated with neutral genetic variation in humans. This method was applied to the EDJ morphology of the human dentition by calculating a pairwise R_ij-matrix using population mean shape generated from aligned Procrustes coordinates and comparing phenetic variation to neutral genetic variation [Rathmann’s genetic R_ij-matrix (21)]. Consistent with findings by Rathmann et al. (21) for the external morphology of the dentition, there is a significant correlation between mean pairwise shape differences (phenetic variation) of the EDJ and neutral genetic variation between populations (Fig. 2) (r = 0.56, P = 0.03). Population pairs that include Australia fall closest to the best-fit line and drive the regression (SI Appendix, Fig. S3). Kinship coefficients from comparison of northern and southern Africa are greater than zero for both phenetic and genetic R-matrices, indicating that these populations are most similar. The distance between two populations in neutral allele frequencies is randomly distributed due to drift. Correlation between neutral genetic variation and phenetic variation thus provides evidence that the phenotype of interest likely also evolved under drift in humans, as selection would have imposed a nonrandom pattern (21).

Out of Africa Dispersal Model.

Variation in EDJ surface anatomy is significantly correlated with neutral genetic variation between populations but deviates from the expectations of an RAO Out of Africa dispersal model. Geographic distance from southern Africa is significantly correlated with allele-sharing distance from southern Africa (r = 0.94, P = 0.02), as to be expected. However, within-population variation in EDJ morphology does not decrease with increasing geographic distance from southern Africa (SI Appendix, Fig. S4) (r = 0.26, P = 0.62), contradicting the predictions of the classic Out of Africa model of trait evolution. Greatest within-population phenetic variation in this sample is among the Asian rather than African populations. Additionally, phenetic distance from southern Africa (measured as Procrustes distance) is not correlated with allele-sharing distance from southern Africa (SI Appendix, Fig. S5) (r = 0.03, P = 0.97). However, there are no significant differences between pairwise Procrustes distances (Kruskal–Wallis rank sum test, P = 0.4159), evidence that variation is greatest between individuals rather than between continental populations. These results may be influenced by unequal sample sizes, and increased sampling in future studies will likely contribute to our understanding of global variation in EDJ morphology.

Morphological Variation.

Patterns of EDJ surface anatomy vary significantly across continental populations (Fig. 3) (Procrustes ANOVA, P = 0.001).

Fig. 3.

Comparison of EDJ morphology across continents as captured by PCs derived from Procrustes superimposition of 3D data. Extremes of EDJ shape are shown along the axes. (A) PC1 plotted against PC2, (B) PC2 plotted against PC3, (C) PC3 plotted against PC4, (D) PC4 plotted against PC5. See figure for legend. PC1 comprises 30.2% of the variation, PC2 comprises 19.1% of the variation, PC3 comprises 8.0% of the variation, PC4 comprises 5.2% of the variation, and PC5 comprises 4.2% of the variation. North and South America were combined for analyses due to small sample sizes. Sample sizes: Africa (n = 51), Americas (n = 8), Asia (n = 30), Australia (n = 28), Europe (n = 27). (E) Variation in EDJ morphology of the left mandibular first molar, exemplified by individuals from Europe (Left) and northern Africa (Right). Note the lack of dentine horn associated with the hypoconulid in the European individual. This figure is available without convex hulls as SI Appendix, Fig. S6.

Quantitative and visual analyses demonstrate significant variation in EDJ morphology across continental populations (see SI Appendix for more details). The dominant morphological difference between continental populations is in positioning of the dentine horns that underlie the cusps, particularly the distal cusps, the hypoconid and entoconid, exemplified by Europeans and potentially associated with variable hypoconulid presence and expression (Fig. 3). Variation in EDJ surface morphology, and positioning of the dentine horns relative to one another and to the cervical line, drives the significant results seen in this study.

However, overall, there is greater morphological variation within rather than across continental populations. The greatest morphological variation in human EDJ morphology is in overall cervical height (captured by principal component 1, PC1): That is, the distance from the inferior intersection between cusps to the cervical line (SI Appendix, Fig. S2). PC1 is significantly correlated with centroid size of EDJ morphology, and PC1 captures variation in EDJ cervical height, suggesting that cervical height is likely associated with overall tooth size and therefore body size (P < 0.001) (69–75). Additionally, multivariate regression of common allometric component scores against log centroid size demonstrates a significant allometric relationship between EDJ size and shape (P < 2.2e-16, r = 0.99). In the largest teeth, the dentine crown outline is larger relative to the cervical outline. As crown size is significantly correlated with body size in primates (76); the relationship between centroid size and the relative size of EDJ dentine crown to cervical outline further highlights the effects of allometry on EDJ morphology in humans.

Discussion

From the results of this study, it can be inferred that variation in EDJ morphology is significantly correlated with neutral genetic variation in humans and evolves neutrally, with variation resulting from genetic drift and migration rather than adaptive processes. Based on the nonrandom distribution of EDJ morphology that is correlated with variation in neutral allele frequencies, the evidence presented here supports the hypothesis that EDJ morphology is heritable, or more similar in more closely related individuals. The geographic structure of neutrally evolving traits reflects population movement on a global scale (16, 61–64). As a neutrally evolving trait, EDJ also reflects population movement, as evidenced by significant correlations between genetic and phenetic distances in this study. Patterns of EDJ variation may be useful for interpreting the archaeological record, as more variation likely reflects greater genetic disparity, and samples can be obtained without requiring destructive genetic testing.

Despite evidence for heritability, EDJ morphology does not follow the predictions of an RAO, serial founder, Out of Africa model of dispersal. This may be a result of reversible and repeatable evolution of trait morphology through time. Alternatively, the finding that EDJ morphology does not follow the classic Out of Africa model’s expectations of decreasing variation may not be particularly surprising, as anthropological genetics research continues to demonstrate a high degree of admixture in past human populations (77–80). There is growing support for a model of multiple range expansions out of Africa over the last 100,000 y, although this hypothesis also predicts decreased diversity with increasing distance from Africa (81, 82). The findings here may be explained by increased admixture outside of Africa, with increased diversity potentially resulting from genetic input from extinct hominid groups (60). Additionally, unequal sample sizes may be contributing to the finding of increased variance outside of Africa, as might successive range expansion signatures or predominantly random selective differences. Humans have a complex population history and range. Greater sampling of continental regions, particularly Asia and the Americas, and additional tests of selection and adaptation, will likely further refine our understanding of EDJ variation and patterns of its distribution on a global scale.

There is a strong allometric effect of tooth size on EDJ shape in this study, and there are significant differences in EDJ surface anatomy across continental populations. Tooth size, which is significantly correlated with body size, plays a strong role in dental variation in humans, affecting not only tooth area but also tooth height and cusp orientation (83–87). Given the strong allometric effect of tooth size on EDJ shape, future work focusing on the relationship between body size and dental variation will continue to be illuminating for our understanding of dental development and global patterns of dental variation in modern and archaeological human populations.

Several factors may underlie EDJ variation, including geographic dispersal, neutral and nonneutral evolutionary processes, and allometry. This study clearly demonstrates the role of neutral genetic variation in shaping global patterns of EDJ surface anatomy in humans. But the impact of other factors on EDJ shape, including selection, adaptation, and developmental constraints, remains to be thoroughly quantified in future studies. Thus, future work focusing on these factors, through studies of asymmetry and metameric variation for example, will help illuminate whether EDJ morphology has been under selection in modern humans. This study substantially increases our knowledge of modern human dental variation and contributes to our understanding of dental development more broadly, with taxonomic implications for the hominid fossil record. Additionally, we suggest that EDJ surface anatomy is an excellent trait in which to pursue genotype–phenotype mapping. As sampling EDJ surface anatomy is a noninvasive procedure, this study emphasizes the utility of this trait for investigating large-scale population history and genetic structure in archaeological and fossil populations.

Materials and Methods

High-definition μ-CT biomedical imaging and geometric morphometrics were used to assess EDJ morphological variation in a sample of late Holocene modern humans (n = 161) from archaeological populations (Table 1 and SI Appendix, Table S1). Of the permanent molars, the first molar is least affected by postnatal environmental factors, because the shape of the EDJ is established before secretion of the enamel matrix (17, 31, 88). Given that environmental effects on the topographical morphology of the EDJ of the first molar (M1) are minimal, and based on previous work on external dental anatomy (21, 22), the present study tests several expectations for human M1 EDJ variation. Only individuals with fully developed permanent molar dentition were included in the study. All individuals were μ-CT–scanned using a Nikon industrial CT in Zürich, Switzerland, and Cambridge, United Kingdom. All CT scans were performed using a 225-kV rotating target with average beam energy 210 kV, average beam current 360 µA, average power 80 W, and average isotropic voxel size 0.049 mm³ (or 49 µm³). The 3D scan reconstructions were completed using Nikon CT Pro-3D software. The data volumes were median-filtered to facilitate threshold segmentation and produce 3D models of the EDJ for each individual using software packages VG Studio Pro and Avizo.

Table 1.

Sample sizes of late Holocene individuals included in this study

Continental area	Region	n	Average dispersal distance, Dd*
Africa	Northern Africa	42	6.2
	Southern Africa	9	1.95
Americas	North America	5	23.47
	South America	3	43.7
Asia	Northern Asia	2	19.5
	Southern Asia	10	19.35
	Western Asia	18	15.9
Australia	Australia	28	28.4
Europe	Europe	27	12.08
Unknown	Unknown	17	NA
Total		161

See SI Appendix, Table S1 for included populations, ID numbers for included individuals, and geological age. n is sample size. Unknown individuals were not included in regional or continental comparisons.

^*Average dispersal distance in 1,000 km.

The 3D mesh cleaning was performed using Meshmixer 3.4.35 (Autodesk, 2017), and the 3D surface data were then imported into FoRM-IT (Fossil Reconstruction and Morphometry Interactive Toolkit) (89) to quantify EDJ shape using a total of k = 102 3D landmarks (fixed landmarks k_f = 7, sliding curve semilandmarks k_c = 95) (SI Appendix, Fig. S2).

A Procrustes superimposition of the 7 fixed and 95 sliding landmarks was performed focusing exclusively on the left permanent mandibular first molars. From the Procrustes superimposition, PC scores and Procrustes distances were exported for further analyses. All statistical analyses were conducted in R v.3.2.3 (90). All data and code for this study are available on GitHub (https://github.com/teslamonson/EDJ) (91).

Although this study focuses on nonallometric shape variation in EDJ morphology, a cursory investigation of allometry was completed in the sample with Procrustes regression using the geomorph package (function procD.allometry) (92) and through regression of the common allometric component against log centroid size using the morpho package (function CAC) (93). The common allometric component was assessed across the sample as a whole.

Through quantification of Procrustes-aligned coordinates and principal components scores, variation in EDJ morphology was compared across populations using Procrustes ANOVA with the procD.lm function in the geomorph package (92), as well as through principal component analyses (prcomp) visualized using ggplot2 (94). Pairwise Procrustes distances were statistically compared using a Kruskal–Wallis rank sum test. Samples from North and South America were combined for analyses due to low sample sizes in these regions. All other comparisons were generated at the continental level. Individuals with unknown affiliation (n = 17) were excluded from continental and regional comparisons. More details about the methods used in this study are available in SI Appendix.

Test of Neutral Evolution.

Previous studies have demonstrated that geographical patterns of phenotypic variation reflect population history when that phenotype has been evolving neutrally or without pressures from strong directional selection (15). To evaluate EDJ morphological variation and expectations of neutral evolution, we calculated a phenetic R-matrix using pairwise comparisons of continental mean shape variation generated from Procrustes coordinates. The phenetic R-matrix was statistically compared to a genetic R-matrix previously calculated by Rathmann et al. (21) using bivariate regression and statistical correlation. Positive values indicate continental populations that are more similar to one another. Protocols for calculating the R-matrices are further outlined in SI Appendix.

Tests of the Out of Africa Hypothesis.

Previous work has repeatedly demonstrated the use of geographic distance as a proxy for neutral genetic evolution of certain phenotypes (16, 64). A dispersal distance for each human population was calculated using longitude and latitude and fixing Johannesburg, South Africa, as the point of dispersal (Table 1). Johannesburg is a standard location of origin in sub-Saharan Africa for studies investigating human dispersal. These data were evaluated with an agent-based reproduction-and-diffusion approach. Distances are in kilometers; they are generally larger than great-circle-segment distances but correlate well with them.

Another prediction of neutral evolution in humans is decreasing phenotypic diversity with increasing distance from southern Africa, a result of the impact of bottlenecks and founder effects during episodes of human dispersal (15). To test for neutral evolution using this proxy, the coefficient of variation (CV) was calculated for each continental population using the Procrustes-aligned coordinates. The CVs were used to represent within-population phenetic variation and compared to geographic distance across continental populations (SI Appendix, Table S3). We also statistically compared phenetic distance from southern Africa to population-mean genetic distances calculated from previously published allele-sharing distances between populations (21). Phenetic distance between populations was calculated as group-mean Procrustes distances from southern Africa.

Data Availability.

All data and code for this study are available on GitHub (https://github.com/teslamonson/EDJ) (91).