Early modern human diversity suggests subdivided population structure and a complex out-of-Africa scenario

Abstract

The interpretation of genetic evidence regarding modern human origins depends, among other things, on assessments of the structure and the variation of ancient populations. Because we lack genetic data from the time when the first anatomically modern humans appeared, between 200,000 and 60,000 years ago, instead we exploit the phenotype of neurocranial geometry to compare the variation in early modern human fossils with that in other groups of fossil Homo and recent modern humans. Variation is assessed as the mean-squared Procrustes distance from the group average shape in a representation based on several hundred neurocranial landmarks and semilandmarks. We find that the early modern group has more shape variation than any other group in our sample, which covers 1.8 million years, and that they are morphologically similar to recent modern humans of diverse geographically dispersed populations but not to archaic groups. Of the currently competing models of modern human origins, some are inconsistent with these findings. Rather than a single out-of-Africa dispersal scenario, we suggest that early modern humans were already divided into different populations in Pleistocene Africa, after which there followed a complex migration pattern. Our conclusions bear implications for the inference of ancient human demography from genetic models and emphasize the importance of focusing research on those early modern humans, in particular, in Africa.

Although there exists general agreement that modern humans emerged in Africa and radiated from there into Eurasia, it is unclear how and when today's morphological diversity evolved. Early anatomically modern Homo (early AMH) first emerged in East Africa ≈200–160 kilo years ago (kya) (1, 2). Other remains from the Levant (Skhūl and Qafzeh) and from northwest Africa (Jebel Irhoud) document the presence of anatomically modern morphology ≈160–100 kya at 2 potential “gateways” into Eurasia. The current modern human origins debate has moved beyond the 2 extreme models, the Out-of-Africa with complete replacement of archaic populations (3, 4) and the classic multiregional model (5). Explanations combining elements from both, for instance the assimilation model (6), attempt to reconcile the discrepancies found in the fossil, archaeological, and genetic record.

The “back-projection” of modern human genetic diversity into a demographic history of its early expansion (7) is insecure. These inferences depend on assumptions about ancient population size and structure over time on which there is no consensus (8–10), and cannot be resolved by appeal to ancient DNA at the present time (because of the high age of the relevant specimens and bad preservation of DNA in Africa's hot climate).

In this article we therefore study morphological diversity of early AMH in relation to that of archaic forms of Homo and modern humans by using the methods of geometric morphometrics (11, 12). These methods make it possible to separate size from shape information, to produce a single summary measure of shape dissimilarity between any 2 specimens, and to ordinate these shape distances by statistical analyses. We focus on overall neurocranial anatomy for 2 reasons: (i) Many Mid- to Late Pleistocene hominid fossils lack faces but have well-preserved cranial vaults. (ii) The face is involved in many functions as diverse as ingestion, breathing, and sensory perception, and thus was likely subject to a larger number of selection regimes than the neurocranium.

Our approach is based on quantitative data obtained directly from fossil evidence. Unlike traditional methods that undersample the form of the cranium in a few qualitative features or a few measured distances, our method treats the neurocranium as one entity, has no a priori assumptions about the value of particular discrete shape features, and comprises a dataset that is unique in its density. Here, we interpret the patterns of neurocranial shape variability. Whereas the overall shape of the bony shell of the brain (more globular vs. elongated) seems to have little functional significance and is thus not under strong selection, this neutral pattern does not hold for brain size and body size. The latter are also strongly influenced by environmental variables, e.g., climate (13, 14), which means that they are less heritable than shape.

Our study compares early AMHs (200–60 kya) with recent humans from diverse geographical origin, Upper Paleolithic (UP) people, Neanderthals, and other archaic forms of the genus Homo (AH) [Table 1; for details of the a priori assignment, see supporting information (SI) Text]. Our sample includes several hundred geometrically homologous anatomical landmarks and semilandmarks measured on 203 modern and fossil neurocrania (Fig. 1). Our inferences are drawn from the patterns of group variability in Procrustes shape space. We study overall patterns of shape variation by using principal component analysis (PCA). Onto the PC scores we overlay a diagram of nearest neighbor connections in terms of shape distance, to further support comparisons of shape similarities between pairs of specimens (see also Fig. S1). Each specimen is connected with its nearest neighbor in full Procrustes shape space; this relation is not symmetric—even if B is the nearest neighbor of A, A not need be the nearest neighbor of B.

Table 1.

Sample description

UP AMH (16)*		Early AMH (7)†		Neanderthal (10)‡		Archaic Homo (11)§
Br2	Brno 2	JI1	Jebel Irhoud 1	Am	Amud 1	Da	Dali
CC	Combe-Capelle	JI2	Jebel Irhoud 2	At	Atapuerca SH5	Ka	Kabwe 1
Cr1	Cro-Magnon 1	LH18	Ngaloba	Gu	Guattari 1	3733	KNM-ER 3733
Cr3	Cro-Magnon 3	Om2	Omo 2	LCS	La Chapelle-aux-Saints	Ng7	Ngandong 7
DV2	Dolní Věstonice 2	Qa6	Qafzeh 6	LF	La Ferrassie 1	Ng14	Ngandong 14
FH	Fish Hoek	Qa9	Qafzeh 9	LQ5	La Quina 5	Pa	Petralona
GE4	Grotte des Enfants 4	Sk5	Skhūl 5	LM	Le Moustier 1	Sa17	Sangiran 17
Mc1	Mladeč 1			Sp1	Spy 1	Tr2	Trinil 2
Mc5	Mladeč 5			Sp2	Spy 2	Zh1	Zhoukoudian 1
Mc6	Mladeč 6			Ta	Tabun C1	Zh11	Zhoukoudian 11
Ok1	Oberkassel 1					Zh12	Zhoukoudian 12
Ok2	Oberkassel 2
Pv	Pavlov 1
Pr3	Předmostí 3
Pr4	Předmostí 4
UC103	Zhoukoudian Upper Cave 103

All specimens were assigned a priori to one of the five groups. Group arrangement is heuristic but in fact results from scientific publications and dating of other authors (see Materials and Methods and SI).

AMH, anatomically modern Homo sapiens.

Recent humans: specimens accepted as anatomically modern H. sapiens from the Holocene (10–0 kya), including also subfossil specimens such as Hohlenstein 1 (Ho1), Hohlenstein 2 (Ho2), Kaufertsberg (Kau), Wahlwies (Ws), Wadjak 1 (Wk1), Cohuna (Co), Kow Swamp 5 (KS5), and Paderborn 1 (Pb).

*UP AMH (Upper Paleolithic AMH): specimens accepted as anatomically modern H. sapiens dating to the Upper Paleolithic period.

^†early AMH: specimens accepted as anatomically modern H. sapiens and predating Upper Paleolithic.

AFH, Archaic Forms of Homo.

^‡Neanderthal: specimens accepted as Homo neanderthalensis as well as Atapuerca SH 5

^§AH (Archaic Homo): specimens classified as archaic forms of Homo, including Homo ergaster/erectus and Homo heidelbergensis.

Fig. 1.

Anatomically modern humans and archaic forms of Homo in shape space. (Upper) Two-dimensional projection of the first 3 principal components of the neurocranial shape coordinates and one example (here Mladeč 1) for the full set of landmarks and semilandmarks measured on each neurocranium. Recent humans in light brown; UP fossils in blue; early AMH in red; Neanderthals in green, archaic Homo in orange. The graph provides 2 different kinds of information: (i) Ordination of each specimen in the first 3 principal components (PCs, together 71% of total variation), and (ii) nearest neighborhood relations according to full Procrustes shape distance (which uses all dimensions, not just the first three). Connections between nearest neighbors from the same group are shown in their group color, connections between nearest neighbors from different groups are drawn as black lines. Equal frequency ellipsoids (75%) are plotted in respective color for all groups. Ellipsoids for recent humans are based on their geographic origin: Africa, Asia, Australia, and Europe. (For a more detailed view see Fig. S1a). (Lower) Log Centroid Size (log CS), a measure for size in our analysis, is plotted for all specimens (color coding as above). Early AMH exhibit the narrowest distribution of log CS in comparison with all other groups.

Results

The result of our statistical shape and form analysis is summarized in 2-dimensional projections of the first 3 principal components (PCs, Fig. 1; see also Figs. S1 and S2) of Procrustes shape space, over which is overlaid our nearest neighbor diagram. Because our group (i.e., color) assignment follows other authors' morphological approaches (which do not attempt to weight all areas of the neurocranial geometry equally), our pairings, based on the high-density data mesh, might or might not comport with the color scheme. Even though the a priori group assignment only affects the color, not the position, nor the connections in our plot, we find that most nearest neighbor pairs are of the same color. This is also true of the modern humans, because crania of similar geographic origin tend to cluster together. Still more interesting are the specimens that are nearest to shapes from another group.

The tightest clustering in this plot is for the Neanderthals and the archaic Homo group (AH); the greatest dispersion is for the early AMH and the modern humans. This is unexpected, because the AH cover a wide geographic and temporal range and are so diverse in other aspects of fossil form as to merit 3 separate species designations (see Table 1 and Materials and Methods). These comparisons of variance are confirmed when we attend to the full shape distances, not just their projections into this 3-dimensional subspace (Fig. 2).

Fig. 2.

Variances in shape space for each group and their bootstrap distributions. Bootstrap distributions for the shape variances of the 5 different groups (20,000 bootstraps each; color coding as in Fig. 1). Because of the small and heterogeneous early AMH sample the distribution of its variance is very wide, i.e., shape variance of early AMH is known with low confidence only. Yet, the shape variances of both early AMH and of recent humans are significantly (P < 0.05) larger than the variances of Neanderthals and of archaic Homo.

One possible source for high levels of shape variability would be allometric effects, because shape covaries with size (small gracile crania vs. large robust ones). However, the group that is the most variable in terms of shape (early AMH) is, at the same time, the least variable with regard to centroid size (see Fig. 1 Lower). This might be related to differences in the temporal, spatial, and taxonomic distribution among the groups, but is beyond the scope of this article because we are not interpreting size per se, only shape. Nevertheless, this observation is important to rule out that the high shape variability of the early AMH group is caused by allometry.

Point clouds of moderns (blue and light brown) and archaics (green and orange) are clearly distinct (Fig. 1 and Fig. S1), with no single nearest neighbor connection between them. This is consistent with the notion that Neanderthals and archaic Homo share a conserved cranial architecture that is different from the one of modern humans (15–17).

Sample-size-insensitive rarefaction analyses and bootstrap tests demonstrated (see Fig. 2 and Fig. S3) that early AMH are significantly more variable than recent Homo sapiens (P = 0.011), Upper Paleolithic people (P = 0.014), Neanderthals (P = 0.008), and archaic forms of Homo (P = 0.045). Our results thus revealed that shape variability of early AMH was highest among all tested groups, i.e., within a sample of the genus Homo embracing the last 1.8 million years. The shortest connections between early AMH are either with other specimens of this group or recent modern humans, for instance, Omo 2 [recently dated to ≈195 ka (1)] and LH 18, two of the earliest east African candidates for the emergence of modern human morphology (18), and the Levantine Qafzeh 6 connect with recent Australian aboriginals (cf. ref. 19). We also find a connection between 3,500-km-distant sites in the Levant and northwest Africa, i.e., between the more archaic looking Jebel Irhoud 1 and Skhūl 5, whereas Jebel Irhoud 2 connects to recent Europeans. Qafzeh 9 (Levant) is linked to a European UP specimen. We find, however, no single link between Neanderthals and AMH, including Upper Paleolithic specimens.

Discussion

Our phenetic analysis confirms doubts raised by genetic studies (7–9, 20) regarding a single-dispersal model proposed by others (21, 22). We interpret the evident heterogeneity of early AMHs as representing multiple temporarily isolated populations in Africa. The diverse nearest neighbor links of the early AMH specimens to various modern populations are consistent with a model of multiple dispersal events out of Africa. This interpretation rests on the assumption that neurocranial shape is not dissociated from true population history (23). No consensus has been achieved on the relative impact of neutral patterns and selective processes on the evolution of human cranial morphology (for summaries see refs. 19 and 23–25). However, recent studies (summarized in ref. 23) indicate that correlations between genetic and phenotypic distances based on crania are moderate to high and several results point toward a more neutral pattern of cranial evolution (23, 25), especially for the neurocranium (13). That neurocranial shape retains a population history signal can also be seen in the nearest neighbor clustering of geographical regions among recent modern human crania (see Fig. S1).

Our nearest neighbor approach finds a closest neighbor for any specimen. This does, of course, not necessarily indicate an ancestor–descendant relationship between each AMH fossil and the specific human it ties to. Except for Omo 2, Procrustes distances from the early AMH to their nearest neighbors in full shape space (detailed list in Fig. S4) do not exceed the upper range of distances from modern humans to their nearest neighbors. The short apparent lengths of these connecting segments are thus not projection artifacts of the principal component analysis.

Although our results cannot pinpoint a specific model of demographic history for the origins of our species, our data are only consistent with a subset of demographic models. Our findings do not support the notion that large genetic variability of modern Africans can be attributed solely to the greater time-depth on the African continent. The morphological variability seen in modern humans is not a recent phenomenon, because we find even higher levels of shape variability already near the emergence of our species in Pleistocene Africa. This high shape variability thus predates modern human culture by tens of millennia and we therefore consider it unlikely that it is an effect of Holocene population expansions or the relaxation of constraints in a modern cultural environment.

Any model consistent with our data requires a more dynamic scenario and a more complex population structure than the one implied by the classic Out-of-Africa model. Our findings on neurocranial shape diversity are consistent with the assumption that intra-African population expansions (21, 22) produced temporarily subdivided and isolated groups (8, 26). In such a metapopulation model, transient populations are connected by migration, subject to extinction and rebirth by colonization, as well as to fluctuation in local size (7, 27). This view is in agreement with recent genetic results (28) suggesting divergence of some recent human populations from the rest of the human mtDNA pool 90–150,000 years ago, and isolation times between 50 and 100,000 years. Separated demes (population subdivisions) might have partly merged again, whereas others left Africa at different times and maybe using different routes, and still others probably also remigrated to Africa.

Our data on neighbors and variability is unsupportive of the strict forms of a single-origin model but does not conflict with another approach, the model of “isolation by distance,” which predicts that genetic and phenotypic dissimilarity increases with geographic distance (24, 29–31). The metapopulation framework would predict the same because frequency and magnitude of genetic exchange would follow the likelihood of 2 populations to meet, which declines with geographical distance from the early AMH epicenter in Africa. Our fossil AMH data, however, suggest that before there was isolation by distance from Africa, there already existed (at least temporally) isolation by distance within Africa during the Pleistocene.

Genetic diversity among living modern humans is known to be very low when compared with extant apes (32, 33). To reconcile this observation with our proposed metapopulation model within Africa, it is necessary to assume that genetic diversity of early AMH (and maybe even earlier fossil groups of Homo) must have been relatively low as well. The only fossil human group for which such genetic data are available, the Neanderthals, support this contention; their level of genetic variability also is low when compared with living apes (34, 35).

Seemingly ancient contributions to the modern human gene pool (36) have been explained by admixture with archaic forms of Homo, e.g., Neanderthals. Although we cannot rule out such admixture (37), the clear morphological distinction between AMH and archaic forms of Homo in the light of the proposed ancestral population structure of early AMH to us suggests another underestimated possibility: the genetic exchange between subdivided populations of early AMH as a potential source for “ancient” contributions to the modern human gene pool (9, 36).

Although more data are needed to corroborate our inferences, we could clearly demonstrate the pronounced variability of early AMH and their morphological relationship to modern humans. It is crucial for any analysis, genetic or phenetic, of modern human origins to take into account this Late Pleistocene African diversity that predates the range expansions into Eurasia. The molecular and fossil evidence of the African continent deserves more attention in the modern human origins debate.

Materials and Methods

Our sample includes 486 geometrically homologous anatomical landmarks and semilandmarks from 203 neurocranial specimens (Fig. 1). Three-dimensional coordinates were measured using a Microscribe 2GX digitizer on the original specimens or research quality casts. Each cranium was measured in 2 orientations, which were later superimposed by using 5 fiducial points. We collected 16 homologous landmarks (Table S1) and closely spaced points along the supraorbital torus, the midsagittal profile from glabella to inion and along the medial part of the superior nuchal line. Furthermore, we digitized a dense point cloud on the neurocranial surface of every specimen. Data on some fossils (Omo 2, LH 18, Atapuerca SH 5, Guattari, Mladeč 1, Petralona, Kabwe, Skhūl 5) were measured on the CT scans of the originals.

Segments along the curves were resampled to get the same number of points (curve-semilandmarks) on each specimen. A mesh of 414 surface semilandmarks was carefully digitized on one cranium and then projected onto all others, by warping them using the thin-plate spline interpolation between the landmarks of the reference specimen and every other specimen and then lofting the points onto each specimen's neurocranial surface. This protocol guarantees the same point count of approximately evenly spaced semilandmarks on every specimen. All semilandmarks were allowed to slide along tangents to the curve or surface so as to minimize the bending energy between each specimen and the Procrustes average shape (12, 38). These tangents were approximated for each curve-semilandmark by using the vector between the 2 neighboring points. For every surface semilandmark we used the first 2 eigenvectors of the covariance matrix of its 5 nearest neighbors (including the semilandmark itself). In the sliding step the thin-plate spline interpolant is used to provide a criterion for geometric homology or correspondence. Thus, after sliding, landmarks and semilandmarks can be treated equivalent in the course of the multivariate analysis.

Anatomical landmarks were measured on the left and right side, curves and surface points only on one side and then mirrored along the midsagittal plane. Usually we measured the curves on the left side; in fossil specimens we measured the better-preserved side and mirrored it. This was done before resampling the curves and sliding the semilandmarks. Because some fossils were incomplete, some reconstruction was necessary before the analysis, because geometric morphometric methods require a full data matrix without missing values. We followed the reconstruction protocol described in ref. 39. Whenever possible, missing parts were reconstructed by mirror imaging. In cases where bilateral landmarks were missing on one side only, they were estimated by reflected relabeling (40), which uses the Procrustes geometry to reflect the paired landmarks without having to specify a mirroring plane. Incomplete specimens were least-squares superimposed with their reflected configurations in Procrustes space and the missing data reconstructed from their homologous counterparts on the other side.

In some cases we reflected the better-preserved side by using a least squares fitting plane through the midsagittal landmarks, rather than by using reflected relabeling: (i) when only one-half of the cranium was preserved, and (ii) when one-half was distorted and the other one correct. In the former case, reflected relabeling could not be computed because of the lack of bilateral points; in the latter case, reflected relabeling would have propagated the error of the distorted side to the other.

In a few cases, landmarks missing on both sides were estimated during the spline relaxation against the Procrustes average; missing points were fully relaxed, i.e., their positions were estimated by minimizing the thin-plate spline bending energy. This yields the configuration with the smoothest interpolation, taking all of the preserved morphology into account. Our sample comprises only specimens that preserve complete calvariae, so the necessary reconstruction was kept to a minimum.

Sample Composition.

In our study, “UP AMH” includes all anatomically modern Homo sapiens (AMH) of our sample that date between ≈45 and 10 kya (cf. ref. 41). AMH specimens predating the Upper Paleolithic are grouped as “early AMH.” “Neanderthals” includes specimens widely accepted as “classic” Homo [sapiens] neanderthalensis (e.g., 42, 43) and one specimen from Atapuerca: SH 5), whereas “archaic Homo” comprises representatives of the genus Homo other than AMH or Neanderthals, e.g., “archaic” Homo sapiens, H. erectus, H. ergaster, or H. heidelbergensis.

Our modern human sample covers a wide range of modern human shape variability. It is not balanced with regard to sex and populations, so the data resolution is not high enough to support any claims about which founding population gave rise to which modern group. Because of the gene flow among modern humans over several millennia, it seems unlikely that such a detailed signal could be recovered, no matter how large the modern human comparative group would be. We would like to point out that a smaller modern human sample is actually biased against our finding that almost all early modern human fossils connect to a recent human. Increasing the modern human sample would only increase the chance that a fossil is close to a modern human in shape space.