Bony labyrinth shape variation in extant Carnivora: a case study of Musteloidea

Abstract

The bony labyrinth provides a proxy for the morphology of the inner ear, a primary cognitive organ involved in hearing, body perception in space, and balance in vertebrates. Bony labyrinth shape variations often are attributed to phylogenetic and ecological factors. Here we use three‐dimensional (3D) geometric morphometrics to examine the phylogenetic and ecological patterns of variation in the bony labyrinth morphology of the most species‐rich and ecologically diversified traditionally recognized superfamily of Carnivora, the Musteloidea (e.g. weasels, otters, badgers, red panda, skunks, raccoons, coatis). We scanned the basicrania of specimens belonging to 31 species using high‐resolution X‐ray computed micro‐tomography (μCT) to virtually reconstruct 3D models of the bony labyrinths. Labyrinth morphology is captured by a set of six fixed landmarks on the vestibular and cochlear systems, and 120 sliding semilandmarks, slid at the center of the semicircular canals and the cochlea. We found that the morphology of this sensory structure is not significantly influenced by bony labyrinth size, in comparisons across all musteloids or in any of the individual traditionally recognized families (Mephitidae, Procyonidae, Mustelidae). PCA (principal components analysis) of shape data revealed that bony labyrinth morphology is clearly distinguishable between musteloid families, and permutation tests of the Kmult statistic confirmed that the bony labyrinth shows a phylogenetic signal in musteloids and in most mustelids. Both the vestibular and cochlear regions display morphological differences among the musteloids sampled, associated with the size and curvature of the semicircular canals, angles between canals, presence or absence of a secondary common crus, degree of lateral compression of the vestibule, orientation of the cochlea relative to the semicircular canals, proportions of the cochlea, and degree of curvature of its turns. We detected a significant ecological signal in the bony labyrinth shape of musteloids, differentiating semi‐aquatic taxa from non‐aquatic ones (the taxa assigned to terrestrial, arboreal, semi‐arboreal, and semi‐fossorial categories), and a significant signal for mustelids, differentiating the bony labyrinths of terrestrial, semi‐arboreal, arboreal, semi‐fossorial and semi‐aquatic species from each other. Otters and minks are distinguished from non‐aquatic musteloids by an oval rather than circular anterior canal, sinuous rather than straight lateral canal, and acute rather than straight angle between the posterior and lateral semicircular canals – each of these morphological characters has been related previously to animal sensitivity for detecting head motion in space.

Introduction

The inner ear is a complex sensory structure located in the petrous temporal bone (petrosal) situated in the basicranial region of the skull in vertebrates. It comprises the cochlear system, associated with hearing capabilities, and the vestibular system (vestibule and semicircular canals), related to detection of head movements in space, control of gaze, and maintaining balance during motion (Fig. 1). The bony labyrinth, housing the inner ear, reflects the shape and relative size of its enclosed soft‐tissue structures (Spoor et al. 1994) and therefore is a good osteological proxy for studying inner ear anatomy. The bony labyrinth in gnathostomes is constituted primarily of three semicircular canals, a vestibule containing otolith organs, and a cochlea that is coiled in therian mammals (Luo et al. 2010). Several studies support the observation that the bony labyrinth morphology of vertebrates bears a significant phylogenetic signal (Schmelzle et al. 2007; Lebrun et al. 2010; Luo et al. 2010; Boistel et al. 2011; Ekdale, 2013), but after accounting for ancestry, some features also can be related to ecological factors (e.g. Spoor et al. 1994, 2002, 2007; Lindenlaub et al. 1995; Alonso et al. 2004; Ladevèze et al. 2010; Ni et al. 2010). In fact, in the vestibular system, the semicircular canals must be functionally significant, as they serve as sensory detectors of head rotational acceleration. They enclose membranous ducts containing a fluid, the endolymph, whose inertial movements during head rotations activate ciliate nervous cells located in ampullae at the base of each duct. Together with the otolith organs, which detect linear motion and gravity, the vestibular nervous cells generate both motor reflexes helping to stabilize gaze, and posture and muscular input commanded by the brain for coordination of movements during locomotion. Because of these properties, some parameters of the vestibular system in mammals, such as the size of the semicircular ducts (membranous structure) and canals (bony structure) (Müller, 1994, 1999; Spoor et al. 1994, 2002, 2007; Yang & Hullar, 2007; Silcox et al. 2009; Macrini et al. 2010; Ryan et al. 2012), and the variations of orthogonality between canals (= 90var of Malinzak et al. 2012), have been correlated with vestibular sensitivity, and by extension with agility or head angular velocity, during locomotion. The configuration of the cochlear system, such as the proportions of the basilar membrane, volume, coiling and number of turns of the cochlea, also provides ecological information, particularly related to determining the limitation of high and low frequency hearing in vertebrates (West, 1985; Manoussaki et al. 2008; Simmons et al. 2008; Kirk & Gosselin‐Ildari, 2009; Ekdale & Racicot, 2015).

Figure 1.

Right bony labyrinth of Lontra canadensis (Lutrinae, Mustelidae): (A) Line drawing of the labyrinth in anterolateral view, illustrating the main anatomical structures referred to in this study, and (B‐D) location of 3D landmarks and sliding semilandmarks, illustrating the dataset used for the musteloid sample in this study (see Table 2 for definition of the landmarks). (B) Anterolateral view. (C) Posterolateral view. (D) Anteromedial view. aam, anterior ampulla; asc, anterior semicircular canal; cc, common crus; co, cochlea; fc, fenestra cochleae (= round window); fv, fenestra vestibuli (= oval window); lam, lateral ampulla; lsc, lateral semicircular canal; pam, posterior ampulla; psc, posterior semicircular canal; vb, vestibule.

The interspecific morphological variation of the bony labyrinth in Carnivora has never been studied, even though this order represents one of the most ecologically diversified and species‐rich clades of extant and fossil placental mammals. Only a few prior studies have figured, described, and/or measured bony labyrinths or inner ears of Carnivora, in sum totaling only about a dozen species (Gray, 1907; Tremble, 1978; Ramprashad et al. 1984; Solntseva, 2001, 2007; Georgi, 2008; Ekdale, 2013). Spoor & Thewissen (2008) presented a review of the most extensive sample of inner ear morphology for the clade, using both new analyses and data from previous studies (a total of 35 species), but they provided only measurements of the semicircular canals. In this paper, we present the first comprehensive investigation of the bony labyrinths of Carnivora, and the influence of allometry, phylogeny and locomotor specializations in shaping its morphology, using the superfamily Musteloidea as a model group. This clade is the most species‐rich superfamily of Carnivora; it comprises 93 modern species, including weasels, martens, badgers, otters, skunks, raccoons, olingos, coatis, the red panda, and their allies, as well as many fossil taxa.

There are several reasons to select Musteloidea as a model group for such a study. First, Musteloidea phylogenetic relationships are well resolved (e.g. Flynn et al. 2005; Finarelli, 2008). Second, musteloids are widely distributed on all major continents (except Australia) and the Pacific Ocean coasts, and occupy a wide array of habitats and environments – ranging from arctic tundras to tropical rainforests, from deserts to mountains, and across inland freshwater systems and marine coasts. Associated with these environments, they have diverse ecological niches and locomotor systems, including numerous terrestrial species along with arboreal, semi‐arboreal or scansorial, aquatic and fossorial specialists (Nowak, 2005).

Using geometric morphometrics for understanding bony labyrinth shape variation is rare (Lebrun et al. 2010, 2012; Boistel et al. 2011; Gunz et al. 2012; Alloing‐Seguier et al. 2013), although these techniques have the potential to capture wider morphological information than traditional linear or two‐dimensional (2D) measurements. Here we characterize bony labyrinth morphology of the species‐rich and ecologically diverse carnivoran clade of musteloids, using high‐resolution X‐ray micro‐computed tomography (μCT) and three‐dimensional (3D) geometric morphometrics. The goal is to investigate whether the bony labyrinth shape of musteloids is influenced by bony labyrinth size variation, phylogenetic relationships, and/or ecology and locomotor differences. We would expect to find an allometric relationship between bony labyrinth shapes and their centroid size, as previously recorded for inner ears of other mammals (e.g. Lebrun et al. 2010; Alloing‐Seguier et al. 2013). We also would expect to find overall variation of the bony labyrinth shape to be related to phylogeny, as well as an influence of locomotor or ecological specializations on vestibular system morphology variations related to head movement differences during distinct locomotor styles.

Material and methods

Sample composition

The study sample is composed of extant musteloid cranial specimens from the following institutions: Muséum National d'Histoire Naturelle (Paris), Institut de Paléontologie Humaine: Évolution et Paléoenvironnements (University of Poitiers), Institut des Sciences de l'Évolution (University of Montpellier II) and American Museum of Natural History (New York; AMNH). These specimens correspond to male and/or female adults representing all of the traditionally recognized subfamilies of musteloids as well as a wide range of ecological behaviors. A total of 31 species were sampled: 21 Mustelidae (five Lutrinae, five Mustelinae, both extant species of Melinae, five Martinae, one Galictinae, and the only extant species of Helictidinae, Mellivorinae and Taxidiinae), six Procyonidae, three Mephitidae, and the only extant species of Ailuridae (Table 1). Ten species included samples of two individuals rather than one, distributed throughout all of the families. We examined the right bony labyrinth for each individual. There has been little study of labyrinth symmetry in mammals, so it is worth noting that Welker et al. (2009) found no significant intra‐individual variation between the right and left bony labyrinths in a sample of shrews.

Table 1.

Sample list and protocol of data acquisition for this study

Family	Subfamily	Genus	species	Specimens	Voxel size (μm)	Scanner
Mustelidae	Lutrinae	Lutra	lutra	UPPal M02.5.005	29.9493	Viscom X8050
		Lutra	lutra	UM 009N	56.4396	Viscom X8050
		Aonyx	cinerea	MNHN MO 1982‐165	24.7008	Viscom X8050
		Enhydra	lutris	MNHN MO 1935‐124	30.2462	Viscom X8050
		Enhydra	lutris	AMNH 24186	70.2102	GE Phoenix Vtome x s240
		Lontra	canadensis	AMNH 254476	55.34	Viscom X8050
		Lontra	felina	MNHN MO 1932‐3019	24.422	Viscom X8050
	Mustelinae	Mustela	frenata	AMNH 60508	31.41	GE Phoenix Vtome x s240
		Mustela	lutreola	UM 670N	27.3632	Viscom X8050
		Mustela	nivalis	MNHN MO 1933‐2153	18.0533	Viscom X8050
		Mustela	putorius	UM 117N	24.8311	Viscom X8050
		Mustela	vison	MNHN MO 1959‐189	31.2082	Viscom X8050
	Melinae	Meles	meles	UPPal M0.2.5.021	43.9893	Viscom X8050
		Arctonyx	collaris	MNHN MO 1962‐153	24.7188	Viscom X8050
	Martinae	Eira	barbara	AMNH 32065	52.7217	GE Phoenix Vtome x s240
		Martes	foina	UPPal M02.5.017A	24.5557	Viscom X8050
		Martes	martes	UPPal M02.5.019A	25.2304	Viscom X8050
		Martes	pennanti	AMNH 121558	50.15	GE Phoenix Vtome x s240
		Gulo	gulo	MNHN MO 1873‐39	38.9639	Viscom X8050
		Gulo	gulo	AMNH 182936	73.81	GE Phoenix Vtome x s240
	Galictinae	Galictis	cuja	MNHN MO 1960‐3811	26.4328	Viscom X8050
	Helictidinae	Melogale	moschata	MNHN MO 1929‐376	24.7188	Viscom X8050
	Mellivorinae	Mellivora	capensis	MNHN MO 1893‐6	38.7596	Viscom X8050
	Taxidiinae	Taxidea	taxus	MNHN MO 1895‐417	27.2056	Viscom X8050
		Taxidea	taxus	AMNH 120577	82.33	GE Phoenix Vtome x s240
Mephitidae		Mephitis	mephitis	MNHN MO 2005‐655	24.7969	Viscom X8050
		Mephitis	mephitis	AMNH 172133	58.33	GE Phoenix Vtome x s240
		Spilogale	putorius	MNHN MO 1962‐961	72	Skyscan 1076
		Spilogale	putorius	AMNH 35207	57.68	GE Phoenix Vtome x s240
		Mydaus	javanensis	AMNH 106635	71.24	GE Phoenix Vtome x s240
Procyonidae		Procyon	cancrivorus	UM 002N	54.7686	Viscom X8050
		Procyon	lotor	UM 091N	28.9534	Viscom X8050
		Procyon	lotor	AMNH 24815	88.14	GE Phoenix Vtome x s240
		Nasua	nasua	UM 141N	32.3281	Viscom X8050
		Nasua	nasua	UPPal M02.5.024A	24.4028	Viscom X8050
		Bassariscus	astutus	AMNH 135964	55.96	GE Phoenix Vtome x s240
		Potos	flavus	UM 124N	73.1389	Viscom X8050
		Potos	flavus	AMNH 239990	72.21	GE Phoenix Vtome x s240
		Bassaricyon	pauli	AMNH 47772	62.5	GE Phoenix Vtome x s240
Ailuridae		Ailurus	fulgens	MNHN MO 1963‐358	24.6453	Viscom X8050
		Ailurus	fulgens	AMNH 185436	94.42	GE Phoenix Vtome x s240

AMNH, American Museum of Natural History, New York, NY, USA; MNHN MO, Muséum National d'Histoire Naturelle de Paris, Laboratoire Mammifères et Oiseaux, Paris, France; UM, Collections of the University of Montpellier II, Montpellier, France; UPPal, Collections of extant and fossil vertebrates of the University of Poitiers, IPHEP, Poitiers, France.

Data acquisition

To extract 3D images of the internal bony labyrinth, we used high‐resolution X‐ray μCTs to collect digital volume data of the basicrania of musteloids. Our sample was scanned with a Viscom X8050 μCT at the University of Poitiers, a SkyScan 1076 μCT at the University of Montpellier II, and a GE Phoenix Vtome x s240 μCT in the Microscopy and Imaging Facility (MIF) at the AMNH (Table 1). Voxel sizes varied between approximately 18 and 94 μm depending on the type of scan and the size of the specimen. Three‐dimensional processing and rendering were performed after semi‐automatic segmentation of the bony labyrinths, using the reconstruction software packages avizo 6.3 (FEI Visualization Sciences Group, Germany) and mimics16.0 (Materialise NV, Belgium). The 3D reconstructions of the right bony labyrinth for each species of musteloid studied here are illustrated in Fig. 2. We generated smoothed 3D surfaces (STL files) and imported them into ISE‐MESH TOOLS 1.0.3 (Lebrun, 2014; free software available at http://morphomuseum.com/meshtools). Because the morphology of the bony labyrinth is complex, with largely smooth and continuous surfaces, it is difficult to assess precisely its variation using only traditional discrete or positionally fixed landmarks. Therefore a total of 126 landmarks and semilandmarks were used to characterize bony labyrinth shape (Fig. 1, Table 2). Six fixed landmark locations were defined on the cochlea (apex of the cochlear helix: Landmark 1) and on the vestibular system (centers of oval and round windows, bifurcation points of the semicircular canals: Landmarks 2–6); five sets of semilandmarks follow the center of the semicircular canals (20 semilandmarks for each semicircular canal and 20 for the common crus: Landmarks 7–86) and the cochlear turns (40 semilandmarks: Landmarks 87–126). Due to differences in the number of cochlear turns among musteloids, the semilandmarks on the cochlea have been placed on the first two turns in all taxa.

Figure 2.

3D reconstructions of the right bony labyrinth for each species of the musteloid sample studied, figured in anterior view. Specimen numbers can be found in Table 1. For species represented by two individuals we illustrate only one: Spilogale putorius AMNH 35207; Mephitis mephitis MNHN MO 2005‐655; Ailurus fulgens AMNH 185436; Potos flavus AMNH 239990; Nasua nasua UM 141N; Procyon lotor UM 091N; Taxidea taxus MNHN MO 1895‐417; Gulo gulo MNHN MO 1873‐39; Enhydra lutris AMNH 24186; Lutra lutra UPPal M02.5.005 (see Table 1 caption for institutional abbreviations). Bony labyrinths are not shown to scale. dors, dorsal; lat, lateral

Table 2.

Definition of the 3D landmarks and semilandmarks in the geometric morphometric analyses (see Fig. 1 for the graphic location)

No	Type	Definition
1	Landmark	Apex of the cochlear helix
2	Landmark	Center of the oval window
3	Landmark	Center of the round window
4	Landmark	Bifurcation point of the lateral and anterior canals
5	Landmark	Bifurcation point of the lateral and posterior canals
6	Landmark	Bifurcation point between the anterior and posterior canals (or common crus)
7 to 26	Semilandmarks	Curve placed at the center of the anterior canal
27 to 46	Semilandmarks	Curve placed at the center of the lateral canal
47 to 66	Semilandmarks	Curve placed at the center of the posterior canal
67 to 86	Semilandmarks	Curve placed at the center of the common crus
87 to 126	Semilandmarks	Curve placed at the center of the cohlear helix

Analyses

The 3D semilandmarks were slid along the curves to establish a geometric correspondence between the different individuals (Bookstein, 1991; Gunz et al. 2005; Gunz & Mitteroecker, 2013). This procedure was performed during superimposition, minimizing either Procrustes distance or bending energy. While minimum Procrustes distance criterion allows semilandmarks to slide on the curve along the direction locally parallel to the line defined by two adjacent landmarks, the bending energy criterion allows semilandmarks to slide along the direction locally tangential to the curve (Bookstein, 1989; Rohlf & Slice, 1990; Bookstein, 1991; see also Perez et al. 2006). Difference of landmark configurations generated by these criteria varies for different morphological features (Perez et al. 2006), and analyses of morphological variation based on data obtained by these two alternative methods could produce different results, notably when studying intra‐group variation (Perez et al. 2006; Tseng & Flynn, 2015). To more fully explore the shape variation of our sample, we analyzed two datasets of 3D Procrustes coordinates obtained from semilandmark sliding using minimum bending energy criterion (Bookstein, 1989, 1991) or minimum Procrustes distance criterion (Rohlf & Slice, 1990). Additional discussion about the sliding process of semilandmarks, when working with surfaces, curves or outlines of biological objects, can be found in the literature (e.g. Bookstein, 1991; Gunz et al. 2005; Gunz & Mitteroecker, 2013). To perform the Procrustes analyses, we imported into r version 3.1.1: (1) a thin‐plate spline file format with the 3D coordinates of the fixed landmarks and semilandmarks of the bony labyrinths exported from ISE‐MESH TOOLS, and (2) an NTSYS file format defining for one bony labyrinth the landmarks that should be treated as semilandmarks vs. those treated as fixed for defining the parallel and tangent directions necessary to the sliding. We ran generalized Procrustes analyses using the function gpagen of the geomorph r package version 2.1.4 (Adams & Otarola‐Castillo, 2013; Adams et al. 2015; see also Sherratt, 2014). We obtained two datasets of 3D Procrustes coordinates (already projected in the tangent space) for our entire sample, and created two new datasets of species mean 3D Procrustes coordinates for the subsequent analyses, each using the minimum bending energy and minimum Procrustes distance criteria detailed above.

We investigated allometric effects of bony labyrinth size on bony labyrinth shape within Musteloidea as a whole, but also independently within Mustelidae, Procyonidae, and Mephitidae. For this purpose, we performed a Procrustes anova to evaluate the relationship between the 3D Procrustes coordinates and the logarithm of the centroid size of the musteloid species analyzed, using the entire set of landmarks, and then ran a permutation test to evaluate the significance of allometry on bony labyrinth shape variation (function procD.lm of geomorph). Shape variability of the entire bony labyrinth among species was summarized by principal components analysis (PCA; function plotTangentSpace of geomorph). Shape variation along PC axes was visualized using 3D warped bony labyrinth surfaces (function warpRefMesh of geomorph).

Because species are not entirely independent from each other phylogenetically, and because the degree of their shared ancestry can influence shape similarity, we investigated the significance of evolutionary relatedness or phylogenetic signal in our dataset. For this purpose, we used a composite phylogenetic consensus tree from morphological and molecular data (Fig. 3). Relationships within families of musteloids follow Koepfli et al. (2007), Helgen et al. (2013) for Procyonidae, and Koepfli et al. (2008) for Mustelidae, each consistent with the relationships reconstructed in phylogenies of Arctoidea by Finarelli & Flynn (2008) and of Carnivora by Flynn et al. (2005) and Eizirik et al. (2010, except the relationship of Ailruidae/Mephitidae within musteloids) for taxa in common with this study. We considered Martes pennanti and Eira barbara close relatives, following Flynn et al. (2005), a relationship which is unresolved in Koepfli et al. (2008). Whereas the close relationship between Mustelidae and Procyonidae relative to other musteloid families is generally accepted (Flynn & Nedbal, 1998; Flynn et al. 2000, 2005; Fulton & Strobeck, 2006; Sato et al. 2006; Finarelli 2006; Sato et al. 2009; Eizirik et al. 2010), the basalmost divergence within Musteloidea as either Mephitidae or Ailuridae is still debated (e.g. Finarelli & Flynn, 2006; Eizirik et al. 2010). Because our sample is a composite of several phylogenies with distinct branch length estimations, we built the reference tree topology in mesquite using a uniform branch length of 1.

Figure 3.

Phylogenetic tree topology used for this study. Branch lengths equal to 1. Respective ecological categories are provided for each species following Nowak (2005) and Helgen et al. (2013; for Bassaricyon pauli). Clades tested in the statistical analyses are numbered in grey (see Tables 5 and 6).

We first calculated phylogenetic signal using the Kmult statistic (Adams, 2014), a multivariate version of the K‐statistic (Blomberg et al. 2003) for high‐dimensional multivariate data. Values of K < 1 indicate that the dataset displays less phylogenetic signal than expected under a Brownian motion model of evolution. Conversely, values of K > 1 indicate that the dataset displays greater phylogenetic signal than expected under a Brownian motion model. The K values also were evaluated statistically via permutation tests, where data at the tips of the phylogeny are randomized relative to the reference tree. Kmult and associated permutation tests were performed for multiple clades (see Fig. 3) with the function physignal of geomorph (Adams & Felice, 2014) and using the r package Ape (Paradis et al. 2004; Paradis, 2012) to import the trees. P‐values were adjusted using a Bonferroni correction to account for the fact that multiple comparisons including shared nodes along a tree can lead to artificially higher P‐values. In addition, we mapped the musteloid phylogeny (Fig. 3) onto the PCA plot using the squared‐change parsimony method, applying r functions described by Perrard et al. (2014; see also McArdle & Rodrigo, 1994; Klingenberg & Gidaszewski, 2010). This method compares the amount of change occurring along the reference tree to the change occurring along random trees generated by random taxon reshuffling. The phylomorphospace obtained helps to visualize the degree of influence of the phylogeny on bony labyrinth morphospace occupation and degree of homoplasy of the dataset.

Following Nowak (2005) and Helgen et al. (2013; for Bassaricyon pauli), we defined five ecological categories (semi‐aquatic, semi‐arboreal, arboreal, semi‐fossorial, and generalized terrestrial forms) based on style of locomotion in different environments (dominated by movement in 3D or 2D space) and distinct viscosity in mediums (in air or water; see Fig. 3). Species were defined as arboreal when mostly living in trees, semi‐arboreal when variably occupying forest and terrestrial habitats, semi‐aquatic when dominantly occupying marine or fresh‐water environments, or semi‐fossorial when they have particularly specialized forelimbs for digging and burrow to create underground dens. Generalized terrestrial taxa can be found in different environments (e.g. Martes foina prefers rocky and open areas rather than forest habitats, unlike Martes martes, which is assigned as a semi‐arboreal species in this study) and might be able to swim, dig and/or climb (e.g. species of Procyon, Gulo gulo, Galictis cuja). To test for ecological signal reflected in the bony labyrinth shape of musteloids, we first performed Goodall's F‐tests based on the 3D Procrustes coordinates of the species and ecological categories by using the function procD.lm of geomorph. We also performed a phylogenetic anova using the reference phylogeny. The function procD.pgls of geomorph is used to analyze the covariation between shape data and discrete variables (i.e. ecological categories) in a phylogenetic context, using a Brownian motion model of evolution. We performed Goodall's F‐tests and phylogenetic anova tests for multiple clades (see Fig. 3) and estimated P‐values with the Bonferroni correction.

Results

Shape variation and allometry of musteloid bony labyrinths

Procrustes anova and permutation tests of bony labyrinth size vs. shape, performed across all musteloids and within each family (except the monospecific Ailuridae, for which we sampled two individuals of the same species), were not statistically significant (Table 3). In contrast to initial predictions, these results indicate that there is no significant effect of the bony labyrinth size on its shape variation. We therefore used the raw 3D Procrustes coordinates for analyzing the relationships between shape variation and ecologic and phylogenetic factors.

Table 3.

Results of Procrustes anova [mean Procrustes coordinates by species vs. log (mean centroid size by species) of bony labyrinth] performed within musteloids and for each family (except Ailuridae). Permutation tests for 10 000 iterations

	R 2	F	P‐value	df
ProcD
Musteloidea	0.043208	1.3096	0.20278	30
Mustelidae	0.07868	1.6226	0.060794	20
Mephitidae	0.56474	1.2975	0.41866	2
Procyonidae	0.23688	1.2416	0.24882	5
BE
Musteloidea	0.038214	1.1522	0.28167	30
Mustelidae	0.076172	1.5666	0.078292	20
Mephitidae	0.56764	1.3129	0.41326	2
Procyonidae	0.25643	1.3795	0.15233	5

BE, minimum bending energy criterion; df, degrees of freedom; F, test statistic; ProcD, minimum Procrustes distance criterion; R ², coefficient of determination.

Morphometric analyses of musteloid bony labyrinths

In PCA of the two datasets of 3D Procrustes coordinates (using minimum bending energy vs. minimum Procrustes distance semilandmark sliding criteria), the first three PC axes explain more than 50% of the shape variation (Table 4). In the first two PC axes, PC1 separates the morphology of bony labyrinths of the four musteloid families (Fig. 4). Mephitids and mustelids display the greatest morphological differences from each other on PC1, being arrayed towards opposite ends of the PC axis. The variance explained by PC1 increased when the Procrustes coordinates from semilandmark sliding using minimum bending energy were used relative to the minimum Procrustes distance criterion (Table 4). In other PC axes, mustelids vary widely in both PC2 (from the bony labyrinth shape of otters to that of the hog badger Arctonyx collaris) and PC3 scores (from the European mink Mustela lutreola to the Chinese ferret‐badger Melogale moschata or the European pine marten Martes martes, depending on the semilandmark sliding criterion used). On these PC axes, mustelids span the morphospace occupied by all other musteloids. Procyonids also vary widely in PC3 scores, although less so than mustelids (from the bony labyrinth shape of the northern raccoon Procyon lotor to the kinkajou Potos flavus or the ringtail Bassariscus astutus, depending on the semilandmark sliding criterion used). The 3D shape changes (warped surfaces) along the first three axes of the PCA are presented in Fig. 5 and summarized in Table 5.

Table 4.

Results of the PCA based on mean Procrustes coordinates by species

ProcD			BE
PC	% Variance	Cumulative %	PC	% Variance	Cumulative %
1	33.83	33.83	1	36.75	36.75
2	11.66	45.50	2	11.37	48.12
3	9.88	55.38	3	10.16	58.28
4	6.67	62.06	4	7.14	65.42

BE, minimum bending energy criterion; ProcD, minimum Procrustes distance criterion.

Figure 4.

Principal components analyses (PCA) based on the mean 3D Procrustes coordinates for each species, obtained by semilandmark sliding using minimum Procrustes distance (A,B) and minimum bending energy (C,D) criteria. The first three axes of each PCA are displayed: (A,C) PC1 and PC2, (B,D) PC1 and PC3. Bony labyrinth morphospaces of families are outlined by grey polygons. Triangle: Mephitidae; Square: Procyonidae; Star: Ailuridae; Circle: Mustelidae.

Figure 5.

Visualizations of shape changes in Musteloidea using 3D warped bony labyrinth surfaces for the first three axes of the PCA (see Fig. 3). Each warped bony labyrinth surface is represented in posterolateral, anterolateral and dorsal views, from top to bottom. (A) Minimum Procrustes distance criterion; (B) minimum bending energy criterion. Arrows indicate the accentuated variation from (A) to (B) for PC2 and PC3. lat, lateral; post, posterior; ventr, ventral.

Table 5.

Summary of the 3D shape changes (warped surfaces) of the bony labyrinths of musteloids along PC1, PC2 and PC3

Negative values	Positive values
PC1
Shorter and slightly curved common crus	Longer and more straight common crus
Circular anterior canal	Oval anterior canal
Smaller angle between lateral and anterior canals (< 90°)	Larger angle between lateral and anterior canals
Lateral canal concave anteriorly	Lateral canal convex anteriorly
Smaller lateral canal	Long U‐shaped lateral canal
More dorsal position of the posterior branch of the lateral canal relative to the posterior ampulla	Less dorsal position of the posterior branch of the lateral canal relative to the posterior ampulla
Posterior canal more extended laterally and shorter dorsally	Posterior canal more extended dorsally and shorter laterally
Laterally compressed vestibule	Laterally wider vestibule
Lower cochlea	Higher cochlea
Greater degree of curvature of the cochlea	Lesser degree of curvature of the cochlea
Cochlea pointing more ventrally in anterior view	Cochlea pointing more anteriorly in anterior view
PC2
Shorter common crus	Longer common crus
Higher angle between lateral and anterior canals	Smaller angle between lateral and anterior canals (< 90°)
Straight lateral canal	Lateral canal concave laterally and convex anteriorly
Circular lateral canal	Maximum axis of the lateral canal displaced posteriorly
Lack of secondary common crus	Presence of secondary common crus
Smaller angle between posterior and lateral canals (< 90°)	Higher angle between posterior and lateral canals
Higher angle between posterior and anterior canals	Smaller angle between posterior and anterior canals
Laterally wide vestibule	Laterally compressed vestibule
Wider and lower cochlea	Narrower and higher cochlea
Greater degree of curvature of the cochlea	Lesser degree of curvature of the cochlea
Cochlea pointing more medially in anterior view	Cochlea pointing more anteriorly and laterally in anterior view
PC3
Slightly curved common crus	More straight common crus
Lateral canal convex anteriorly and concave laterally	Lateral canal concave anteriorly and convex laterally
Higher angle between anterior and posterior canals (≥ 90°)	Smaller angle between anterior and posterior canals (< 90°)
Lateral canal more laterally extended	Lateral canal less laterally extended
Posterior canal less laterally extended	Posterior canal more laterally extended
More lateral connection between lateral and posterior canals	More medial connection between lateral and posterior canals
Smaller angle between posterior and anterior canals	Higher angle between posterior and anterior canals
Wider and lower cochlea	Narrower and higher cochlea
Greater degree of curvature of the cochlea	Lesser degree of curvature of the cochlea
Cochlea pointing more ventrally in anterior view	Cochlea pointing more anteriorly in anterior view

See Fig. 5 for illustration.

The first three principal components in both PCA correspond to complex 3D variations of the shapes of the bony vestibular and cochlear systems. Negative PC1 values are associated with: shorter and slightly curved common crus; circular anterior canal with an increased dorsal and lateral curvature (reduction of the angle between the lateral and anterior canals); lateral canal slightly convex laterally and concave anteriorly; small lateral canal (less extended both laterally and anteriorly); more dorsal position of the posterior branch of the lateral canal with respect to the ampullar entrance of the posterior canal (associated with the absence of a secondary common crus in mephitids and some procyonids); posterior canal more extended laterally and shorter dorsally; cochlea shorter in height; greater degree of curvature (i.e. greater diameter difference between the basal and apical turns); and cochlea twisted ventrally relative to the semicircular canals, associated with closer oval and round windows and a more laterally compressed vestibule. Positive PC1 values are associated with: longer and straighter common crus; rather oval anterior canal that is more extended dorsally than anteriorly; less sinusoidal anterior and posterior canals; higher angle between the anterior and lateral canals; lateral canal slightly concave laterally and convex anteriorly; long, U‐shaped lateral canal in dorsal view (more extended laterally than the posterior canal); more ventral position of the posterior branch of the lateral canal relative to the ampullar entrance of the posterior canal (presence of a secondary common crus in some mustelids); posterior canal more extended dorsally and slightly shorter laterally; higher cochlea; cochlea with a lesser degree of curvature (i.e. lesser diameter difference between the basal and apical turns); cochlea pointing more anteriorly; and laterally wider vestibule, with more distantly spaced oval and round windows.

In the PCA of superimposed shape data from the Procrustes minimum distance criterion, more negative PC2 values are associated with the following: shorter common crus with less dorsally extended anterior and posterior canals; increase of the angle between the anterior and lateral canals; rather straight lateral canal; circular lateral canal; lack of a secondary common crus with the posterior branch of the lateral canal anterior to the ampulla of the posterior canal; smaller angle between the posterior and lateral canals, and greater angle between the posterior and anterior canals; laterally wide vestibule, with the oval and round windows more separated from each other; wider and lower cochlea pointing more medially; and cochlear turns with a greater curvature degree. More positive PC2 values are associated with: longer common crus, associated with longer anterior and posterior canals; smaller angle between the anterior and lateral canals; lateral canal convex in anterior view and concave in lateral view; maximum axis of the lateral canal displaced posteriorly; presence of a secondary common crus; greater angle between the posterior and lateral canals, and weaker angle between the posterior and anterior canals; laterally compressed vestibule, with the round and oval windows close together; narrower and higher cochlea; cochlea apex pointing more anteriorly and laterally; and cochlear turns displaying a lesser degree of curvature. The same, but more accentuated, morphological changes along PC2 can be observed for the PCA, resulting from the minimum bending energy criterion. For the negative values of PC2, the lateral canal is more laterally extended relative to the posterior canal when using the minimum bending energy criterion dataset rather than the Procrustes distance criterion dataset (Fig. 5).

For principal component 3 in the PCA of the Procrustes distance criterion dataset, a more negative value is associated with: slightly curved common crus; lateral canal oriented almost horizontally, slightly convex anteriorly and concave laterally; posterior canal positioned perpendicular to the lateral canal; more laterally extended lateral canal and less laterally extended posterior canal, resulting in more lateral connection between the posterior branch of the lateral canal and the ventral branch of the posterior canal; smaller angle between the posterior and anterior canals; lower and wider cochlea situated more ventral relative to the semicircular canals; and greater degree of curvature of the cochlear turns. A more positive value is associated with: straighter common crus; markedly curved lateral canal, concave anteriorly and convex laterally; smaller angle between the posterior and lateral canals (< 90°); less laterally extended lateral canal and more laterally extended posterior canal, resulting in more medial connection between the posterior branch of the lateral canal and the ventral branch of the posterior canal; greater angle between the posterior and anterior canals; narrower and higher cochlea, more anteriorly pointed relative to the semicircular canals (so that the oval window is facing more anteriorly and the round window more ventrally); and lesser degree of curvature of the cochlear turns. The same morphological changes along PC3 can be observed for the PCA resulting from the minimum bending energy criterion, but again are more accentuated in the latter analyses. For the negative values of PC3, the lateral canal is more markedly curved (concave laterally and convex anteriorly), and the posterior canal and lateral canal exhibit a greater angle (> 90°) compared with the bony labyrinth shape, corresponding to the negative values of PC3 using the Procrustes distance criterion (Fig. 5).

Phylogeny and bony labyrinth shape

We calculated the degree of phylogenetic signal in the bony labyrinth shape for Musteloidea and multiple clades within the superfamily (Fig. 3, Table 6) using the Kmult statistic (Adams, 2014; see also Blomberg et al. 2003). Kmult values are lower than those expected by a Brownian model of evolution (K < 1), except for Mephitidae (clade 1, Fig. 3). However, the randomization test for mephitids indicates that even this phylogenetic signal is not statistically significant. The phylogenetic signal of the bony labyrinth shape obtained with the Procrustes distance criterion is significant for musteloids (clade 0, Fig. 3), for the clade Mustelidae + Procyonidae (clade 2, Fig. 3) and for the clade Mustelidae, excluding Taxidea taxus and Mellivora capensis (clade 6, Fig. 3). Using the minimum bending energy criterion, the phylogenetic signal is only significant for the entire Musteloidea and for the clade of Mustelidae + Procyonidae. These results suggest that the bony labyrinth morphology of Musteloidea, Mustelidae + Procyonidae, and Mustelidae (excluding Taxidea and Mellivora) can all be partially explained by phylogeny.

Table 6.

Phylogenetic signal as estimated with Kmult statistic for multiple clades of the reference tree (see Fig. 3)

Clades	ProcD		BE
	Kmult	P‐value corrected	Kmult	P‐value corrected
Clade 0	0.536949	0.00139986 **	0.5182461	0.00419958 **
Clade 1	1.012053	1	1.012867	1
Clade 2	0.505285	0.00279972 **	0.4858896	0.00979902 *
Clade 3	0.769863	1	0.7393746	1
Clade 4	0.533878	0.49275072	0.5037824	0.7769223
Clade 5	0.559807	0.78112188	0.5132381	1
Clade 6	0.545737	0.00839916 *	0.5044495	0.0769923
Clade 7	0.688826	0.79652034	0.6260761	1
Clade 8	0.692983	1	0.6662269	1
Clade 9	0.649099	0.32476752	0.6153896	0.92810718
Clade 10	0.65512	1	0.6423236	1
Clade 11	0.645157	0.51234876	0.6445374	0.6579342
Clade 12	0.681203	1	0.6698507	1
Clade 13	0.718854	1	0.7122635	1

BE, minimum bending energy criterion; ProcD, minimum Procrustes distance criterion.

Associated P‐values with Bonferroni correction applied for 14 tests; asterisks and bold indicate significant P‐values: *P < 0.05; **P < 0.005.

Moreover, the phylomorphospaces for Musteloidea (Fig. 6) show that species of the same family are clustered together, which confirms the relevance of the phylogenetic signal underlying the shape variation of the bony labyrinth within this sample. However, the phylomorphospaces for Mustelidae show several phylogeny‐shape crossings or morphospace overlap between subfamilies, which suggest a higher degree of bony labyrinth shape homoplasy within species of Mustelidae than within the other families of Musteloidea. For instance, the Mustelinae and Martinae occupy the same general morphospace in the first two axes of the PCA, with relatively short branches from their common ancestor to the terminal nodes leading to the taxa. In contrast, Lutra lutra diverges from the other otter species within Lutrinae, and instead converges on the Mustela species morphospace on PC2, while within the Martinae, Martes martes and Martes foina are divergent on PC3 (as well as on PC1, when using the minimum bending energy criterion). Each of these divergent taxa display long branches between their hypothetical ancestor and the terminal nodes in the phylomorphospace plots.

Figure 6.

Phylomorphospace of the mean bony labyrinths for each species of Musteloidea. Individuals are colored by ecological habitus (red, semi‐fossorial; blue, semi‐aquatic; brown, terrestrial; light green, arboreal; dark green, semi‐arboreal). (A,B) Minimum Procrustes distance criterion. (C,D) Minimum bending energy criterion.

Ecology and bony labyrinth shape

Taking into consideration all of the ecological categories, Goodall's F‐tests on the bony labyrinth 3D Procrustes coordinates show a significant ecological signal in the Mustelidae clade (clade 4, Fig. 3, Table 7), in the less inclusive mustelid clade excluding Taxidea taxus (clade 5, Fig. 3, Table 7), and in the clade that excludes both Taxidea taxus and Mellivora capensis (clade 6, Fig. 3, Table 7) when using minimum Procrustes distance criterion. The same tests performed on the data that used the minimum bending energy criterion for the semilandmark sliding are also significant for clades 5 and 6, but not for the Mustelidae clade (Fig. 3, Table 7). However, the phylogenetic anova fail to show any significant ecological signal, meaning that the shape of the bony labyrinth in the mustelid clades 4, 5 and 6 (Fig. 3, Table 7) is explained both by species ecology and their shared evolutionary history.

Table 7.

Results of Goodall's F‐tests and phylogenetic anova (iteration = 10 000), evaluating the relationship between ecological categories and bony labyrinth shapes (mean 3D Procrustes coordinates by species), with and without taking into account the phylogeny. Ecological signal is evaluated for multiple clades of the reference tree and for different set of ecologies (all ecologies, and each ecology compared with other categories)

	ProcD						BE						df
	Goodall's F‐test			Phylogenetic anova			Goodall's F‐test			Phylogenetic anova
	R 2	F	P‐value corrected	R 2	F	P‐value corrected	R 2	F	P‐value corrected	R 2	F	P‐value corrected
All ecologies
Clade 0	0.21079	1.7361	0.232674	0.14706	1.1207	1	0.19088	1.5334	0.737022	0.15881	1.2272	1	4
Clade 1	0.63995	1.7774	1	0.51607	1.0664	1	0.64095	1.7851	1	0.51716	1.0711	1	1
Clade 2	0.24779	1.8118	0.0805922	0.16203	1.0635	1	0.23767	1.7147	0.167687	0.16185	1.0621	1	4
Clade 3	0.49432	1.4663	1	0.49638	1.4785	1	0.44162	1.1863	1	0.45091	1.2318	1	2
Clade 4	0.23903	1.7799	0.0233974 *	0.18587	1.2937	1	0.22883	1.6814	0.0545948	0.19908	1.4085	1	3
Clade 5	0.26865	1.9591	0.00649935 **	0.20162	1.3469	1	0.25227	1.7994	0.0350961 *	0.21274	1.4412	1	3
Clade 6	0.30295	2.1731	0.00129987 **	0.22834	1.4796	1	0.27857	1.9307	0.0259974 *	0.23449	1.5316	0.945009	3
Clade 7	0.4725	1.7914	0.1104896	0.32806	0.9765	1	0.4178	1.4352	1	0.306	0.8818	1	2
Clade 8	0.32348	1.4344	0.856609	0.41847	2.1588	0.500448	0.31423	1.3747	1	0.41529	2.1307	1	1
Clade 9	0.25508	1.5409	0.597935	0.2345	1.3785	1	0.26217	1.5989	0.625885	0.25243	1.5195	1	2
Clade 10	0.15224	1.6163	1	0.15184	1.6112	1	0.17721	1.9385	0.581048	0.18474	2.0394	1	1
Clade 11	0.17984	1.7542	0.729222	0.18064	1.7637	1	0.20058	2.0072	0.510848	0.20911	2.1152	1	1
Clade 12	0.40524	2.044	0.170937	0.48769	2.8559	0.998296	0.45251	2.4795	0.320411	0.54471	3.5893	0.657735	1
Semi‐aquatic and other categories
Clade 0	0.10823	3.5197	0.0318967 *	0.049281	1.5032	1	0.096795	3.1079	0.0923912	0.058097	1.7887	1	1
Clade 2	0.035301	0.9148	1	0.042977	1.1227	1	0.033974	0.8792	1	0.047084	1.2353	1	1
Clade 4	0.04671	0.931	1	0.087189	1.8148	1	0.047823	0.9543	1	0.09034	1.8869	1	1
Clade 5	0.038998	0.7304	1	0.045047	0.8491	1	0.037499	0.7013	1	0.046035	0.8686	1	1
Clade 6	0.051444	0.922	1	0.070386	1.2872	1	0.048059	0.8582	1	0.073885	1.3562	1	1
Clade 9	0.056486	0.5987	1	0.057182	0.6065	1	0.051547	0.5435	1	0.053074	0.5605	1	1
Clade 9+ Melinae	0.034671	0.431	1	0.088572	1.1661	1	0.038587	0.4816	1	0.090277	1.1908	1	1
Clade 9+ Martinae	0.039664	0.6195	1	0.059188	0.9437	1	0.037983	0.5922	1	0.056381	0.8962	1	1
Clade 10	0.095686	0.9523	1	0.11759	1.1994	1	0.12009	1.2284	1	0.13821	1.4434	1	1
Clade 11	0.1395	1.2969	1	0.093135	0.8216	1	0.15682	1.4879	1	0.092622	0.8166	1	1
Clade 12	0.20703	0.7833	1	0.24533	0.9752	1	0.20525	0.7747	1	0.23874	0.9408	1	1
Semi‐fossorial and other categories
Clade 0	0.046084	1.401	1	0.029091	0.8689	1	0.042063	1.2734	1	0.031919	0.9562	1	1
Clade 1	0.48442	0.9396	1	0.53802	1.1646	1	0.48108	0.9271	1	0.53478	1.1495	1	1
Clade 2	0.024313	0.623	1	0.057851	1.5351	1	0.023778	0.6089	1	0.06067	1.6147	1	1
Clade 4	0.045101	0.8974	1	0.042256	0.8383	1	0.039814	0.7878	1	0.036509	0.72	1	1
Clade 5	0.053751	1.0225	1	0.077039	1.5024	1	0.060143	1.1518	1	0.085024	1.6726	1	1
Clade 6	0.068046	1.2412	1	0.08891	1.6586	0.9599	0.065604	1.1936	1	0.087845	1.6372	1	1
Clade 7	0.19406	1.204	1	0.3494	2.6852	0.86091	0.20766	1.3104	1	0.37194	2.961	0.53295	1
Clade 9+ Melinae	0.087508	1.1508	1	0.10862	1.4622	1	0.086723	1.1395	1	0.10941	1.4743	1	1
Clade 9+ Martinae	0.037803	0.5893	1	0.040653	0.6356	1	0.037413	0.583	1	0.037408	0.5829	1	1
Clade 9	0.050509	0.532	1	0.054727	0.5789	1	0.052625	0.5555	1	0.051884	0.5472	1	1
Semi‐arboreal and other categories
Clade 0	0.025779	0.7674	1	0.038725	1.1683	1	0.022483	0.667	1	0.033913	1.018	1	1
Clade 2	0.023696	0.6068	1	0.031441	0.8116	1	0.023901	0.6122	1	0.027606	0.7097	1	1
Clade 3	0.24977	1.3317	1	0.15251	0.7198	1	0.2332	1.2165	1	0.13363	0.617	1	1
Clade 4	0.036652	0.7229	1	0.052695	1.0569	1	0.041631	0.8254	1	0.057992	1.1697	1	1
Clade 5	0.096013	1.9118	0.373014	0.11204	2.2711	0.600237	0.099118	1.9804	0.261423	0.11789	2.4056	0.489555	1
Clade 6	0.044482	0.7914	1	0.034937	0.6154	1	0.039403	0.6973	1	0.030367	0.5324	1	1
Clade 7	0.22298	1.4348	0.832869	0.35299	2.7278	0.188082	0.2375	1.5574	0.648387	0.36614	2.8882	0.090891	1
Clade 8	0.18397	0.6763	1	0.18743	0.692	1	0.1809	0.6625	1	0.16648	0.5992	1	1
Clade 9+ Martinae	0.05583	0.887	1	0.049927	0.7883	1	0.050356	0.7954	1	0.042071	0.6588	1	1
Arboreal and other categories
Clade 0	0.053894	1.652	0.256473	0.042112	1.2749	1	0.049403	1.5072	0.39897	0.050291	1.5357	0.64014	1
Clade 2	0.037744	0.9806	1	0.045781	1.1994	1	0.040308	1.05	1	0.04732	1.2418	1	1
Clade 3	0.1655	0.7933	1	0.20136	1.0085	1	0.19999	1	1	0.24452	1.2946	1	1
Semi‐arboreal‐arboreal and other categories
Clade 0	0.048947	1.4925	0.39717	0.03206	0.9605	1	0.04381	1.3287	0.58404	0.032993	0.9895	1	1
Clade 2	0.038234	0.9938	1	0.032083	0.8287	1	0.041134	1.0725	1	0.028189	0.7252	1	1
Clade 3	0.18569	0.9121	1	0.21765	1.1128	1	0.16283	0.778	1	0.24849	1.3227	1	1

BE, minimum bending energy criterion; df, degrees of freedom; F, test statistic; ProcD, minimum Procrustes distance criterion; R ², coefficient of determination.

Associated P‐values with Bonferroni correction applied for 13, 11, 10, 9, 3 and 3 tests; asterisks and bold indicate significant P‐values: *P < 0.05; **P < 0.005.

Taking into consideration each ecological category independently, Goodall's F‐tests indicate that semi‐aquatic taxa (comprising Mustela lutreola, Mustela vison, Lontra canadensis, Lontra felina, Enhydra lutris, Lutra lutra, Aonyx cinerea) are significantly differentiated from the non‐aquatic musteloids when using the Procrustes distance criterion (clade 0, Fig. 3, Table 7). Here again, after accounting for the phylogeny (phylogenetic anova), there is no significant difference for the semi‐aquatic category in musteloids, indicating that the difference of bony labyrinth shapes between semi‐aquatic and non‐aquatic musteloids is also partly linked to their shared common ancestry. Finally, Goodall's F‐tests and phylogenetic anova together indicate that, for both the Musteloidea clade as a whole and for its subclades, none of the other distinct ecologies alone [semi‐fossorial, semi‐arboreal, arboreal, and a newly created, clustered category that includes both semi‐arboreal and arboreal taxa (to distinguish taxa preferring trees vs. land substrates)] exhibit a bony labyrinth shape significantly different from the rest of the ecological categories (Fig. 3, Table 7).

Because the bony labyrinths of semi‐aquatic forms differ from those of the non‐aquatic musteloids, we estimated the mean shape of the bony labyrinth of all non‐aquatic musteloids vs. the semi‐aquatic musteloids [the five otters (Lo. canadensis, Lo. felina, E. lutris, L. lutra, A. cinerea) and the two minks (M. lutreola, M. vison)]. Morphological differences between the two average bony labyrinths for these groupings are related to the semicircular canals and orientation of the cochlea relative to the semicircular canals (Fig. 7). The bony labyrinth of semi‐aquatic musteloids displays an oval rather than a circular anterior canal, a sinuous rather than a straight lateral canal, with an acute rather than a right angle between the posterior and the lateral canals, and a cochlea facing more anteriorly compared with non‐aquatic musteloids. Physiological attributes that could be related to those morphological features, and by extension the link between the bony labyrinth morphological characters and ecological specializations in the particular case of the semi‐aquatic adaptation in otters and minks, are discussed in the next section.

Figure 7.

Mean shape of non‐aquatic musteloids (A) and aquatic musteloids (B) in anterolateral (top) and posterolateral (bottom) views. Arrows indicate the location of the morphological differences between the mean shapes.

Discussion

Our results show that the bony labyrinth shape is not influenced by bony labyrinth size variation in Musteloidea, or in the clades Mustelidae, Mephitidae or Procyonidae. Several studies of mammals demonstrated a negative allometric relationship between the bony labyrinth and its centroid size (e.g. Spoor et al. 2007; Lebrun et al. 2010; Alloing‐Seguier et al. 2013). Lebrun et al. (2010) and Alloing‐Seguier et al. (2013) indicate that in strepsirrhine primates and diprotodontian marsupials, the ‘larger specimens possess anterior and posterior semicircular canals proportionally more developed and a lateral semicircular canal proportionally less developed’; they also have a higher position of the posterior canal relative to the lateral, longer common crus, and ‘smaller and laterally oriented cochlea’. None of those morphological tendencies applies to our musteloid carnivoran sample, suggesting that there is no allometric relationship between the bony labyrinth and its centroid size in this clade, and potentially for other Carnivora. It is possible, however, that such allometric effects will be observed when analyzing bony labyrinths across a broader sample of Carnivora, incorporating a wider variation of body size, phylogenetic relationships and ecologies, a potential association that can and should be tested in future studies.

Our analysis of bony labyrinth shape variation in Musteloidea shows a clear distinction between the morphological traits of the commonly recognized families (clades) of mephitids, ailurid, procyonids and mustelids. These morphological changes affect the entire bony labyrinth: size and curvature of the semicircular canals; angles between the canals; presence or absence of a secondary common crus; degree of lateral compression of the vestibule and orientation of the cochlea relative to the semicircular canals; and degree of coiling and proportions of the cochlea. Such phylogenetic signal is also confirmed by the significance of the permutation test for the Kmult statistic in Musteloidea. The phylogenetic signal also is significant for the musteloid subclades (see Fig. 3), as long as the number of taxa included in these tests is > 12, reflecting the importance of large sample sizes in permitting a test outcome that can overturn the null hypothesis of random association between shape and the variable being tested. The exceptions are for clades 4 and 5 (Mustelidae, and Mustelidae excluding T. taxus; Fig. 3). Indeed, the morphological variation of bony labyrinths observed in these two clades instead could be more influenced by species ecology, as suggested by the significant P‐values of Goodall's F‐tests. For clade 6 (Mustelidae excluding T. taxus and M. capensis; Fig. 3), the shape variation is influenced significantly by both phylogeny and ecological categories. Again, clades with fewer than 12 taxa do not show significant P‐values for any ecological signal. At least some of these negative results could be due to the small sample size (e.g. Blomberg et al. 2003), underlining the challenge of testing the correlation between shape and ecology, both independently and in covariance with the phylogeny, when working with discrete ecological factors and moderate sample sizes.

The phylomorphospace analyses indicate different degrees of convergence, divergence and reversion of bony labyrinth shapes in Mustelidae, probably closely linked to their wide diversity of locomotor and ecological specializations, and rapid diversification history of the clade. Several bursts of diversification of mustelids occurred during the Miocene and Pliocene. New ecological opportunities for mustelids may be related to environmental changes, such as that potentially linked to the onset of permanent Antarctic and Arctic ice sheets and to the strengthening of the Asian monsoons (Koepfli et al. 2008). For example, the bony labyrinth of the common otter L. lutra exhibits a narrower and higher cochlea compared with the other otters of our sample, and appears morphologically closer to the Mustela species on PC2. As Lutra is not the most basally divergent taxon of the Lutrinae, this could demonstrate a reversion of the bony labyrinth morphology towards the ancestral condition of the clade Mustelinae + Lutrinae. The stone marten M. foina and the European pine marten M. martes are divergent from each other on PC3 (and PC1 for the minimum bending energy criterion), differing substantially in proportions of the cochlea, lateral extension and shape of the lateral canal, and presence/absence of a secondary common crus. This labyrinth morphology divergence could be related at least in part to their ecological differentiation, with the stone marten being more terrestrial and occupying more rocky and open areas, compared with the European pine marten, which occurs in forest environments and is more adapted for an arboreal locomotor ecology (Nowak, 2005).

Using the minimum Procrustes distance or minimum bending energy for semilandmark sliding in our study resulted in differences in the distribution of bony labyrinth shapes along PC axes, in the percentage of variance explained by PC axes, and in the significance of our phylogenetical and ecological statistical tests. In both semilandmark identification approaches, the goal is to minimize the differences between each specimen and the average shape of the entire sample studied, in order to create landmarks comparable between individuals. However, whereas the Procrustes distance criterion tends to slide semilandmarks along the direction locally parallel to the line defined by two adjacent landmarks on the curve, the bending energy criterion makes them slide on the tangential direction of the curve, leading to substantial differences in the final semilandmark conformations. Perez et al. (2006) and Tseng & Flynn (2015) performed PCA and statistical analyses based on outlines of dental and facial structures in humans and on skull surfaces in Carnivora, respectively. As in our study, their results showed that the percentages of variation explained by the PC axes using the minimum bending energy were greater than when using the minimum Procrustes distance criterion. Moreover, in Perez et al. (2006), the distribution of morphospaces differs along PC axes. One of the consequences of the morphospace variation is related to the identification of morphological changes along PC axes. The reconstructions of bony labyrinth shapes along the extremes of PC2 and PC3 (warped bony labyrinth surfaces) using the alternative semilandmark identification criteria indicate a greater morphological variation, notably in the degree of curvature of canals and angles between canals, when using the minimum bending energy instead of the minimum Procrustes distance criterion. Moreover, the significance of the three distinct types of statistical tests performed in this study (Kmult statistic, Goodall's F‐test, phylogenetic anova) is more sensitive in each case when using the minimum Procrustes distance criterion. In sum, in our study, the minimum bending energy seems to capture a greater variation between individuals, but the more important intra‐ and intergroup variation might obscure the significance of the statistical tests. However, these results could change depending on the variation within a particular taxonomic sample, and notably the taxonomic level studied, making it difficult to choose definitively one sliding criterion as always preferable to the other in geometric morphometric studies.

We found that the morphological differences between the average bony labyrinths of semi‐aquatic and non‐aquatic musteloids are related to their semicircular canals and the orientations of the cochleas relative to the semicircular canals (Fig. 7). Compared with non‐aquatic musteloids, the bony labyrinth of minks and otters displays an oval rather than a circular anterior canal, sinuous rather than straight lateral canal, acute rather than right angle between the posterior and the lateral canals, and cochlea facing more anteriorly when compared with non‐aquatic musteloids. The deviation of the semicircular canals from circularity might affect the dimensions of the membranous ducts and therefore the flow dynamics of the endolymph and the afferent nervous sensitivity during head motion while locomoting in aquatic environments. In this regard it is noteworthy that the anterior semicircular canal of another clade of aquatic Carnivora, the Pinnipedia (seals, sea lions, walrus and their relatives), has been described as more elliptical compared with terrestrial members of the order such as the dog Canis familiaris (Georgi, 2008; Ekdale, 2013, 2015). The degree of curvature of the semicircular canals can help to maximize vestibular sensitivity to the primary rotational directions, by picking up accelerations from different planes, as they do not align with the anatomical canal planes (Rabbitt, 1999; Malinzak et al. 2012). It has been suggested that both the deviation from circularity and the degree of curvature of semicircular canals in mammals have a weak or insignificant effect on the ratio of canal plane area to streamline length (P/L) of the semicircular canal, which is strongly correlated with vestibular sensitivity and agility in mammals (McVean, 1999; Cox & Jeffery, 2010). Among the three canals, however, the lateral semicircular canal is the most closely related to locomotor agility, in particular in aquatic, arboreal, aerial and saltatorial mammals (Spoor et al. 2007; Cox & Jeffery, 2010; Ekdale, 2013), underlying the importance of a better understanding of labyrinth shape variation for making reliable locomotor and ecological inferences. The deviation from orthogonality of the semicircular canals could be associated with the speed of head rotations (Malinzak et al. 2012), as mammals with faster head rotations have canals that are closer to right angles. This would imply that otters and minks have slower head movements compared with non‐aquatic musteloids, perhaps related to a slower motion in the more viscous medium of water rather than air. It is worth noting that the morphological characters of the semicircular canals contribute together to the overall sensitivity of the vestibular system, so that one shape character alone cannot adequately define a specific association with the animal's agility (the measurement of which also is contingent on the definition of agility). Determining the relationship between bony labyrinth morphology and agility also is difficult, because the relationships between afferent sensitivity and the agility is challenging to determine empirically (Malinzak et al. 2012; see also David et al. 2010 for functional characters of the bony labyrinth).

The significance of the semi‐aquatic signal in the bony labyrinth shape of musteloids is also influenced by phylogenetic history. This highlights the necessity to perform ancestral state reconstructions of the bony labyrinth, especially in analyses that also include fossils, as incorporating fossils may markedly change inferences of ancestral states relative to analyses that include only extant taxa (e.g. Finarelli & Flynn, 2006). Moreover, further investigation can better identify the range of morphological characters of the bony labyrinth that may be closely linked to adaptation to an aquatic environment, by including a broader sample of terrestrial vs. aquatic Carnivora taxa, representing more independent origins of aquatic specialization across clades.

Concluding remarks

Our results show that the bony labyrinth shape variation in Musteloidea and in each of the traditionally recognized families (Mephitidae, Procyonidae, Mustelidae) is not influenced by the bony labyrinth size range of our sample. The shape of the vestibular and cochlear regions of the bony labyrinth is significantly related to phylogenetic relationships in musteloids and in most mustelids. Moreover, locomotor ecology differences between terrestrial, semi‐arboreal, arboreal, semi‐aquatic and semi‐fossorial species affect the shape of the bony labyrinth of musteloids. More particularly, the bony labyrinth of otters and minks can be distinguished from those of non‐aquatic musteloids, mainly by morphological differences of the semicircular canal angles and shapes. These results should motivate further analysis of the evolution of bony labyrinth morphological traits in musteloids and other Carnivora, as well as the investigation of sensorial adaptations of Carnivora to aquatic environments.

Author contributions

Camille Grohé designed the study, acquired, analyzed and interpreted the data, and wrote the manuscript. Z. Jack Tseng, Renaud Lebrun and Renaud Boistel acquired the data. Z. Jack Tseng, Renaud Lebrun and John J. Flynn participated in designing the data analysis and interpreting the data. All the authors revised and improved the writing of this manuscript.