Analysis of Fiber Strain in the Human Tongue During Speech

Abstract

The goal of this study is to investigate mechanical cooperation between tongue muscles and to demonstrate the application of model-based reconstruction of dynamic medical imaging. Five healthy volunteers were imaged using tagged magnetic resonance imaging while speaking, enabling extraction of 4D strain (volume and time) via model-based reconstruction. Strain in the line of action of fibers (SLAF) was extracted from the imaging-based strain estimation by projecting the strain tensor onto a model of fiber directionality. SLAF was averaged across each volume, producing time history waveforms, which were temporally aligned to determine consistency across subjects and correlation across muscles. The results indicate evidence of consistent muscular contractions and extensions across subjects. The strongest positive correlations involved the anterior genioglossus, superior longitudinal, and hyoglossus muscles. The strongest negative correlations involved the transverse, verticalis, anterior genioglossus, superior longitudinal, and hyoglossus muscles. These results suggest that the investigated tongue motion is achieved through opposing action (involving the transverse muscle) and bending (involving the genioglossus). While the mechanism of fiber strain is biologically constrained to linear fiber families, patterns of simultaneous contraction or extension of fibers provide means to produce bending and shearing motion, and to intensify deformation via volume conservation.

Introduction

Deformation of the tongue plays a key role in several everyday functions, including breathing, swallowing, and speech generation—a principal contributor of quality of life [1,2]. Deformation, quantified by the strain tensor, arises from biomechanical interactions including interfaces with organs such as bones, contraction of several myofiber families, and a delicate temporal orchestration between substructures in the tongue [3–5]. In speech production, the tongue modulates the reverberating volume within the vocal tract by surface shape changes, enabling the articulation of different sounds [6]. Surface changes result from volumetric shape changes achieved via time-variant and spatially inhomogeneous deformation patterns [7–9]. These patterns are of scientific and clinical interest, and although they can be interrogated using medical imaging (which yields 4D strain tensor fields), the results are difficult to visualize and interpret [3,10]. Thus, interpretation with respect to tissue anisotropy, conservation of volume, and boundary conditions is necessary to take advantage of experimental strain measurements.

To interpret the biomechanics of tongue motion, physiologists leverage a priori knowledge of the tongue’s structure. The tongue includes hierarchal arrangements observed in other types of muscle as well as unique structural elements that support both heterogeneous activation patterns along the organ’s volume as well as myofibril contraction in one or two (almost perpendicular) directions [4,5,11,12]. Thus, the tongue is not a uniform muscle, but a collection of several interdigitated muscles, which leverage conservation of volume for producing strain perpendicular to the line of action of contractile fibers [3,13]. Tongue muscles can be organized into two categories, intrinsic muscles, which are defined as coursing entirely within the tongue body with no insertions on any bones, having no insertions, and extrinsic muscles, which have such insertions [4,5]: The extrinsic muscles are different from the intrinsic muscles because they connect to neighboring bones in the head and neck. Because of this characteristic, the direction of motion produced by independent activation of extrinsic muscles is fairly obvious. However, there is little empirical evidence to suggest that independent activation is the modus operandi in speech generation or other tongue functions. In fact, the opposite is far more likely [8,9,14,15]. Therefore, while it is an important prerequisite, anatomical characterization alone is insufficient for understanding the biomechanics of tongue movement.

A more precise characterization of tongue motion comes from noninvasive measurements of muscular activity, tissue deformation, and computer models of the vocal tract. Data from electromyography have been previously used to estimate muscular activation by mapping of surface electrical potentials to individual tongue regions [16]. Although these estimates are coarse (due to the unreliable electrode contact and the diffusive nature of electrical conductivity), they suggest strong cooperations between muscles across most voluntary tongue movements [17]. Visualization of tissue movement can be performed with tagged magnetic resonance imaging (MRI) and ultrasound imaging [10,18]. Of these, MRI measurements offer the highest quality and have the added ability to characterize morphology via high-resolution imaging and muscle structure with diffusion tensor MRI (DT-MRI) [19,20]. Previous studies have used this information to establish a link between fiber arrangement and the mechanical configuration of the tongue in controlled motion and swallowing [3,21], as well as noted differences in motion patterns between controls and tumor patients [21,22]. However, little specific information about cooperative motion of tongue substructures has been produced based on experimental data. Instead, mechanical cooperation has been implied using simulated kinematics, which suggest muscular cooperation schemes based on similarity between observed and simulated deformations [8,9,13,15,23]. Thus, to complement previous research, it would be advantageous to obtain experimental measurements that can be used to evaluate and improve simulations and deepen our understanding of the production of tongue movement.

Previous imaging studies point to challenges that limit the use and interpretation of experimental data, particularly (1) the quality of fiber characterization (especially in areas of crossing fibers) [20,24] and (2) inconsistency in measurements due to the lack of simultaneous acquisition of motion, anatomical, and structural measurements during speech generation [25,26]. Unfortunately, overcoming both of these limitations involves technical problems, which most likely will remain unsolved for years to come. At the same time, methods comparison of spatiotemporal tensor fields remain limited, and although novel techniques are beginning to be explored [27], there is no consensus on what may constitute the best approach. Therefore, the goal of this study is to use computational methods to reduce the influence of experimental limitations and reduce their dimensionality, enabling interpretation of the existent data with respect to an anatomical reference. In particular, we use a model-based approach to enforce physical relationships in the motion and structure of the tongue, and we use the resulting data to obtain insights on muscular cooperation based on strain on the line of action of fibers (SLAF) associated with the local arrangement of muscular tissue.

Materials and Methods

The overall methodological approach is illustrated in Figure 1. We start with (in vivo) tagged MRI of the tongue in motion and use it to quantify deformation. Our main strategy was to use an approximation of local muscle fiber directionality to interpret this data by reducing the dimensionality of tensorial data from a 4D tensor field (volume and time) to a measure of relative muscular elongation over time. Throughout this work we expressed this strain measure using a stretch ratio where a value of one signifies no strain and values above or below one respectively represent stretch or contraction with respect to the reference configuration defined in the following sections.

Figure 1.

Overall methodological approach.

Imaging Data

Tagged MRI of healthy individuals (n=5, 2 male, 3 female) was obtained using a Siemens Prisma Fit 3.0 T scanner (Siemens Healthcare, Erlangen, Germany) as part of an auxiliary study (see [10] for additional information, including informed consent and approved protocol information). The volunteers were trained to articulate the utterance “asuk” (Figure 2), which is composed of four sounds: /ə/ as is the sound “uh” in the word “hum”, /s/ is the sound of the letter “s” in the world “sierra”, /u/ is the sound of the letter “u” in the word “luke”, and /k/ as in the first letter of the word “kiosk”. The utterance was chosen to focus on the tongue movement evolving from a state of relative tonal relaxation, /ə/, into protrusion, /s/, and eventually into retraction, /k/. Imaging information consisted of anatomical CINE images and tagged MRI, which were acquired as part of a larger imaging study. The CINE images were obtained using a gradient echo pulse sequence, and tagged MRI were obtained via a CSPAMM sequence [28]. Magnetic resonance tags were applied during the first part of the utterance—the sound /ə/. In that sense, this time point bears some similarity to end diastole, which is used as the reference configuration in cardiac imaging [29]. The resulting data consisted of 10 coronal slices and 8 sagittal slices, each composing a 256 × 256 matrix of 1.9 ×1.9 mm resolution and 6 mm thick (for each of the sequences). Both temporal sequences involved 26 frames over 1 s, and were obtained in Siemens 3.0T Tim Trio scanner (Siemens, Malvern, PA). More information about medical imaging acquisition can be found in [30].

Figure 2.

CINE sequence during phonemes of interest.

Model-Based Displacement Reconstruction

Two types of models were used to approximate strain in the line of action of muscle fibers from tagged MRI: a biomechanical model and a structural model (Figure 1). The former was used to extract spatiotemporal displacements, while the latter was used to approximate fiber directionality and muscular placement. Displacements were obtained from tagged MRI data via harmonic phase analysis with finite elements (HARP-FE) [31]. Briefly, HARP-FE method employs a finite-element mesh (of the tongue, the mandible, and hyoid bone), and converts tagged MRI information into a pseudo force field that deforms the mesh in order to conform to the observed changes in the images. The subsequent reconstruction of 4D motion is obtained while enforcing physical qualities, including mechanical compatibility (one to one mapping across time), incompressibility, and boundary conditions, as well as reducing the effect of noise [31]. Note that deformation is extracted directly from images, and images in tagged in orthogonal directions were used to track tissue; thus, simulated contraction was not needed for tracking in areas of low contrast, as done in the literature [32]. The mesh used in this study is composed of 1025 nodes arranged in 300 tetrahedral elements (bones) and 255 quadratic hexahedral elements with 20 nodes each (tongue), and was described in [12]. From this original mesh, we created subject-specific meshes topologically identical, but geodetically altered to conform to the anatomy of each of the subjects. The alteration was achieved using a manually assisted approach that included an affine component to capture most of the change in shape, as well as a non-rigid component for smaller adjustments. A structural model of the tongue was used to approximate fiber directionality and muscular placement. To this end, the directional “flow” of myofibers was approximated using solutions to the Laplace equation obtained in each of the subject-specific meshes [12]. These solutions are obtained based on muscular compartments (shown in Figure 3 in schematic form) and boundary conditions that mimic muscular placement and insertions [4,5]. This approach has shown agreement with DT-MRI in areas where this measurement is available, and produces an approximation informed by a priori anatomical information (obtained from dissection and an anatomical imaging atlas) in areas where these measurements are insufficient to resolve the underlying directional data [12]. To calculate SLAF, elementwise strain measurements were projected onto the local fiber orientation evaluated at the element centroid. Because tongue muscles are interdigitated, two (approximately orthogonal) fiber families share the same volume. Based on physiological studies, we assume that the shared volume consists of the intersection of two muscles, as opposed to a single, multi-directional muscle. Thus, each muscle was assumed to act only along a primary fiber direction, even though there are elements in the mesh that belong simultaneously to two muscles. When two muscles shared the same volume two SLAF measurements were obtained along each direction. The SLAF measurements were associated with eight separate muscles, which are described below.

Figure 3.

Muscular compartments. Volumetric regions were used to calculate the mean strain on the line of action of fibers associated with local muscular tissue. Each color shows the approximate location of the compartment on a midsagittal view of the tongue.

Biomechanical Analysis

As in the heart (also a hydrostat), any given reference configuration is not purely strain-free due to the effect of biological processes such as residual tone and stresses, which make obtaining initial stretch information intractable [33]. For this reason, and based on previous studies on motion analysis in the tongue and the heart [10,29], strain was measured with respect to the initial time frame in the time sequence, when the tongue was in a relatively relaxed configuration while the mandible opened as the /ə/ sound was initiated. All the analysis in this study leveraged SLAF approximations averaged across a volume associated with particular muscles. This averaging strategy enables us to see if structures that have been grouped together via gross dissection, do indeed correspond to mechanical groups that act together over time. The genioglossus was subdivided into anterior (GGA) and posterior (GGP) parts. The other muscle representations consist of the following muscles: superior longitudinal (SL), inferior longitudinal (IL), geniohyoid (GH), hyoglossus (HG), transverse (T), vertical (V), and styloglossus (SG) (Figure 3). The mean SLAF approximations enabled a visual semiquantitative analysis of consistency, and a quantitative correlation-based assessment of whether two muscles acted together determining mechanical cooperation. As a reference, the spatially distributed SLAF evaluated at discrete locations can be seen in Figure 4.

Figure 4.

Visualization of strain in the line of action of selected fiber families (shown in Figure 3). From the top: longitudinal (SL, IL, HG), transverse (T) and towards the styloid process (SG), radial (GGA, GGP, and GH), and vertical (V). Major elliptical diameter shows fiber direction. The tongue model is shown in the sagittal view with the anterior to the left of the figure.

To observe trends of motion, we visualized the time history (waveform) of the SLAF averaged across each of the muscle volumes as a function of time. The estimations of SLAF were temporally aligned, so each the temporal location of each phoneme occurred at the same time. The timing of each sound was identified by two vocal tract visualization experts based on CINE-MRI visualization of the utterance. Once this timing was established, the SLAF time histories were interpolated via spline interpolation to obtained temporally aligned waveforms. The consistency of mean SLAF across subjects was evaluated based on overall trajectory in time. Measurements in a given muscle were deemed consistent if individual time histories had similar trajectories in all subjects. Waveforms were considered similar if the relative extension-contraction pattern between consecutive phonemes was the same in more than 3 out of the 4 phonemes (75% of the time). For instance, the GGA was considered consistent because all subjects exhibited contraction followed by extension in three phonemes, /e/, /s/, and /u/ (although in /k/ one of the subjects showed extension instead of contraction). Partial consistency was defined when similarities were only present in four subjects. The waveforms were deemed inconsistent if the conditions for similarity or partial similarity were not met.

To determine whether SLAF changes in two muscles occurred together, we calculated the Spearman rank-order correlation between time history pairs in each subject. The significance level in the correlation coefficients was 0.01. (Note that the Spearman rank-order correlation does not assume a constant rate of variation between variables.) Correlation analysis was global, i.e., across all time points in the sequence; thus, it resulted in a single global correlation matrix per subject. Each matrix was visualized as a colormap with “x” denoting significance, or a connectogram where the thickness of the connecting line was inversely proportional to the p-value and the color represented the sign of the correlation, as demonstrated in Figure 5. To determine the consistency between correlation matrices across all subjects, we created a cumulative connectogram where the thickness of the connecting line was directly proportional to the number of significant correlations between muscle pairs, and the color represented the percentage of positive correlations within all the significant pairs. For instance, if four out of the five subjects exhibited significance and three of these were positive, then the percentage of positive significance was 75%.

Figure 5.

Visualization of correlations between strain in the line of action of muscle fibers. The color in the cells of the table (A) encode the Spearman correlation coefficient (Cp) between SLAF time histories across muscles. An “x” marks instances where the correlation is statistically significant (p<0.01). The connectogram (B) consists of drawing a connecting line between significantly correlated muscles, using a color to identify Cp. The thickness of the connecting line is inversely proportional to the p-value.

Results

The time histories of average SLAF, which are shown in Figure 6, are most consistent among subjects in the GGA, IL, and T muscles. Note, for instance, that the GGA muscle starts with a relatively mild contraction of fibers of roughly −3% (mean +/− std. dev. across subjects), evolving almost linearly to about 20%, after which, it decays. In the T muscle, SLAF shows extension followed by increasing amounts of contraction peaking at 10% towards the end of the sequence. The IL muscle shows a sequence of extension, contraction, followed by extension. Partial consistency across subjects is observed in the GH muscle, which features similarities only in the first half of the time sequence, and in the SL, V, HG, and GGP muscles, where only three of the subjects show waveform trajectory similarities. The remaining time history, the SG, is not consistent because there is too much variation across subjects. One of the subjects (labeled as subject #2 in Figure 6), shows a different behavior, particularly in V, SL, HG and GGP. The time histories also offer some visual evidence of correlation between muscles, such as the T and GGA traces, which suggest that one muscle contracts as the other extends.

Figure 6.

Average SLAF time histories. Each solid line represents the spatial mean strain in the line of action of myofibers across a muscle volume. The dotted lines show the standard error SEM of the time history.

Quantitative analysis shows that SLAF correlations vary individually but also include common patterns across subjects. Individually, the correlations show that the most evident pattern of differentiation between subjects is the number of statistically significant correlations, which varies between individuals from 18 to 24, as shown in Figure 7. Positive correlation was more common in the subjects with a larger number of significant correlations, where over 70% of statistically significant correlations were positive. In the subjects with fewer significant correlations, positive correlations occur in nearly 50% of cases. In terms of common patterns, our analysis shows that the T muscle has the largest number of negative correlations with respect to three muscles, the SL, the GGA, and the GH. In addition, T is involved with V, although one of the subjects exhibits a weak positive correlation instead of a negative correlation (which appears in the other four subjects). The largest number of positive correlations involve the GGA muscle with respect to the two muscles, SL and HG. The V muscle is positively correlated with SL, GGA, and SG, but only in some (at least three) of the subjects. The V muscle is also involved with HG, even though one of the subjects exhibits a weak positive correlation instead of a negative correlation. Positive correlation is also observed in SL and GH and the SL and HG muscles, where positive correlations are common in at least three of the subjects. There are also situations in which three or more correlations are observed, but there is no consensus on the sign of the correlations. This is the case when the relationships involve the SG muscle, and include correlations with the T, SL, GGA, and GH muscles.

Figure 7.

Individual and cumulative correlations of average SLAF waveforms. The connectograms of individual subjects (top) show average SLAF correlation according to Figure 5. The cumulative connectogram (bottom) consists of drawing a connecting line between muscles that exhibit at least 3 significant correlations in the individual subjects. The color of the connecting line is derived from the proportion of positive (or negative) correlations.

Discussion

The most important finding in this study is a quantitative insight about the nature of mechanical cooperation between tongue muscles during speech generation. In particular, we found clear signs of simultaneous and antagonistic contraction and extension in both the SLAF estimation waveforms and their correlations.

Waveform analysis enabled us to compare the way muscles contract or extend across study subjects. Despite some inconsistencies, different tongue muscles show similarities, which can be associated with conformational changes necessary to produce the phonemes in the utterance. Based on visual inspection and motion analysis, the motion experience during the utterance in this study can be described as follows: The /ə/ sound is characterized by a mild jaw rotation and a mild widening of the tongue in the left-to-right direction, and the phoneme /s/ consists largely of tongue protrusion. After that, increasing amounts of elevation are added in order to achieve /u/, culminating with /k/— a combination of retraction and tongue elevation. Although some subjects appear to depart from this description (such as Subject #2 in Figure 6, which appears to maintain some tongue protrusion during /u/ and /k/, balancing with additional IL and SG shortening), our findings suggest a mechanism for the generation of the most common deformations: In order to produce /ə/, the tongue experiences a small retraction in the form of relative motion between the GGA and GGP muscles, while the side-to-side widening is apparent in the initial extension of the T muscle (GGA shortens to bring the tongue down and back to the middle of the vocal tract, causing extension in two directions, the GGP and T). Deformation patterns are mild because /ə/ results in the least amount of deformation compared to the other phonemes (generally less than 5% in most muscles). Development of the /s/ appears to be accomplished largely by volume-preserving shift through narrowing of the tongue via T muscle contraction and support by the V muscle, which likely activates to maintain its length or where length is maintained as a result of palate contact. As a result, superior and inferior longitudinal extension is observed. Tongue elevation i.e., extension in the radial direction (as seen in V muscle fibers), is also elicited by volume-preserving shift with a noticeable contraction of the T muscle, which is larger in /k/ than /u/ given than the latter experiences less elevation. Evidence for retraction can be seen in the IL muscle, which during /u/ and /k/ maintains an extended length compared to the reference configuration, but contracts with respect to the /s/ conformation. The transition from protrusion (/s/) towards elevation (/u/) and retraction (/k/) is largely modulated by relative motion of the GGA and GGP, which changes sense i.e., forward bending during /s/ and backward bending during /k/. Interestingly the SL remains extended during these deformations, which indicates the presence of elevation of the tongue, forcing the dorsal surface to remain extended either anteriorly, or as it bends during retraction.

Larger deformations are characterized by larger extensions than contractions, with the latter reaching more than 25% in some muscles compared to the former, which remain less than 10%, suggesting some limit on sarcomere contraction. Similarity of motion was particularly clear in muscles occupying a relatively large volume (primarily the GGA and T muscles). This observation can be explained from both a mechanical and an experimental perspective. Mechanically, relatively uniform motion across a large muscle facilitates deformation stemming from volume-preserving shifts in order to achieve large changes in length with a relatively small amount of sarcomeric contraction. Larger muscles may also be more efficient at producing motions associated with a common set of target phonemes. Thus, we speculate that the overall tongue shape is driven by the larger muscles, while more idiosyncratic shape variations may be associated with smaller muscles—those that also exhibited more differences across participants. From an experimental perspective, larger volumes are less susceptible to imaging artifacts such as partial volume, and are less prone to errors in muscular placement that may stem from the model. Based on the experimental viewpoint, we can explain why smaller muscles (particularly the SG muscle, but also the IL muscle) exhibited the least amount of consistency across subjects and greater variance on the waveforms.

With correlation analysis, we were able to observe patterns of mechanical cooperation evidenced by simultaneous action of muscles within each subject, and we determined whether these patterns persist across subjects. Globally, the strongest correlation patterns, i.e., the ones with the most significant correlations of the same sign, involved the T muscle, whose action serves as a primary driver of tongue elevation, resulting in simultaneous extension of the GGA, HG, and SL muscle fibers (note that these last three muscles were positively correlated to each other). Negative correlation was also detected between the T and V muscles, indicating antagonistic action stemming from the elevation mechanism (because vertical and radial fibers in the GGA muscle have similar directionality). Relative motion or bending via antagonistic deformation of the GGA and GGP can also be observed, but is not as consistent across participants as the strongest interactions. Despite their physical proximity, GGP and HG muscles are not correlated; instead, the HG is strongly correlated to the GGA, most likely because these two muscles are influenced by GGP. Evidence for this type of synchrony within the genioglossus has been found via electromyography and numerically [13,34]. Other correlations are weaker in terms of the number of subjects in which significant correlations are observed or their signs are in agreement.

This study is limited, but provides motivation for further research aimed at improving basic knowledge of the biomechanics of speech generation through a combination of modeling and comprehensive interpretation of experimental data. Although the presented analysis allowed us to dissect the image-derived 4D strain fields with a more accessible measure of deformation along a local fiber direction, some inconsistency appeared on the average SLAF waveforms due to imaging artifacts or areas of suboptimal modeling. Although the numerical means to improve morphological and structural modeling with improved geometrical representation and better fiber orientation models do exist, there is much improvement to be made in terms of extracting accurate experimental data in vivo in order to create more accurate subject-specific models. These approaches largely involve acquisition and processing techniques of MRI data, including faster and more consistent CINE imaging to lower the amount of independent motion repetitions as well as improved structural characterization by means of DT-MRI or similar techniques. Nevertheless, our model-based interpretation of experimental data can be used to complement anatomy-driven and computational speculations regarding the production of tongue motion. For instance, literature featuring the anatomy of the tongue puts forward two mechanisms for producing forward motion, bulk motion and extension [5], and our observations indicate that the latter is in involved in the /s/ sound via joint motion of the T and V muscles. This result is consistent with computational results [13], which point to these two muscles, particularly in the posterior portion of the tongue and relative motion or bending via the anterior and posterior portions of the genioglossus (although glossectomy patients likely rely on bulk motion instead). Likewise, the mechanism for /ə/ likely involves mild muscular contraction by action of the jaw, which carries the whole tongue with minimum amount of strain or muscular activation [5,13]. We also found some disagreement with numerical results, which suggest that the SG muscle plays a large role in retracting the tongue [14]. Instead, retraction during /k/ appears to involve both bending via the GGA and GGP muscles (in the opposite direction of that seen during the production of the /s/ sound), although our results in the SG are subject to some experimental inconsistency. Finally, we showed that mechanical cooperation between contraction (or extension) in the line of action of fibers is present, although uniformity is not strict across individuals. The existence of strong correlations as well as dispersion in the average SLAF (which forgets all the muscle contraction dynamics focusing on change of shape from a reference configuration) substantiates a fundamental link between the overall motion of the tongue—which includes visibly similar configurations at given phonemes—and the idiosyncratic dynamics necessary to achieve these configurations in the presence of subject-specific differences; that is, source of variability can also include physical differences, such as volume and morphology as well as learned patterns of motion, both of which warrant more investigation.

Table 1.

Consistency results of average SLAF waveforms.

Waveform	Consistency	Comments
GGA	consistent	subject 5 shows an increase (all other subjects show a decrease) from /u/ to /k/
IL	consistent	subjects 3 and 5 show increases, while 1 and 2 remain constant from /u/ to /k/
T	consistent	steady decrease, although subjects 1 and 2 show constant values from /u/ to /k/
GGP	partially consistent	subject 5 shows noticeably different waveform
SL	partially consistent	subject 2 shows a relatively constant value from /s/ to /u/, subject 1 shows a decrease from /u/ to /k/
GH	partially consistent	subjects 2 and 5 show constant values during /k/, subjects 1 and 4 show increases from /s/ to /u/
HG	partially consistent	subjects 2 and 3 show decreases from /e/ to /s/, subject 5 remains constant from /u/ to /k/
SG	inconsistent	similar trajectory only from /u/ to /k/ across all subjects
V	partially consistent	subjects 1 and 2 shows increase from /s/ to /u/, and decrease from /u/ to /k/