Skip to main content
The Journal of Deaf Studies and Deaf Education logoLink to The Journal of Deaf Studies and Deaf Education
. 2015 Dec 6;21(2):171–186. doi: 10.1093/deafed/env055

Impacts of Visual Sonority and Handshape Markedness on Second Language Learning of American Sign Language

Joshua T Williams 1,✉, Sharlene D Newman 1
PMCID: PMC4886314  PMID: 26644551

Abstract

The roles of visual sonority and handshape markedness in sign language acquisition and production were investigated. In Experiment 1, learners were taught sign-nonobject correspondences that varied in sign movement sonority and handshape markedness. Results from a sign-picture matching task revealed that high sonority signs were more accurately matched, especially when the sign contained a marked handshape. In Experiment 2, learners produced these familiar signs in addition to novel signs, which differed based on sonority and markedness. Results from a key-release reaction time reproduction task showed that learners tended to produce high sonority signs much more quickly than low sonority signs, especially when the sign contained an unmarked handshape. This effect was only present in familiar signs. Sign production accuracy rates revealed that high sonority signs were more accurate than low sonority signs. Similarly, signs with unmarked handshapes were produced more accurately than those with marked handshapes. Together, results from Experiments 1 and 2 suggested that signs that contain high sonority movements are more easily processed, both perceptually and productively, and handshape markedness plays a differential role in perception and production.


Learning a new language late in adulthood can be a difficult experience. Learning novel sounds (Best & Tyler, 2007), word segmentation (Field, 2003), and a myriad of other features (Birdsong, 1992) can create many roadblocks along a learner’s acquisition path. However, there are many characteristics of the first language that can facilitate acquisition of a second language (Gass & Selinker, 1992). Many of the phenomena that have been documented to either facilitate or hinder second language acquisition are largely restricted to our knowledge of how two spoken languages interact within a bilingual system. Learners whose first and second languages are both spoken are referred to as unimodal bilinguals. On the other hand, those learners whose first language is spoken but are acquiring a sign language are referred to as bimodal bilinguals. A distinction between the types of language modalities that bilinguals use is important to our understanding of how knowledge of one language can influence the acquisition of another. Most studies that examine transfer effects investigate two spoken languages. One could imagine how general knowledge of the universal structure of phonology (e.g., sonority) could influence sign language acquisition, however, regardless of the divergence between the two language modalities.

Evidence for amodal transfer between languages during L2 acquisition comes from studies that have demonstrated that learners of a sign language use knowledge of their first language co-speech gesture system (Brentari, Nadolske, & Wolford, 2012; Chen Pichler, 2009) as well as other sources (Chen Pichler & Koulidobrova, 2015) to aid in sign language acquisition. As such, it is likely the case that bimodal bilinguals can use such knowledge to help attune to salient features in their sign language. It has been hypothesized that there are modality-independent phonological characteristics of language (Berent, Dupuis, & Brentari, 2013). In fact, sonority, or the perceptual salience of a phonetic feature, has been implicated as an amodal feature that is pervasive in both spoken and sign languages. It is possible that sonority can also be processed at the perceptual level regardless of L1 phonological knowledge. As such, the present study aims to investigate the role of visual salience on sign language learning in hearing adults.

Multidimensional perceptual salience, or the ability for a feature to stand out in the input based on some dimension, has been shown to be important during many cognitive processes, including language acquisition (Goldschneider & Dekeyser, 2001; Yantis & Egeth, 1999). Multidimensional salience in language can arise in what is termed sonority (Ohala and Kawasaki, 1984). Sonority in spoken language has a phonetic correlate of amplitude, or loudness, of a given speech sound. In other words, sonority is often thought of as the relative degree of constriction of the oral cavity (Chin, 1996). In fact, sonority can be ranked on a hierarchy based on its relative amplitude (Hankamer & Aissen, 1974). In addition to having phonetic correlates, sonority can also be important phonologically. For instance, sonority is important in the syllabification of languages insofar as languages often arrange their sound sequences based on constraints of sonority (see Sonority Sequencing Principle, Clements, 1990). As such, the syllable and its sonority have representational power within the phonological system of any given language (Blevins, 1995). The representational power of the syllable (i.e., sonority) in spoken language can affect child language acquisition and unimodal second language acquisition. Children are aware of sonority restrictions early in acquisition (Berent, Harder, & Lennertz, 2011) and attune to speech as a function of sonority (Yavas & Gogate, 1999). Children also produce sound clusters with the greatest sonority rise earlier (Ohala, 1999) and show greater generalizability of learning patterns based on sonority complexity (Gierut, 1999). Additionally, unimodal second language (L2) learners show increased variability in L2 production as a function of sonority (Broselow & Finer, 1991; Eckman & Iverson, 1993; Tropf, 1987). It is important to investigate whether sonority impacts sign language learning given positive evidence that sonority may impact spoken language processing and learning; however, we must first explicate how sonority is defined in sign languages.

Based on the fact that sonority is the conceptual representation of perceptual salience in language, it is unsurprising that sign languages also have a visual correlate (Brentari, 1993, 1998). Sign phonologists agree that movement is the most sonorous element in a well-formed sign (Brentari, 1993, 1998; Corina, 1996; Liddell & Johnson, 1989; Perlmutter, 1992; Sandler, 1993; Wilbur, 1993). Signs have similar syllable structure as spoken words insofar as movement accounts for the syllable nucleus, similar to vowels (Brentari, 1998, 2002; Sandler, 1993). However, there is some debate as to how best to quantify sonority. For instance, visual sonority can be derived from the proximity of the articulating joint (e.g., shoulder, elbow, fingers, etc.; Brentari, 1998), the type of movement during a sign’s production (Sandler, 1993), or by other perceptual variation like movement size (Crasborn, 2001). On the other hand, sonority has also been posited to be derived based on phonotactics and movement deletion patterns, which are separable from the aforementioned theories of movement-type sonority (Corina, 1996). These accounts differ based on whether sonority is treated as phonetic or phonological in nature, respectively. For instance, the sonority hierarchy proposed in Sandler (1993) is purely phonetic given that not all contrasts based on movement characteristics are phonologically motivated (see Corina, 1996 for discussion). Although theories of sonority in sign languages, like spoken language, are complex and still unresolved, the present study adopts the stance that sonority can be phonetic in nature and thus movement characteristics (e.g., articulating joint, path and hand-internal movements, etc.) are important to the perceptual salience of the sign (see Brentari, 1998; Sandler, 1993).

According to the particular theory adapted in this study, movements distinguish the syllable complexity and are the most sonorous elements of the sign. Thus, sign languages can create sonority by the perceptual visibility of the articulating joint (e.g., signs with shoulder movements are more visible than those with interphalangeal movements; Brentari, 1998, 217). It could be assumed based on this account of sonority that greater visibility, which implies greater sonority, could provide advantages for some signs over others in terms of identification and subsequent processing. Motion (or movement) has been shown to enhance visual perception in other domains (e.g., Ambadar, Schooler, & Cohn, 2005). As such, the same may apply to sign language learning. In fact, many studies have shown that native deaf signers and second language learners of sign language often acquire and identify movement features much later and with more errors than the other phonological parameters (Bochner, Christie, Hauser, & Searls, 2011). Moreover, deaf children often use less motorically complex proximal articulators (i.e., shoulder; high sonority) than the complex adult-target distal articulators (i.e., phalangeal joints; low sonority) during early sign language acquisition (Meier, 2006; Meier, Mauk, Cheek, & Moreland, 2008). No study has yet to examine the role of sonority on movement identification or production in adult learners. Due to the perceptibility of high sonority signs and the fact that hearing second language learners have fully developed motor systems, it is possible the high sonority signs may be acquired more easily.

It should be noted that perceptual salience in sign language may not be restricted to movement sonority. Based on a number of studies, there is evidence that learners may have difficulty in the perception of other sublexical features (or parameters) based on their perceptual salience and psycholinguistic properties (Bochner et al., 2011; Emmorey, McCullough, & Brentari, 2003; Grosvald, Lachaud, & Corina, 2012; Morford & Carlson, 2011; Morford, Grieve-Smith, MacFarlane, Staley, & Waters, 2008). It might be the case that the perceptibility of a given sign is a function of multiple features; that is, a combination of a salient handshape and a salient movement (i.e., sonority) might provide greater perceptibility.

Multiple cues can be advantageous in cognitive processing across multiple domains. For instance, multiple auditory cues can aid in auditory processing (Schröger & Widmann, 1998). Furthermore, when listeners are shown visual information in conjunction with auditory information, there is expedited processing both in behavioral performance and neural processing (Du et al., 2011; van Wassenhove, Grant, & Poeppel, 2005). Multiple cues in visual processing are also advantageous (Itti & Koch, 1999). Given that multiple cues aid in cognitive processing across domains, it may be the case that beginner learners of sign language also use multiple perceptual cues (e.g., sonority, handshape markedness) during sign language learning. In order to develop a more nuanced understanding of the interaction between perception and sign language acquisition, it’s important to begin moving away from a simplistic conception of sign parameters as a linear hierarchy of features and to investigate whether it is in fact a dynamic system where multiple features interact to contribute to overall saliency. As such, it is important to examine the role of different types of salience.

Markedness may provide another source of salience. Unmarked features are common features that occur relatively often, whereas marked features are unusual and occur rarely; markedness can be thought of as the relative frequency of a given feature (Jakobson, 1968). Handshape, or the configuration of the selected fingers of a sign, can be delineated into a group of marked and unmarked handshapes. Typically, unmarked handshapes are limited to a small group of handshapes (B, A, S, C, O, 1, and 5; Battison, 1978; Boyes-Braem, 1990; Grosvald et al., 2012; Siedlecki & Bonvillian, 1997; see Appendix for depictions of handshapes). Deaf children acquire unmarked handshapes earlier, which is thought to be a result of motoric simplicity (Ann, 2006; Siedlecki & Bonvillian, 1997). This points to another account of markedness such that it is a result of motoric complexity. Ann (2006) computed handshape markedness in Taiwanese Sign Language based on ease of articulation as determined by several anatomical criteria (e.g., muscle opposition in handshape configuration, support for extension and flexion, tendency to oppose thumb, and tendency to spread), which roughly, but not exclusively, correlated with many findings in the American Sign Language (ASL) literature (see Boyes-Braem, 1990). Additionally, given that markedness can be accounted for by either frequency or motoric complexity, and that hearing nonsigners or naive learners are not attuned to the frequency characteristics of the language, learners may process markedness based solely on motoric complexity. However, handshape markedness may not be relegated to only motoric complexity, but also visual complexity. In a phoneme-monitoring task, deaf signers perceived marked handshapes better than unmarked handshapes; however, hearing nonsigners perceived unmarked handshapes better (Grosvald et al., 2012). This reversal is thought to be driven by perceptual salience such that deaf signers attune to information that stand out in their input (i.e., marked features), whereas hearing nonsigners attune to less complex structures (i.e., unmarked features). Thus, it might be hypothesized that signs that contain unmarked handshapes would be less complex and be more easily acquired by hearing second language learners. However, it is not clear whether the complexity that drives these differences lies within the visual or motoric systems.

Taking these factors into consideration together with theories of sonority, it was predicted that the acquisition of signs depends on multiple saliency features. It was hypothesized that multiple features and their visual saliency values influence sign language learning. Despite learners perceiving unmarked handshapes better than marked, we predicted that signs that contain marked handshapes and high sonority movements increase perceptibility. This prediction is due to marked handshapes being visually distinctive, especially when paired with high sonority movements. Hence, greater perceptibility of the sign will have an additive effect on the phonological specification during acquisition. It is possible, however, that unmarked handshapes with high sonority movements are easier to acquire. In that vein, the roles of motoric and visual complexity were investigated by examining the subsequent production of these signs. Since both sonority and handshape markedness can be derived from both motoric and visual complexity, these differential effects of sonority and handshape markedness may change when signers are required to produce these signs. It is hypothesized that marked handshapes with high sonority movements are more easily perceived due to the increase in visual salience; it can be conversely hypothesized that signs with high sonority (i.e., high visual salience) and marked handshapes (i.e., high motoric complexity) are harder to produce overall, biasing production to be faster for unmarked handshapes (see the influence of articulatory complexity on speech sound and lexical acquisition in children: Cairns, 1996; Sander, 1972; Schwartz & Leonard, 1982). We hypothesized that the benefits of certain salient features (i.e., sonority, handshape markedness) play differential roles in perception and production during the acquisition process.

To summarize, this study aimed to explore the role of visual salience (i.e., sonority and markedness) on the perception and production of ASL. We tested these hypotheses using a sign-picture matching paradigm in Experiment 1 and tested a subsequent reproduction task using a key-release measure in Experiment 2. A sign-picture matching task was chosen because previous studies have used the method effectively to investigate how certain linguistic aspects influence both child language and adult second language learning (Escudero, Hayes-Harb, & Mitterer, 2008; Showalter & Hayes-Harb, 2013; Storkel & Adlof, 2009; Storkel & Lee, 2011). Given that we were interested in how sonority and handshape markedness influence sign acquisition in adult learners, this task is quite useful because we are able to explicitly test learners’ accuracy in acquiring a sign across these conditions. Additionally, the sign learning task provided a method to explicitly test how these features influence recognition (or encoding). On the other hand, a sign reproduction task was chosen to test whether sonority and markedness would differentially affect production. A distinction between these two processes provides insights into how salient phonological features influence encoding and retrieval differentially and whether sonority and markedness are more visually or motorically salient.

Experiment 1: Sign–Picture Matching Task

Methods

Participants

Twenty-five English-speaking participants (four males) were recruited from an introductory psychology course. All participants received course credit for their participation. The university Internal Review Board approved all procedures. The participants’ ages ranged from 18 to 21 years (M = 18.32; SD = 0.69). All participants scored as right-handed on the Edinburg Handedness Inventory (Oldfield, 1971; M = 72.5; SD = 16.1). Eighteen participants reported at least one spoken second language (Spanish = 12; Japanese = 2; Latin = 1; German = 1; Tamil = 1; Hindi = 1), but no participants reported experience or exposure to any sign language (including ASL). Additionally, all participants reported no speech, hearing, or neurological disorders. Hearing nonsigners were selected for this study in order to simulate initial stages of learning. This also allowed us to make conclusions based on the perceptual processing of signs with no interaction with established lexical items.

Materials

Sixteen to-be-learned ASL signs were selected. These 16 signs were split into high and low sonority groups (n = 8 each). Sonority was rated using both the Brentari (1998) and Sandler (1989) models of ASL sonority scales. Based on these models, a general sonority hierarchy was constructed for this study in which sonority was determined by a combination of articulating joint (i.e., shoulder [5] > elbow > wrist > base > nonbase [1]) and movement type (i.e., path movements with trilled internal movements [5] > path movements with internal movements > local internal movements > contacting movements > trilled stationary [1]). For example, a sign that is articulated with the shoulder joint with a trilled path movement would theoretically be the most sonorous and a sign that is articulated with base and nonbase joints with a trilled stationary movement would be least sonorous. The group of high sonority signs included communication, vomit, show, sorry, decide, join, ask, and sympathize. The low sonority signs included cookie, fine, huh, audiology, high school, puzzled, and hate (refer to Figure 1 for a sample of the stimuli). There was a significant difference of sonority ranking between the low and high sonority groups (t(7) = 4.710, p < .05; high = 4.25 [0.46], low = 2.38 [0.74]). In order to capture other possible explanations of visual saliency, signs in both low and high sonority groups were split into two subsequent groups based on handshape markedness. Markedness was based on several studies that have documented the acquisition of handshape by children and adults (Ann, 2006; Boyes-Braem, 1990; Brentari, 1998; Grosvald et al., 2012; Siedlecki & Bonvillian, 1997). Signs were classified as unmarked if they contained the unmarked handshapes B, A, C, and 1 and as marked if they contained the marked handshapes F, H, X, or 8 (see Appendix for depictions of handshapes). A native ASL signer signed the stimuli at a slow but naturalistic rate in front of a blue-gray backdrop. Video clips of the signs were edited to one frame before lift of the hands and one frame after the drop of the hands. The durations of the signs did not differ across sonority (high = 1,663 [171] ms; low = 1,538 [207] ms; F < 1), markedness (unmarked = 1,600 [184] ms; marked = 1,600 [143] ms; F < 1), or an interaction between the two (F < 1). All other aspects of sign phonology (e.g., number of hands, location, etc.) were randomly varied.

Figure 1.

Figure 1.

A sample of the stimuli included in this experiment. Signs with high sonority are on the top row (e.g., vomit, ask) and those with low sonority are on the bottom row (e.g., fine, puzzled). As static 2D pictures, it is hard to intuitively derive the sonority; however, the high sonority signs are characterized by path movements that span the neutral space (e.g., vomit: mouth to neutral; ask: mouth to neutral). The low sonority signs have either internal handshape movements (e.g., puzzled is stationary at the forehead but the handshape changes from 1 to X) or local movements (e.g., fine: slight repeated taps to the chest). The stimuli are further delineated by markedness across columns, with signs with unmarked handshapes (e.g., B) to the left and marked handshapes (e.g., X) to the right. Photo released with permission.

All of the signs were paired with a novel nonobject. Sixteen imageable gray scale line drawings of nonobjects were pseudo-randomly selected from Kroll and Potter (1984). Nonobjects were selected such that there was no iconic mapping between the phonology of the sign and the nonobject’s representation (see Figure 2 for examples of nonobjects). Iconicity has been shown to influence sign acquisition and processing such that native deaf signers and late L2 learners are often faster at naming highly iconic signs (Ormel, Hermans, Knoors, & Verhoeven, 2009; Thompson, 2011; Thompson, Vinson, & Vigliocco, 2009), but the number of arbitrary, noniconic signs that are acquired early in language acquisition outnumber iconic signs (Orlansky & Bonvillian, 1984) and sign acquisition often does not follow iconic principles (Emmorey, 2001; Meier, 1982). Additionally, iconicity does not have a privileged role in lexical access (Bosworth & Emmorey, 2010). Therefore, shielding against iconicity does not invalidate the learning of these signs. Nonobjects were also selected so that participants would be required to create a new semantic representation as well as to shield against imagability between sign and semantic representations. All subjects saw the same sign-nonobject pairs, which is similar to the use of nonobjects in language learning that has been demonstrated in a number of other successful child language learning paradigms (Storkel & Adlof, 2009; Storkel & Lee, 2011) and L2 learning paradigms (Escudero et al., 2008; Showalter & Hayes-Harb, 2013).

Figure 2.

Figure 2.

Study design.

Procedure

The procedure used was similar to previous studies examining L2 phonological acquisition (see Showalter & Hayes-Harb, 2013, 2015). Participants were seated at a 27-inch widescreen iMac computer. The experiment was controlled by PsychoPy software (Pierce, 2007). There were two phases: the learning phase and the final sign-picture matching test. The participants were presented with a 500-ms fixation cross before each trial. During the learning phase, participants were exposed to both ASL sign and nonobject representations. The ASL signs were presented on the right side of the screen. To the left of the ASL sign appeared the matching nonobject representation. Previous spoken L2 studies have simultaneously presented the word aurally and the semantic representations visually in a cross-modal learning paradigm. Since all of the stimuli in the present study are visual, the ASL sign was presented for the duration of the sign and the nonobject was presented for 1,000ms longer than the sign (e.g., ask = 1,600ms, nonobject = 2,600ms; see Figure 2). This method of presentation was assumed to provide the participants enough time to visually encode all of the information on the screen. Participants were also instructed to look at the right side of the screen (for ASL sign) first and then look to the left (for the nonobject). Each of the 16 signs was randomly presented once per block for 3 block repetitions.

The final sign-picture matching test consisted of all 16 signs randomly presented. Immediately after the presentation of the sign, a two alternative force choice nonobject referent-matching paradigm was presented to the participants. Two nonobjects appeared on the screen. The correct nonobject was randomly assigned to either the left or the right. Another nonobject (that was the correct answer for another sign representation) was randomly presented in the other location. Participants were instructed to select the nonobject that matched the sign they previously had seen. If the correctly matching nonobject was on the left, the participants were to press the “1” key with their left index finger. If the correctly matching nonobject was on the right, the participants were to press the “0” key with their right index finger. All selections were instructed to be as fast as possible, while being as accurate as possible. Reaction times (RT) were measured at the onset of the trial. Given that there was no significant difference in video lengths across all conditions, RT should not be colored by video lengths. The test phase only presented each sign once. None of the alternative nonobject choices (the foils) were shown more than once as a foil.

Data Analysis

Data analysis was conducted using mixed-effects models (R Statistics v.3.1.2; Bates, Maechler, Bolker, & Walker, 2013) that included both fixed effects (i.e., sonority and markedness) and random effects (i.e., participants and items). Mixed-effect modeling is now commonplace in psycholinguistic literature in light of many arguments against traditional analysis of variance. Specifically, mixed-effects models allow for the modeling of random effects that are caused by participant and item variance. Additionally, mixed-effects models can account for both continuous (e.g., RT) and binary outcomes (e.g., accuracy counts; see Baayen, Davidson, & Bates, 2008 and Jaeger, 2008 for discussion). Each model investigated the main effects of the fixed effects (i.e., sonority and markedness) as well as their interaction at both the participant/group (F1) and item (F2) levels by including these as random effects.

Results

RT measured from the onset of correct trials were filtered for outliers that fell 2 SDs above or below the mean (1.4%). Descriptive statistics can be found in Table 1. The linear mixed-effects model revealed no significant main effects of either sonority (F1(1,775) = 0.022, p = .881; F2(1,16) = 0.003, p = .995) or markedness (F1(1,775) = 0.626, p = .429; F2(1,16) = 0.093, p = .764). There was a significant interaction observed between sonority and markedness at the group level (F1(1,775) = 7.803, p = .005; F2(1,16) = 1.164, p = .297) (Figure 3).

Table 1.

Statistics for Experiment 1 learning note

Predictor Condition Mean SE F1 | F2 p1 | p2
Sonority Markedness
Reaction times
Sonority High 4,131 81 0.022 | 0.003 .881 | .995
Low 4,093 92
Markedness Unmarked 4,093 85 0.626 | 0.093 .429 | .764
Marked 4,131 90
Sonority × Markednessa High Unmarked 4,148 111 7.803 | 1.164 .005 | .297
Marked 4,115 88
Low Unmarked 4,037 92
Marked 4,148 109
Accuracy
Sonoritya High 89.0% 2.5 7.592 | 2.067 .006 | .170
Low 82.5% 2.5
Markedness Unmarked 86.5% 2.2 0.404 | 0.110 .525 | .110
Marked 85.0% 2.8
Sonority × Markednessa High Unmarked 87.0% 2.6 5.436 | 1.480 .020 | .241
Marked 91.0% 3.3
Low Unmarked 86.0% 2.4
Marked 79.0% 3.4

Note. F1 = group, F2 = item.

aSignificant.

Figure 3.

Figure 3.

The reaction time (in milliseconds; left) and accuracy (right) results for the sign–picture matching task split by sonority and markedness. Error bars represent ± 1 SE.

Accuracy results revealed a significant main effect of sonority at the group level (F1(1,775) = 7.592, p = .006; F2(1,16) = 2.067, p = .170) such that high sonority signs (89% [2.5]) were more accurately learned than low sonority signs (82.5% [2.5]). There was no main effect of markedness (F1(1,775) = 0.404, p = .525; F2(1,16) = 0.110, p = .467) such that both unmarked (86.5% [2.2]) and marked (85% [2.8]) signs were learned equally well. There was an interaction of sonority and markedness at the group level (F1(1,775) = 5.436, p = .020; F2(1,16) = 1.480, p = .241).

Recall, this study aimed to investigate: (a) whether visual sonority provides greater intelligibility for marked handshapes and (b) whether there are additive effects of sonority and markedness on learning such that unmarked high sonority (i.e., high salience, low complexity) signs are easier to acquire than marked low sonority signs (i.e., low salience, high complexity). Planned t tests were performed to investigate these outstanding hypotheses and to further explore the interaction effects in both RTs and accuracy. A comparison of low and high sonority signs that both contained marked handshapes revealed a significant effect only for accuracy (t(24) = 3.361, p < .01) such that high sonority signs (91.0% [3.3]) were more easily learned than low sonority signs (79.0% [3.4]) when they contained a marked handshape (t RT < 1); however, they did not differ when they contained an unmarked handshape for either RTs or accuracy (ts < 1). A comparison of unmarked high sonority and marked low sonority signs revealed a significant difference for both RT (t(24) = 2.144, p < .05) and accuracy (t(24) = 2.486, p < .05) such that unmarked high sonority signs (87.0% [2.6]) were learned more easily and responded to more quickly (4,036ms [38]) than marked low sonority signs (accuracy: 79% [3.0]; RT: 4,148ms [54]). However, this was not the case for high sonority signs that differed in markedness (accuracy: t(24) = 1.138, p = .266; RT: t < 1).

Given that the planned comparisons showed a hierarchy of confusability in accuracy (e.g., high + marked [91%] > high + unmarked [87%] = low + unmarked [86%] > low + marked [79%]), it was important to delineate confusability more explicitly. A confusion matrix was computed in order to capture qualitative insight into how signs were learned based on their sonority and markedness. How often a given sign was classified as another sign in the learning test phase was calculated. In Figure 4, the signs are plotted and divided by their sonority and markedness. Along the diagonal is how often a given sign was correctly identified as itself with the proportion indicated in the box. Here, we can qualitatively capture how signs were confused based on their sonority and markedness by summing the confusion value for each sign within a condition and dividing by the number of confused signs. For sonority, results from the confusion matrix revealed that low sonority signs were mistaken for other low sonority signs (14.0%) less often than high sonority signs (18.0%) and high sonority signs are mistaken for other high sonority signs (0%) less often than for low sonority signs (11.0%). Taken together, this suggests that low sonority signs were mistaken for any other sign 16% of the time, whereas high sonority signs were mistaken for any other sign only 5.5% of the time. For markedness, results indicated unmarked signs were mistaken for marked signs (16.0%) more often than unmarked signs (12.0%). Marked signs were mistaken for marked signs (18.8%) more than unmarked signs (8.6%). Collectively, this pattern of results indicated that unmarked signs were mistaken for any other signs (14%) only slightly more often than marked signs (13%). A conjunction of the two conditions indicated a similar hierarchy as mentioned above such that high sonority signs with marked handshape were misidentified only 9.0% of the time relative to high sonority signs with unmarked handshapes (13.0%), low sonority signs with unmarked handshapes (14.0%), and low sonority signs with marked handshapes (21.0%). Given that the confusion matrix demonstrates the predicted inverse relationship to the accuracy data (i.e., the more accurate, the less confusion), then we can confidently say that the same general hierarchical pattern is robust.

Figure 4.

Figure 4.

A confusion matrix was constructed to qualitatively characterize the confusions between signs in the learning phase of Experiment 1. The signs are ordered based on their sonority and markedness. The boxes are colored based on how often a sign was identified as that sign, where greater identification with a given sign was weighted by a darker color. The values along the diagonal represent correct identification and other values are misidentifications. Values are proportions (0–1) and can be converted to percentages by multiplying by 100. It should be noted that this confusion matrix should be read left to right, with the confusion summing up to 1 (or 100%) across the columns in any given row. For example, ask was only categorized as either ask (88%) or cookie (12%), but not puzzled (0%); however puzzled was in fact misidentified as ask 38% of the time.

Discussion

Participants attempted to learn novel ASL sign-nonobject mappings in a repetition nonobject referent-mapping task in Experiment 1. It was our aim to investigate the role of visual salience on the acquisition of novel signs. It was hypothesized that high visual sonority would facilitate sign-picture matching. Additionally, the visual salience of marked handshapes was also expected to facilitate acquisition. Thus, it was predicted that to-be-learned signs that contained low sonority movements and unmarked handshapes would be harder to acquire due to their low perceptual salience. The data presented indicated that there were no main effects in RT across conditions; however, accuracy results revealed that participants were more successful at matching high sonority signs with their nonobject representations than low sonority signs. This pattern of results may indicate that visual sonority is crucial during sign language learning. It is likely that learners attuned to signs that contain high sonority movements and were better able to encode their phonological features. Since previous studies have shown that the handshape parameter is often more difficult to acquire for second language learners (Morford & Carlson, 2011) and handshape markedness differentially affects processing (Grosvald et al., 2012), the question of whether the effects of markedness are diminished (or highlighted) with greater visual sonority remained.

It was also found that signs were learned more easily when marked handshapes were embedded in high sonority signs. This advantage was not seen for unmarked handshapes, where signs containing unmarked handshapes in high sonority signs (87%) were matched to those in low sonority signs (86%). The salient feature of a marked handshape likely drew attention to the handshape parameter. Attention directed to a marked handshape was then highlighted by the high visual saliency of the sign to provide distinct features to encode the sign representation (and its phonetic parameters). This interaction between sonority and markedness was further demonstrated by a second comparison of unmarked high sonority and marked low sonority signs, which revealed that participants matched unmarked high sonority signs with their nonobject better than marked low sonority signs. Despite the additive facilitation by high sonority when learning signs with marked handshape, this pattern of results indicates detrimental effects of both low sonority and marked handshapes. These results can be explained by the fact that learners are burdened by marked handshapes in a low visually distinctive signal (i.e., low sonority); whereas, learning is facilitated by high sonority and is not negated by marked handshapes. This supports previous research that shows that learners have difficulty with handshape identification, discrimination, and learning (Bochner et al., 2011; Morford & Carlson, 2011; Morford et al., 2008), but do better with unmarked handshapes (Grosvald et al., 2012). Thus, possibly the ideal combination of visual features that aids in learning seems to arise when the handshape parameter is marked and the sign movement is highlighted with high sonority.

Experiment 2: Sign Reproduction

The pattern of results from Experiment 1 showed that visual sonority impacts sign language acquisition. While Experiment 1 addressed perceptual learning, there is still the question of whether visual sonority impacts the production of familiar and novel signs in hearing nonsigners. Here, a reproduction paradigm was used, which allowed for the tracking of RT for sign language production. These RT provide a psycholinguistic account of the role of sonority and handshape markedness on the assembly of motor programs to initiate sign production. Additionally, production accuracy can reveal the phonetic specificity of the underlying sign representation. It was hypothesized that low sonority signs and signs that contain marked handshapes will be slower to produce and contain more errors due to their increased complexity in motor programming. Additionally, since already-learned signs are going to be tested, if the phonological representation of previous learned signs are underspecified due to their sonority or markedness (as seen in Experiment 1 results), we would expect reproduction of low sonority signs that contain a marked handshape to be more prone to error. As such, it was predicted that motoric complexity as well as underspecified representations during learning would produce slower and more erroneous sign productions.

Methods

Participants

Twenty-three of the same participants from Experiment 1 participated in this experiment immediately following Experiment 1. Two were omitted from the following analyses due to technical difficulties in video recording responses.

Materials

Thirty-two signs were included in this experiment. Sixteen of the familiar signs from Experiment 1 were included. Additionally 16 novel signs were included. The additional novel signs were delineated by high and low sonority and unmarked and marked constraints, similar to those in Experiment 1. The novel signs and familiar signs were not systemically different in any way.

Procedure

The procedure outlined here is a paradigm that records RT in sign production. Older sign language production studies have captured RT by laser beam triggering (Corina & Hildebrandt, 2002) or motion capture (Lupton & Zelaznik, 1990). In this study, we used PsychoPy (Pierce, 2007) in order to capture button releases before sign production, which is similar to what more recent production studies have implemented (see Emmorey, Petrich, & Gollan, 2012; Secora & Emmorey, 2015). At the beginning of every trial, the participants saw a prompt to place their dominant signing hand on the space bar. Once the space bar was held down for 1s, the video would begin to play. After the video finished, a prompt appeared and participants were provided a 3,000-ms period to make their productions. The next trial did not begin until the participant pressed down the space bar (see Figure 5 for design). Although a prompt was given after the video played, participants were instructed that they could produce the sign as soon as they knew how to produce it. In order to control for participants who might lift their hands immediately and then delay their sign production, participants were additionally instructed not to lift their hands to sign without immediately producing the sign (i.e., “only lift your hands when you are completely ready to sign; do not lift your hands if you have to think about how to produce the sign”). Reaction times were calculated at the offset of the sign video. Thus, negative RT indicate production during the sign video and positive RT indicate production initiated after the video had finished (during the sign production period). In other words, negative RT are faster than positive RT. All participants were video recorded to capture sign productions for accuracy measures. Signs were given a binary accuracy score by two judges, a native signer and a proficient L2 signer (first author). A sign production was deemed accurate if the sign was produced exactly as shown by the sign model (barring any fine phonetic variation; e.g., greater finger flexion, location differences within a couple of centimeters, etc.). This means that the participants were required to produce the target signs with no handshape, movement, or location substitutions or distortions. If judgments differed, a 100% consensus on accuracy scores was reached after discussion between the two judges.

Figure 5.

Figure 5.

The design of the reproduction study. Participants were shown a sign video after holding down the space bar to begin. The participants could lift their hand and sign any time but must be within a 3-s timeout after video offset. Reaction times (RT) were recorded relative to the video offset.

Data Analysis

A similar analysis was performed as in Experiment 1; however, an additional fixed effect of familiarity was added to the model in order to investigate how learners differ in their reproductions of familiar and novel signs.

Results

RT were filtered for trials where the subject lifted the hands but did not produce the sign immediately and for those that fell 2 SDs above or below the mean (2.3%). Filtered RT from only correct trials were analyzed using the linear mixed-effect model. A significant main effect of sonority was found at the group level (F1(1,713) = 6.984, p = .008; F2(1,32) = 0.847, p = .364] such that high sonority signs (−265 [61] ms) were produced more quickly than low sonority signs (−233 [60] ms). There was no main effect of markedness (F1(1,713) = 0.019, p = .890; F2(1,32) = 0.002, p = .969) insofar as both unmarked (−259 [67] ms) and marked (−239 [65] ms) signs were produced equally as fast. However, there was a highly significant effect of familiarity (F1(1,713) = 1407.624, p < .0001; F2(1,32) = 170.778, p < .0001), where familiar signs (−688 [52] ms) were produced more quickly than novel signs (+170 [69] ms). There was no significant interaction between sonority and markedness (F1(1,713) = 1.312, p = .251; F2(1,32) = 0.160, p = .692]. There was a significant interaction between sonority and familiarity (F1(1,713) = 5.221, p = .023; F2(1,32) = 0.633, p = .432). There was no interaction between markedness and familiarity or a three-way interaction (Fs < 1) (Figures 6 and 7; Table 2).

Figure 6.

Figure 6.

The mean reproduction times collapsed across familiar and novel signs in milliseconds relative to stimulus offset. Thus, the more negative the reaction time the faster the sign was produced (i.e., before stimulus offset), whereas the more positive the reaction time the slower the sign was produced (i.e., poststimulus offset). Error bars represent ± 1 SE.

Figure 7.

Figure 7.

The mean reaction times (in milliseconds) for low and high sonority signs grouped by whether or not they were familiar (in Experiment 1) or novel. Reaction times are relative to the stimulus offset. Error bars represent ± 1 SE.

Table 2.

Statistics for Experiment 2 (reproduction)

Predictor Condition Mean SE F1 | F2 p1 | p2
Sonority Markedness
Reaction times
Sonoritya High −265 61 6.984 | 0.847 .008 | .364
Low −233 60
Markedness Unmarked −259 67 0.019 | 0.002 .890 | .969
Marked −239 65
Sonority × markedness High Unmarked −298 56 1.312 | 0.160 .251 | .692
Marked −232 68
Low Unmarked −220 61
Marked −247 62
Accuracy
Sonoritya High 81.3% 2.3 60.734 | 15.352 <.001 | <.001
Low 56.5% 2.3
Markednessa Unmarked 85.1% 1.6 104.744 | 26.476 <.001 | <.001
Marked 52.7% 3.2
Sonority × markednessa High Unmarked 93.5% 1.9 6.327 | 1.625 .011 | .212
Marked 69.0% 4.3
Low Unmarked 76.8% 3.1
Marked 36.3% 3.0

Note. F1 = group, F2 = item.

aSignificant.

Planned t tests were performed in order to tease apart the interaction effects. In regards to the familiarity effect, it is important to know if there was a high sonority advantage for only familiar signs compared to novel signs. There was a significant effect of sonority for the familiar signs (t(22) = 4.127, p < .001) such that high sonority signs (−709 [52] ms) were produced much faster than low sonority signs (−627 [54] ms). On the other hand, there was no sonority advantage in the reproduction of novel signs (high = 179 [71] ms, low = 160 [67] ms; t < 1). There was also a difference between familiar and unfamiliar signs for both high (t(22) = 35.825, p < .0001) and low sonority (t(22) = 35.242, p < .0001), where familiar signs were produced faster than unfamiliar signs (Figure 8).

Figure 8.

Figure 8.

The mean accuracy in production for signs collapsed across familiarity but separated between sonority and markedness.

Accuracy rates were also analyzed using a mixed-effects model, which revealed a main effect of sonority at both levels (F1(1,713) = 60.734, p < .001; F2(1,32) = 15.352, p < .001) such that high sonority signs (81.3% [2.3]) were produced more accurately than low sonority signs (56.5% [2.3]). There was a main effect of markedness at both levels (F1(1,713) = 104.744, p < .001; F2(1,32) = 26.476, p < .001) such that unmarked (85.1% [1.6]) were more accurately reproduced than marked (52.7% [3.2]) signs. There was no main effect of familiarity (F1(1,713) = 1.067, p = .302; F2(1,32) = 0.270, p = .607), where familiar signs (67.3% [2.3]) were produced relatively as accurately as the novel signs (70.5% [2.6]). There was a significant interaction between sonority and markedness at the group level (F1(1,713) = 6.327, p = .011; F2(1,32) = 1.625, p = .212) such that unmarked high sonority signs were qualitatively reproduced the most accurately (93.5% [1.9]) with unmarked low sonority signs (76.8% [3.1]), marked high sonority signs (69.0% [4.3]), and marked low sonority signs (36.3% [3.0]) being less accurate. There was a trending interaction of sonority and familiarity (F1(1,713) = 3.183, p = .075; F2(1,32) = 0.804, p = .376), due to a larger difference for the reproduction of high sonority signs compared to low sonority signs for familiar relative to novel signs. There were no other significant interactions (Fs < 1).

To further tease apart the interaction between sonority and markedness, planned t tests were performed. Results indicated that all interactions were significant. First, there was a difference between high sonority and low sonority for unmarked handshapes (t(22) = 4.183, p < .001) such that high sonority signs (93.4%) were more accurate than low sonority (76.8%) signs when they contained an unmarked handshape. The same was true when they contained a marked handshape (high: 69.1%, low: 36.3%; t(22) = 8.830, p < .001). Second, high sonority signs that contained unmarked handshapes (93.5%) were produced more accurately than marked (69.1%) handshapes (t(22) = 5.125 p < .001). The same held true for low sonority signs (unmarked: 76.7%, marked: 36.3%; t(22) = 9.805, p < .001).

Discussion

In Experiment 2, the roles of sonority and handshape markedness on the reproduction of familiar and novel signs by hearing nonsigners were investigated. It was hypothesized that both sonority and markedness would influence the reproduction of the signs insofar as signs that contained high sonority movements and unmarked handshapes would be easier to produce due to their motoric complexity. By testing the reproduction of familiar signs and the reproduction of novel signs, we were able to test how language experience might influence the role of sonority. Results revealed an effect such that high sonority signs were reproduced faster than those with low sonority, especially when the signs were familiar. This effect was heightened when the signs contained an unmarked handshape. Markedness significantly affected production accuracy. Interactions between handshape markedness and sonority suggest that these two interact in a coordinated way. Accuracy data revealed that motoric complexity may also impact sign language production such that marked and low sonority signs were less accurately produced than unmarked and high sonority signs, respectively. In the General Discussion section, the implications of both results and how this is important to second language acquisition and sign language processing more generally are discussed.

General Discussion

The goal of the current study was to investigate the role of sonority and handshape markedness on the perception and reproduction of signs. In Experiment 1, it was found that nonsigners were more accurate at matching signs that contained high sonority movements to their nonobject representations than those with low sonority movements. Additionally, the increased accuracy due to sonority was differentially modulated by handshape markedness. Another important finding is the role of sonority and markedness on sign reproduction. It was found that learners tended to produce signs with high sonority movements faster and more accurately than those with low sonority movements. As seen previously, handshape markedness affected reproduction. The results additionally showed that sonority effects were only apparent for the reproduction of familiar signs.

Sonority, or the perceptual salience of a linguistic unit, has been shown to influence spoken language acquisition and processing (Broselow & Finer, 1991; Eckman & Iverson, 1993; Gierut, 1999; Ohala, 1999; Tropf, 1987; Yavas & Gogate, 1999). The influence of sonority has also been found to be cross-modal (Brentari, 2002). While there has been a great deal of research investigating the impact of sonority on spoken language, there are a number of outstanding questions regarding the relationship between sonority and sign language learning and processing. In the present study, a phonetic account of sign language sonority was adopted and the characteristic of the movement parameter was varied to investigate its impact on acquisition and recall. It was hypothesized here that greater visual salience (i.e., high sonority) would facilitate acquisition by providing salient cues that would aid in sign-picture mapping in a learning paradigm. Indeed, this was the case. Learners were better at acquiring sign-nonobject mappings of novel signs that contained high sonority movements compared to those that contained lower sonority movements. Concomitant increased accuracy as a function of greater sonority suggests that learners are attuned to the most salient features in the input and exploit these features during learning.

Similarly, sonority demonstrated a facilitative role in the reproduction of familiar signs. When learners were asked to produce signs that they had just learned, learners showed faster RT in the reproduction of familiar signs that contained high sonority movements compared to those that contained low sonority movements. There are at least two possible explanations. First, learners may have encoded the movement features better due to increased attention to salient cues. This would provide for greater feature specificity and learning which would facilitate subsequent sign production. A second possibility is that sonority and motoric complexity are highly correlated such that signs with high sonority movements are less motorically complex. Examining sonority alone may not be able to disambiguate the impact of motor complexity and sonority. However, handshape markedness may allow us to disentangle visual and motoric complexity.

Sonority was not the only perceptual factor that influenced sign acquisition. Handshape markedness provided additional visual and motoric complexity, which interacted with sonority during sign acquisition. The results presented show an interaction between sonority and markedness with high sonority movements and unmarked handshapes being easier to process than signs with low sonority movements and marked handshapes. From Experiment 1 when sonority is low and the handshape is marked, there is greater confusion, which was largely supported by an ad-hoc confusion matrix analysis. This perceptual hierarchy is similar to Storkel’s (2006) saliency ratio. Storkel found that novel words were better acquired if the novel words contained sound sequences that were not already in the child’s phonological repertoire, because uncommon sounds are more salient than common sounds and they facilitate child language acquisition. Storkel’s saliency ratio points to a more general process for contrastive abstraction that the human learner uses to acquire various types of knowledge, such as language. In other words, salient features, like uncommon sounds or sonority, are only salient when they are compared to the distribution of other features. When there are varying degrees of saliency, then the learner can pick out those features that are most salient. This general process underlies statistical word learning in children (Yu & Smith, 2007) and adult L2 learning (Laufer & Girsai, 2008).

This saliency ratio can be reconceptualized in terms of movement sonority and handshape markedness by including results from this study as well as other studies (e.g., Grosvald et al., 2012). For sign learners, when the full distribution of salient features is present (i.e., high vs. low sonority; marked vs. unmarked handshapes), high sonority signs allow for movement features 1 to pop out in the input since high sonority requires more space to be used (e.g., path movements) and larger articulatory gestures (e.g., signs articulated with the should joint) relative to low sonority signs. Increased handshape markedness may additively contribute to perception when contrasted with unmarked handshapes. However, with decreased saliency, marked forms create greater confusability for learners. It may be that this saliency ratio changes with proficiency. Just as phonotactic probabilities highlight novelty and then allow for easier learning of uncommon sounds/lexical items during language development, as a learner becomes more proficient the marked handshapes may become more salient and easier to learn. This may explain why hearing learners do better on unmarked handshapes whereas deaf signers do better with marked handshapes (i.e., marked handshapes pop out in the input for deaf signers but are confusing at low levels of sonority for hearing sign learners).

The interaction between handshape markedness and sonority was also seen in the reproduction study. The reproduction results mirror the findings in Experiment 1, showing an interaction between sonority and handshape markedness. This suggests that visual characteristics not only influence the perception of signs, but also their reproduction. Given no difference in novel signs in their production as a function of sonority, we can say that sonority is unlikely to be treated as motoric in nature; rather, it is likely that sonority provides visual salience that allows for enhanced encoding during learning, which aids in faster recall during production. There was a notable difference insofar as sonority improved reproduction for only signs with unmarked handshape, whereas marked handshapes in high sonority signs improved learning. During perception, there is greater reliance on markedness as a perceptual constraint, while it acts more like a motoric constraint during production. Thus, it may be the case that the visual salience helps with identification and perceptual encoding, but motoric complexity is the important feature for production. Although the acquisition study alone cannot distinguish between visual and motoric influences, it seems that the difference between the two is driven by the fact that motoric complexity is higher for marked signs (Ann, 2006; Boyes-Braem, 1990), which might decrease motor assembly and execution rates. Therefore, we can posit that the visual salience of marked handshapes is beneficial during learning, but handshape markedness is detrimental during production. These findings support some previous findings that show handshape markedness may pose challenges in adult L2 sign production (Rosen, 2004).

The combined effects of sonority and handshape in the present study, for both learning and reproduction, support a theory of additivity in visual processing. Given that learners must attend to cues in order to encode (and reproduce) signs, it may be advantageous to attend to multiple cues that maximize encoding. Just as combined auditory and visual information expedite processing (as well as many other examples of multiple cues being advantageous to cognitive processing), both sonority and handshape markedness directly (and additively) impact encoding. As such, it is important for studies to start to move away from sign language processing at the single parameter level, and begin also investigating how phonetic salience and co-occurrences of cues may impact learning and processing. This will allow our field to move toward a grander, unified theory of sign language acquisition and processing.

It is important to mention that there may be a possible confound in the present study—a correlation between sonority and number of active articulators (i.e., one-handed versus two-handed). Many of the high sonority signs were also two-handed signs. As such, it may be the case that seeing two hands as active articulators was the driving force behind improved encoding. However, this does not necessarily impact the present results insofar as increased visual salience due to two hands falls in line with our theory of sign language processing. Additionally, sign language theories have rather neglected the use of handedness in their respective sonority hierarchies. As such, further theoretical investigation is necessary to better specify these theories and de-correlate this interaction.

Additionally, there may be an alternative explanation for the present results. Given that there are various factors that were uncontrolled in the stimulus set (e.g., number of hands, body contact, etc.), learners might have followed a strategy to minimize effort by noting the most important distinguishing characteristics needed to succeed in learning, and not the salient phonetic features that were being tested. However, this is highly unlikely given that the learner would have to remember a large number of different features that are unique to each sign, which would tax the memory system and likely reduce overall accuracy and speed. Yet, overall accuracy was pretty high. The confusion matrix also provides strong evidence against this alternative explanation. If learners were able to provide one contrastive feature (e.g., location), then when two signs shared that feature (e.g., high school and furniture), there would be a high confusion rate. As we see, this is not the case. In fact, inspection of those signs that were confused with one another revealed that often there were various differences between them. The sign pairs that were most confused did not have a common overlapping feature; instead each pair had a different feature in common. For example, puzzled and ask share handshape (i.e., F), high school and join share major location (i.e., neutral space), and communication and puzzled share no common feature. Furthermore, there is no confusion between two signs that share location (communication and vomit) or handshape (audiology and sorry). As such, the confusion matrix seems to rule out any common phonetic feature that learners were attuning to which could explain our results. Nevertheless, more experimental investigations are needed in order to advance our knowledge on the impact of salient features in sign acquisition.

The results of the present study may also inform the overarching theories of sign language phonology. Given the phonetic account of sonority (based on articulating joint and movement characteristics; Brentari, 1993, 1998; Sandler, 1993), it may be the case that naive learners attune to movement features more specifically. In other words, naive learners may be attuned to the phonetic correlates of sonority. Although we cannot rule out the possibility that learners are also sensitive to typological or allowable patterns, it seems to be the case that learners can still use phonetic (and not necessarily phonological) cues when learning, especially given that they are unaware of the frequency distributions or phonotactics of sign language (cf. Corina, 1996). Again, more research is needed to further explore the cues used during the initial stages of acquisition.

Not only do the results from the present study inform theories of sign phonology, but also theories on lexical access in sign language. Gating studies have demonstrated that access to phonetic-phonological information occurs early within the sign, with lexical access occurring before the entire sign in completed—often within the first 300ms (Emmorey & Corina, 1990; Morford & Carlson, 2011). In the present study, familiar signs were produced before the end of the target stimulus, while novel signs were produced after the target was finished. This finding supports theories that sufficient phonological information is stored within the first portion of the sign, which triggers recognition, but the learner must wait until the entire sign is produced before reproduction when there is no lexical representation to recall. Although this was a tangential finding, we argue that this finding extends previous research on lexical access in recognition to production.

Beyond theoretical insights, the present study may also have a practical impact. The results from the present study lend themselves to future research on second language acquisition and pedagogy. Previous studies have shown that movement is harder to acquire than other sublexical features (Bochner et al., 2011). However, this study has shown that signs are accurately acquired (~91%) if they contain high sonority movements. Thus, if high sonority signs can be taught first, learners may be able to encode movement characteristics much easier. The encoding of movement features may help with the acquisition of certain signs. Moreover, this may have positive effects downstream in terms of production development. Along the same vein, late L2 learners master handshape features much later (Bochner et al., 2011; Morford & Carlson, 2011). Therefore, if marked handshapes are paired with high sonority movements, learners may acquire the phonetic characteristics earlier. The findings presented herein provide new ways to conceptualize the teaching of lexical items based on phonological characteristics. Furthermore, this study demonstrates the need to control for sonority and handshape markedness in future sign language perception research.

Funding

National Science Foundation (NSF) Graduate Research Fellowship #1342962 (J. T. Williams).

Conflicts of Interest

No conflicts of interest were reported.

Acknowledgments

A special thanks is dedicated to Edwin Rivera for his work on recording the stimuli and Shelby Farmer for her diligent work on collecting the data.

Appendix

Figure A1.

Figure A1.

Depictions of the handshapes used in this study.

Note

1. Movement features are likely not the only features that are salient in high sonority signs. For instance, path movements inherently require location changes. Given the location is often very salient in L2 acquisition, it could be the case that high sonority signs also allow location features to pop out. However, given the design of the present study, we are unable to directly address this theoretical point.

References

  1. Ambadar Z. Schooler J. W., & Cohn J. F (2005). Deciphering the enigmatic face: The importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16, 403–410. doi:10.1111/j.0956-7976.2005.01548.x [DOI] [PubMed] [Google Scholar]
  2. Ann J. (2006). Frequency of occurrence and ease of articulation of sign language handshapes. Washington, D.C: Gallaudet University Press. [Google Scholar]
  3. Baayen R. H. Davidson D. J., & Bates D. M (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412. doi:10.1016/j.jml.2007.12.005 [Google Scholar]
  4. Bates D. Maechler M. Bolker B., & Walker S (2013). lme4: Linear mixed-effects models using Eigen and S4. R Package Version, 1. doi:10.18637/jss.v067.i01 [Google Scholar]
  5. Battison R. (1978). Lexical borrowing in American Sign Language. Silver Spring, MD: Linstok Press. [Google Scholar]
  6. Berent I. Dupuis A., & Brentari D (2013). Amodal aspects of linguistic design. PLoS One, 8, e60617. doi:10.1371/journal.pone.0060617 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Berent I. Harder K., & Lennertz T (2011). Phonological universals in early childhood: Evidence from sonority restrictions. Language Acquisition, 18, 281–293. doi:10.1080/10489223.2011.580676 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Best C. T., & Tyler M. D (2007). Nonnative and second-language speech perception: Commonalities and complementarities. In Bohn O. S., Munro M. J. (Eds.), Language experience in second language speech learning: In honor of James Emil Flege (pp. 13–34). Philadelphia, PA: John Benjamins Publishing. [Google Scholar]
  9. Birdsong D. (1992). Ultimate attainment in second language acquisition. Language, 68, 706–755. doi:10.1353/lan.1992.0035 [Google Scholar]
  10. Blevins J. (1995). The syllable in phonological theory. In Goldsmith J. A. (Ed.), The handbook of phonological theory (pp. 206–244). Cambridge, MA: Blackwell. [Google Scholar]
  11. Bochner J. H. Christie K. Hauser P. C., & Searls J. M (2011). When is a difference really different? Learners’ discrimination of linguistic contrasts in American Sign Language. Language Learning, 61, 1302–1327. doi:10.1111/j.1467-9922.2011.00671.x [Google Scholar]
  12. Bosworth R. G., & Emmorey K (2010). Effects of iconicity and semantic relatedness on lexical access in American Sign Language. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 1573–1581. doi:10.1037/a0020934 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Boyes-Braem P. B. (1990). Acquisition of the handshape in American Sign Language: A preliminary analysis. In V. Volterra & C. Erting (Eds.), From gesture to language in hearing and deaf children (pp. 107–127). Berlin, Germany: Springer. doi:10.1007/978-3-642-74859-2_10. [Google Scholar]
  14. Brentari D. (1993). Establishing a sonority hierarchy in American Sign Language: The use of simultaneous structure in phonology. Phonology, 10, 281–306. doi:10.1017/S0952675700000063 [Google Scholar]
  15. Brentari D. (1998). A prosodic model of sign language phonology. Cambridge, MA: MIT Press. [Google Scholar]
  16. Brentari D. (2002). Modality differences in sign language phonology and morphophonemics. In R. Meier, K. Cormier, & D. Quinto-Pozos (Eds.), Modality and structure in signed and spoken languages (pp. 35–64). Cambridge, UK: Cambridge University Press. [Google Scholar]
  17. Brentari D. Nadolske M. A., & Wolford G (2012). Can experience with co-speech gesture influence the prosody of a sign language? Sign language prosodic cues in bimodal bilinguals. Bilingualism: Language and Cognition, 15, 402–412. doi:10.1017/S1366728911000587 [Google Scholar]
  18. Broselow E., & Finer D (1991). Parameter setting in second language phonology and syntax. Second Language Research, 7, 35–59. doi:10.1177/026765839100700102 [Google Scholar]
  19. Cairns H. S. (1996). The acquisition of language (2nd Ed.) Austin, TX: Pro-Ed. [Google Scholar]
  20. Chen Pichler D. (2009). Sign production in first-time hearing signers: A closer look at handshape accuracy. Cadernos de Saúde, Número especial, Línguas gestuais, 2, 37–50. [Google Scholar]
  21. Chen Pichler D., & Koulidobrova E (2015). Acquisition of sign language as a second language. In Marschark M., Spencer P. (Eds.), The Oxford handbook of deaf studies in language: Research, policy, and practice (pp. 218–230). Cambridge, UK: Oxford University Press. [Google Scholar]
  22. Chin S. B. (1996). The role of the sonority hierarchy in delayed phonological systems. In Powell T. W. (Ed.), Pathologies of speech and language: Contributions of clinical phonetics and linguistics (pp. 109–117). New Orleans, LA: International Clinical Phonetics and Linguistics Association. [Google Scholar]
  23. Clements G. N. (1990). The role of the sonority cycle in core syllabification. In Kingston J., Beckman M. E. (Eds.), Papers in laboratory phonology I: Between the grammar and physics of speech (pp. 283–333). Cambridge, UK: Cambridge University Press. [Google Scholar]
  24. Crasborn O. (2001). Phonetic implementation of phonological categories in sign language of the Netherlands (Doctoral dissertation). Netherlands Graduate School of Linguistics, Utrecht, the Netherlands. [Google Scholar]
  25. Corina D. P. (1996). ASL syllables and prosodic constraints. Lingua, 98, 73–102. doi:10.1016/0024-3841(95)00033-X [Google Scholar]
  26. Corina D. P., & Hildebrandt U. C (2002). Psycholinguistic investigations of phonological structure in ASL. In Meier R. P., Cormier K., Quinto-Pozos D. (Eds.), Modality and structure in signed and spoken languages (pp. 88–111). Cambridge, UK: Cambridge University Press. [Google Scholar]
  27. Du Y. He Y. Ross B. Bardouille T. Wu X. Li L., & Alain C (2011). Human auditory cortex activity shows additive effects of spectral and spatial cues during speech segregation. Cerebral Cortex, 21, 698–707. doi:10.1093/cercor/bhq136 [DOI] [PubMed] [Google Scholar]
  28. Eckman F. R., & Iverson G. K (1993). Sonority and markedness among onset clusters in the interlanguage of ESL learners. Second Language Research, 9, 234–252. doi: 10.1177/026765839300900302 [Google Scholar]
  29. Emmorey K. (2001). Language, cognition, and the brain: Insights from sign language research. Mahwah, NJ: Lawrence Erlbaum Associates. [Google Scholar]
  30. Emmorey K., & Corina D (1990). Lexical recognition in sign language: Effects of phonetic structure and morphology. Perceptual and Motor Skills, 71, 1227–1252. doi:10.2466/pms.1990.71.3f.1227 [DOI] [PubMed] [Google Scholar]
  31. Emmorey K. McCullough S., & Brentari D (2003). Categorical perception in American Sign Language. Language and Cognitive Processes, 18, 21–45. doi:10.1080/01690960143000416 [Google Scholar]
  32. Emmorey K. Petrich J., & Gollan T. H (2012). Bilingual processing of ASL-English code-blends: The consequences of accessing two lexical representations simultaneously. Journal of Memory and Language, 67, 199–210. doi:10.1016/j.jml.2012.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Escudero P. Hayes-Harb R., & Mitterer H (2008). Novel second-language words and asymmetric lexical access. Journal of Phonetics, 36, 345–360. doi:10.1016/j.wocn.2007.11.002 [Google Scholar]
  34. Field J. (2003). Promoting perception: Lexical segmentation in L2 listening. ELT Journal, 57, 325–334. doi:10.1093/elt/57.4.325 [Google Scholar]
  35. Gass S. M., & Selinker L (1992). Language transfer in language learning: Revised edition (Vol. 5). Philadelphia, PA: John Benjamins Publishing. [Google Scholar]
  36. Gierut J. A. (1999). Syllable onsets clusters and adjuncts in acquisition. Journal of Speech, Language, and Hearing Research, 42, 708–726. doi:10.1044/jslhr.4203.708 [DOI] [PubMed] [Google Scholar]
  37. Goldschneider J. M., & DeKeyser R. M (2001). Explaining the natural order of L2 morpheme acquisition in English: A meta-analysis of multiple determinants. Language Learning, 51, 1–50. doi:10.1111/1467–9922.00147 [Google Scholar]
  38. Grosvald M. Lachaud C., & Corina D (2012). Handshape monitoring: Evaluation of linguistic and perceptual factors in the processing of American Sign Language. Language and Cognitive Processes, 27, 117–141. doi:10.1080/01690965.2010.549667 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Hankamer J., & Aissen J (1974). The sonority hierarchy. In Bruck A. et al. (Eds.), Papers from the Parasession on natural phonology. Chicago, IL: Chicago Linguistic Society. [Google Scholar]
  40. Itti L., & Koch C (1999). Comparison of feature combination strategies for saliency-based visual attention systems. Human Vision and Electronic Imaging IV, 3644, 473–482. doi:10.1117/12.348467. [Google Scholar]
  41. Jaeger T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59, 434–446. doi:10.1016/j.jml.2007.11.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Jakobson R. (1968). Child language: Aphasia and phonological universals (No. 72). Hague, the Netherlands: Walter de Gruyter. [Google Scholar]
  43. Kroll J. F., & Potter M. C (1984). Recognizing words, pictures, and concepts: A comparison of lexical, object, and reality decisions. Journal of Verbal Learning and Verbal Behavior, 23, 39–66. doi:10.1016/S0022-5371(84)90499-7 [Google Scholar]
  44. Laufer B., & Girsai N (2008). Form-focused instruction in second language vocabulary learning: A case for contrastive analysis and translation. Applied Linguistics, 29, 694–716. doi:10.1093/applin/amn018 [Google Scholar]
  45. Liddell S. K., & Johnson R. E (1989). American Sign Language: The phonological base. Sign Language Studies, 64, 195–277. doi:10.1353/sls.1989.0027 [Google Scholar]
  46. Lupton L. K., & Zelaznik H. N (1990). Motor learning in sign language students. Sign Language Studies, 67, 153–174. doi:10.1353/sls.1990.0020 [Google Scholar]
  47. Meier R. (2006). The form of early signs: Explaining signing children’s articulatory development. In Schick B., Marschark M., Spencer P. E. (Eds.), Advances in sign language development by deaf children (pp. 202–230). New York, NY: Oxford University. [Google Scholar]
  48. Meier R. Mauk C. Cheek A., & Moreland C (2008). The form of children’s early signs: Iconic or motoric determinants? Language Learning and Development, 4, 393–405. doi:10.1080/15475440701377618 [Google Scholar]
  49. Meier R. P. (1982). Icons, analogues, and morphemes: The acquisition of verb agreement in American Sign Language (Doctoral dissertation). University of California, San Diego, CA. [Google Scholar]
  50. Morford J. P., & Carlson M. L (2011). Sign perception and recognition in non-native signers of ASL. Language Learning and Development, 7, 149–168. doi:10.1080/15475441.2011.543393 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Morford J. P. Grieve-Smith A. B. MacFarlane J. Staley J., & Waters G (2008). Effects of language experience on the perception of American Sign Language. Cognition, 109, 41–53. doi:10.1016/j.cognition.2008.07.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Ohala D. K. (1999). The influence of sonority on children’s cluster reductions. Journal of Communication Disorders, 32, 397–422. [DOI] [PubMed] [Google Scholar]
  53. Ohala J. J., & Kawasaki H (1984). Prosodic phonology and phonetics. Phonology, 1, 113–127. doi:10.1016/S0021-9924(99)00018-0 [Google Scholar]
  54. Oldfield R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9, 97–113. doi:10.1016/0028-3932(71)90067-4 [DOI] [PubMed] [Google Scholar]
  55. Orlansky M. D., & Bonvillian J. D (1984). The role of iconicity in early sign language acquisition. Journal of Speech and Hearing Disorders, 49, 287–292. doi:10.1044/jshd.4903.287 [DOI] [PubMed] [Google Scholar]
  56. Ormel E. Hermans D. Knoors H., & Verhoeven L (2009). The role of sign phonology and iconicity during sign processing: The case of deaf children. Journal of Deaf Studies and Deaf Education, 14, 436–448. doi:10.1093/deafed/enp021 [DOI] [PubMed] [Google Scholar]
  57. Pierce J. W. (2007). PsychoPy–Psychophysics software in Python. Journal of Neuroscience Methods, 162, 8–13. doi:10.1016/j.jneumeth.2006.11.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Perlmutter D. M. (1992). Sonority and syllable structure in American Sign Language. Linguistic Inquiry, 4, 407–442. doi:10.1016/b978-0-12-193270-1.50016-9 [Google Scholar]
  59. Rosen R. (2004). Beginning L2 production errors in ASL lexical phonology. Sign Language and Linguistics, 7, 31–61. doi:10.1075/sll.7.1.04beg [Google Scholar]
  60. Sander E. (1972). When are speech sounds learned? Journal of Speech and Hearing Disorders, 37, 55–63. doi:10.1044/jshd.3701.55 [DOI] [PubMed] [Google Scholar]
  61. Sandler W. (1989). Phonological representation of the sign: Linearity and nonlinearity in American Sign Language (Vol. 32). Dordrecht, the Netherlands: Walter de Gruyter. [Google Scholar]
  62. Sandler W. (1993). A sonority cycle in American Sign Language. Phonology, 10, 243–279. doi:10.1017/S0952675700000051 [Google Scholar]
  63. Schröger E., & Widmann A (1998). Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology, 35, 755–759. doi:10.1111/1469–8986.3560755 [PubMed] [Google Scholar]
  64. Schwartz R. G., & Leonard L. B (1982). Do children pick and choose? An examination of phonological selection and avoidance in early lexical acquisition. Journal of Child Language, 9, 319–336. doi:10.1017/S0305000900004748 [DOI] [PubMed] [Google Scholar]
  65. Secora K., & Emmorey K (2015). The action-sentence compatibility effect in ASL: The role of semantics vs. perception. Language and Cognition, 7, 305–318. doi:10.1017/langcog.2014.40 [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Showalter C. E., & Hayes-Harb R (2013). Unfamiliar orthographic information and second language word learning: A novel lexicon study. Second Language Research, 29, 185–200. doi:10.1177/0267658313480154 [Google Scholar]
  67. Showalter C. E., & Hayes-Harb R (2015). Native English speakers learning Arabic: The influence of novel orthographic information on second language phonological acquisition. Applied Psycholinguistics, 36, 23–42. doi:10.1017/S0142716414000411 [Google Scholar]
  68. Siedlecki T., & Bonvillian J. D (1997). Young children’s acquisition of the handshape aspect of American Sign Language signs: Parental report findings. Applied Psycholinguistics, 18, 17–39. doi:10.1017/S0142716400009851 [Google Scholar]
  69. Storkel H. L. (2006). Do children still pick and choose? The relationship between phonological knowledge and lexical acquisition beyond 50 words. Clinical Linguistics & Phonetics, 20, 523–529. doi:10.1080/02699200500266349 [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Storkel H. L., & Adlof S. M (2009). The effect of semantic set size on word learning by preschool children. Journal of Speech, Language, and Hearing Research: JSLHR, 52, 306–320. doi:10.1044/1092-4388(2009/07-0175) [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Storkel H. L., & Lee S. Y (2011). The independent effects of phonotactic probability and neighborhood density on lexical acquisition by preschool children. Language and Cognitive Processes, 26, 191–211. doi:10.1080/01690961003787609 [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Thompson R. L. (2011). Iconicity in language processing and acquisition: What signed languages reveal. Language and Linguistics Compass, 5, 603–616. doi:10.1111/j.1749-818X.2011.00301.x [Google Scholar]
  73. Thompson R. L. Vinson D. P., & Vigliocco G (2009). The link between form and meaning in American Sign Language: Lexical processing effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 550–557. doi:10.1037/a0014547 [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. Tropf H. S. (1987). Sonority as a variability factor in second language phonology in sound patterns in second language acquisition. Studies on Language Acquisition (SOLA), 5, 173–191. [Google Scholar]
  75. van Wassenhove V. Grant K. W., & Poeppel D (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences of the United States of America, 102, 1181–1186. doi:10.1073/pnas. 0408949102 [DOI] [PMC free article] [PubMed] [Google Scholar]
  76. Wilbur R. B. (1993). Syllables and segments: Hold the movement and move the holds. Current Issues in ASL Phonology, 3, 135–168. doi:10.1016/b978-0-12-193270-1.50012-1 [Google Scholar]
  77. Yantis S., & Egeth H. E (1999). On the distinction between visual salience and stimulus-driven attentional capture. Journal of Experimental Psychology: Human Perception and Performance, 25, 661. doi:10.1037/0096-1523.25.3.661 [DOI] [PubMed] [Google Scholar]
  78. Yavas M. S., & Gogate L. J (1999). Phoneme awareness in children: A function of sonority. Journal of Psycholinguistic Research, 28, 245–260. doi:10.1023/A:1023254114696 [DOI] [PubMed] [Google Scholar]
  79. Yu C., & Smith L. B (2007). Rapid word learning under uncertainty via cross-situational statistics. Psychological Science, 18, 414–420. doi:10.1111/j.1467-9280.2007. 01915.x [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Deaf Studies and Deaf Education are provided here courtesy of Oxford University Press

RESOURCES