Analysis of the Visual Spatiotemporal Properties of American Sign Language

Abstract

Careful measurements of the temporal dynamics of speech have provided important insights into phonetic properties of spoken languages, which are important for understanding auditory perception. By contrast, analytic quantification of the visual properties of signed languages is still largely uncharted. Exposure to sign language is a unique experience that could shape and modify low-level visual processing for those who use it regularly (i.e., what we refer to as the Enhanced Exposure Hypothesis). The purpose of the current study was to characterize the visual spatiotemporal properties of American Sign Language (ASL) so that future studies can test the enhanced exposure hypothesis in signers, with the prediction that altered vision should be observed within, more so than outside, the range of properties found in ASL. Using an ultrasonic motion tracking system, we recorded the hand position in 3-dimensional space over time during sign language production of signs, sentences, and narratives. From these data, we calculated several metrics: hand position and eccentricity in space and hand motion speed. For individual signs, we also measured total distance traveled by the dominant hand and total duration of each sign. These metrics were found to fall within a selective range, suggesting that exposure to signs is a specific and unique visual experience, which might alter visual perceptual abilities in signers for visual information within the experienced range, even for non-language stimuli.

1. INTRODUCTION

Several lines of experimental evidence suggest that visual experience plays a role in shaping visual abilities during development (Kiorpes & Movshon, 2004). Generally, it is thought that human perceptual systems are most efficient at processing the signals that occur most frequently within the environment (Simoncelli & Olshausen, 2001). One of the best examples of this is found for the domain of orientation processing; animals raised in restrictive environments containing only horizontal or vertical contours have heightened sensitivity for orientations they experienced and poor sensitivity for those they did not experience (Blakemore & Cooper, 1970, 1971; Hirsch & Spinelli, 1970; Stryker, Sherk, Leventhal & Hirsch, 1978). The effects of restrictive visual experience are also seen in humans who had an astigmatism as children, which distorts visual input due to a corneal aberration. If this condition remains uncorrected, these children later develop meridional amblyopia, a condition of decreased visual sensitivity for orientations blurred by their astigmatism that originates in the neural visual pathway (Gwiazda, Mohindra, Brill & Held, 1985; Mitchell, Freeman, Millodot & Haegerstrom, 1973; Mitchell & Wilkinson, 1974). There is also suggestion that even typically-developing humans show anisotropies in sensitivity for orientations based on the frequencies of orientations in their environment. Specifically, cardinal orientations (vertical and horizontal) are more prevalent in natural scenes than are oblique orientations, as shown by Fourier analyses of natural scenes (Baddeley & Hancock, 1991; Coppola, Purves, Mccoy & Purves, 1998; Keil & Cristobal, 2000; Switkes, Mayer & Sloan, 1978; Van Der Schaaf & Van Hateren, 1996). This is offered to explain the well-known phenomenon in which humans have better sensitivity for cardinal orientations than for oblique orientations, referred to as the “oblique effect” (Appelle, 1972; Campbell, Kulikowski & Levinson, 1966; Mitchell, Freeman & Westheimer, 1967). Indeed, the cardinal bias measured with Fourier analysis is stronger for scenes of man-made or “carpentered” environments that contain structures and buildings than for naturalistic scenes of landscapes and bodies of water (Hansen & Essock, 2004; Keil & Cristobal, 2000; Torralba & Oliva, 2003). This difference has been suggested to explain why people who live in less carpentered environments, such as the Cree Indians who live in prairie regions, exhibit a smaller oblique effect than people who live in highly carpentered environments (Annis & Frost, 1973). Together, these results observed for orientation sensitivity suggest that the visual system is modified by, and tailors to, visual statistics within the environment.

In the current study, we consider the case of daily, enriched exposure to a visual-manual signed language for individuals who use it as their primary means of communication, with the notion that exposure to the unique statistical properties inherent in the sign language signal might similarly shape low-level visual sensitivity in individuals who use it regularly. Sign language comprehension requires detailed perceptual processing of motion, form, orientation and shape cues inherent in the hands and arms, as well as on the face, and enriched exposure to these cues could enhance signers’ perceptual abilities (reviewed in Emmorey, 2001). To illustrate, often very slight changes in a sign’s hand movement, while all other features such as handshape and location are held constant, can change meaning (for example, the signs, SERIOUS and MISS in ASL are very similar, both involving pointing with an index finger on the chin, with slightly different movement patterns). Likewise, subtle changes in only location can also confer large changes in meaning (e.g., APPLE vs. ONION, conveyed on the lower vs. upper cheek, respectively, with identical handshape, orientation, and movement). There are over 40 handshape variants in ASL, that require the observer to attend to fine differences in the configurations of the fingers to distinguish between them (Battison, 1978). Supporting the effects of experience with ASL, there are several studies showing that expert signers who have been signing since infancy (both deaf and hearing) exhibit altered and/or enhanced visual abilities for aspects of visual processing that might be important for sign language, such as categorical perception for facial expressions, visual motion perception, and face discrimination (Bavelier, Brozinsky, Tomann, Mitchell, Neville & Liu, 2001; Bavelier, Tomann, Hutton, Mitchell, Corina, Liu & Neville, 2000; Bosworth & Dobkins, 1999, 2002; Brozinsky & Bavelier, 2004; Emmorey, Klima & Hickok, 1998; Emmorey & Kosslyn, 1996; Emmorey, McCullough & Brentari, 2003; Mccullough & Emmorey, 1997; Mccullough, Emmorey & Sereno, 2005; Poizner, 1983).

Given that life-long experience with sign language alters visual processing, it is reasonable to predict that differences in visual processing between signers and non-signers might be greatest for visual stimulus properties that reflect the statistical range encountered in the perceived sign language signal. For example, visual processing might be altered only for the speeds of motion or the orientations that represent those most frequently occurring in sign language and not those outside this range. To investigate this hypothesis, however, the visual properties of sign language signal must be characterized, and surprisingly, despite a long history of evidence showing visual alterations in signers, this has yet to be fully done. We initially addressed this in a previous study, where we quantified the spatial frequency and orientation content of the articulators (hands and arms) during sign production by conducting Fourier analysis on a set of static photograph images of many signs (Bosworth, Bartlett & Dobkins, 2006; Bosworth, Wright, Bartlett, Corina & Dobkins, 2003). The results revealed differences between the sign images and two other image sets (faces and natural scenes), particularly for orientation properties. Specifically, sign images were found to contain more amplitude for vertical than for horizontal contours, while images of faces and natural landscape scenes showed an opposite pattern. This stimulus specificity of orientation content in signs predicts that, when tested in perceptual and/or imaging studies, signers (compared to non-signers) might show enhanced/altered visual sensitivity to vertical, but not horizontal, orientations. We refer to this prediction as the “Enhanced Exposure Hypothesis”.

In order to further document the visual image statistics of the sign language signal, in the current study we measured spatiotemporal properties, focusing on location and motion of the signing hands through space. To determine the ranges of these two properties, we used small ultrasonic position trackers placed on the hands to measure hand position in three-dimensional (3D) space over time from deaf signers who were fluent in ASL as they produced signed stimuli. Three sign types were analyzed: First, the signers produced 42 signs chosen to represent a diverse sample of lexical and phonological forms (embedded in a carrier phrase to provide a more natural context, compared to isolated words), 6 elicited sentences with various grammatical structures, and two spontaneous narratives. From the sampled position coordinates over time, we calculated retinal eccentricity, which is the average distance of the signer’s dominant hand from the viewer’s fixation, speed as the hand moves through 3D space, distance traveled by the hand for each sign, and duration of each sign. Across all signed stimuli, we report the means and distributions of these measures. This provides a corpus of image statistics that can be used in designing future visual processing studies to test the Enhanced Exposure Hypotheses in signers, with a particular emphasis on location and speed of visual stimuli, as these stimulus parameters can be easily manipulated in studies of visual processing. Like the prediction mentioned above for orientation, the Enhanced Exposure Hypothesis predicts that differences in visual processing between signers and non-signers will be greatest for speeds and locations that fall within the range encountered in the sign language signal.

In addition to providing image statistics that can be used to test the Enhanced Exposure Hypothesis, the spatiotemporal properties of sign language are interesting in their own right, similar to studies describing the temporal characteristics of spoken languages or across several signed languages (e.g., Bellugi & Fischer, 1972; Börstell, Hörberg & Östling, 2016; Grosjean, 1980; Klima, Bellugi, Fischer & Newkirk, 1979; Wilbur, 1999; Wilbur & Nolkn, 1986; and with regards to the temporal properties of fingerspelling, Jerde, Soechting & Flanders, 2003; Wilcox, 1992; Zakia & Haber, 1971). To this end, we explored a secondary and conceptual question about the spatiotemporal properties of signs, which is whether signers might modulate the timing of their hand/arm movements to maintain some degree of constancy in either the speed or the duration of signs (or a combination of both). Although not the main purpose of this paper, these data could speak to a highly debated topic of whether articulatory isochrony exists in languages, a term that refers to the concept that production (or perception) of language units occurs regular intervals in time (Klima et al., 1979; Pike, 1945; Tuller & Fowler, 1980), perhaps in order to accommodate perceptual ease for the viewer, and/or articulatory constraints (such as muscle contraction or respiratory rates).

2. METHOD

2.1. Apparatus and Stimuli

Hand position and movement of the hands in space were recorded for three deaf signers as they signed words, sentences and narratives. All three were right-handed, fluent in ASL, had been signing for approximately 20 years, and used ASL daily. Two signers (RB and DH) learned ASL in early childhood in school settings (with exposure at the age of 5 years) and one (VM) was a second-generation signer, who had deaf signing parents, and was exposed to ASL at birth. Hand position was measured using an InterSense 3-D motion measurement system at the Virtual Reality Laboratory at the University of California, Irvine. The 3 signers (RB, DH, and VM) wore thin, flexible, fingerless gloves with a small ultrasonic position tracker (a 1 inch cube) placed firmly on the back of each hand. These devices emitted ultrasonic signals at a rate of 15 Hertz, which were recorded remotely by a receiver placed on the ceiling above the signer. These signals provided the x (horizontal), y (vertical), and z (depth) position of the hands every 66.7 milliseconds, as the subject signed (see example in Figure 1). Signers were asked to stand under the sensor which was mounted on the ceiling and produce each signed stimulus item at natural pace. For the purpose of this paper, we analyzed data only from the right (dominant) hand of each subject, since one-handed signs are produced with only the dominant hand, and in two-handed signs, the dominant hand moves while the non-dominant hand remains either stationary or mirrors the dominant hand’s movement.

Figure 1. Example 2-D motion trajectory.

Position (x, y) of the right dominant hand for the ASL phrase, sign know easy (English translation: “To sign the word ‘know’ is easy”) is plotted. In this example, the target sign is know, represented by the solid line, while the carrier phrase is represented by the dashed line, with larger dashes used for sign and smaller dashes for easy. (The z dimension, not shown here, was also recorded.)

Signers were instructed to stand, keep feet positioned in the same spot, and sign naturally, while they made eye contact with the experimenter who was seated in front. For each signer, we collected hand position data for three different types of sign production: 1) Excised Signs. We used the same 42 signs used in our previous study looking at the spatial frequency and orientation content of signs (Bosworth et al., 2006), which were chosen with the goal of creating a diverse sample of lexical items that represent various common phonological features (see Appendix for the list of signs used.) Each signer read each word item from a printed sheet, one at a time, and produced the item to the experimenter at her own comfortable speed. Signers were instructed to reproduce each of the 42 signs embedded within a carrier phrase, “sign x easy”, where X represents the sign of interest (which we refer to as the “target” sign). The English translation of this sentence is “To sign the word “X” is easy to do.” The purpose of employing a carrier phrase was to make the production of that word more natural as well as to remove the ballistic movement that would otherwise be present at the onset and offset of a sign in isolation. The carrier signs, SIGN and EASY were chosen because of their distinctive movement patterns that allowed the target sign to be easily extracted (described below). For each phrase, the signer began and ended with her hands resting at her sides. Signers were asked to sign each carrier phrase three times. The purpose of this repetition was to calculate reliability in the signer’s reproductions of each target sign (See results). 2) Sentences. Six sentences were presented in ASL by the experimenter to the signer, who was instructed to repeat them back, and these sentences are listed in the Appendix. 3) Narratives. Signers responded to two prompts, first, “tell me a childhood memory” and, second, “describe how you celebrate a major holiday”. No attempt was made to restrict or coach the signer and the narratives were not transcribed. The research protocol observed the tenets of the Declaration of Helsinki and was approved by the Institutional Review Boards (IRB) of the University of California, San Diego.

2.2.1. Excising Target Signs for Analysis.

For the analysis of signs, we needed to remove the carrier phrase, leaving only the target signs for analysis, which was done with script written in Matlab as follows. First, the x, y, z position over time for each carrier phrase was plotted using MATLAB 3-D plotting tools (Matlab, 2015b). The Matlab script served to demarcate movement patterns that were consistent across all samples (within each signer) for the non-target signs (sign and easy) of the carrier phrase. The start of the carrier phrase was characterized by a large initial change in the vertical position of the hands, resulting from both hands rising from the resting position (i.e., signer’s hands at sides), followed by cyclic repetition in the vertical dimension, resulting from generating the word “sign”. Likewise, the end carrier phase was characterized by two rapid changes in vertical position, resulting from generating the word “easy”, followed by a large change in vertical position, resulting from the hands returning to their resting state (see Figure 1). After the carrier words were removed, the co-authors independently evaluated each remaining excised sign and were in agreement as to the start/end points of the target sign. In the rare case of disagreement, the authors analyzed the carrier phrase together and agreed upon a solution.

2.3. Measures

For each signer, we recorded position coordinates of the hands at each time sample every 68 msec), where x is a “lateral” plane in front of the signer that moves to the left or the right of the signer, y is height of the hand, as the hand moves up and down, and z is the plane that moves in front of versus behind the signer’s body. We defined the origin (0, 0, 0) as the point in between the signer’s eyes, which was chosen with the assumption that this is an estimate of where a viewer looks when watching another person sign. Positive values were y values that are above the eyes, x values that were to the right of the body midline, and z values that were in front of the body. From these position coordinates, we calculated the following measures:

1. Position and eccentricity in space of the signers’ dominant (right) hand (degrees). Each time sample had an x and y distance (in cm) from the origin (midpoint between the signer’s eyes, assuming this is where the viewer generally fixates), which was used to compute eccentricity for each time sample: $2\text{D eccentricity}=\sqrt{{\text{x}}^2+{\text{y}}^2}$. Greater values indicate greater distances of the hand from the origin. Eccentricity values were then converted to degrees of visual angle (described below), separately for signs, sentences and narratives.

2. Total 3D Distance of words (cm). This was calculated by summing the instantaneous distances traversed from one time sample to the next. 3D instantaneous distances were calculated as $\sqrt{{\text{dx}}^2+{\text{dy}}^2+{\text{dz}}^2}$, where dx, dy, and dz represents change in position between two consecutive time samples. For example, if there were 9 time samples within an excised sign, this would mean summing 8 instantaneous distances. Total 3D distance was calculated for excised signs, to be used in our analysis of Constraints on Signing Speed (described in section 2.3.2, below). For comparison, we also calculated the average distance of words for sentences, by summing instantaneous 3D distances for the entire sentence and dividing by the number of words in the sentence (separately for each of the six sentences). This was not calculated for narratives as we did not keep track of the number of words.

3. Speed of the hand (cm/sec and degrees/sec). This was calculated by averaging the instantaneous speed across samples. Instantaneous 3D speed (cm/sec) was calculated as the instantaneous 3D distance (described above) divided by the time elapsed from one sample to the next (on average, 68 msec). Instantaneous 2D speed distances/speeds were calculated in the same way, using just the X and Y dimensions (and converting into degrees/sec). 2D and 3D speeds were calculated for signs, sentences and narratives.

4. Duration of words (seconds). For excised signs, this was calculated as the time elapsed from the start to finish of the target word. For comparison, we also calculated the average duration of words in sentences, by dividing the duration from the start to the finish of the sentence by the number of words in the sentence (separately for each of the six sentences). Duration of words was not calculated for narratives as we did not keep track of the number of words.

For eccentricity and speed, we present the data in centimeters. We also present data in degrees of visual angle because the visual system decodes sizes of objects in the world, which is defined in terms of visual degrees, and, therefore, this is the relevant dimension (not absolute size in cm) when referring to a signer’s visual experience. Equally important, if future studies test the Enhanced Exposure Hypothesis, the properties of signs in degrees are needed to recreate those conditions on a video monitor. Only the 2D, frontoparallel (x,y) plane is presented in degrees since this is the plane projected on (and “experienced” by) the retina. Moreover, 2D motion, and not 3D motion is encoded at the level of the retina (Bonnen, Huk & Cormack, 2017; in addition, future studies that test the Enhanced Exposure Hypothesis will likely use 2D monitors, which can only replicate the x, y spatiotemporal properties of signs). As in our previous study, to determine degrees we assumed a viewing distance of 1.52 meters in front of the signer, with the estimate that signers stand roughly 1.52 meters apart when conversing (see Discussion for more details). Degrees of visual angle (in degrees) was calculated as tan⁻¹ (w/152), where w = distance in cm, assuming a viewing distance of 1.52 meters (i.e., 5 feet).

2.3.1. Means and Distributions.

For each of our measures, and for each stimulus type (signs, sentences, and narratives), we present means and distributions for each of the three signers. First, for eccentricity and speed, for each stimulus type, we used all time samples to compute means and standard deviations. This was performed separately for the three signers as well as combined across all time samples and signers, for a grand mean.¹ The total number of eccentricity and speed samples is presented in Table 1. Second, for duration and distance, for excised signs, we calculated means and standard deviations across all the 42 words. This was performed separately for the three signers, as well as combined across signers for a grand mean. Finally, for sentences, we reported the mean duration and distance of words averaged across the six sentences, but did not report the standard deviation, because we calculated an average word duration/distance for each sentence.

Table 1.

Number of samples per signer and stimulus type

	Signs		Sentences		Narratives
	Eccentricity Samples	Speed Samples	Eccentricity Samples	Speed Samples	Eccentricity Samples	Speed Samples
Signer 1 (RB)	529	487	315	309	467	465
Signer 2 (DH)	452	410	259	253	587	585
Signer 3 (VM)	406	337	213	207	478	476
Total Samples	1387	1234	787	769	1532	1526
Average per signer	462	411	262	256	511	509

2.3.2. Modeling Constraints on 3D Signing Speed.

It is expected (and the data confirm) that sign durations and hand speeds vary within and across signs (and therefore, distance traversed by the hands necessarily varies across signs, as well). With our distribution of 3D speeds/distances for signs, we asked whether signers modulate their hand/arm movements in a systematic way that maintains some degree of invariance in either the average speed or the total duration of signs. There is reason to believe that signers might try to maintain a constant duration, in line with the concept of articulatory isochrony (Tuller & Fowler, 1980). If this were the case, hand speed should be faster for signs where the hands traverse a larger distance (and vice versa). To address this question, for the excised sign data (from 42 signs), we plotted speed vs. distance, separately for each signer. This allowed us to ask whether the resulting function was more in line with a constant duration (that is, a non-zero slope, with the slope equal to the mean duration of signs, for a given signer) or a constant speed (that is, a slope of 0, with the mean equal to the mean speed of signs, for a given signer).

3. RESULTS

First, for the individual signs database, we asked whether there was internal consistency across the three repetitions for each of the 42 signs by calculating Cronbach’s alpha on the results. Indeed, there was very high internal consistency across the repetitions (2D eccentricity: RB = 0.96; DH: α = 0.96, VM: α = 0.86; 2D speed: RB: α = 0.92, DH: α = 0.88, VM: α = 0.79; duration: RB: α = 0.95, DH: α = 0.96, VM: α = 0.89). As such, for the rest of the analysis presented here, only the first production of each sign was used.²

3.1. Eccentricity

Figure 2 presents a scatterplot of x, y position coordinates (in both cm and degrees, assuming a 1.52 meter viewing distance between the signer and the viewer) of all time samples (relative to the origin, which was the midpoint between the two eyes), separately for a) signs, b) sentences, and c) narratives. Data are shown in separate colors for the three different signers. For each signer, the grand mean position (across all their time points) is also shown. Means and standard deviations are presented in Table 2. Here we focus on the 2D results, as that is relevant for visual perception.

Figure 2. Scatterplot of hand position over time.

Position coordinates are shown for a) signs, b) sentences and c) narratives. All samples from each stimulus type are presented, separately for the three signers, in each figure. These position coordinates are presented both in terms of centimeters and, for x and y planes only, in degrees from the origin (between the signer’s eyes, defined as 0,0,0). Values are plotted for the frontoparallel plane, i.e., height (y) and width (x), assuming one is facing the signer. On the bottom and left axes, the metric is in centimeters. On the top and right axes, the metric is in visual degrees, assuming a 1.52 meter viewing distance. If one is facing the signer from twice the distance (e.g., 3 meters), the x and y degrees labels would simply be halved, are plotted for the frontoparallel plane, i.e., height (y) and width (x), i.e., assuming one is facing the signer. For each figure, a larger circle depicts the average position for each signer.

Table 2.

Averages and standard deviations for position coordinates and eccentricity, calculated from all samples for signs, sentences, and narratives

	Signs
	3D space (centimeters)		2D space (degrees)
	X	Y	X	Y	Eccentricity from origin
Signer 1 (RB)	−0.9 (6.3)	−11.9 (7.7)	−0.3 (2.4)	−4.5 (2.9)	5.1 (2.7)
Signer 2 (DH)	2.6 (6.2)	−16.3 (8.1)	1.0 (2.3)	−6.1 (3.1)	6.6 (2.9)
Signer 3 (VM)	−7.2 (7.1)	−6.0 (11.1)	−2.7 (2.7)	−2.2 (4.2)	5.3 (2.8)
Average	−1.5 (7.6)	−11.7 (9.8)	−0.6 (2.9)	−4.4 (3.7)	5.7 (2.9)
	Sentences
	3D space (centimeters)		2D space (degrees)
	X	Y	X	Y	Eccentricity from origin
Signer 1 (RB)	0.5 (5.6)	−22.5 (20.1)	0.2 (2.1)	−8.4 (7.5)	8.9 (7.2)
Signer 2 (DH)	2.3 (6.7)	−30.2 (18.2)	0.9 (2.5)	−11.3 (6.8)	11.9 (6.2)
Signer 3 (VM)	−4.7 (9.0)	−30.0 (28.6)	−1.8 (3.4)	−11.2 (10.7)	13.0 (9.0)
Average	0.6 (7.5)	−27.6 (22.5)	−0.2 (2.8)	−10.3 (8.4)	11.3 (7.7)
	Narratives
	3D space (centimeters)		2D space (degrees)
	X	Y	X	Y	Eccentricity from origin
Signer 1 (RB)	−4.0 (6.9)	−13.7 (12.6)	−1.5 (2.6)	−5.1 (4.7)	6.2 (4.4)
Signer 2 (DH)	2.9 (4.7)	−18.0 (9.9)	1.1 (1.8)	−6.8 (3.7)	7.1 (3.6)
Signer 3 (VM)	−1.8 (7.7)	−9.6 (20.1)	−0.7 (2.9)	−3.6 (7.5)	6.1 (6.4)
Average	−1.0 (7.1)	−13.8 (15.0)	−0.4 (2.7)	−5.2 (5.6)	6.5 (4.9)

For signs, across all three signers, the grand mean 2D (x, y) eccentricity from origin is 5.7° (SD = 2.9°) in the lower visual field, assuming the viewer is fixating between the signer’s eyes. Results for narratives were very similar to signs, with a mean eccentricity average of 6.5° (SD = 4.9°). Sentences (which are listed in the Appendix) were, on average, much farther, falling 11.3° (SD = 7.7°) below fixation. One possible reason the sentences were lower than signs and narratives is because the sentences contain the starting and ending positions with the arms at the sides.

All three signed stimuli types had mean horizontal x positions less than 1° from origin, which means they fell very close to the midline of the signer’s body (mean x position for signs: − 0.6 (SD = 2.9); sentences: −0.2 (SD = 2.8); narratives: −0.4 (SD = 2.7)). As can be seen in Figure 2, the hand position samples were dispersed to both the left and right side of the signer’s body midline. Across all signers, the percentages of time samples that fell in the viewer’s left visual field (i.e., the same side as the signer’s right hand) was 61% for sign, 55% for sentences, and 53% for narratives. Hence, the three stimuli types were similar in their visual field placements.

The vertical y position varied over a larger range from near fixation to far below fixation, with signs occurring on average 4.4° (SD = 3.7°) below the signer’s eyes. The signing hand during narratives occurred at 5.2° (SD = 5.6°) below origin, and sentences fell 10.3° (SD = 8.4°) below the origin. Across the three signers, the percentages of time samples that fell in the lower visual field (i.e., below the signer’s eyes) was 89% for signs, 94% for sentences, and 88% for narratives.³ In sum, across all stimulus types and signers, signs fell below origin 90% of all time samples, and in the viewer’s left visual field, 56% of all time samples. This represents a slight bias for the right hand to remain in the ipsilateral side of the body. We return to these differences in the Discussion.

3.2. Speed:

Mean speeds (both 3D, cm/sec and 2D, degrees/sec) and standard deviations are presented in Table 3.² Like the position and eccentricity data (above), speed data are shown separately for the three different signers, as well as averaged across the three signers. We focus on 2D speeds, as that is most relevant for visual perception.

Table 3.

Averages and standard deviations of speeds are calculated for signs, sentences, and narratives

	Signs		Sentences		Narratives
	2D speed (deg/sec)	3D speed (cm/sec)	2D speed (deg/sec)	3D speed (cm/sec)	2D speed (deg/sec)	3D speed (cm/sec)
Signer 1 (RB)	18.7 (113)	72.0 (44.0)	18.9 (13.8)	65.6 (49.1)	13.5 (9.7)	47.4 (32.4)
Signer 2 (DH)	16.8 (10.0)	67.1 (40.2)	18.1 (13.9)	62.8 (47.9)	10.8 (9.7)	40.6 (36.6)
Signer 3 (VM)	25.5 (15.5)	115.7 (82.3)	30.4 (21.0)	141.1 (123.1)	24.0 (16.6)	106.1 (102.0)
average	19.9 (12.7)	82.2 (59.6)	22.5 (16.9)	89.9 (83.5)	15.7 (13.5)	63.0 (70.0)

Results for 2D speeds were very similar for the three sign types. For signs, across all three signers, the grand mean 2D speed was 19.9 deg/sec (SD = 12.7) for signs, 22.5 deg/sec (SD =16.9) for sentences and 15.7 deg/sec (SD = 13.5) for narratives. It is possible that narratives were a bit slower than signs and sentences, perhaps because narratives have greater use of prosodic elements and pauses, as signers recall episodic memories from their past lives. VM, who was the native signer exposed to ASL from birth, was the fastest, compared to RB and DH who learned ASL in late childhood, suggesting a possible age of acquisition impact on articulation speeds. We address these differences in the Discussion.

3.2. Distance and Duration of Words:

Statistics for distance and duration of words are presented in Table 4. Like the position, eccentricity and speed data (above), distance and duration data are shown separately for the three different signers, as well as averaged across the three signers.

Table 4.

Averages and standard deviations for distance and duration for signs produced in carrier sentences and for signs within elicited sentences.

	Excised Signs		Per Sign in Sentence
	3D Distance traveled (cm)	Duration in time (sec)	3D Distance traveled (cm)	Duration in time (sec)
Signer 1 (RB)	58.2 (28.3)	831 (292)	62.7	808
Signer 2 (DH)	46.4 (26.6)	782 (420)	50.0	992
Signer 3 (VM)	65.7 (40.5)	725 (422)	91.8	622
average	56.7 (33.1)	779 (382)	68.0	821

For signs, across all three signers, the mean distance traversed across a single sign was 56.7 cm (SD = 33.1) and the mean duration of a single sign was 779 msec (SD = 382). Distance and duration for each sentence was divided by the number of words to provide an estimate per word. (For this reason, only means and not standard deviations are presented.) For sentences, the mean distance traversed was 68.0 cm per word, and the mean duration was 821 msec per word, which agree quite reasonably with the excised sign data.

3.3. Modeling Constraints on Signing Duration/Speed

To address whether signers might try to constrain either the speed or the duration (or both) across variations in total signing distance, we plotted speed vs. distance for each signer (across the 42 signs), asking whether the resulting function was more in line with a constant duration (i.e., a non-zero slope), or a constant speed (i.e., a slope of 0, with the mean equal to the mean speed of signs, determined separately for each signer), or some combination of the two. The plots are shown in Figure 4, separately for each of the three signers. For each signer (and each figure), a constant speed is modeled by the dotted diagonal line, calculated for each signer based on the average duration and distance traveled across all signed samples, whereas a constant duration is modeled by the horizontal dashed line, also calculated based on the average duration for each signer. For all three signers, a logarithmic fit provided a very good fit, as follows: RB: r = 0.63; DH: r = 0.48, VM: r = 0.66, with all fits highly significant (p < 0.001). It may be that for signs of shorter distance, signers try to constrain duration (i.e., the slope relating speed vs. distance is close to the mean duration of signs), yet for signs of longer distances, they constrain speed (i.e., the function relating speed vs. distance starts to flatten out). We address this, and other possibilities, further in the following Discussion.

Figure 4. Speed vs. Distance Plots.

3D Speed (in centimeters per second) and distance (centimeters) values across all signs are plotted in separate figures for the three signers, RB, DH, and VM. For each signer, each dot represents the average speed value of a single sign as a function of the sign’s cumulative distance traveled by the hand. The dashed line is the model of constant speed, the thin line is the model of constant duration (see text). The bold line is a logarithmic fit, and the correlation coefficient is presented for this fit.

4. DISCUSSION

The results of this study provide statistics about the visual spatiotemporal properties of signs in sign language. We were interested in quantifying these properties so that future studies could test whether frequent exposure to sign language alters visual processing, i.e., the Enhanced Exposure Hypothesis, in deaf signers. The data from this study also allowed us to ask whether signers might modulate the timing of their hand/arm movements to maintain some degree of constancy in either the speed or the duration of signs (or a combination of both). We address each of these, in turn, below, as well as addressing whether or not spatiotemporal properties of signs may be a truly unique experience for signers.

4.1. Testing the Enhanced Exposure Hypothesis.

The Enhanced Exposure Hypothesis predicts that differences in visual processing between signers and non-signers are predicted to be greatest for the visual properties that fall within, versus outside, those encountered in sign language. Generally, studies have tested visual perception in signers versus non-signers, but we know of no explicit attempt to test both aspects of vision that are expected to be enhanced, i.e. within the range of what would be considered the signer’s unique experience, and outside this range, where signers and non-signers are expected not to differ. Although studies directly testing this hypothesis for sign language have yet to be performed, there does exist some data from previous studies that allow us to take a first step in addressing this. Specifically, we can ask whether previous studies that observed differences in visual processing between signers and non-signers used stimuli whose properties fell within the range of those observed for sign language in the current study. For this question, the most obvious visual measures to explore are speed and retinal eccentricity in studies of motion processing, as these are often well-controlled in visual psychophysical studies.

In the domain of motion processing, perhaps one of the most robust differences between signers (both deaf and hearing) and non-signers are reported for visual field asymmetries in performance. First, with regards to lateral (left-right) visual field asymmetries, while non-signers show either no visual field asymmetry or a slight left visual field (LVF) advantage, signers show a strong and significant right visual field (RVF) advantage for motion tasks (Bosworth & Dobkins, 1999; Neville & Lawson, 1987a). This effect for motion processing has been shown using lateralized stimuli for a leftward vs. rightward direction-of-motion discrimination task (Bosworth & Dobkins, 1999, 2002; Samar & Parasnis, 2005), an apparent motion task (Neville & Lawson, 1987a; Neville & Lawson, 1987b), and a speed discrimination task (Brozinsky & Bavelier, 2004). Supporting these behavioral results, deaf and hearing signers show greater brain activation in the left hemisphere while viewing moving stimuli compared to hearing non-signers (Bavelier et al., 2001; Neville & Lawson, 1987b). Since the left hemisphere is believed to be dominant for sign language processing (Corina, Vaid & Bellugi, 1992; Poizner, Battison & Lane, 1979), the RVF (i.e., left hemisphere) advantage in signers has been attributed to a “language capture” effect, wherein motion processing gets usurped by the left, language-dominant hemisphere because motion is an integral part of comprehending sign language. Asymmetries have also been found for superior-inferior visual fields. Studies have found that signers, but not non-signers, are better at detecting visual stimuli in the inferior visual field, compared to the superior visual field, presumably because signs tend to fall in the lower visual field (Bosworth & Dobkins, 2002; Dye, Seymour & Hauser, 2016; Stoll, Palluel-Germain, Gueriot, Chiquet, Pascalis & Aptel, 2018).

Given the altered visual field asymmetries seen in deaf and hearing signers for motion tasks, we are in a place to ask whether the speeds and eccentricities of the stimuli used in those studies were within the range of those observed for sign language in the current study. To this end, we looked at the speeds and eccentricities used in empirical studies that reported altered visual processing in signers, in the form of a right visual field advantage. In terms of speed, values in these previous empirical studies ranged from 3 to 10 deg/sec. In the current study, we found that the mean speed (in the x, y plane) across the three signers and the three sign stimuli types was 19.4 deg/sec, with a 95% CI of 11 to 27 deg/sec. In terms of eccentricity, past studies used values that ranged from 4 to 18 degrees in the x dimension (i.e., stimuli tested at both left and right of fixation), and from 0 (i.e., aligned with fixation) to 13 degrees (i.e., above/below fixation) in the y dimension. We report here that the 95% CI range of eccentricity of excised signs falls from 5° to the left and 6° to right of the signer’s body midline, and 3° above and 12° below the signer’s eyes. From this exercise, we conclude that the speeds used in previous studies of visual processing in signers were in the low range of speeds encountered in sign language. For eccentricity, those used in previous studies of visual processing in signers were in the range of those encountered in the current study. Of course, this comparison between parameters used in previous empirical studies and those observed in sign language depends on what assumptions the current study makes when converting physical distance (cm) to viewing distance (in degrees of visual angle). In the current study, we converted cm to degrees, assuming that signers converse at about 1.52 meters from one another. If, for example, the conversing distance were closer to 3 meters, then our calculations of speeds and eccentricities get halved (i.e., 95% CI ranges from about 3.8 to 34.5 deg/sec), and then the speeds used in previous studies of visual processing in signers (i.e., 3 to 10 deg/sec) overlap quite well with those encountered in sign language.

Given that there is in fact, overlap with previous studies, then at least one aspect of the Enhanced Exposure Hypothesis appears to be true, that signers exhibit altered visual processing for spatiotemporal parameters that fall within those encountered in sign language. What has yet to be tested (within the same study) is the converse hypothesis, i.e., signers will not exhibit altered visual processing for spatiotemporal parameters that fall outside those encountered in sign language (for example, speeds of 90 degrees/sec, or eccentricities of 25°). Future studies will be needed to test this hypothesis further. The strongest test of the hypothesis will involve testing two sets of spatiotemporal parameters; one within, and one outside, the range encountered in sign language. In addition, it will be important to test both deaf and hearing signers, to determine whether differences are due to sign language experience vs. deafness.

4.2. Constraints on Signs.

In our analysis that addressed whether signers might try to constrain their arm/hand movements as they sign, we found evidence for systematic variation in both the speed and duration of signs in our correlation analyses of speed vs. distance. Because the data were well fit with a logarithmic function, this suggest that signers may try to constrain duration for signs of shorter distances, yet constrain speed for signs of longer distances. The results of our analysis suggest that the variance we observed in the speed and duration of signs is systematic, rather than random, in nature.

If there is systematicity in rate of signing, the interesting question arises as to why this might be the case. On the one hand, it might be the case that the speed of arm/hand movements in sign language is limited by biological constraints (i.e., how fast the muscles can move), and as such, is not under the volition or cognitive control of the signer. Research on the speed of arm movement find an upper limit of around 150 – 250 cm/sec when participants must quickly raise an arm to stop an oncoming obstacle (Degoede, Ashton-Miller, Liao & Alexander, 2001). Because this is well above the hand speeds observed in the current study, we do not think the speed of signs is under a biological constraint. On the other hand, it might be that signers use speeds that stay within the bounds of those that are comprehensible to a viewer, and that this is under the volition of the signer. It is intuitive that signers will attempt (volitionally or not) to sign at a speed that is within the bounds of those that are comprehensible for the viewer. As is likely the case for spoken language too, presumably the goal for signers is to sign as fast as they can, but not so fast that the listener/viewer cannot follow. In a relevant study by Fischer, Delhorne and Reed (Fischer, Delhorne & Reed, 1999), the relationship between speed and comprehensibility was investigated by presenting signers with videos of people signing at different playback speeds. To this end, they videotaped native signers signing 98 different words.³ The researchers then tested comprehension in fluent signers, who were asked to watch the videotapes of the signs and report each word they saw, at different playback speeds. The results of this study showed that comprehension fell from 98% to 46% as signs went from the normal speed/duration to 6×, with impairments seen at about 3× normal rate.⁴ This result is consistent with the possibility that signers use speeds that are within the bounds of those that are comprehensible in sign language.

4.3. Are the Speeds Inherent in Sign Language Unique?

As a final point, we address how the speeds of signs compare to speeds of other common objects in the environment (people walking, flying birds, cars, etc.) to get a sense of whether signing speeds are a unique experience. For this, we start with estimating cm/sec, and then, address the conversion of speed into degrees/sec. Perhaps the two most common objects we see move in our environment are walking people and moving cars. For people walking, it is estimated that a common walking speed is 3 miles/hour, which converts to 134 cm/sec. For cars, we estimate that they move between 30 – 60 miles/hour, which translates to 4 – 8K cm/sec. Determining degrees/sec for signs, walking people and moving cars requires making assumptions about viewing distance. For sign language, viewing distance ought to be largely constrained (and we assume a distance of about 1.52 meters), for two reasons. First, social etiquette dictates a comfortable distance between conversers (which is true for both signed and spoken language). Second, too far of a distance between conversers will hinder comprehension, either because of occlusion from other objects (e.g., if someone walks in between the two conversers) or an inability to resolve the articulators (fingers, hands, arms) at a far distance. By contrast, viewing distance for walking people or moving cars is far less constrained (i.e., people/cars can be very nearby or very far away). As such, degrees/sec of walking people and moving cars can vary substantially, with a faraway person at 60 meters moving as slowly as 1.3 degrees/sec and a nearby car, 3 meters away on a city street, moving as fast as 85 degrees/sec. This large speed range (about 1 – 85 degrees/sec) for other common moving objects in the environment encompasses those encountered in sign language determined from the current study (across 3 signers, we found a mean 2D (x, y) speed of 19 degrees/sec).

Given the large speed range in ecologically relevant stimuli (such as people walking and cars moving), it seems unlikely that the speed of hand movement in sign language provide a unique experience for signers. We have previously addressed the significance of non-uniqueness in our study that characterized the spatial frequency and orientation makeup of signs (using Fourier analysis, Bosworth et al., 2006), because in that study, we observed a unique orientation bias, but not a unique spatial frequency bias, for signs. Specifically, compared to faces and natural scenes, which contained more amplitude for horizontal than vertical contours, signs showed the opposite pattern. However, like the current analysis of speed, the Bosworth et al. study did not find evidence for a unique spatial frequency bias in signs (i.e., signs, faces, natural scenes all showed the classic 1/f curve, where f is spatial frequency, which describes the fine to course level of detail in an image). We argued in that paper, as we will argue here, that uniqueness, while interesting if it exists, is not a necessary prerequisite for the Enhanced Exposure Hypothesis, which is why we did not refer to it as the “selective exposure hypothesis”. In other words, we argue that — whether or not the visual properties of sign language are unique, signers will get more exposure to these properties than do non-signers (and of course, rely heavily on these signals for comprehension). According, we propose that whether or not the spatiotemporal properties of sign language are unique, the “enhanced exposure hypothesis” is an important hypothesis to test.

It will be important to compare variation in signing rates and articulatory (and hence, perceptual) properties of signing across multiple signers who differ in gender and body size, and in age of sign language acquisition. Likewise, future studies should examine various situational contexts such as naturalistic settings outside the laboratory, because it is likely that situational context can affect how one converses (true for both signing and spoken language), for example, the articulatory characteristics of signing are likely to vary for relaxed versus formal settings (such as at home, in a group, or lecturing to an audience). Finally, future studies should be done to compare across multiple signed languages of the world.

6. APPENDIX

Sign Stimuli:

One-handed signs: CANADA, FOOD, GOAT, HEART-FELT, KNOW, MINE, ASK, FIND, SHUT-UP, THROW, CAT, MAIL, SPIT, SUMMER, FACE, GIVE, REJECT, SMART, TELL, VOMIT, GIVE-continuously, TELL- continuously

Two-handed signs: ABORTION, DOCTOR, BICYCLE, ENJOY, GESTURE, LONG-AGO, WASH, HAVE, SICK, HATE, DAMAGE, STEAL, ARREST, SEND, IMPROVE, READ, UNTIL, YEAR, READ-continuously, SICK-continuously

Sentence Stimuli, presented as English glosses and translations:

1) SORRY TRAIN-GONE	Sorry, you are too late.
2) LAST-NIGHT [topic], MOTHER LEFT	Mother left last night.
3) YOU ENJOY TRAVELING?	Do you enjoy traveling?
4) WHO YOUR TEACHER?	Who is your teacher?
5) HE #P-I-L-O-T LIVE N.Y., FLY-COMMUTE, ALL-OVER-WORLD.	He is a pilot who flies all over the world for his job.
6) LAST-WEEK I-GO SEE MOVIE CROWDED SOLD-OUT. WHAT-Do? GO #P-O-O-L	Last week, I went to see a movie, but the line was so long, so I went to the pool instead.

Sign	One or Two handed	Symmetry of two hands	Average duration (msec)	Average total distance traversed (cm)	Average 2D Speed (deg/sec)	Average 3D Speed (cm/sec)
Canada	one	--	745	68	29	118
cat	one	--	756	70	20	82
face	one	--	757	39	20	84
find	one	--	653	85	24	96
food	one	--	563	47	15	62
give	one	--	628	28	15	59
give-continually	one	--	1546	65	24	88
give-them-all	one	--	945	29	17	53
goat	one	--	936	71	23	79
heart-felt	one	--	522	87	28	108
know	one	--	714	50	24	83
my/mine	one	--	750	35	12	57
shut-up	one	--	624	59	23	90
smart	one	--	707	53	17	98
spit	one	--	598	117	24	96
summer	one	--	522	113	28	127
throw	one	--	930	48	14	53
vomit	one	--	522	29	12	65
about	two	asymmetrical	747	26	12	74
arrest (a person)	two	asymmetrical	680	29	14	58
doctor	two	asymmetrical	554	54	22	83
improve	two	asymmetrical	991	56	19	90
read	two	asymmetrical	612	60	20	86
read-casually	two	asymmetrical	1358	46	12	63
read-continuously	two	asymmetrical	1599	41	16	64
read-emphatically	two	asymmetrical	783	96	19	72
remove	two	asymmetrical	599	118	22	84
send (via mail)	two	asymmetrical	445	61	24	91
steal	two	asymmetrical	321	28	13	52
year	two	asymmetrical	760	32	12	74
bicycle	two	symmetrical	950	33	11	57
destroy	two	symmetrical	825	44	16	65
enjoy	two	symmetrical	919	117	21	83
gesture	two	symmetrical	645	48	19	70
hate	two	symmetrical	449	44	18	71
have	two	symmetrical	368	39	14	73
long-ago	two	symmetrical	1027	48	35	127
reject	two	symmetrical	542	33	19	67
sick	two	symmetrical	678	42	10	49
sick-continually	two	symmetrical	1481	42	18	101
sick-emphatically (very)	two	symmetrical	714	70	21	70
wash (e.g., dishes)	two	symmetrical	1272	83	28	105