Perception of Tactile Graphics: Embossings Versus Cutouts

Abstract

Graphical information, such as illustrations, graphs, and diagrams, are an essential complement to text for conveying knowledge about the world. Although graphics can be communicated well via the visual modality, conveying this information via touch has proven to be challenging. The lack of easily comprehensible tactile graphics poses a problem for the blind. In this paper, we advance a hypothesis for the limited effectiveness of tactile graphics. The hypothesis contends that conventional graphics that rely upon embossings on two-dimensional surfaces do not allow the deployment of tactile exploratory procedures that are crucial for assessing global shape. Besides potentially accounting for some of the shortcomings of current approaches, this hypothesis also serves a prescriptive purpose by suggesting a different strategy for conveying graphical information via touch, one based on cutouts. We describe experiments demonstrating the greater effectiveness of this approach for conveying shape and identity information. These results hold the potential for creating more comprehensible tactile drawings for the visually impaired while also providing insights into shape estimation processes in the tactile modality.

1. Introduction

“The more techniques we can develop to access visual information, the better off we will be, not just in science classes but in all areas of study.”

Dr Cary Supalo, Purdue University (Dr Supalo is blind and a prominent proponent of developing tools for enhancing access to science, math, and technology education for blind students.)

Visual graphics, such as illustrations, graphs, and diagrams, are an essential complement to text for conveying information about the world. Blind individuals are disadvantaged by the lack of access to visual graphics, which is most apparent in the realm of education. The National Science Foundation in the United States acknowledges that people with disabilities are underrepresented in the fields of math and science compared to the general workforce (Ebert, 2005). Because these fields make extensive use of visual graphics, they are especially challenging for individuals with impaired vision. Therefore, making graphical information accessible to blind people would significantly increase their educational and career opportunities.

One method of creating accessible graphics for blind individuals is to convert them into two-dimensional tactile graphics. Tactile graphics are advantageous compared to verbal descriptions of graphics because they allow users to actively explore the image. Consequently, tactile graphics are especially appealing for educational purposes in which students need to learn new information from a potentially complex image.

Although it seems that the translation of visual graphics into tactile images should be straightforward, the rehabilitation and education communities find it quite challenging to provide effective tactile graphics to blind individuals. The physical conversion of visual into tactile graphics can be accomplished in a variety of ways. Graphics can be scanned and printed on Braille embossers or swell paper. An embossed template can be made on thin metal sheets and the pattern can then be transferred to plastic (thermoform). Even more straightforwardly, an instructor can trace the graphics by hand on to specialized tactile drawing film. All of these superficially distinct techniques for creating tactile graphics lead to a similar outcome — an embossed version of a visual line drawing. Given the ease with which the visual line drawings can be recognized, one might expect that this recognizability would transfer to the tactile modality as well. However, systematic experimental tests suggest otherwise.

Several researchers (Heller et al., 1996; Kalia and Sinha, 2011; Klatzky et al., 1993; Lederman et al., 1990; Loomis, 1981; Magee and Kennedy, 1980) have evaluated the recognizability of tactile drawings of common objects (such as those in the Snodgrass and Vanderwart (1980) set shown in Fig. 1A). Whereas the drawings are trivially easy to recognize visually, average performance with their tactile counterparts (by blindfolded sighted individuals) drops to less than 35%. Performance of congenitally blind subjects on these drawings is comparable (Heller, 1989) or worse still (Heller et al., 1996; Lederman et al., 1990). Evidently, simply creating an embossed version of a line drawing is not sufficient to ensure its recognition via touch. What might be the source of this difficulty?

Figure 1.

(A) Examples of line drawings of common objects used in studies of visual and tactile recognition. (B) Subjects’ drawings (lower row) of felt embossings (upper row) highlight inaccuracies in shape perception (Kalia and Sinha, 2011).

1.1. Hypothesis

Results from our recent studies suggest that the difficulty in recognizing embossed tactile graphics is largely due to inaccurate perception of shape (Kalia and Sinha, 2011). Figure 1B illustrates this point. As their drawings demonstrate, even with extensive time for exploration (two minutes per tactile drawing), subjects are often unable to acquire accurate shape estimates of the embossed images. The poor recognition performance with tactile drawings could be a direct consequence of inaccurate shape perception. If this is the case, then how might shape estimation of tactile drawings be improved?

A possible answer may be found in the pioneering work on hand movements conducted by Lederman and Klatzky (1987). The researchers observed that when people were required to extract specific object properties using only their haptic sense, their hand movements fell into clear categories, which they called ‘exploratory procedures’ (EPs). Figure 2 shows the primary EPs that Lederman and Klatzky identified. Each EP was “defined by its invariant and typical properties” and was used for a particular task not simply because it was sufficient, but because it was optimal or necessary for acquiring the appropriate knowledge about the object in hand. Tactile receptor distributions and response properties help explain the linkages between the EPs and the object properties they are best suited to assess (Johnson, 2002). For instance, lateral motion stimulates the slowly adapting mechanoreceptors (the SA I fibers) that contribute to assessments of surface roughness. Interestingly, the authors found that the optimal EP for acquiring global shape information was the ‘enclosure’ EP, which requires the integration of proprioceptive information from the relative spatial locations of the fingers.

Figure 2.

Exploratory procedures used by observers for assessing different attributes of tactile stimuli (Lederman and Klatzky, 1987). This figure is published in colour in the online version.

A consideration of the EPs that can feasibly be deployed with embossed tactile drawings suggests a tentative hypothesis regarding why such drawings might not be very effective at conveying global shape information. Embossings permit the use of only a subset of EPs. Specifically, they allow lateral motion, pressure, static contact and contour following. Notably, none of these EPs are optimal for assessing global shape. Texture, hardness and temperature are largely irrelevant attributes in the context of embossed drawings. Even contour following, which provides some shape information, is geared towards an estimation of ‘exact shape’, i.e. the precise local details, rather than global shape. In our own studies with embossings, we have observed that participants spend most of their time executing lateral motion or fragmentary contour-following EPs. The enclosure EP, considered to be the most important for estimating global shape, simply cannot be deployed with embossed drawings. Given their inability to support the enclosure EP, it follows logically that embossings will prove to be poor conveyors of global shape.

This simple hypothesis not only helps explain the limitations of embossings, but more importantly, it provides a constructive suggestion for how one might design more effective tactile graphics. They should, critically, permit enclosure EPs.

To execute the enclosure EP, one necessarily has to dissociate the object from its background so that its extremities can be in contact with, and enclosed by, the fingers. The most straightforward way of accomplishing this is by creating cutouts of the object silhouette. If our hypothesis is correct, then simply lifting a graphic off the page, i.e. rendering it as a cutout rather than as an embossing, would improve global shape estimation. The two representations, although theoretically equivalent in their information content, would thus yield different levels of performance. This simple idea is the crux of this work.

Below we describe two experiments designed to test whether cutouts do indeed provide better shape estimates than the corresponding embossings. In Experiment 1, we tested tactile identification of cutout versus embossed depictions of familiar objects. In Experiment 2, we explored whether tactile shape assessment improved with cutout versus embossed depictions of unfamiliar objects.

2. Experiment 1

The goal of the first experiment was to assess relative efficacies of embossed and cutout depictions for recognition of familiar objects. This experiment builds on the results that have accumulated in the field of tactile object recognition. Several past studies have assessed recognition of embossed versions of simple line drawings, such as those in the Snodgrass and Vanderwart (1980) set. The typical finding has been that although the line drawings are trivially easy to recognize visually, tactile exploration of their embossed versions yields much lower performance. Would presenting these tactile stimuli as cutouts enhance recognition performance? In this experiment, subjects were presented with cutout versions of object stimuli, and their recognition performance was compared to recognition of embossed drawings reported previously (Kalia and Sinha, 2011).

2.1. Method

2.1.1. Stimuli

Our stimulus set comprised cutout versions of 28 line drawings of common objects (Fig. 1A) obtained from the Snodgrass and Vanderwart (1980) set. We selected images that we have previously used in tests of tactile embossing recognizability (Kalia and Sinha, 2011). The cutout images were produced using a 3D printer.

2.1.2. Subjects

The subject pool comprised 21 normally sighted college students in the Boston area with a mean age of 20 years. All subjects were right-handed except for one. These subjects had not been tested with the embossed drawings in the study reported previously (Kalia and Sinha, 2011).

2.1.3. Procedure

Each subject was presented with all 28 cutout stimuli. The experimenter gave the cutout stimuli to the subjects in the correct orientation. Subjects were blindfolded during the experimental session and had two minutes to tactually explore and recognize a given stimulus. They were asked to name the object verbally and were not provided any feedback. The experimenter recorded their responses and also the time elapsed between the start of the exploration and the response.

2.1.4. Data Analysis

We compared recognition of the cutout objects to the recognition of embossed drawings reported previously, and also the time taken to explore both types of stimuli. We conducted a paired t-test across all 28 objects to determine whether recognition accuracy and exploration time differed significantly between the two types of depictions. Also, we correlated the difference in recognition between the cutouts and embossings with the complexity of the object, determined by the ratings of naïve observers described by Kalia and Sinha (2011). Observers rated object complexity (described as the level of detail in the image) on a scale of 1 (low complexity) to 5 (high complexity); because ratings of complexity by vision and touch were highly correlated (Kalia and Sinha, 2011), we used the visual complexity ratings for our analyses. According to our hypothesis, there should be a greater improvement in recognition for high complexity images when presented as cutouts versus embossings.

2.2. Results

Our results reveal significantly higher recognition rates with cutouts compared to embossings (Fig. 3). On average, subjects who were presented with the cutouts recognized a significantly greater number of objects compared to subjects who were presented with embossings [t (39) = 4.055, p < 0.001]. An object-by-object comparison also shows that the cutout versions of the objects were recognized by more subjects compared to the embossed versions of the objects [t (27) = 6.744, p < 0.001], although some objects show a greater improvement in recognizability compared to other objects. This variability can be partially accounted for by the complexity of the image; the improvement in recognition for a particular object when presented as a cutout was significantly correlated with its complexity (Fig. 4, r = 0.437, p = 0.02). Furthermore, subjects spent significantly less time exploring the cutout objects compared to the embossings [t (27) = 5.042, p < 0.001; Fig. 5].

Figure 3.

(Top bars) Recognition performance (proportion of subjects who successfully recognized a given object) in the embossed condition and cutout condition. (Bottom bars) Average performance (proportion of objects correctly recognized) of subjects when presented with embossed or cutout objects. Error bars represent standard error. Data with embossings are derived from Kalia and Sinha (2011).

Figure 4.

Improvement in recognition with the cutouts as a function of the complexity of the object image. Ratings of object complexity were obtained from Kalia and Sinha (2011). This figure is published in colour in the online version.

Figure 5.

(Top bars) Amount of time subjects spent exploring the stimuli before recognizing the depicted object in the embossed and cutout conditions. (Bottom bars) Average time until recognition across subjects when presented with embossed or cutout objects. Error bars represent standard error. Data with embossings are derived from Kalia and Sinha (2011).

2.3. Discussion

These results indicate that presenting the same tactile images as cutouts rather than embossings significantly improves recognition of the depicted objects. The fact that the improvement in recognition is correlated with the complexity of the image suggests that the cutouts improve access to the global shape of the image. Better access to the global shape may also guide exploration of local shape with the fingertips, thereby allowing observers to better understand details in complex images. In Experiment 2, we further probed how perception of shape is altered when exploring cutouts versus embossings.

3. Experiment 2

The goal of the second experiment was to determine whether cutout graphics better enable subjects to perceive the shape of images via touch compared to embossings, thereby accounting for the improved recognition performance in Experiment 1. In principle, an observer can gain precisely equivalent information about the shape of an object by tracing its contour in an embossing, or by exploring its cutout. And yet, by allowing enclosure EPs, does the latter depiction style improve shape estimation relative to the former? The decision to work with unfamiliar objects rather than recognizable ones is driven by the need to prevent shape estimation processes from being influenced in a top-down manner by knowledge about specific objects.

3.1. Method

3.1.1. Stimuli

We chose to use the alphabet of a fictional language (Klinzhai, from the television serial Star Trek) as our corpus of shapes (Fig. 6A). This alphabet has the advantages of being unfamiliar, and having shapes that are made of both curves and straight edges like drawings of real world objects. Using a 3D printer, we fabricated two versions of this alphabet. The first depicted each letter as an embossed figure. The second comprised cutouts of each letter (Fig. 6B). The letters were of the same size in both depictions and averaged ten cm in height and width.

Figure 6.

(A) The set of unfamiliar shapes used in Experiment 2. (B) Stimuli fabricated using a 3D printer. This figure is published in colour in the online version.

3.1.2. Subjects

We enlisted nine normally sighted participants who had not participated in Experiment 1. Participants had no prior artistic training.

3.1.3. Procedure

Each participant underwent sixteen experimental trials. The sixteen trials corresponded to a subset of eight of the twelve letters from the alphabet, each in two depictions (embossing and cutout). The stimuli and the order in which they were presented was randomized across subjects; therefore, data for all twelve stimuli were collected across subjects. During each trial, a subject was handed a shape in it’s correct orientation and was instructed to tactually explore it for one minute (no restrictions were place on using one or two hands). All subjects were blindfolded during the exploration and did not see the tactile stimuli at any point during the experimental session. After each exploration, their blindfold was removed and they were asked to draw the felt shape. Thus, each experimental session yielded sixteen drawings.

3.1.4. Data Analysis

Eleven individuals, naïve as to the purpose of the study served as ‘judges’. For each subject, they were presented with one of the original shapes (i.e. drawn from Fig. 6A) and also the two drawings produced by the subject corresponding to the embossed and cutout depictions. The judges were not told which depiction style the drawings corresponded to. Each judge individually picked the drawing from a given pair that appeared more similar to the original. Each pair of drawings was rated three times by a rater in a randomized sequence. In this way, each pair of drawings was scored 33 times (11 judges × 3 ratings). Summing the scores across all subjects and all judges associated the number of ‘wins’ with each drawing. Using a χ²-test, we then determined if drawings corresponding to one style of stimulus (embossings or cutouts) were rated significantly higher than others.

3.2. Results

The ratings revealed a highly consistent trend: drawings corresponding to the cutouts were chosen as the best match to the original image more often compared to drawings corresponding to the embossings (68.6% vs. 31.4%; χ² = 109.14, p < 0.001). Figure 7 shows samples of a few pairs of drawings from this experiment and underscores the greater effectiveness of cutouts for providing a veridical sense of global shape. The ratings given to the drawings for each object are shown in Fig. 8.

Figure 7.

(Top row) Original shapes. (Lower left in each panel) Drawings made after exploring embossed versions of original shapes. (Lower right in each panel) Drawings made after exploring cutout versions of original shapes. This figure is published in colour in the online version.

Figure 8.

The proportion of times that raters chose drawings from the embossing versus cutout conditions as more closely matching the original object.

3.3. Discussion

These results demonstrate that displaying graphical information as cutouts allows observers to better acquire global shape information compared to embossings. By using enclosure EPs, the observers more accurately understood, and therefore could replicate, the shape of the images. Importantly, because we tested subjects with nonsense shapes, the results cannot be attributed to previous object knowledge.

4. General Discussion

The studies described here tackle a pressing question in the domain of visual impairment — how can we effectively convey graphical information via touch? A consideration of exploratory procedures has led us to a hypothesis that potentially accounts for some of the limitations of current embossing-based graphics and points towards cutouts as a more promising approach. Our results lend support to the hypothesis and, specifically, the effectiveness of allowing better access to the global shape of graphics.

It is interesting to consider why the enclosure EPs make them better at accessing global shape. One possibility is that by grasping, observers can take advantage of proprioceptive information about the relative positions of fingers to more accurately assess shape. Although information about relative finger position is also available when following contours with multiple fingers, it may be less accurate in a 2D plane compared to 3D space. Another possibility is that grasping allows access to more area of the image at a time, thereby increasing the effective haptic field of view. For embossings, increasing the haptic field of view by using both hands (Wijntjes et al., 2008) or multiple fingers on a hand (Klatzky et al., 1993; Loomis et al., 1991) to explore an image also improves recognition of tactile graphics. Furthermore, the enclosure EP may facilitate the information acquired with other EPs, such as contour following, that provide local shape information. Using the hand to access global shape information may provide a frame of reference to guide and contextualize local exploration with the fingertips, which may be especially helpful when exploring more complex tactile graphics.

We must temper our enthusiasm for these findings with a few notes of caution. First, we acknowledge that subjects could potentially have used other exploration strategies instead of or in addition to the enclosure EP to explore the stimuli. Observations during testing and a video recording of one subject exploring a cutout object indicate that the lateral motion EP is used to explore the internal lines with one hand while the object is grasped in the other hand. With embossings, lateral motion is used to explore both the internal and external contours of the image. Without video recordings of all subjects, we relied on our observations during testing to confirm that subjects used enclosure EPs more with the cutouts than with the embossings.

Secondly, the effectiveness of the cutout-based approach is likely dependent on the type of graphical information being displayed. The field would be best served by knowing which approach is best suited for a given type of graphic. For instance, it is possible that one approach is better for conveying mathematical graphs, while another is more effective for depicting real-world objects. The effectiveness of the approach may also depend on the availability of supplementary text descriptions of the graphic. We hope that future studies will provide guidance regarding which approaches will be best suited for conveying a particular kind of graphical information. There are also logistical challenges with creating cutout graphics and providing ways for blind users to explore them without assistance. For instance, blind users would need a method to determine the correct orientation of the cutout, perhaps with an additional tactile marker. Such issues should be solvable, especially as 3D printers are becoming more easily available.

Furthermore, these studies need to be replicated with blind individuals. Previous literature has been mixed regarding whether blind and sighted individuals differ in haptic shape perception and recognition of tactile graphics. For instance, while there is some evidence (Theurel et al., 2013) that greater experience with tactile graphics improves haptic recognition performance (suggesting that blind individuals should perform better on the task than sighted individuals), there are other reports showing that late-blind individuals are better at recognizing embossed tactile graphics compared to congenitally blind people, arguing that previous visual experience may aid recognition (Heller, 1989). A promising avenue for further research is to examine the relative performance of blindfolded sighted and blind individuals on shape estimation with cutouts and embossed depictions. Preliminary data that we collected from two congenitally blind subjects (Supplementary Table S1), who ‘drew’ the haptically felt shapes using bendable sticks, are consistent with the results from the blindfolded sighted subjects (Supplementary Fig. S1); reproductions corresponding to the cutouts were chosen as the best match to the original image more often compared to reproductions corresponding to the embossings (61.8% vs. 38.2%; χ² = 5.65, p = 0.017; see online Supplementary Material for a description of the procedure).

Finally, we need to titrate out other ways in which cutouts make it easier to access shape information besides allowing the use of enclosure EPs. The cutout graphics make it easier to distinguish between internal and external contours, and to segment the inside of the figure from the background compared to the embossings. It has been found previously that clearly distinguishing the inside from the outside of the shape, by filling the inside of an embossed shape with texture, improves recognition of the depicted object (Thompson et al., 2003). Therefore, this difference between cutouts and embossings could be responsible for improving shape acquisition with the cutouts rather than the ability to use the enclosure EP. Preliminary data from two additional congenitally blind subjects (Supplementary Table S1) show that adding a textured background to an embossing does not improve their global shape understanding; reproductions made after exploring embossings without a textured background were more often chosen to more closely match the original image compared to reproductions that followed exploration of embossings with a textured background (62.7% vs. 37.3%; χ² = 6.63, p 0.010; see online Supplementary Material for a description of the procedure). Although we only tested a small sample, these results likely differ from Thompson et al. (2003) because: (1) we assessed global shape acquisition rather than recognition of the images, and (2) we tested congenitally blind subjects who may be able to better separate the interior of the shape from the background. Further studies with more subjects are required to determine whether these types of embossings result in similar perceptual benefits as cutouts.

Taken together, results from these studies point the way towards the design of tactile graphics that can better convey the content they are intended to. In order to best serve the interests of the community of visually impaired individuals, it is imperative that we examine whether currently used approaches are able to adequately convey graphical information via the tactile modality. If these approaches are found to have any shortcomings, then we need to understand the factors underlying their limitations. Such understanding would be a precursor to devising new approaches that can more effectively convey graphical information. Improving tactile graphics would translate to the development of better educational material and improved access to scholastic avenues, especially those related to science, mathematics, and technology subjects, for visually impaired children and adults.