Health Websites: Accessibility and Usability for American Sign Language Users

Abstract

To date, there have been efforts towards creating better health information access for Deaf American Sign Language (ASL) users. However, the usability of websites with access to health information in ASL has not been evaluated. Our paper focuses on the usability of four health websites that include ASL videos. We seek to obtain ASL users’ perspectives on the navigation of these ASL-accessible websites, finding the health information that they needed, and perceived ease of understanding ASL video content.

ASL users (N=32) were instructed to find specific information on four ASL-accessible websites, and answered questions related to: 1) navigation to find the task, 2) website usability, and 3) ease of understanding ASL video content for each of the four websites. Participants also gave feedback on what they would like to see in an ASL health library website, including the benefit of added captioning and/or signer model to medical illustration of health videos.

Participants who had lower health literacy had greater difficulty in finding information on ASL-accessible health websites. This paper also describes the participants’ preferences for an ideal ASL-accessible health website, and concludes with a discussion on the role of accessible websites in promoting health literacy in ASL users.

Background

According to the Integrated Conceptual Model for Health Literacy, being health literate requires not only accessing health information but also understanding and utilizing health information to appraise and use health-related information to maintain and improve health (Sorensen et al., 2012). In the Deaf health field, health literacy at the basic levels of accessibility and comprehension is a problem, which we describe in a review of the literature on the health knowledge and literacy in the Deaf population. We define “Deaf health” as the health of people who are Deaf and use sign language to communicate.

Researchers of Deaf health agree that poor health literacy, language barriers to health care, and other factors contribute to increased risks for poor health outcomes (Margellos-Anst, Estarziau & Kaufman, 2006; Barnett, McKee, Smith, & Pearson, 2011; Smith, Massey-Stokes, & Lieberth, 2012). Generally, Deaf American Sign Language (ASL) users report difficulties with understanding health information in print, accessing health services and understanding treatment-related procedures (McKee & Paasche-Orlow, 2012). A review of seven articles on Deaf patient-physician communication issues suggested that Deaf adults experience health disparities due to the lack in linguistic and cultural competency on the part of health care practitioners in treating Deaf patients (Barnett, 2002). As expected, data from the Deaf Health Survey revealed that patients whose communication was concordant with the provider were more likely to utilize preventive services (McKee, Barnett, Block, & Pearson, 2011). However, the number of physicians who are culturally competent to care for Deaf patients is small. As a result, Deaf patients are forced to rely on alternate sources of health information (i.e. health websites; books; trusted peers or family members) for preventive care and better understanding of diagnoses (Margellos-Anast, Hedding, & Miller, 2004). Such an approach increases the risk of misunderstanding and using health information inappropriately, particularly if the user is not highly literate in English. This can negatively impact how the user incorporates the newly learned health information in making decisions related to their own health and preventive care.

Clearly, there is a need to develop effective health information materials and programs that are accessible to and benefit Deaf individuals who use ASL. Smith and colleagues (2012) recommend “planning, implementing, and evaluating a free, web-based repository of reliable health resources specifically designed for delivery in both ASL and English print” as the next steps toward promoting accessible health information for this population. To address this recommendation, we seek to obtain ASL users’ perspectives on the navigation of four ASL-accessible websites, ease of finding the health information that they needed, and the quality of ASL videos. These perspectives are aligned with two basic components of the Integrated Conceptual Model for Health Literacy (Sorenson et al., 2012): 1) accessing health information and 2) understanding health information. Both components were assessed subjectively, where the participants were asked to rate the accessibility and ease of understanding health information in ASL. If one perceives health information to be inaccessible or difficult to understand, then one might not be able to appraise and apply this health information in health care and disease prevention which are important steps in developing one’s health literacy. Although important, the objective assessment of participants’ level of understanding health information is outside the scope of the present study.

Method

Sampling and Participants

Approval for all aspects of this research was received from the Institutional Review Board at the National Technical Institute for the Deaf (NTID) in Rochester, New York. Purposive sampling of 32 Deaf students (18 or older) who know ASL was planned to provide a range of health literacy (minimal to adequate). While these students also know English, their preference for using ASL or English depended on the situation (e.g. academic environment; casual environment that involves communicating with a hearing person or deaf person). We asked the participants to select which of the following languages (ASL, English, Equal preference for both languages) they felt most comfortable using.

Recruitment for the first half of the sample was open to Deaf students in Associate degree programs at NTID in Fall 2012. After reviewing the participants’ characteristics and health literacy scores, we purposively sampled the remaining participants who were enrolled in an intensive English course based on English placement test scores in Spring 2013. These participants were recruited through brief classroom presentations by research staff, which took place from November 2012 to March 2013. Participants were given a $20 cash compensation for 1-hour participation time and effort and were afforded the opportunity to withdraw of their own volition at any time during the process.

Design

Step 1: Identify websites for evaluation

To address Smith and colleagues’ recommendation to evaluate web-based repository of reliable health information in ASL (2012), we focused on only websites that contained ASL health videos. As the initial step to selecting four websites for usability evaluation in our study, a Google search was conducted using key words, “Deaf health information” or “Deaf medical information.” Only the first few pages of the search results were reviewed, which included the ASL health websites that were already linked in the resource section at the National Center for Deaf Health Research (NCDHR). We considered the links posted at the NCDHR website to have been reviewed by experts in the Deaf health field before listing them in the recommended resource section. All of these searches were narrowed down to the final four websites, which we will not identify here. Our goal was not to critique those websites, but to solicit Deaf participants’ perspectives on the websites usability as well as what an ideal ASL health website should look like. The four websites met the following criteria: 1) be directed at providing health information access to ASL users and 2) include ASL videos about health information.

Step 2: Disorder/disease-specific information seeking activities

For the current study, we assigned a disorder- or disease-specific health information seeking activity for each website. The assignments were as follow: anxiety (Website 1), atherosclerosis (Website 2), diabetes (Website 3), and skin cancer (Website 4). Information seeking activity procedure is further described in “Task and Procedure” section of this paper.

Step 2: Expert website evaluation

A researcher with demonstrated expertise in website, interactive media and game design, development and usability conducted a user evaluation and ranking (hardest to easiest) of these websites. The following criteria were applied in the evaluation: 1) navigation to find the task, 2) website usability, and 3) video content. Each criterion was assigned a score from 1 (easiest) to 4 (hardest), followed by a detailed rationale for the score assignment. The overall score for each website served as a baseline for the participants’ ratings of the four websites. While these evaluation efforts are not comprehensive, this study is exploratory, and takes important first steps to understanding how online health information can be presented to Deaf populations.

Website 1 (anxiety task) was aimed at being a general pediatric health site with a wider range of medically related information for the Deaf, including what to expect at physician visits. Website 2 (artherosclerosis task) had the largest collection of ASL informational videos but was focused purely on information about diseases and tests. The textural content of the site seemed to come from a variety of health sources on the Internet. Content was translated by an ASL narrator and posted in video format on this website. Website 3 (diabetes task) was designed to be a general information/encyclopedia site on a wide range of topics specifically for ASL users that happens to have some medical information on it. The layout was similar to an “Ask.com” type of site rather than a “Web MD” type of site. This might account for the limited depth on the pertinent topic in each ASL informational video.

Website 4 (skin cancer task) contained a large collection of detailed, text-based articles on types of cancer, symptoms and treatments. Information for ASL users consisted primarily of seven videos with text transcripts created specifically for this audience. Some streamed lectures were also available, but were created with an outdated streaming software package and a player must be downloaded before they can be viewed.

Website 4 (skin cancer task) was the only site out of four that was produced by a large, professional medical organization. The rest (Websites 1–3) were independent efforts. It was clear that all these sites could benefit from being redesigned. The reasons for the need of redesigning include: 1) multiple pages of text with small fonts, 2) dated web layout with crawling banners and blocky sponsor ads, 3) older generation of space-consuming drop down menu system, and 4) inefficient requirement of accepting terms of use prior to web browsing.

Ranked 1 to 4 (1 as best and 4 as worst), we found that Website 1 (anxiety task), with original content produced exclusively for ASL was the best of the group in navigation and usability and video content. Website 3 (diabetes task) was ranked 2^nd due to its ease of finding diabetic-specific information in this study. Generally, this website had limited content, barriers to entry for some of the video content and its use of a transcript of a video for text, rather than revising and summarizing the content. Website 4 (skin cancer task) was ranked 3^rd due to a generalist approach with an outdated technology. Although this website was orphaned and/or discontinued site at this point, one could find skin cancer information on this website if given the link as was done in this study. Website 2 (atherosclerosis task) was as navigable as site one but less usable due to its large quantity of text pulled from other sources across the Internet, despite the fact that some of the content was translated to ASL in video. Condensing and revising that content, or creating original textural content for the Deaf would have made it more usable. For this reason, we ranked Website 2 as the hardest site to get information on atherosclerosis.

Task and Procedure: Disorder/disease-specific information seeking activities

In our study, the first half of the participants (n=16) began with anxiety (Website 1) and ended with skin cancer (Website 4). This order was reversed for the second half of the group (n=16, Website 4 to Website 1). Each participant was given a link to a website, and then told to navigate and find disorder- or disease-specific health information in this website.

If the participant was told to find information on atherosclerosis on Website 1, the participant had to figure out where to start searching for this information. Specifically, in a typical scenario the participant needed to:

1. keep track of the goal to find the disorder- or disease-specific information,

2. find out what resources are accessible at the website,

3. use search features (e.g. search box) or click on the website navigation links to arrive at the webpage that contains the target disorder- or disease-specific information, and

4. watch ASL video.

5. answer questions about the ease of finding the information, using the website, and understanding the ASL video.

After navigating each website and completing the pertinent health information task, the participant was asked to answer a set of questions in the following domains: 1) navigation to find the task, 2) website usability, and 3) ease of understanding ASL video content. We elected to solicit subjective ratings from the participants, and did not conduct objective comprehension testing of the participants’ knowledge on the pertinent topic. After completing all tasks, all participants gave feedback on what they would like to see in an ASL health library website.

A new question about the video delivery of an animated medical illustration was added later in the study. Only the second half group viewed an animated medical illustration (cancer or diabetes) at Focus Medical Health Animation Collection CREDO (http://www.focusmedica.com/) and provided feedback on whether they preferred to add captions and/or signer model overlay to this medical illustration video. This question was not included in the analyses, and the responses were used to supplement other information that was gathered in the interview from all participants.

Materials

Participants were invited to take part in a confidential, electronic survey consisting of four different tasks. We used Flesch-Kincaid readability analysis to ensure that the survey items were wordsmithed at 3^rd grade level. A Deaf research staff assisted with ASL translation of the survey items when needed. Follow-up interviews with all participants were done in ASL.

To ensure participants were kept anonymous, no names were collected or documented on any of the forms. After completing background questionnaire, participants took a 7-minute Short form of the Test of Functional Health Literacy in Adults (S-TOFHLA; 36 items; Baker, et al., 1999). The S-TOFHLA is a gold standard, widely used test of functional health literacy, which is a test of participants’ ability to read and comprehend health information in the English language. The reading level for the first passage is at the 4^th grade level, and 10^th grade level for the second passage. Internal consistency for the S-TOFHLA is high (α = 0.97). This test was used in a recent pilot study with 94 Deaf college students, but no reliability information was reported (Thew, 2013).

We initially used standard cut-offs in which scores from 0–16 represented inadequate HL, 17–22 marginal HL, and 23–36 adequate HL (Sarkar et al., 2011). In the current study, 63% of our sample had adequate HL and the remaining participants had marginal (15%) and inadequate (22%) HL. For the purpose of the current study, S-TOFHLA scores of 0–22 were classified as low health literacy group. S-TOFHLA scores of 23–36 were classified as adequate health literacy group.

Analysis

Using SPSS (version 17.0; SPSS, Inc., Chicago, IL, USA), bivariate correlation and t-tests were used to provide descriptive analyses of the participants’ perception of health websites with ASL videos. Tests of homogeneity were carried out to ensure consistency of the distributions across health literacy groups. Bivariate correlation was used to examine relationships among the domains: 1) navigation to find the task, 2) website usability, and 3) ease of understanding ASL video content. Mean differences between health literacy groups were explored with unpaired t-test.

All participants participated in an open-ended brief interview following study completion. We first conducted an inventory of the responses and then codified those using key words that relate accessing and understanding health information. For example, for the question relating to an ideal ASL-accessible health website, a response such as “ASL video overlay with health images” was classified in the category “accessing health information” for this question. The responses were then summarized within each category and described in the paper.

Results

Out of 32 participants who knew ASL and English, only 3 students expressed a general preference for English in daily communication. These 3 students’ health literacy scores did not significantly differ from the group of students who had equal preference for both English and ASL. These two sub-groups were then combined into one group (n=13), and compared with a group of students who expressed a general preference for ASL in daily communication (n=19).

Table 1 presents a comparison of participants’ demographics using t-test and chi-square. Consistent with the health literacy literature on the general population (Kutner, Greenberg, Jin, and Paulsen, 2006), race was associated with health literacy category (X²=3.70; p<.05). Participants who chose ASL as their preferred language in general had lower health literacy score compared to other participants who reported equal preference for both languages. The lower health literacy among primary ASL users was likely to be a consequence of longstanding restrictions on accessibility to health communication and health materials in the ASL user’s environment.

Table I.

Comparison of demographics among ASL-preferred users and ASL-English equal preference users using t-test and chi-square

Variable	Mean (95% CI)
	ASL-preferred	ASL-English equal preference	Difference or X2 value	p-value (2-sided)
Age (year)	22.6	22.0	0.579
	18.0, 27.1	18.0, 26.0	−5.31, 6.46	0.842
Gender:			0.130	0.719
Female	6	10
Male	7	9
Race:			8.016	0.005
Caucasian (reference group)	2	13
Black/African American	3	2
Asian/Pacific Islander	2	1
Biracial	1	1
Hispanic American	4	2
Did not answer	1	0
Family History of Deafness:			1.166	0.280
Yes	8	8
No	5	11
Health Literacy Group:			5.398	0.020
Adequate	5	15
Low	8	4

T-test and bivariate correlation analyses were conducted on website usability between the lower and adequate health literacy groups. No significant differences across health literacy groups emerged for any of the websites. However, there was a tendency for the lower health literacy group to report greater difficulty with using the website than the adequate health literacy group. With exception for the website with the Atherosclerosis task (link to the video was broken and not seen by any of the 2^nd half participants), the responses were similar for the ASL video content in three other websites. When the effects of race and language preference were partialed out, finding information (navigation) was positively correlated with perceived helpfulness of this information (r^partial ranged from .48 to .76 across all four websites).

Participants’ overall ranking for the four websites were compared to the usability testing conducted by an expert who was blind to the study results. Table II shows that the adequate health literacy group’s rankings were similar to the expert’s ranking for the most difficult and easiest website to navigate and use. Interestingly, the low health literacy group’s ranking was very different from both expert and adequate health literacy group’s rankings. According to the expert evaluator, Website 1 (anxiety) was judged to be easy to navigate and find information on anxiety. However, the participants with low level of health literacy rated the same website as difficult to find information. One possible explanation for these group differences might be related to how people utilize prior knowledge and current health literacy skills to navigate through websites. We did not record or track web navigation activity for each participant, which would have provided useful clues to the group discrepancy. Nevertheless, this discrepancy suggests that the perceptions of people with adequate health literacy skills cannot be generalized to people with low health literacy skills. ASL-accessible website usability studies should include at minimum people with low health literacy skills. In the following sections, we provide a summary of participants’ feedback on designing an ideal ASL-accessible health website for the Deaf community.

Table II.

Ranking of website navigation from hardest to easiest

Expert Evaluation	Adequate Literacy	Low Health Literacy
1. Website 2 (Atherosclerosis)	1. Website 2	1. Website 3
2. Website 4 (Cancer)	2. Website 3	2. Website 1
3. Website 3 (Diabetes)	3. Website 4	3. Website 4
4. Website 1 (Anxiety)	4. Website 1	4. Website 2

Accessing Health Information

A popular suggestion was to have an A–Z search format for easy access to health information. Many expressed a preference for the A–Z format over the other websites. These participants found the other websites challenging to navigate, especially if one did not know the correct spelling of the disorder.

For every disorder or disease-specific information, there should be a video made available on that page so users could immediately access pertinent information in ASL. Participants agreed that there should be an option for caption and other video preferences.

Using the Website

Besides the video, participants had many suggestions for how the website should be structured. Participants expressed that the website navigation was equally important to accessing health information in ASL. The navigation of the website should be easily accessible on the homepage to save time in browsing.

A clean, simple interface was desirable for maintaining one’s attention to the webpage. Some of the participants commented that they gave up looking for information on several of the other websites that had complex interface. This comment tended to come from participants who had lower health literacy.

As is expected from any user, the participants in our study frequently commented that the website should be attractive with appropriate colors and be user-friendly. A search box on the homepage was highly desirable.

Perceived Ease of Understanding Health Information in ASL

Participants generally agreed that clear and engaging videos in ASL were critical to understanding the health information. Professional attire and neutral background in the videos were suggested to minimize visual distractors and maximize attention to the health information in ASL.

If the captions were available on the ASL video, 71% of the sample reported that they read the captions. They also preferred that both ASL and captions be offered as options in the videos.

The lower health literacy group demonstrated a greater preference for one person explaining the health problem. For the adequate health literacy group, some preferred one person explaining the health problem and others preferred an informal group discussing the health problem. Less than 10% of the total sample chose “two people discussing the health problem” as preferred response.

Participants in our study preferred young, skilled signers as narrators and were more willing to watch videos that were shorter. These participants also commented that they gave up after a few minutes of searching for information on the websites. If they found the disorder, they watched the video briefly and replayed the video if they found it difficult to comprehend or had more information than they originally had intended to find which led to information overload.

Participants provided some suggestions to prevent an overload of information on health websites: 1) information about the disease or disorder should be specific, 2) information should be brief, 3) a link should be available for more detailed information about the disease or disorder, and 4) a transcript and/or captions should be made available. Although the participants watched the ASL video, there was a desire to have transcripts or captions accompany the video so they could also read the information in English.

Some participants suggested making contact information available for those who wish to speak with a doctor or ask questions about a medical issue that was not available on the website.

Discussion

The results of this study suggest that simply making a health website accessible in ASL is not enough. ASL-accessible health website must also be user-friendly and easy to navigate. It is important that the ASL users perceive the video content to be easy to understand. This finding was pronounced for ASL users who demonstrated low health literacy level. Future development or improvement to existing ASL-accessible health websites would benefit from usability testing experience and feedback by ASL users with low health literacy level. If the websites with ASL videos are perceived as easy to access and understand by the low health literacy sub-group, then anyone should also be able to utilize information from these ASL-accessible health websites to appraise and use health-related information to maintain and improve health.

In the literature and websites that were reviewed in this study, there was a wide variety in how information was presented in ASL. A discussion format was used in some videos, in which two to four people sat around to talk about a particular health topic. Also, scenarios with actors were used to present health information, such as a person visiting a doctor who could sign, two friends chatting at a cafe, and more. These types of videos tended to be lengthy due to the nature of discussions and scenarios. Feedback from the current study participants suggested that brief and well-focused videos with signers from similar backgrounds would have positive impact on capturing the intended ASL-user audience’s attention and reinforcing knowledge of health information.

For some videos that had dull color or distracting images, participants suggested that the video background should be neutral with an attractive color. Such background should help reduce cognitive load associated with trying to ignore distractors or over-focusing on the signing in videos with dull color. If the videos were too lengthy, ASL users might be inclined to give up and seek information elsewhere that might not be accurate. Follow up health literacy studies with ASL users are needed to examine the impact of video focus and brevity on understanding and utilizing health information to maintain and improve health.

In the current study, participants were instructed to seek out information about a pre-determined health condition and therefore motivated to carry out this task on ASL-accessible health websites. As we consider the intrinsic factors such as motivation and knowledge in the Integrated Health Literacy conceptual model (Sorenson et al., 2012), participants had basic knowledge such as the name of the predetermined health condition and were motivated by the task itself to watch educational videos about the predetermined health condition to successfully accomplish this task. Such health-promotion behavior that drives one to seek out information on the Internet is an important component of health literacy (Office of Disease Promotion and Prevention, 2004 Office of Disease Promotion and Prevention, 2006). If the basic knowledge such as the name of the disease was not given to the ASL user, this user might not have the motivation or reason to seek out and watch educational videos about managing a particular health condition. For example, preventive education on diabetes for ASL users would require an informative video that was visually appealing and engaging enough to encourage sustained attention so the message can have an indirect impact on improving health literacy through health-promoting behavior. This type of informative video might be distributed through various means such as, but not limited to social media and advertising (in the form of Public Service Announcements). The target population that might benefit from such social media and advertising would be those who have low health literacy and are less likely to actively seek out for health information on the Internet. Ideally, an ASL-accessible health website would include both educational and informative videos intended to improve the overall health of ASL users, both for those who are seeking information about a specific health condition and for those who would benefit from informative videos distributed via social networks. Those informative videos would serve as a portal steering the ASL user to the main website source that contains more ASL-accessible information about a variety of health conditions.

An ideal ASL-accessible website would include the recommendations based on the results of this study. Informative videos distributed via social networks is one avenue for alerting and recruiting ASL users with low health literacy to the ASL-accessible websites, which we hope will lead to improving the overall health of the Deaf community. Due to the changing nature of the Internet and new findings in health-related research, it is necessary to continually redesign websites and to update videos to meet the needs of ASL users. The demographics of the ASL community such as age groups and targeting particular topics that affect those age groups should also be considered. For example, the older adults might be more prone to diabetes type 2 and heart disease. Websites that include ASL accessible videos on these topics would benefit from having narrators from the older age group, as the target audience is more likely to be responsive to information presented by their peers. Further study is needed to determine the type of content as well as the depth and breadth of health related information that have direct impact on health-promoting behaviors and health literacy in ASL users. As future research explores whether ethnic sub-groups within the ASL community are more prone to specific health issues and whether there are cultural aspects, new findings from such research should be considered in developing and utilizing the videos to best meet the sub-groups’ health literacy needs.

A limitation in this study is the administration of STOFHLA and survey items in English. Although the survey items were written at the 3^rd grade level, future studies might benefit from making these items available in ASL for those who might prefer this. An additional limitation is that the college student sample might not be representative of the general Deaf ASL population.

For everyone who use ASL, but more so for those with lower health literacy, it is critically important that health websites are not only ASL- accessible but also are free of broken links and poor design at both page and site levels. Good navigation is the cornerstone of a successful ASL-accessible health website, which has strong potential to promote preventive health and improve health literacy among ASL users. This in addition to brief, clear ASL videos can motivate the ASL users to seek health care and discuss any concerns with their providers. When these necessary changes are implemented for future websites, there should be an increase in benefits for ASL users with predicted improvements in health literacy. Therefore, in the long-term, improving health literacy among ASL users can reduce hospitalization, poor disease outcomes, and mortality rates.