Bilingual cancer genetic education modules for the Deaf community: Development and evaluation of the online video material

Abstract

Health information about inherited forms of cancer and the role of family history in cancer risk for the American Sign Language (ASL) Deaf community, a linguistic and cultural community, needs improvement. Cancer genetic education materials available in English print format are not accessible for many sign language users because English is not their native or primary language. Per Center for Disease Control and Prevention recommendations, the level of literacy for printed health education materials should not be higher than 6^th grade level (~11 to 12 years old), and even with this recommendation, printed materials are still not accessible to sign language users or other non-native English speakers. Genetic counseling is becoming an integral part of healthcare but often ASL users are not considered when health education materials are developed. As a result, there are few genetic counseling materials available in ASL. Online tools such as video and closed captioning offer opportunities for educators and genetic counselors to provide digital access to genetic information in ASL to the Deaf community. The Deaf Genetics Project team used a bilingual approach to develop a 37-minute interactive Cancer Genetics Education Module (CGEM) video in ASL with closed captions and quizzes, and demonstrated that this approach resulted in greater cancer genetic knowledge and increased intentions to obtain counseling or testing, compared to standard English text information (Palmer et al., 2017). Though visually enhanced educational materials have been developed for sign language users with multi-modal/lingual approach, little is known about design features that can accommodate a diverse audience of sign language users so the material is engaging to a wide audience. The main objectives of this paper are to describe the development of the CGEM and to determine if viewer demographic characteristics are associated with two measurable aspects of CGEM viewing behavior: 1) length of time spent viewing, and 2) number of pause, play, and seek events. These objectives are important to address, especially for Deaf individuals because the amount of simultaneous content (video, print) requires cross-modal cognitive processing of visual and textual materials. Use of technology and presentational strategies are needed that enhance, not interfere with health learning in this population.

1 Introduction

Health information about inherited cancer and the role of family history in cancer risk for the American Sign Language (ASL) Deaf community, a linguistic and cultural community, is lacking. Between 500,000 to one million Deaf Americans communicate in ASL as their native or primary language in their daily life, and English is their second language (Mitchell, 2006). They are part of a marginalized subpopulation that lacks access to health information in their own language, thus they are more at risk for poor health outcomes compared to the society in general (Barnett, McKee, Smith, & Pearson, 2011). As online and digital access has become increasingly widespread within the broader community, ASL users have particularly benefited from the availability of video streaming content because it allows for visual-spatial communication. Online tools and technologies such as video and closed captioning offer opportunities for educators and genetic counselors to provide genetic information in ASL and in English as a bilingual format to the Deaf community remotely and digitally. The goals of this paper are to describe the development and evaluation of online cancer genetics educational modules (CGEM) for the Deaf community using a bilingual approach. This research is important because there is limited empirical data on best practices for designing online health or genetic counseling-related information for the Deaf community.

Considerable effort is made by health educators to make health materials accessible to the average American English speaker whose level of reading proficiency is between 6^th and 7^th grade, ~11–12 years old (Neuhauser & Kreps, 2008) following the recommended level of literacy for printed health education materials (Centers for Disease Control and Prevention, 2009; McInnes & Haglund, 2011; National Cancer Institute Office of Communications, 2001; Safeer & Keenan, 2005). A general consensus, but still an ongoing debate, is that the average reading level of the US Deaf ASL community has hovered around 4^th to 6^th grade for more than 20 decades (Karchmer & Mitchell, 2003; Traxler, 2000; Zazove, Meader, Reed, & Gorenflo, 2013). This reading level is lower than the US national average due to Deaf individuals’ varied language acquisition experience and, often, their lack of language foundation due to language deprivation at early ages (T. Humphries et al., 2012). Compounded with the level of literacy, half of the Deaf community holds a high school degree or less (Erickson, Lee, & von Schrader, 2015). It is important to note that Deaf individuals are routinely deprived of incidental learning opportunities during their lifetime and it impacts their access to the information readily available to the general population (Hauser, O’Hearn, McKee, Steider, & Thew, 2010). Access to health literacy online is becoming more critical as an alternate source of information from print.

There are materials about cancer and cancer prevention in ASL. As examples, the UC San Diego Moores Cancer Center Research and Training has an extensive collection of DVDs as part of their ASL Cancer Education Program for the Deaf and Hard of Hearing (UCSD, n.d.), an online public resource (DeafHealth.org) provides a list of technical medical terms along with explanations relevant to cancer in ASL, and there are many sites that provide a glossary of ASL science and technology signs (ASL-STEM, 2009; DEAFSTEM, 2017; Johnson & Lang, 2016), but there are no in-depth educational materials about cancer genetics and genetic counseling for the Deaf population. The widely available resources are based on text only and do not necessarily benefit Deaf individuals whose native language is ASL.

To address this informational gap, we developed cancer genetics educational material using a bilingual approach of ASL with English captions (available at: aslcancergenetics.org). The content for the final CGEM product was developed by a team of experts in genetic counseling, cancer genetics, Deaf health education, and ASL. The main objective of the new education material is to inform the Deaf community about the potential benefits of genetic counseling and testing, importance of family health history, risk factors for inherited cancers and how cancer predisposition can be inherited. Accessible and tailored materials for Deaf ASL-users is paramount for preparing for a potential genetic counseling session. In a randomized study comparing the bilingual approach to an English text only version of the material, we demonstrated that the bilingual approach resulted in greater cancer genetics knowledge among individuals with less than high school education, and greater confidence developing a family tree and increased intentions to obtain counseling or testing regardless of educational level (Palmer et al., 2017).

Visually enhanced educational materials have been developed for sign language users with multi-modal/lingual approach, but little is known about factors that affect website navigation, or that facilitate or hinder learning outcomes with the online digital platform. This topic is important to address, especially for Deaf individuals, because the amount of simultaneous content (video, print) used in a bilingual approach requires cross-modal cognitive processing of visual and textual materials. There is some research indicating that Deaf individuals’ health information website navigation is associated with health literacy (Kushalnagar et al., 2015) suggesting that user characteristics may influence users’ online viewing behaviors. Thus, the potential association between user characteristics and users’ viewing behaviors of the CGEM is another key area to address because our intention was to create materials that would appeal and be accessible to a wide audience of ASL-users.

To further understand the benefit of the ASL education material that our team developed, this paper will address two topics. First, we describe the process we used to develop effective user interface for education modules to introduce new genetics knowledge in ASL (CGEM Development). Second, we evaluate the relationship between viewer characteristics and CGEM viewing behavior (CGEM Evaluation), with a hypothesis that there are no substantive differences in viewing behaviors by viewer characteristics.

2 CGEM Development

Development of the CGEM occurred in four phases: 1) focus groups to evaluate models for delivering health education information online, 2) prototype creation and assessment, 3) content development, and 4) full CGEM development (final product). This paper focuses on phases 1, 2, and 4. Details on content development including translation process, education modules content, and narration will be provided in a separate paper.

2.1 Focus Groups

Three focus groups were conducted during Fall 2012 to discuss a variety of aspects for delivering health-related information, particularly genetics and genetic counseling-related content, in ASL in an on-line format. Eligible individuals met the following inclusion criteria: 18 years or older, self-identified as Deaf or hard-of-hearing, ASL-user, and able to meet in person for the study within the county of Los Angeles. Individuals were recruited from a study website, local Deaf organizations using study flyers, and from a previous study on genetic counseling and testing (Boudreault et al., 2010) using a recruitment letter. A total of nineteen Deaf or hard-of-hearing participants (9 male, 10 female) participated in the focus groups. Two groups of participants with average cancer risk (based on self-reported family history) were formed (group 1: n=7, 4 males, group 2: n=6, 3 males). A third group included participants with greater than average risk of cancer based on family history, (n=6, 2 males), defined as at least two family members reported to have developed cancer (breast, ovarian, and colon). The majority of the participants were non-Hispanic Caucasian and college educated, with overall mean age of 51.7 years. One participant in the greater than average risk group had a personal history of prostate cancer. The majority of participants had had genetic counseling or genetic testing in the past, and although the purpose of counseling and testing was not ascertained, this finding may represent a bias from recruiting from our earlier study on genetic counseling and testing for deaf genes. The majority reported identifying with the Deaf community. Supplementary Table 1 provides additional detail on focus group sample characteristics.

The focus groups were conducted with two moderators, a hearing certified genetic counselor with cancer expertise and a Deaf expert in content and language delivery of the materials online. A team of two certified ASL/English interpreters was also present. Each focus group ran for approximately two hours with breaks and participants were compensated $100 for their time. Focus group discussions were recorded with three digital cameras and an audio recording to allow for cross-referencing between the two language modalities for greater transcription accuracy. Audio recordings were transcribed and cross-checked with video recordings for accuracy. Transcripts were reviewed by a team member and comments were categorized by focus group topic. In this paper we focus on topics related to messaging delivery, visual images, and video quality. The focus group study and prototype assessment phase were approved by UCLA, University of San Francisco, and Salt Lake City Veteran Affairs Institutional Review Boards.

Demographic information and responses to a brief written survey assessing familiarity with eight terms relevant to cancer genetics were collected individually prior to start of the focus group sessions. The sessions were divided into two parts: 1) content knowledge, and 2) content presentation. For content knowledge, we discussed participants’ experiences with cancer, the meaning of genetics and family history of cancer to participants, how biology and science classes discuss these topics, how the general media helps participants learn about these topics, Mendelian vs. complex cancer, knowledge of genetic services, genetic testing, genetic counseling for cancer, and genetic services from the Deaf perspective. For content presentation, the facilitators presented different illustrations from a variety of publicly available resources depicting family trees and genetics concepts such as dominant inheritance and the two-hit hypothesis to generate discussion and elicit feedback on the usefulness of the images. In addition, participants evaluated 12 ASL videoclips and approximately 15 images/graphics from publicly available materials aimed at ASL-users. The videoclips described various topics including health-related (e.g. general health, cancer) and financial-related (e.g. IRS) topics, and used different presentation styles such as narration and storytelling (e.g. documentary, presentational). The videoclips were evaluated on messaging delivery (content clarity, ASL quality, English option), usage of visual images (graphics, figures, and illustrative text), and video quality. Participants also were encouraged to provide additional comments on each videoclip.

2.1.1 Genetic Terminology

Overall, focus group participants were generally familiar with and comfortable explaining the terms: “genetic counseling,” “gene,” and “genetic testing”. There was less familiarity with terms “dominant inheritance,” “recessive inheritance,” “mutation,” and “predisposition”. None of the participants was familiar with “BRCA1”. Overall, these participants were uncomfortable trying to explain technical terminology for which they were less frequently exposed. See table 1 for familiarity of genetic terms. These findings were instrumental in developing strategies for describing genetics terms and concepts in the CGEM.

Table 1.

Familiarity and level of comfort of genetics terms. N=12

Genetics Term	Seen term in printa (% yes)	Mean level of comfort explaining term to a friendb (1=low; 10=high)
Genetic counseling	91.7%	7.17 (SD=2.62)
Gene	75%	7.1 (SD=3.45)
Genetic testing	75%	6.1 (SD=3.02)
Dominant inheritance	58.3%	5.0 (3.29)
Recessive inheritance	41.7%	4.0 (3.52)
Mutation	33.3%	3.6 (3.5)
Predisposition	25%	3.3 (2.98)
BRCA1	0%	1.1 (0.32)

^aPhrasing of item on the survey: Have you seen the word _____ in written materials?

^bPhrasing of item on the survey: How comfortable would you be explaining this word to a friend?

2.1.2 Messaging Delivery

Messaging delivery was evaluated on three attributes: message clarity, ASL clarity, and English option. There was general consensus among focus group participants that, to learn new concepts, it is essential that the narrator be Deaf, a professional presenter with considerable experience to deliver ASL content in front of the camera, and knowledgeable about the topic. While viewing example videoclips, the participants were able to recognize presenters who are not deaf: “I thought it was an interpreter. I prefer it to be someone who is deaf.” (FG1-1), and “It was obvious it was someone who was a hearing interpreter.” (FG1–2)

It was considered most important that ASL usage conform to that language’s grammatical structure and that ample usage be made of spatial-syntax features inherent in ASL. Signed content that aligned with English based or linear spoken grammar was considered inappropriate. The emphasis on the proper use of ASL was important for the participants, with several participants commenting on how English interfered with ASL grammar in some of the example videoclips: “Too English. It’s in English word order.” (FG1–3), and “This is not very ASL at all.” (FG1-1)

Participants recommended that information be broken down into modules averaging five minutes each to facilitate users’ attention. The increments and the duration were important for facilitating more effective retention of new materials as shown by these comments: “… to get better results you need to go 5 minute increments. Meaning you do something and then the second half would be another 5 minutes…” (FG3–4), and “One thing that we learned was with 20 minutes, your eyes cannot watch something for more than 20 minutes. That’s key. That’s #1.” (FG3–4)

In addition, as illustrated by the following exchange, a video should apply a consistent and normal pacing with frequent pauses so the viewer can process the information while the graphics or text are being shown on the video.

• -“Did you like back and forth between the pictures?” (Facilitator)
• -“I liked it. You had time to look back and forth.” (FG3–4)
• -“It was easy on the eyes. I was able to adjust. Your eyes have to be able to adjust.” (FG3-unknown)
• -“For too long they have been boring. This was just perfect. This was just right. You could see the explanation with the picture that came up, going between the two. I could watch that for a long time, I could.” (FG3–5)

Limited use of English text prompts, coupled with balanced delivery between ASL and English, were considered acceptable to support the messaging and to recall new learned concepts, as shown in the following quotations: “Simple words were put up there for emphasis. I thought it was fine.” (FG3–6), and “I think this is fabulous. For me, looking up and down at the signs I didn’t have to pick one. But with the captioning next to it I like that the sign and all that. I only had to look at one area within the video. Whenever she was signing the word came up on the side of her head. I thought it was perfect.” (FG1–7)

Some participants commented that captioning speed should be consistent and time-aligned with the narrator using ASL: “The words themselves didn ‘t always match. I had a hard time predicting.” (FG1–2) Also, participants preferred captioning that they could turn on and off rather than subtitles which do not offer this feature: “I know the captions were hard to see but if we have the on and off, we could turn off the captions, but that was the plus the positive about this” (FG3–5)

2.1.3. Visual images

Overall, the focus group participants were comfortable with the visual images showing basic pedigree concepts/diagrams and family relationships. As shown by the following quotations, a simple schema was preferred with identifying shapes for gender identification (showing faces before the symbols) and different colors for type of cancer, titles (including age) to denote relationships among family members, display of both sides of the family to denote paternal/maternal family members, and an indicator to identify the status of the family members who had cancer (living or passed away). “… Both aunt and uncle should show both spelled out. And then on the grandfather side you should say aunt and uncle so that you have two on each side.” (FG2–13); “I’m wondering what is the order of brothers and sisters. Who comes first, second, third, fourth. There should be a birth order. That’s important for the genealogy also.” (FG3–6); “I prefer the other diagram better with the actual picture of the people in the box” (FG1–7); and “You can use different colors to identify different diseases. Like blue for that cancer, and a different one for another” (FG2–8)

Participants preferred to avoid complex pedigree nomenclature and placed an emphasis on simplicity, to allow them to grasp the main information: “If it was kept simple, it could be easier to understand rather than it being overly complicated and busy, something more simple.” (FG2–9) We used the same simplicity concept when developing other visual images to introduce or explain new concepts such as DNA and the process of mutated genes resulting in cancerous cells.

2.1.4. Video Quality

Video quality deals with considerations for video production and video editing. Based on focus group feedback on the example videoclips we drew the following conclusions:

1. There was a range of preference for background color but a clear contrast is the main element: “The white background was too bright. It was too contrasting. He looked like a cut out. It was too hard to focus on him, the signer.” (FG1–10), and “I think that it was a little too dark. It was hard to see the signs and the fingerspelling on the background as well as the color of his shirts.” (FG1–11).

2. The narrator should wear solid color clothing contrasting with background color to allow view of the hand and finger movements in detail. “I don’t like the clothing. The green. It’s hard to look at and see her hands in the background. I think she needs a darker background, darker clothing to contrast to her skin tone.” (FG1–10)

3. The narrator should avoid placing the hands over the face because the skin color overlap will make it difficult to understand the message being signed. “She was signing right in front of her face. It should have been on the side of her body.” (FG3–12)

4. The video speed must be appropriate to capture all the sign movements without producing any blurry images. The smoothness of the signing is critical as indicated by several participants:

   • -
     “Their signing wasn’t smooth. It wasn’t clear. It was choppy. Their signing, the camera. There’s a picture of a fade away.” (FG3–4)
   • -
     “Blurry?” (Facilitator)
   • -
     “There was a trail from the sign.” (FG3–5)
   • -
     “It was the speed. The camera speed wasn’t capturing the image.” (FG3–4)

2.2. Prototype Development and Assessment

Focus group participants’ evaluation of the 12 video clips yielded productive input to produce a suitable CGEM prototype to evaluate how effectively the content could be delivered in a YouTube video. The content addressed how to create a family tree and identify risk factors for inherited cancer. The prototype video included interactive features (narration, visual images, textual content, and quiz). Because we planned to use a bilingual approach (ASL and English) for content delivery, for this phase, we created two versions of the prototype video, differing in the method to implement the bilingual approach: ASL video with English text that accompanies the video below the viewing frame (Sample A: video + text), and ASL video with English caption where the information is delivered simultaneously but the captioning can be turned off by the user (Sample B: video + caption).

The content and delivery methods were evaluated by a total of 43 individuals: 1) two Deaf and two hearing project consultants with expertise in health education, cancer genetics, genetics, or genetic counseling, each of whom received links to view all three sample videos and a request for feedback, 2) nine participants representing all three original focus groups, each of whom received links to view all three videos and a survey to complete, and 3) 30 Deaf community members attending the MATA Expo in Fall 2012 (mataexpo.com), a popular event held in Southern California that typically attracts Deaf individuals, where we purchased exhibit booth space to invite expo attendees to participate in the prototype evaluation. It was our intent to randomly assign community member volunteers to one of the three prototype versions to view and evaluate using a brief survey. However, space constraints and general crowding at the event, as well as disinterest in evaluating an English text version of the prototype resulted in a non-random assignment to Sample A and Sample B of the prototype (n=14 Sample A; n =16 Sample B). The survey completed after viewing the videos assessed comprehension of the educational material, viewer engagement and motivation, and design features of the prototype (Supplementary Table 2), and encouraged additional feedback. Volunteers in this phase of the work were not compensated.

2.2.1 Learning New Genetic Concepts

Based on the questionnaire assessing participants’ understanding, the material conveyed in the prototype was not fully comprehended. A quotation from one of the participants illuminated the need to develop simpler messaging for the final product. “While I understood clearly what the information [was] and how it is being presented; for others it may be helpful to simplify the language used and reduce the amount of information (more direct)” (FG-2)

As shown in Supplementary Table 2, the percent of correct responses to identical or analogous questions embedded as quiz questions in the final product is generally higher than the percent correct on the prototype survey, suggesting that the final product was more successful at promoting comprehension of the material. The majority of prototype participants were motivated by the information to become interested in making their own family tree and to tell their family and friends about the video, a result that was observed in an evaluation of the final product as well (Supplementary Table 2).

2.2.2 Messaging delivery

In terms of design features, the majority of participants evaluating the prototype indicated that quizzes embedded in the videos helped them understand the content (note that the survey comprehension questions were different from the quiz questions embedded in the videos) and only a minority found the quiz questions to be annoying. Participants reported that the video length (2 minutes 50 seconds), and the pace of ASL delivery were good. See Supplementary Table 2 for details on participant responses on messaging delivery.

In open-ended comments, participants noted the clarity of the narration delivered in ASL by a Deaf signer, and the narrative style was described as essential in drawing attention to the program content.

• -“Yea, and it clear with ASL to describe in family tree and with picture too” (FG-5)
• -“Nice move to sign very smooth” (ComA-1)
• -“Easily understood” (ComB-3)
• -“Presented by Deaf [person] in ASL with captioning. Good narrative story in the beginning, can get people’ attention.” (consultant 3, Deaf)

At least one participant preferred the narrator to engage with the viewer more interactively, which could be accomplished by splicing the video into shorter segments within the module: “May shorten the time while [the narrator] is not showing on screen.” (consultant 3, Deaf)

In general, prototype participants reported that the English text/closed captions were easy to read (Supplementary Table 2). When viewing Sample B, some participants preferred to watch only the ASL: “I do not like the caption on the video at all. Too confusing not knowing which to look at the video or read the captions. Much prefer just the video in sign language only.” (FG-7). Overall, there was a slight bias in favor of the captioned version that was observed through survey responses (Supplementary Table 2) and open-ended comments: “Overall, the content of video looks so great and I like it very much. As I believe more people like Group B, I like this Group B since this group include video with caption, easily to read and learn on what family tree means” (consultant 3, Deaf).

However, some participants preferred the version with English text (Sample A). A hearing consultant indicated that a positive feature of this version is that it provides a complete transcript: “It’s nice to have the transcript there on the right, to have the option to read it through, separate from watching the video.” (Consultant 1, hearing)

In their open response feedback, some participants emphasized that ASL narration and visual graphics needed to be presented alternately, not simultaneously, for greatest clarity. “I like it as well but it’s too much [of a distraction] to [view the] ASL and picture all [at] the same time.” (ComB-11)

In addition, more time was needed to look at the graphics or bulleted text shown in the video in order to process the information: “Enlarge family history tree and then each pointed square and circle while the narrator is not on screen.” (consultant 3, Deaf) “When bullet appears on screen, have the narrator pause briefly as each new bullet appears to give viewers time to read it.” (consultant 1, hearing)

However, a long pause or gap between narration and visual images was discouraged; thus, determining an ideal timing was considered important: “May shorten the time while [the narrator] is not showing on screen.” (consultant 3, Deaf).

Maximizing the spatial depiction and the narrator pointing to the graphics or the English text to guide the user was recommended.

“While graphics are helpful, it may be even more helpful to perhaps highlight the areas being “pointed out” while [the narrator] discuss a specific area/part.” (FG-2)

“The narrator’s ASL could have specifically described Anna’s family tree (such spatial representations are a particular strength of ASL), but instead his signing describes a generic pedigree. This gave me a feeling of disconnect.” (consultant 2, Deaf)

It was suggested that the concepts, such as technical and genetics concepts, be presented and explained first, before showing the graphics to facilitate recalling the content. “My advice to you, you need ASL to be [shown] clearly first then show picture. I think it would help you.” (ComB-11)

This is known as the sandwiching approach: explain new concept for the first time, depict the graphic/text, and briefly review. This process also would apply to use of English fingerspelling genetics-related terminology, where the new concept word would be displayed after it is fingerspelled to help the user recall the terminology. The “sandwiching approach” is commonly used in the bilingual education for deaf children (T. Humphries & MacDougall, 1999).

2.2.3 Video Quality and Visual Images

Of note, the prototype video used Chroma key screen which, by including a grey background during the post-production caused blurry lines or feathers around the fingers due to low frame rate and the speed of the signer. The blurriness may have affected the level of comprehension of the material for some participants. This underscored that the speed of the video must be appropriate to capture all the sign movements without creating any blurry images.

Focus group and community participants reported that they benefited from the use of the graphics or pictures presented in the prototype (see Supplementary Table 2) and they were described in the open-ended comments as making the video more interesting and compelling.

“Graphics make the video more interesting and compelling.” (FG-2), and “Good video with proficient pictures that depict various cancers within a family.” (FG-4)

Although the use of graphics was considered helpful, some users indicated that they would have preferred animated graphics to the still images in the video: “Be a bit more animated with graphs” (ComB-9), and “Need to add high visual communication using animation” (ComB-4)

However, the Community group participants were also more likely to report that the graphics were distracting than the Focus group participants found them (Supplementary Table 2). One possible explanation for this discrepancy is that the Focus group participants had already learned new concepts from previous focus group sessions making the cancer genetic related concepts easier to follow, and suggesting that a first time exposure is critical for the learner to follow complex graphics, especially the family tree.

2.3 Final CGEM Product

Our project team’s expertise, in combination with two rounds of evaluation from focus group and prototype evaluation resulted in the production of a 37.3 minutes long video with captioning with on/off feature comprised of 6 modules averaging 5–6 minutes each and a total of 82 scenes. The content modules are as follows. Module 1: “Introduction” explains that although the focus is on hereditary breast, ovarian, uterine, and colon cancers, the information is applicable to other hereditary diseases. Module 2: “Creating a Family Tree” explains how to develop a pedigree. Module 3: “Risk Factors for Inherited Cancer” identifies what a high risk family tree for cancer might look like. Module 4: “How Cancer is Inherited” focuses on concepts of genetic predisposition and autosomal dominant inheritance. Module 5: “Role of Genetic Counseling and Testing” explains the process of genetic counseling and testing, including the purpose, benefits, and possible outcomes of genetic testing. Module 6: “Review” is a brief review of the modules. Two to four quiz items are included at the end of each module, and a correct answer with explanation is provided if incorrectly answered. Although it is typical to translate English source material into ASL when developing health education materials (Pollard, Dean, O’Hearn, & Haynes, 2009), doing so too often impedes the ability to generate an authentic discourse around the visual materials (Jones, Mallinson, Phillips, & Kang, 2006). To facilitate authentic discourse, CGEM content was developed by team members using ASL as the source text, and then translated into English for both English captions and the English text version evaluated in Palmer et al (2017). The Flesch-Kincaid reading level for the English captioning using Office Word was calibrated for 7^th grade (13 year olds) or lower level.

Information was provided in each module by a Caucasian male Deaf native signer, a member of the project team. He worked jointly with another project team member, a native signer who is a bilingual teacher at a deaf school in California. We used several strategies to ensure the content accuracy and clarity of messaging to optimize acquisition of material for ASL-users. To address feedback about the importance of simple messaging we minimized the use of technical signs by substituting with visual description and maximum usage of ASL depiction to explain each new concept. Also, we incorporated more time and attention to explain genetics concepts, such as gene, mutation, and predisposition with supporting visual content, typically using illustrative graphics. To promote appropriate timing to align with the visual content presented in the video, the narrator points to the visual content and pauses a few seconds to allow the viewer the look at the visual content. One goal of the CGEM is to empower Deaf individuals to collect and understand their family health history. In our Review Module, we encourage Deaf individuals to prepare and collect the information to assess their own risk and bring all the information to the visit with a genetic counselor. Although many genetic counseling clients would not be familiar with the conventional symbols used by genetic counselors, we wanted to include a visual image of family and provide our targeted audience with some tools for collating their family health history information. Based on focus group feedback about various ways to display family history, we concluded that conventional symbols with labels, presented first as faces that then faded into the symbols, would address much of the feedback we received. The final content also was reviewed with members of the research team to ensure accuracy of content in both ASL and English.

The video was filmed with 60 frames per second full HD (1080p) with solid light white background with the narrator wearing a solid and dark colored shirt. The CGEM makes use of a number of different angles: close center for emphasis or connecting with the viewer, mid center for general messaging, left or right side next to graphics or texts, and walking into the frame for introductory scenes. Between scenes, different angles or close ups with contrast cut rather than the commonly used dissolve technique or continuous delivery was used to create an interactive effect. We filmed close up as much as possible without impeding or cutting the hands from the edges of the video to exhibit the grammatical facial markers.

Gaps between scenes or clips were monitored to avoid long pauses. Caption delivery aligns with the ASL narration with the default option set at ‘on’ and the user is able to turn the captions off at any time. The video dimension is large enough (i.e. at least 1280 × 720) or scalable based on the user’s screen resolution to maximize the viewing experience. The website navigation guide allows the user to move between modules and quizzes, and page by page, with “back” or “next” buttons. The visual images were designed to enhance the visual representation of the complex concepts (Refer to video, and Figures 1 and 2 in the supplemental file).

A Deaf software developer created specific coding using Ruby on Rails, a web application that supports databasing, web service, and web pages using web standards (e.g. JavaScript, XML, HTML) along with Vimeo.com as a video delivery platform to create the website. In addition to providing educational content, the website also records digital trace data (Howison, Wiggins, & Crowston, 2011) of user’s viewing behaviors in terms of keystroke and cursor activity and movement within the CGEM (bilingual version) with the web events monitoring feature, capturing duration of viewing, user-browsing activity (closed caption with off/on function; pause, play, and seek (fast-forward/rewind)). A MySQL database records the trace data using event listener in the JavaScript code.

3 CGEM Evaluation: Is there an association between viewer characteristics and viewing behavior?

3.1 Participants and Procedures

Data to address this question were collected as part of a larger study comparing the effectiveness of a bilingual CGEM (ASL with English captions) to a monolingual CGEM (English text only) (Palmer et al., 2017). Participants for that study were recruited nationally, between November 2013 and May 2014, through deaf and hard-of-hearing clubs and organizations, community events, and from the participant database of a previous study involving genetic counseling and genetic testing for deaf genes (Boudreault et al., 2010). Recruitment materials indicated that the study was available online and focused on learning about family health history. Deaf/hard-of-hearing ASL-using individuals at least 18 years old were eligible to participate.

See Palmer et al (2017) for details of the larger study. Briefly, participants completed the study online. They received a link to a pre-test survey requesting demographic information, and assessing cancer genetics knowledge, attitudes toward genetics and genetic counseling, and language proficiencies. Upon completion of the pre-test survey, they received a link to either the bilingual or monolingual CGEM, and upon completion of viewing the CGEM, participants received a link to a post-test survey assessing cancer genetics knowledge, attitudes toward genetics and genetic counseling, and opinions on the CGEM. Those individuals who completed the study received a $40 gift card. The study was approved by UCLA and Gallaudet University Institutional Review Boards. An information sheet was used to facilitate the consent process and all participants gave implied consent upon opening the first survey link.

In the larger study, 100 participants were assigned to the ASL-English bilingual version and 50 were assigned to the English text only monolingual version. The focus of this paper is on 97 of the 100 participants assigned to the ASL-English bilingual version for whom useable trace data are available (two individuals withdrew prior to viewing the CGEM, and one individual had outlier trace data). Refer to Supplementary Table 3 for a complete description of sample characteristics for the CGEM evaluation.

3.2 Measures

Dependent variables

The two dependent variables (viewing behaviors) of interest are number of pause, play, and seek events (labeled as PPS) and time spent viewing the CGEM (labeled TIME). These variables were captured as trace data described in Section 2.3. Both are continuous variables.

Independent variables

Demographic data were collected from participants, including gender; age; race/ethnicity; education level; and personal or family history of breast, ovarian or colon cancer. Two variables measuring cancer family history were constructed (Cancer History: yes/no, where yes applies if at least self or one family member reported to have had cancer; and Cancer Risk: average/greater than average). ASL proficiency was assessed using the TGJASL test (Boudrealt, 2006; Boudreault & Mayberry, 2006), and English reading proficiency was assessed by asking participants to self-report their level of ease reading English on a scale of 1–10. Participants’ cancer genetic knowledge score at pre-test was assessed as the sum of correct responses to 25 true/false items.

3.3. Data analysis

T-tests were performed to assess association of the dependent variables with dichotomous independent variables: gender (male/female), ethnicity (white/people of color), education level (up through high school diploma/beyond high school diploma), breast, ovarian or colon cancer family history (yes/no; or average risk/greater than average risk). Pearson’s correlations were performed to assess association of the dependent variables with age, ASL proficiency score (TGJASL), English reading self-reported score, and pre-test cancer genetic knowledge score. Statistical significance was set at p ≤ 0.05.

3.4 RESULTS

The actual complete viewing time, excluding the quizzes, is 37.3 minutes, and in the sample of 97 individuals, the average time spent viewing the CGEM was 48.4 minutes (SD 18.7). The expected minimum number of PPS events is 6, reflecting the number of clicks to play each video. In this sample, the median number of PPS events (play, pause, and seek) was 17, with average of 49.2 (SD=84.0). The individual excluded from analysis as an outlier had 2,555 PPS events, which was orders of magnitude greater than the other participants. Due to non-normal distribution of PPS events, we performed a log transformation of PPS (LogPPS) before doing further analysis. We checked for collinearity between LogPPS and TIME. As expected, as the number of PPS events increased, so did the amount of time viewing the CGEM (r=0.53, p < 0.0001). However, because only about 25% of the variance in one variable is explained by the other variable, we investigated both dependent variables separately in subsequent analyses. Table 2 presents details of these analyses, described briefly below.

Table 2.

Results of tests examining relationship of viewer behavior characteristics by their demographic characteristics

		LogPPSa			Timeb
Variable	N	Mean (SD)	Difference (95% CI)	p	Mean (SD)	Difference (95% CI)	p
Gender			−0.12 (−0.34, 0.10)	0.28		−9.41 (−16.96, −1.86)	0.02*
Males	37	1.28 (0.47)			42.55 (21.04)
Females	60	1.40 (0.56)			51.96 (16.20)
Education			0.04 (−0.19, 0.27)	0.72		−1.40 (−9.58, 6.78)	0.73
High school or less	30	1.38 (0.55)			47.4 (18.40)
College	67	1.34 (0.52)			48.8 (18.90)
Ancestry			−0.09 (−0.32, 0.14)	0.43		1.85 (−6.19, 9.90)	0.65
People of color	32	1.28 (0.56)			49.17 (19.81)
Non-Hispanic Caucasian	63	1.37 (0.51)			47.31 (18.05)
Cancer history (self-report)			0.07 (−0.15, 0.29)	0.52		−0.85 (−8.15, 6.81)	0.83
No	41	1.39 (0.58)			47.88 (21.60)
Yes	56	1.32 (0.48)			48.73 (16.44)
Cancer risk			0.10 (−0.13, 0.33))	0.38		−3.50 (−11.58, 4.58)	0.39
Average	66	1.38 (0.54)			47.25 (20.34)
Greater than average	31	1.28 (0.50)			50.75 (14.48)
		rf (95% CI)			r (95% CI)
Age	97	44.7 (14.6)	0.12 (−0.08, 0.31)	0.24	44.7 (14.6)	0.05 (−0.16, 0.24)	0.66
TGJASL scorec	97	0.75 (0.15)	0.05 (−0.15, 0.25)	0.62	0.75 (0.15)	−0.03 (−0.23, 0.17)	0.77
English reading ease scored	94	7.95 (1.85)	−0.14 (−0.33, 0.07)	0.18	7.95 (1.85)	−0.20 (−0.39, −0.001)	0.05*
Pre-test genetics knowledge scoree	97	18.44 (3.31)	0.04 (−0.16, 0.24)	0.71	18.44 (3.31)	−0.003 (−0.20, 0.20)	0.98

^*p < 0.05

^aLogPPS = log of the sum of play, pause, and seek clicks made while viewing the Cancer Genetics Education Material

^bTime = number of minutes viewing the Cancer Genetics Education Modules

^cTGJASL-R: ASL Grammar Judgement Task. >0.5–1 reflects increasing grammatical accuracy; 0.5 reflects random guessing; <0.5 reflects systematic bias

^dself-rated on scale of 0 (low) – 10 (high)

^ebased on 25 true/false items (range 0–25)

^fr is the Pearson’s correlation coefficient

3.4.1 Demographic variables

There was no significant association between logPPS and gender (p= 0.28), education level (p= 0.72), ethnicity (p= 0.43), or age (p= 0.24). Consistent with Palmer et al (2017), there was no significant association between TIME and education level in this slightly reduced sample analyzed here (p= 0.73). In addition, there was no significant association between TIME and ethnicity (p= 0.65) or age (p= 0.66). However, there was a statistically significant association between gender and TIME such that women spent more time viewing the CGEM than men (p= 0.02).

3.4.2 Family History of Cancer

There was no significant association between LogPPS and cancer history (p= 0.77), or cancer risk (p= 0.38). There was no significant association between TIME and cancer history (p= 0.83), or cancer risk (p= 0.39).

3.4.3 Language

There was no significant association between LogPPS and TGJASL scores (p= 0.62), or self-reported English reading ease scores (p= 0.18). Similarly, there was no significant association between TIME and TGJASL scores (p= 0.77). However, there was a significant negative association between self-reported English reading ease scores and TIME (r= −0.20, p= 0.05). This result indicates that individuals who reported greater English reading ease spent less time in the CGEM than individuals who reported lower English reading ease.

3.4.4 Knowledge

There was no correlation between pre-test cancer genetics knowledge and LogPPS (p= 0.71) or TIME (p= 0.98).

4 Discussion

This paper is the first to describe the development of online genetic counseling-related materials using a bilingual approach for the US Deaf ASL community and evaluate their appeal and accessibility to a wide audience of ASL-users using digital trace data. Sign language has mechanisms to deliver complex concepts, including genetics concepts, as efficiently as other languages.

Sign language relies on the use of spatio-visual modality that allows the signer to explain complex concepts such as genetics concepts where the terms are not commonly used in the mainstream. The language allows indexing (to point in the space for anaphoric reference in mental space or literal reference in real space), enumerating a list of ideas, and depicting verbs with iconic representation using the signer’s spatial-reference to provide information (Liddell, 2003), and gestures to represent complex concepts (Meir, Cormier & Quinto-Ponzos, 2002; Sandler & Lillo-Martin, 2006). The CGEM final product was made possible with insightful inputs coming from a wide range of members from the Deaf ASL community, genetic counselors, and language experts. The data presented from this study allowed us to establish a framework for developing genetic education materials that are solely delivered digitally on-line. This proposed framework will also have a wider range of application for developing accessible genetic counseling-related and other health education materials for the Deaf ASL community.

Design Features

The highlight of the focus group findings is that the level of comfort with a genetics term declines when the concepts become more complex or are unfamiliar. Thus, materials should incorporate more time or attention to explain those concepts with supporting visual content, either in English text or illustrative graphics. Focus group participants also reported that the effectiveness of the messaging delivery in ASL depends on who is narrating and in the way the message is delivered. As examples, the narrator should use the spatial-syntax grammar inherent to ASL instead of using an English based syntax structure. The content should be delivered by a fluent Deaf narrator who is knowledgeable about the content and is mindful of the audience with varied level of education and linguistic proficiency. The narrator should put emphasis on visualizing and describing complex genetic concepts along with the appropriate use of depiction to reach less fluent English users. Use of supporting graphic content also benefits the learners to retain the learned concepts and the technical terminology. The visual presentation must be carefully timed to avoid any overlap between the narrator and the images or text to thwart the multiple cognitive efforts. The inclusion of closed caption is a highly desirable feature and it should be offered with on/off option instead of a subtitle that is ‘burned’ on the screen. In addition, a transcript should be included to allow users to refer to the video content for an immediate and holistic reference of the material instead of having to go back to a specific time in the video content after having viewed the video. The transcript will also serve as a reference for hearing family members (as shown by the interest of the hearing consultant to have a transcript) and DeafBlind individuals who often prefer to have a transcript for large text viewing.

For an effective video presentation, the sign language narrator’s hands should be clear and finger movements should be crisply captured. The narrator should wear a contrastive color to highlight the hands and grammatical facial expression. Video editing should focus on producing dynamic and engaging material that is delivered in “chunks” of 5–6 minutes or less for better attention and retention from the users.

Audience engagement is critical for the success of the material delivery. To facilitate audience engagement, the user interface must be simple and should not be impeded by other visually distracting elements. The user interface should allow the user to have an intuitive engagement with the educational modules, with the ability to navigate the site effectively by guiding them step-by-step along the modules. Including quizzes between modules can benefit learners in two ways; it allows them to process and review the learned concepts in the modules, and it gives a short break between modules from the cognitive efforts of attending to each module. Overall, those findings align with the conclusions of a study investigating a Deaf college student sample that developing ASL-accessible health resources is not sufficient if we do not implement appropriate features for web usability and content understandability to engage Deaf users (Kushalnagar et al., 2015). Moreover, our study extends this principle to the general Deaf population.

User Viewing Behavior

To our knowledge, there is no other study that has evaluated the user web behavior online in ASL using systematic digital trace data tracking. Doing so allowed this study to provide insight into the end-user activity while viewing the CGEM. Overall, there were no statistically significant associations between viewing behavior (amount of time spent viewing CGEM or number of play, pause, or seek clicks) and participant characteristics. The one exception is that we found that women spent more time viewing the CGEM than men. This result suggests that women may be more motivated or attentive to learn about breast, ovarian, and colon cancer genetic information compared to men. Although this interest may be because breast cancer occurs more commonly in women, and ovarian cancer occurs only in women, we had hoped that men would be equally motivated or attentive to learn about inherited forms of these cancers since the genetic predisposition can be passed through both men and women to their children. This gender difference warrants further attention in future studies to assess if this result would be the same with other genetic diseases that have no discrimination between genders.

In this study, we found that participants had the same web behavior regardless of their level of education and their ASL proficiency. However, it is important to note that the relationship between English reading ease and TIME is significant (r= −0.20; p= 0.05) in this sample where the average self-reported English reading ease score is 8 out of 10 (SD 1.9), which is likely higher than the 4^th-6^th grade average reading comprehension level of the general Deaf population. This indicates a possibility that in a larger sample of Deaf ASL-users we would find a greater differential time response as a function of self-reported English proficiency. Of note, Palmer et al (2017) found that the lower education group had lower pre-test genetics knowledge compared to the higher education group. This finding might suggest that the lower education group would demonstrate a significantly increased level of activity to process the new content in the CGEM, analogous to the finding by Kushalnagar et al (2015) that Deaf individuals with lower levels of health literacy reported more difficulty navigating health information websites. But it was not the case in our study as measured by our participants’ online activity, suggesting that the CGEM design does not require differential viewing behavior on the basis of users’ education or pre-existing knowledge in order to produce increased genetics knowledge.

Finally, one might assume that individuals who have a personal or family history of cancer or the potential risk of developing cancer would have more web activity due to the possible undesirable outcomes that may impact their lives. However, we found no significant differences in viewer behavior as measured in this study and their family history cancer risk, again suggesting that the CGEM engages all participants in a manner that is essentially the same.

One potential concern with our study is that our sample size could be too small to detect meaningful differences in viewing behavior. With 97 participants, we have 80% power to detect a difference of 25% or greater in logPPS (a 35% change in PPS on the untransformed scale) between any of the dichotomized variables such as education level, cancer history, or ethnicity. We also have 80% power to detect a correlation of 0.28 or greater with logPPS and a continuous variable such as age. Likewise we have 80% power to detect a difference of 11 minutes or greater in viewing time between any of the dichotomized variables or a correlation of 0.28 or greater with viewing time and any of the continuous variables. Thus, although we would fail to detect small to moderate differences in CGEM viewing behaviors by participant characteristics, we have sufficient power to detect meaningfully large differences which, had they existed, could be indicative of accessibility barriers or lack of appeal to a particular group of ASL users. In other words, the overall absence of differences in viewing behaviors as measured in our study suggests that we were successful in our goal of developing materials that are sufficiently neutral to differences in viewer demographic characteristics, general education level, cancer history, and pre-test cancer genetics knowledge.

Limitations

This study used digital trace data, specifically, viewing time and number of plays, pauses, and seek clicks, to measure user engagement with the CGEM. It is possible that these measures are not sensitive enough to effectively measure user engagement, however, frequency of activity (ie, counts) and time on task are good indicators of the extent to which individuals use a tool (Gašević, Dawson, & Siemens, 2015).

Future Directions

The process of developing the CGEM provided much-needed details regarding the necessary framework for designing and producing appropriate genetic counseling materials for Deaf ASL users. The use of widely available technology at low cost facilitates the educational opportunities for the Deaf community with varied background. We believe that implementation of the best practices as outlined in this paper will allow the end user to gain a sense of control and confidence when it comes to learning new complex information. Therefore, sign language users will come to genetic counseling with more background knowledge, enhancing the clinical counseling experience. Using video technologies for providing genetics education in the clinic setting as an adjunct to face-to-face genetic counseling can also be considered, though additional research would be needed to evaluate the success of this approach. In addition to this, the sign language interpreters who work in genetic counseling settings will also benefit from viewing the online materials in order to apply the most effective ways to communicate the genetic information in ASL based on various important linguistic features (i.e. lexical, grammar, and discourse).

The CGEM content messaging development was solely based on the language expert’s experience in producing and translating educational materials. The messaging and translation for both ASL and English warrants further investigation to determine how the level of grammatical difficulty in ASL is being measured in conjunction with the formalized measure of index of English readability, e.g., Flesch-Kincaid reading level. In addition, this study did not include a measure of health literacy, which is an important factor to consider when designing and evaluating the effectiveness of online health information in ASL. Future research should include health literacy in conjunction with ASL proficiency and education level of participants to gain a more robust understanding of the critical factors for developing effective educational materials for the Deaf community. To facilitate such research, health literacy measures should be translated into ASL (McKee et al., 2015).

This study also did not include a group of DeafBlind individuals to evaluate the effectiveness of the CGEM. There is no study on the preferences of DeafBlind individuals when it comes viewing videos, such having specific color background (i.e. dark vs. light) or the delivery of the visual text and graphics. However, there are several best practices for delivering materials in ASL online based on DeafBlind community forums and workshops, and there is a wide range of preference depending on their needs. In general, the recommended features are to include an accessible transcript in various formats that provides video and image descriptions including tagged structured content. The contrastive feature should be carefully applied in diagrams, background colors, preferably to have multiple color option from the same video, the use of plain long-sleeved shirts with a clear skin contrast and no facial hairs. The inclusion of DeafBlind individuals for future study is critical for a comprehensive health education material in ASL. Finally, there is a dearth of educational materials that are being produced and presented by ethnically diverse group of experts, prompting careful consideration for inclusiveness for future material development.

The increasing discoveries in the realm of genetics lead to ongoing needs to develop educational materials for the general population. When it comes to producing materials for the Deaf ASL community, there is a considerable delay due to the reliance on technology that is more complex compared to the production of print text. This discrepancy puts the content material at risk to be outdated, and warrants a wider cross-collaboration among the genetic counselors as content experts and Deaf studies and language experts to develop and produce materials in a timely fashion. A uniform mechanism is needed to produce the materials on a regular basis in varied genetic counseling topics and in health education in general.