Informational content, literacy demands, and usability of websites offering health-related genetic tests directly to consumers

Abstract

Purpose

As direct to consumer (DTC) genetic testing becomes more available, a diverse group of consumers, including those with limited health literacy, may consider testing. In light of concerns raised about DTC genetic testing, this study sought to critically examine whether the informational content, literacy demands, and usability of health-related DTC websites met existing recommendations.

Methods

A content analysis was performed on 29 health-related DTC websites. Two coders independently evaluated each website for informational content (e.g., benefits, limitations), literacy demands (e.g., reading level), and usability (e.g., ease of navigation).

Results

Most sites presented health conditions and some markers for which they tested, benefits of testing, a description of the testing process, and their privacy policy. Fewer cited scientific literature, explained test limitations or provided an opportunity to consult a health professional. Key informational content was difficult to locate on most sites. Few sites gave sample disease risk estimates, or used common language and explained technical terms consistently. Average reading level was grade 15.

Conclusion

The quality of informational content, literacy demands, and usability across health-related DTC websites varied widely. Many users would struggle to find and understand the important information. In order for consumers to better understand the content on these sites and evaluate the meaning of the tests for their health, sites should lower the demands placed on users by distilling and prioritizing the key informational content while simultaneously attending to the reading level and usability elements. In the absence of regulation compelling such changes, government agencies or professional organizations may need to increase consumer and provider awareness of these issues.

INTRODUCTION

Since the completion of the human genome project in 2003, there has been much discussion of a new era of personalized medicine that will transform both health care and the way we interact with our genetic information.^{1, 2, 3} While the promise that genomics will revolutionize modern medicine is likely still some years off in the future, another revolution has taken place outside the doctor’s office: the promotion and sale direct to consumers (DTC) of health-related genetic tests over the internet. Today, anyone with an internet connection and a credit card can purchase online a wide number of health-related genetic tests from companies who claim it will give them not only insight into their genomes, but also information about future disease risk that will enable them to better make important life decisions. The number of companies offering DTC genetic testing has grown rapidly along with coverage of the phenomenon in both the mainstream and scientific media and recent data have shown that awareness of this type of testing is relatively high both among consumers and providers.^{4, 5}

DTC companies offer various types of genetic tests, including whole genome sequencing, single nucleotide polymorphism (SNP)-based tests for susceptibility to complex, multifactorial diseases, and tests for known mutations in disease-associated genes. In addition to genetic results, some companies suggest behavioral, dietary, and other lifestyle changes that consumers can make to decrease their risk of disease. Some of the more prominent companies have made claims that DTC testing not only educates the public about genetics and disease risk, but also democratizes access to health information, since there is no doctor’s visit involved.⁶ These companies use empowerment rhetoric that encourages consumers to take control of their health by accessing their genetic information.⁷

In response to these companies, a number of professional associations and academics in public health and genetics have vocally criticized the clinical utility and validity of many DTC genetic tests and caution against their use for health-related purposes.⁸ Such experts assert that current tests being offered have not been validated in diverse populations and that more research is needed before these tests are used by consumers.^{8, 9} Many researchers have also raised concerns about consumers interpreting complex genetic information without the involvement of a healthcare provider who has specialized knowledge of genetics, given the potential for misinterpretation of disease risks.^{10, 8, 11, 12}

One group of consumers that has been overlooked in the current debate about the potential of DTC genetic testing to inform and empower is that of adults with limited genetic literacy. As the cost of sequencing continues to decrease and DTC companies lower the price of their services, DTC genetic testing is becoming increasingly affordable and accessible to broader segments of the population. Over one-third of U.S. adults have limited health literacy,¹³ and many more are likely to have limited genetic literacy.¹⁴ Prior research has shown that even those with high educational attainment have difficulty understanding the probabilistic information that is characteristic of genetic information.¹⁵ In addition, while adults may have some familiarity with genetics-related terms, they are unlikely to understand fully the underlying concepts.^14,16,17 Therefore, because DTC genetic tests are becoming increasingly accessible, because the Internet is an important health information source for consumers,¹³ and because internet use is increasing in every demographic group including among those likely to have limited literacy,¹⁸ it is important to examine whether DTC companies present web-based information that can be understood and used by consumers with a range of health literacy skills. One way to assess the appropriateness of such information is to examine the content, reading level and usability of DTC websites.

Guidelines for content

Guidelines and recommendations that have been issued by various professional organizations are one standard for assessing the content of DTC websites. In the U.S., a number of professional associations (e.g., American Society of Human Genetics, American College of Medical Genetics, American College of Clinical Pharmacology) have developed guidelines that seek to establish standards for the responsible provision of DTC genetic testing services. These bodies recommend that companies offering health-related DTC genetic testing provide information regarding test validity and the current limitations of the science, disclose the risks and benefits of undergoing testing, use CLIA-certified laboratories, carefully maintain the privacy of consumers’ information, disclose the clinical evidence for behavior change recommendations, and involve a health professional in the testing process.^{9,19, 20}

In addition, health literacy experts have created guidelines for the design of health-related materials to improve their reading level and increase their usability by adults with varying health literacy skills. These recommendations describe optimal strategies for organization, writing style and layout of health information.²¹ One widely used tool, the Suitability Assessment of Materials (SAM), includes an objective rating scale designed to assess the quality of a material’s informational content, reading level, writing style, sentence construction, use of vocabulary, graphics, organizational cues (e.g., headers, captions), typography, and layout.²¹ Similarly, web usability experts have issued user-centered guidelines for optimizing online health information.²² For example, they recommend the use of navigational elements (e.g., ensuring each page has a unique title, using visual cueing devices such as boxes and arrows to direct attention) that help users easily move through a website and an organizational structure that prominently places the most important information on a page.^{23 22}

Recent research on DTC genetic testing has predominantly focused on its ethical and policy implications and its potential impact on consumers.^{24,25,26,27, 28} No study to date has examined the features of DTC genetic testing websites in light of health literacy considerations. The primary objectives of this study were therefore to assess the informational content, literacy demands, and usability of health-related websites offering genetic testing direct to consumer. We critically examined these websites in order to assess whether the content met existing guidelines for information, health literacy, and web usability.

METHODS

Site selection

In this analysis, we examined websites in which consumers could directly purchase and receive genetic testing without the mandatory involvement of a healthcare provider. We focused only on tests related to health conditions or diseases, and therefore excluded any websites or portions of websites offering testing exclusively for paternity, ancestry, and/or pharmacogenomics. We also excluded websites that used genetic information solely for the purposes of providing advice on nutritional or behavior-related lifestyle choices without identifying a specific health risk.

We identified DTC websites for this analysis using two methods. First, we consulted a list of DTC companies prepared by the Genetics and Public Policy Center (version date 11/23/08).²⁹ Three of the 32 companies on the list were excluded because they did not test for health conditions (e.g., testing for athletic performance, fetal gender). Second, two of the authors independently conducted comprehensive web searches for additional DTC genetic testing websites using four popular search engines (Google.com, Ask.com, Altavista.com, and MSN Live Search) and one meta-search engine (Metacrawler.com). Adapting keywords used by Gollust and colleagues,³⁰ the following search terms were utilized: “genetics OR DNA + test OR testing + home OR kit OR disease,” “direct to consumer genetic OR gene + testing company,” “order/ing a DNA/genetic test online,” “personal genome service,” “online genetic testing,” “my DNA,” “learning about my genes,” and “my genome.” Valid DTC genetic testing company URLs appearing within the first 30 search results were considered for inclusion in the sample.²² The comprehensive web search identified one additional health-related DTC genetic testing website. Because one website was excluded due to its high degree of overlap with another website in the sample, 29 websites were included in the analysis.

Codebook development

The codebook for analysis was based upon the informational content, health literacy, and usability criteria described above, as well as prior health literacy-related content analysis research.³¹ The complete coding dimensions are presented in the Technical Appendix. Coding dimensions were grouped into two domains. Group 1 contained items evaluating the key informational content of the websites (e.g., health conditions, markers, testing process, scientific evidence, benefits, limitations, involvement of healthcare providers, privacy). These coding dimensions were based on the recommendations set forth by the professional associations regarding the key pieces of information that should appear on DTC genetic testing websites. Group 2 contained items evaluating the literacy and usability demands of the websites. For the literacy demand (i.e., level of health literacy required to use the sites) and usability domains (e.g., ease of locating the informational content, text density, navigational cues, layout and format, table structure), coders were asked to base their assessments on the sections of the website that contained the key informational content defined in the Group 1 domains. Coding dimensions for ease of location (e.g., evaluation of whether the content was displayed on the homepage, second-level or lower-level page) were adapted from published measures by the research team. These adapted dimensions used an objective pre-defined formula, based on page level and prominence of content, which the coders then referenced in order to determine the ease or difficulty of locating key content. Literacy demands were assessed using the SMOG and Fry readability formulas, as described below. Three trained independent coders refined the codebook in two rounds of pilot testing using DTC websites that had been excluded from the main sample (e.g., ancestry sites).

Technical Appendix.

Coding dimensions to assess the informational content, literacy demands and usability of health-related DTC genetic testing websites.

Construct	Sample Item	Response options	Source
Content characteristics
Tested health conditions	List the health conditions tested by the company	Open ended	Hudson et al., 2007
Tested markers	Listed SNPs or genes tested by the company	Open ended	Hudson et al., 2007
Scientific evidence	Is scientific evidence or data supporting marker selection provided?	Yes/No	ACMG 2008 & Hudson et al., 2007
Cited publications	Do they cite scientific publications to support marker selection?	3 point scale from “Yes, one or more citation is specifically linked to marker selection” to “No”	Hudson et al., 2007
Explanation of risk estimation	Does the site explain how it estimates/calculates risk?	Yes/No	ACMG 2008
DNA sample collection	Do they describe how the DNA sample is collected?	3 point scale from “Yes, detailed description” to“No”	ACMG 2008
DNA sample analysis	Do they describe how the sample is analyzed?	3 point scale from “Yes, detailed description” to“No”	ACMG 2008
CLIA laboratory	Do they state that they use a CLIA certified laboratory?	Yes/No	Hudson et al., 2007 & ACMG 2008
Health care provider consult	Does the site offer the chance to speak with a genetic counselor or other health care provider before and/or after testing?	Yes/No	Hudson et al., 2007, ACMG 2008, and Ameer & Krivoy, 2009
Health care provider explanation	Does the site explain to providers how they should use the results?	Yes/No	Hudson et al., 2007 & HHS Usability Guideline 16:8
Privacy policy	Does the site mention privacy or have a privacy policy?	3 point scale from “Yes, it describes its entire policy” to “No, it mentions nothing”	Hudson et al., 2007 & ACMG 2008
People behind company	Does the site identify the people behind the company offering testing?	Yes/No	Investigator created
Scientific collaborators	Does the site identify any scientific collaborators?	4 point scale from “Yes. The site clearly identifies collaborators and what they will do with consumers’ biologic samples and data.” to “No collaborators are identified”	ACMG 2008
Benefits: later	Is at least one benefit realized later?	Yes/No	Investigator created
Benefits: information	Does the site explicitly mention the benefit of information for its own sake?	Yes/No	Investigator created
Benefits: health decision	Does the site state that the test can inform a health decision?	Yes/No	Investigator created
Benefits: family	Is it beneficial to one’s children or family members?	Yes/No	Hudson et al., 2007
Benefits: scientific research	Is it beneficial to scientific research?	Yes/No	Investigator created
Benefits: other	Additional benefits to testing described by the site	Open ended	Investigator created
Limitations: limited marker effects	Do they describe that marker effects will be limited due to the effects of other genes, environmental factors or behaviors?	Yes/No	Hudson et al., 2007, ACMG 2008, and Ameer & Krivoy, 2009
Limitations: science uncertain	Does the site mention that the science is uncertain?	Yes/No	Hudson et al., 2007, ACMG 2008, and Ameer & Krivoy, 2009
Limitations: other	Does the site mention other limitations?	Open ended	Investigator created
Information updates	Does the site offer a mechanism for information updates?	Yes/No	Investigator created
Science changing	Does the site mention that the science is new and/or quickly changing?	Yes/No	Investigator created
Literacy demand and usability criteria
Ease of Location
In reference to: health conditions, markers, testing process, benefits, limitations, privacy, and involvement of health care providers	How easy or difficult is it to find information about tested health conditions?	4 point scale from“It is very easy (e.g., on the home page or prominent link on home page)” to“It is very difficult (e.g. 3rd level page or lower, not labeled)”	Adapted from Bock et al., 2004 and HHS Usability Guidelines 6:3, 6:5 and 16:5
Organization
Website purpose	Is the purpose of the website explicitly or implicitly stated on the home page?	3 point scale from “Yes, the purpose is explicitly stated” to “No purpose is stated”	Adapted from Doak et al., 1996
Relevance	How much of the content would help a consumer make a decision about the genetic test?	4 point scale from “All/most of the content is very relevant/useful” to“None/very little is relevant/useful”	Adapted from Bock et al., 2004
Risk presentation	Are any risks presented in the form of a percentage/ratio/graph or qualitatively?	Yes/No	Adapted from Doak et al., 1996 and HHS Usability Guideline 16:6
Literacy demand
Plain language	Does the site use common language and explain technical terms?	3 point scale from“Extensive use of common language; very little unexplained jargon” to“Extensive unexplained jargon; very few common words”	Adapted from Doak et al., 1996 and HHS Usability Guidelines 15:2 & 15:3
Active voice	Is the site written in active voice?	4 point scale from “The site mostly uses active voice” to“The site uses passive voice throughout”	Adapted from Doak et al., 1996 & HHS usability guideline 15:9
Sentence structure	Is the site written with simple sentences?	4 point scale from “Simple sentences, without embedded information, are almost always used” to“Most sentences have embedded information”	Adapted from Doak et al., 1996
Graphic illustrations, Lists, Tables and Charts
Content of graphics, animation and video	Does the web site use graphics in ways that support or add to the key informational content?	Yes/No	Adapted from Doak et al., 1996 and HHS Usability Guideline 14:4
Introduction of graphics	Are graphics, video, audio, and animated materials introduced with captions?	3 point scale from“Explanations are used with all/nearly all graphics” to “No explanations or captions are used”	Adapted from Doak et al., 1996 and HHS Usability Guideline 14:12
Familiarity	Do the images/graphics/videos portray real-world objects?	3 point scale from“Familiar, real world objects are used in all/most images” to “Images generally do not show familiar, real world objects”	Adapted from Doak et al., 1996 and HHS Usability Guideline 14:13
Table item density	For the most complex table on the web site, how many items are there?	5 point scale from “more than 225 items” to “75 items or fewer”	Mosenthal & Kirsch, 1998
Table dependency	In order to use the most complex table, do you have to look at information in another document or on another page within the website?	Yes/No	Mosenthal & Kirsch, 1998
Clickable images	Does the site generally use graphics with no labels as clickable items?	Yes/No	Bock et al., 2004 and HHS Usability Guideline 14:2
Learning stimulation and motivation
Glossary	Does the site have a glossary that links to technical terms?	3 point scale from “Yes, has linked glossary” to “No glossary”	HHS Usability Guideline 2:16
Tutorials	Does the site have educational tutorials?	Yes/No	HHS Usability Guideline 10:14
Web navigation
Search engine	Does the site have a search engine?	Yes/No	Bock et al., 2004

Sources

Ameer B, Krivoy N. Direct-to-Consumer/Patient Advertising of Genetic Testing: A Position Statement of the American College of Clinical Pharmacology. J. Clin. Pharmacol. 2009;49:886-888.

American College of Medical Genetics Board of Directors. ACMG statement on direct-to-consumer genetic testing, April 7, 2008. Available at: www.acmg.net/StaticContent/StaticPages/DTC_Statement.pdf. Accessed July 17, 2009.

Bock BC, Graham AL, Sciamanna CN, et al. Smoking cessation treatment on the Internet: Content, quality, and usability. Nicotine & Tobacco Research. 2004;6(2):207-219.

Doak CC, Doak LG, Root JH. “Suitability Assessment of Materials (SAM).” Teaching Patients with Low Literacy Skills. 2nd ed. Philadelphia: J.B. Lippincott Company; 1996: 49-60.

Hudson K, Javitt G, Burke W, Byers P, Committee ASI. ASHG statement on direct-to-consumer genetic testing in the United States. Obstetrics and Gynecology. 2007;110(6):1392-1395.

Mosenthal, Peter B. and Kirsch, Irwin. A new measure for assessing document complexity: The PMOSE/IKIRSCH document readability formula. Journal of Adolescent & Adult Literacy, 41:8, May 1998: 638-657.

U.S. Department of Health and Human Services. Research-Based Web Design & Usability Guidelines. Washington, DC: U.S. Government Printing Office; 2006. Available at: http://usability.gov/pdfs/guidelines_book.pdf. Accessed 2008.

Coding and analysis

From March through June 2009 two coders independently coded each of the websites in the study sample and discussed any discrepant scores in order to achieve consensus. Sites were coded in a predetermined order; the coders completed coding a given site within 3–5 days of each other. We conducted the analysis of the close-ended items based on the consensus codes using SPSS 15.0 for Windows (Chicago, IL). Percent agreement was calculated for close-ended codes. The following variables did not meet the pre-established threshold of 70% agreement between coders: description of DNA sample collection (59%) and analysis (59%); ease of locating stated benefits (69%), limitations (62%), and information about the involvement of health care providers (66%); description of privacy policy (69%); website purpose (66%); and relevance of content to making a health decision (69%). Because the level of agreement was, in most cases, close to 70% and because of their central importance as key content variables according to the recommendations of the professional associations, we present data on these items below, with appropriate notations regarding their reliability.

We assessed the readability of each website using two formulas: the SMOG and the Fry. The SMOG readability formula yields grade-level estimates of reading difficulty based on the number of words with three or more syllables in a sample of 30 sentences.³² The Fry formula measures sentences per 100 words and syllables per 100 words and plots the approximate reading grade level on a graph.³³ To perform the readability assessment, coders independently sampled three passages of text from three different key content areas on the website based on the same priority ordering: home page, health conditions, markers, testing process, benefits, limitations, involvement of health care provider, and privacy. Each coder then calculated both an average SMOG and an average Fry readability score independently for each site, and the scores from each coder were averaged to create a consensus score for each readability formula. Using a scoring tolerance of within two grade levels, coders achieved 76% agreement on the Fry and 83% agreement on the SMOG.

RESULTS

Test type offered

Given that the sites varied both in their purpose and in the type of testing they offered, it would be expected that the information presented on the sites relating to the benefits and limitations of the tests would vary accordingly. We therefore grouped the sites according to the type of testing they offered, breaking them down into three main types: whole genome sequencing or whole genome scans (7 sites), tests for one or more SNPs (15 sites), and tests for known gene mutations (e.g., cystic fibrosis, hemochromatosis) (7 sites). Four sites offered tests encompassing more than one type; these were grouped according to the type of test they offered most frequently.

Across the three main types of tests, the websites offered testing for a wide range of health-related conditions and diseases; in total, 255 health conditions and traits representing twenty different categories (Table 1). Websites varied in terms of the categories of conditions for which tests were offered; some offered tests from only one category, and one offered tests from eighteen of the categories. On average, websites in our sample offered tests from four different categories of conditions. The most frequently offered genetic tests included those for cardiovascular disease risk and those related to metabolism of or response to pharmaceuticals or other substances. Several categories of genetic tests (tests for susceptibility to diseases of the urinary or immune systems, skin diseases, and risks related to infectious disease) were only sold on sites that also offered tests from at least five other condition categories. Otherwise, there were few discernible patterns to the categories of testing offered by the included sites.

Table 1.

Categories of health conditions for which sites offered testing

Category	Examples of Conditions	Sites selling tests (N=29)
Chromosome analysis/FISH	recurrent pregnancy loss, 22q deletion	2 (7%)
Single gene diseases	cystic fibrosis, hemochromatosis	7 (24%)
Cancer Susceptibility
High Penetrance	BRCA1/2, PTEN	3 (10%)
Low Penetrance	lung cancer, prostate cancer	7 (24%)
Susceptibility to non-cancerous common complex diseases
Cardiovascular	cardiovascular disease, thrombosis	15 (52%)
Digestive	Crohn’s disease, gallbladder disease	8 (28%)
Endocrine	obesity, Type 2 diabetes	6 (21%)
Immune	allergies, lupus	6 (21%)
Nervous	ALS, epilepsy	10 (34%)
Reproductive	endometriosis, infertility	4 (14%)
Respiratory	asthma, emphysema	4 (14%)
Skeletal	arthritis, back pain	9 (31%)
Skin	psoriasis	3 (10%)
Urinary	kidney stones	2 (7%)
Susceptibility to psychiatric conditions	depression, schizophrenia	4 (14%)
Risk due to oxidative stress	coQ10 efficiency, oxidative stress	3 (10%)
Metabolism of or response to pharmaceuticals or other substances	caffeine metabolism, beta blocker response	13 (45%)
Substance dependence	nicotine dependence, heroin addiction	4 (14%)
Infectious disease risk or progression	HIV progression, norovirus resistance	3 (10%)
Non-disease related health profiles	fatigue, body composition	9 (31%)

Website content

The key informational content of these health-related DTC genetic testing websites is described in Table 2. Almost all (97%) of the websites listed at least one of the health conditions for which they offered testing and 79% listed at least one of the tested markers (SNPs). Results for these two items did not vary substantially according to the type of test offered on the website. About a third of sites (38%) provided scientific evidence to support the tested markers and 21% cited specific scientific publications to support marker selection, although this varied substantially by type of website. 21% of sites explained how they calculated the risk estimates given to consumers in their test results; this was most commonly done by the whole genome sites (43%).

Table 2.

Content characteristics of health-related DTC genetic testing websites

Content Item	Overall N = 29	Whole genome N = 7	SNP- based tests N =15	Known mutations N = 7
Listed one or more health conditions	28 (97%)	6 (86%)	15 (100%)	7 (100%)
Listed one or more tested markers	23 (79%)	5 (71%)	12 (80%)	6 (86%)
Provided scientific evidence or data to support marker selection	11 (38%)	2 (29%)	8 (53%)	1 (14%)
Cited specific publications to support marker selection	6 (21%)	3 (43%)	2 (13%)	1 (14%)
Explained how risk is estimated/calculated risk	6 (21%)	3 (43%)	2 (13%)	1 (14%)
Described how the DNA sample is collected*	27 (93%)	6 (86%)	14 (93%)	7 (100%)
Described how the DNA sample is analyzed*	22 (76%)	6 (86%)	10 (67%)	6 (86%)
Stated use of a CLIA certified laboratory	13 (45%)	4 (57%)	7 (47%)	2 (29%)
Provided an opportunity to talk to a health care provider
Before testing	4 (14%)	1 (14%)	1 (7%)	2 (29%)
After testing	8 (28%)	2 (29%)	3 (20%)	3 (43%)
Explained to providers how to interpret/use test results	6 (21%)	2 (29%)	3 (20%)	1 (14%)
Mentioned privacy or provided privacy policy*	23 (79%)	6 (86%)	11 (73%)	6 (86%)
Identified people behind the company	19 (65%)	5 (71%)	10 (67%)	4 (57%)
Identified scientific collaborators, but role not stated	7 (24%)	3 (43%)	2 (13%)	2 (29%)
Listed at least one benefit related to testing	26 (90%)	7 (100%)	15 (100%)	4 (57%)
Listed at least one limitation related to testing	16 (55%)	7 (100%)	6 (40%)	3 (43%)
Offered a mechanism for updates on test results	10 (34%)	6 (86%)	2 (13%)	2 (29%)
Stated that the science is new or changing	13 (45%)	6 (86%)	4 (27%)	3 (43%)

^*Intercoder agreement for this variable fell below 70%

For the testing process, almost all (93%) of the sites gave at least a brief description of the process by which the consumer’s DNA sample is collected and three-quarters (76%) at least briefly described how the sample is analyzed. About half (45%) of sites stated that they used a CLIA-certified laboratory to analyze the DNA samples. Only 14% of sites provided an opportunity for consumers to consult with a health care provider before testing, while 28% provided that opportunity after testing; this was most common among sites offering tests for known mutations. Less than one-quarter (21%) of sites gave information for health care providers about how they could interpret and use a patient’s DTC genetic test results.

In terms of privacy, nearly 80% of sites gave some information about their privacy policy. Two-thirds (65%) provided information on the people behind the company, although only 24% clearly identified any scientific collaborators and described their access to consumers’ DNA samples.

Benefits

Almost all of the sites (90%) listed at least one benefit to consumers of undergoing testing, although this was less common among sites offering testing for known mutations (57%). Of those sites that stated a benefit, the one most frequently stated (76%) was that the results of a test can inform a health decision (data not shown). Other commonly stated benefits were that there would be a benefit to the consumer in the future (35%); being tested could be beneficial to one’s family or children (24%); information itself is beneficial (24%); and there would be a benefit to scientific research (21%).

Limitations

About half (55%) of the sites overall presented at least one limitation related to undergoing testing, although all (100%) of the whole genome sites listed limitations. Of those sites that stated a limitation, the limitations most frequently stated were that other factors are important for disease risk (45%); the science surrounding testing is still uncertain (17%); the tests cannot diagnose diseases (14%); and testing was provided for only a limited number of mutations (14%) (data not shown). About one-third (34%) of sites offered some mechanism to allow consumers to receive updated information over time. About half of the sites (45%) stated that the science underlying the tests is new and/or changing, although this was often not described as a limitation of the test.

Literacy and usability

The results for the health literacy and usability criteria are shown in Tables 3 and 4.

Table 3.

Percentage of health-related DTC websites on which key informational content was easy to locate

Item	Overall N = 29	Whole genome N = 7	SNP-based tests N=15	Known mutations N=7
Tested health conditions	18 (62%)	2 (29%)	10 (67%)	6 (86%)
Tested markers	5 (17%)	0 (0%)	5 (33%)	0 (0%)
Information regarding benefits*	12 (41%)	4 (57%)	7 (47%)	1 (14%)
Information regarding limitations*	4 (14%)	1 (14%)	3 (20%)	0 (0%)
Information regarding privacy	14 (48%)	3 (43%)	6 (40%)	5 (71%)
Information about DNA sample collection and analysis	11 (38%)	3 (43%)	5 (33%)	3 (43%)
Information regarding the involvement of health care providers*	8 (28%)	3 (43%)	3 (20%)	2 (29%)

^*Intercoder agreement for this variable fell below 70%

Table 4.

Literacy demands and usability of health-related DTC genetic testing websites

Literacy Demand or Usability Item	Overall N = 29	Whole genome N = 7	SNP-based tests N=15	Known mutations N=7
Organization
Explicitly stated purpose of website on home page*	17 (59%)	4 (57%)	8 (53%)	5 (71%)
More than half of the presented content was relevant to making a testing decision*	12 (41%)	5 (71%)	3 (20%)	4 (57%)
Risk estimates for at least one disease were presented	8 (28%)	4 (57%)	3 (20%)	1 (14%)
Literacy demand
Mostly used common language and explained technical terms	2 (7%)	0 (0%)	2 (13%)	0 (0%)
Mostly used active voice	15 (52%)	4 (57%)	6 (40%)	5 (71%)
Mostly used simple sentences without embedded structures	9 (31%)	1 (14%)	6 (40%)	2 (29%)
Mean SMOG level of readability	14.7 (12–18)	15 (13.5–18)	14.8 (12–17.5)	13.9 (12–15)
Mean Fry level of readability	15.2 (10.5–17)	15.5 (13–17)	15.7 (12–17)	14 (10.5–17)
Graphic illustrations, Lists, Tables and Charts
Used graphics (picture, video or animation) that supported the key information	14 (48%)	5 (71%)	4 (27%)	5 (71%)
Introduced graphics with brief captions	18 (62%)	6 (86%)	7 (47%)	5 (71%)
Graphics almost always portrayed familiar, real-world objects	10 (35%)	2 (29%)	5 (33%)	3 (43%)
Used tables with less than 75 items	24 (83%)	5 (71%)	13 (87%)	6 (86%)
Used tables that were not dependent on information located elsewhere on the site	17 (59%)	2 (29%)	9 (60%)	6 (86%)
Did not use unlabeled graphics as clickable items	25 (86%)	4 (57%)	14 (93%)	7 (100%)
Learning stimulation and motivation
Provided a glossary	10 (35%)	3 (43%)	4 (27%)	3 (43%)
Provided one or more tutorials	12 (41%)	5 (71%)	3 (20%)	4 (57%)
Web navigation
Provided a search engine	10 (35%)	3 (43%)	4 (27%)	3 (43%)

^*Intercoder agreement for this variable fell below 70%

Ease of location

We assessed how easy information about each of the informational content topics was to locate on the websites (see Table 3). Overall, information about health conditions was easy to locate on 62% of the websites, although this varied greatly by type of site. In contrast, information about tested markers was easy to locate on only 17% of sites. Information regarding benefits was easy to locate on 41% of sites, although this was notably lower among sites offering testing for known mutations (14%). Information regarding limitations was easy to locate on only 14% of sites. Privacy information was easy to locate for just under half of the sites (48%), although this was higher among sites offering testing for known mutations (71%).

Additional literacy and usability dimensions

For organizational criteria (Table 4), about 60% of the sites explicitly stated their purpose on the home page. In 40% of the sites, more than half of the content related to key informational content, although this varied greatly across website type. About one-fourth (28%) of sites gave pre-test risk estimates for at least one disease and this was most common among the whole genome sites (57%),

In assessing the literacy demand of the sites, very few (7%) used mostly common language and consistently explained technical terms. About half (52%) used mostly active voice and 31% used mostly simple sentences, although this varied across site type. The sites had a very high level of reading difficulty. The mean SMOG score was 14.7 (range 12–18) and the mean Fry score was 15.2 (10.5–17), which can be interpreted as the sites requiring college-level reading skills, on average.

For the graphics and visuals used on the sites, in about half of the sites (48%) these were used in ways that supported the main content; this was highest among the whole genome sites (72%). A majority of the sites (62%) had at least brief captions explaining graphics, although this was less common among the SNP-based sites (47%). However, on only about a third of the sites (35%) did graphics consistently portray familiar objects. A majority of sites (83%) used relatively simple tables.

For learning stimulation and motivation, only about one-third of the sites (37%) provided a glossary to assist users with technical terms. 41% provided a tutorial; this was most common among the whole genome sites (71%). About one-third of sites (35%) provided a search engine to assist web navigation. Most (93%) displayed a link to the home page from 2^nd level pages and many (59%) displayed a link to the home page from 3^rd level pages (data not shown).

DISCUSSION

In this content analysis, we observed a wide range in the quality of informational content as well as the literacy demands and usability, across health-related DTC genetic testing websites. Although guidelines for these areas have been widely published, they have been variably adopted in the design of these websites. It is further apparent that most users would struggle to find and understand the important information on most sites. Our findings therefore provide empirical evidence for many of the recent critiques of DTC genetic testing websites, although they also lend some support to arguments that websites offering different types of genetic tests should not be lumped together in assessments of their content and quality.

Content findings

Overall, most sites presented health conditions and at least some markers for which they offered testing, described their process of collecting and analyzing DNA samples, and stated their privacy policy. Privacy policies were often filled with jargon and, while most sites identified the people who ran the company, there was far less transparency regarding the role of scientific collaborators (whether academic or private industry). The standard of clearly describing which collaborators would have access to consumer data at which points in the testing process and their relationship to the testing company was largely unmet. As described above, the recommendations from professional bodies are for consumer privacy policies to be clearly stated and for consumer data to be highly guarded. This is clearly an area in need of improvement.

Our finding that few sites provided scientific evidence to support the selection of the markers for which they offered testing provides empirical support to many of the previously described academic critiques of these testing services. While data regarding clinical validity and utility is not yet available for all of the markers for which companies offered testing, in order to meet the recommendation of providing consumers with accurate and transparent information, efforts should be made to reveal this information when it is available and a disclosure made when such information does not exist. The absence of this information could lead consumers to weigh equally the credibility of all of the tests, when, in fact, some are more clinically valid than others.

The results also showed that there was an imbalance on most sites between the provision of benefit information and limitations. Likely due to their promotional nature, a much greater proportion of sites described the benefits of testing than the limitations inherent to these tests and their interpretation. Berg and Fryer-Edwards’s²⁶ recent ethical analysis of 13 DTC genetic testing sites examined whether or not the sites provided consumers with enough information about the risks and benefits of testing in order to promote informed decision-making and found, similarly, that just four sites mentioned any risks associated with their tests. Our analysis further revealed that few sites stated that the science surrounding this type of testing is still uncertain. While the limitations of testing have been established in the scientific literature,^{34, 35} it does not appear that consumers would learn about these limitations if they relied only upon the information provided to them by the websites. Furthermore, despite a disclaimer that the information found on the sites was for informational purposes only, our analysis found that more than three-quarters (76%) of the sites simultaneously stated that the results of the test could be used to inform a health decision. As others have noted,³⁶ such conflicting statements are not only confusing to consumers, but may affect the sites’ credibility. The findings therefore suggest that the sites may not be providing adequate education to truly empower consumers.³⁷

Our results also show that the companies generally do not offer much assistance at the time of testing or following testing in interpreting results. Few of the websites offered consultations with health professionals before or after testing and few provided information that could be used by the consumer’s own health care provider in interpreting their patient’s test results. This is consistent with the findings of Goddard and colleagues²⁷ in their analysis of websites providing DTC testing for thrombosis, which found low adherence to professional recommendations regarding the provision of information and counseling from a health care provider. Furthermore, only about a third of sites offered a mechanism for updating consumers’ test results. As the science is still evolving and new discoveries are rapidly being made, the meaning of genetic test results could certainly change over time. Thus, as Shirts and Parker³⁸ have observed, the burden of keeping up with the science may largely fall on the consumers themselves who likely do not have the skills to seek out and interpret updated information. In addition, one recent study³⁹ found that even when risk information is updated, contradictory results might be given, which can adversely affect the lifestyle and behavioral choices that could already have been adopted by a consumer, further arguing for the importance of skilled interpretation by a health professional. Finally, these companies are very volatile in nature. In the year since our sample was identified, at least three of the sites have since disappeared from the web or have merged with other DTC companies. It is not clear either how consumers’ data is protected in these circumstances or how they would be notified as to a change in the interpretation of their test results when a company no longer exists.

Literacy demand and usability findings

The results presented here identify major areas in literacy demand and usability that could be improved on the websites in order to enhance their usefulness. While most websites in our sample provided information across the majority of the essential content areas, this important information was difficult to locate on many of the sites, often requiring users to navigate to third and lower level pages or scroll through lengthy PDF files. Since these websites are persuasive platforms seeking to convince a consumer to make a purchase, this selective emphasis of some information over other information may not be surprising, nor may it be surprising that a substantial proportion of the provided content is unrelated to helping consumers make an informed testing decision. While active voice was used on about half of the sites, common language and explanations of technical terms were not used consistently. In addition, the reading level of the sites was far above the 8^th to 9^th grade reading level of the average U.S. adult.⁴⁰ While individuals with limited literacy frequently encounter substantial barriers when seeking health information on the internet,^41,42 these findings indicate that the vast majority of consumers would have difficulty using these sites. Less than half of the sites used graphics in a way that would support consumers in understanding the key informational content. We had anticipated that sites would make use of tools such as glossaries, tutorials, and internal search engines as these are practical online instruments that facilitate both consumer understanding and navigation, but such assistive devices were not common. In sum, our results indicate that major changes to the literacy demands and usability of the sites would be needed to make the content understandable and usable to most U.S. adults. A logical next step to this line of research would be to conduct more formal usability assessments with users with diverse literacy and web navigational skills.

Differences across site types

Our analysis of results by site type did not identify one type of site as being a clear leader in the field in regard to both informational content and usability, but the results did indicate substantial variability across the key content criteria. Some authors have suggested that sites offering whole genome scans are making an effort to distinguish themselves as offering a higher quality service.³⁶ Our analysis found that while these sites scored better in certain areas (e.g., stating limitations, stating the science is changing, offering a mechanism for updates, presenting risk estimates, providing tutorials), there is still room for improvement on a number of key content variables (e.g., providing scientific evidence for marker selection, explaining how risk is estimated).

Recommendations

The public health potential of each of the examined DTC genetic testing websites will depend on the clinical utility and validity of the test and on individuals’ abilities to understand the implications of information that they receive. For consumers to make informed choices about genetic testing, they need to have access to all of the relevant information in an understandable format. The results presented here suggest the following concrete areas for improvement of health-related DTC genetic testing websites. The sites should aim to provide and distill the scientific evidence they rely on to select their markers and more clearly explain to consumers and their physicians how to interpret the meaning of their results. Furthermore, coders found that navigating the sites was often a highly complex task, and designers could do a better job of making the most important information, such as the limitations of testing, easier to locate. The sites had a very high reading level, used relatively few simple sentences, and used common language or clear explanations for technical terms inconsistently. Incorporating established plain language, clear communication, and web usability strategies across all important content areas would decrease the reading level and benefit both current and potential customers by making the information more accessible. Unlike print formats, websites are tools that lend themselves well to layering information according to different information-seeking needs and preferences. This unique feature should be utilized to its utmost advantage to prioritize important information and then facilitate further exploration if a consumer seeks greater detail. These types of easily implemented changes would likely enhance consumer comprehension of the content on these sites and better enable them to evaluate the meaning of these tests for their health. If implemented together, such changes would also allay some of the concerns raised by academics and could further enhance the educational, and, ultimately, the public health potential of websites offering health-related DTC genetic testing websites.

Policy Implications

Because these companies are for-profit rather than public health or educational entities, however, it is unlikely that they will be motivated to more fully disclose the limitations of the tests they offer or to improve the accessibility of the information on their websites for a broader range of users. Further, at least one study has shown that including more balanced risk information on a website offering DTC genetic testing for breast cancer decreased participants’ intentions to purchase online testing,⁴³ and such data would likely further disincentive companies from taking steps towards fuller disclosure. Therefore, as in the case of the FDA requiring pharmaceutical companies to disclose drug side effects in their advertisements,⁴⁴ it is likely that change will require that a state or federal regulatory body compel DTC genetic testing companies to take the measures that we and others recommend. Currently, no such agency monitors the sale and promotion of DTC genetic tests, though there have been some efforts on the part of individual states to more closely regulate the companies who sell these tests.²⁴ Further policy work should identify the strengths and weaknesses of the various agencies that could be called upon to implement this level of oversight.

In the absence of either regulation or voluntary change on the behalf of DTC companies, it may be necessary for government agencies, professional organizations and others to increase public awareness, and thus informed choice, through alternative means. Recently, others have suggested that the creation of a mandatory national genetic test registry administered by an agency within the U.S. Department of Health and Human Services would provide consumers and health care providers with objective information regarding the analytic validity, clinical validity, and clinical utility of all genetic tests, including genetic tests marketed and sold directly to consumers.⁴⁵ If such a registry was widely recognized by the public, information contained within could appropriately supplement the marketing materials produced by the companies themselves. The guidelines used throughout our study indicate that any such resource, including the DTC websites, should simultaneously attend to the scope of educational content, reading level, and usability to optimize the chance that a consumer could make a more fully-informed decision regarding whether or not to invest in this type of testing.

Limitations

The limitations of this study should be considered in interpreting the results. We assessed only English-language sites. We did not examine the validity of the tests being offered because our primary focus was to analyze the information being presented to consumers. The readability tools may overestimate reading difficulty when the same polysyllabic word is used frequently on a web page. Although we examined the literacy demands of the sites, we did not directly assess comprehension among consumers. Finally, because of the rapidly changing nature of the websites, our results present a snapshot of the sites at a particular time, although there has been no indication of major changes on key coding criteria subsequently.

Conclusion

DTC genetic testing sites will likely reach a consumer base who is more diverse in terms of education and socioeconomic status as the technology becomes more affordable for larger numbers of interested users. Yet, currently, there is no evidence that even the early adopters of these services understand the information being given to them by the sites. Future research efforts should seek to directly assess the understandability of the information presented on these websites with a diverse sample of consumers, including those with limited health or genetic literacy. There is also a need to investigate how consumers who have already purchased DTC genetic testing are using the information from their test results. Are they using it to make important health decisions, to supplement existing health and family history information or is their objective in testing simply recreational? As others have commented,³⁵ it will also be important for both researchers and those within DTC companies to track the lifestyle and behavior changes that consumers may be making as a result of the information they receive from these tests. If the information from tests with known clinical validity and utility can motivate people to make and sustain healthy lifestyle changes, their public health utility would be strengthened.