Efficacy of a Web-based Intelligent Tutoring System for Communicating Genetic Risk of Breast Cancer: A Fuzzy-Trace Theory Approach

Abstract

Background

Many healthy women consider genetic testing for breast cancer risk, yet BRCA testing issues are complex.

Objective

Determining whether an intelligent tutor, BRCA Gist, grounded in fuzzy-trace theory (FTT), increases gist comprehension and knowledge about genetic testing for breast cancer risk, improving decision-making.

Design

In two experiments, 410 healthy undergraduate women were randomly assigned to one of three groups: an online module using a web-based tutoring system (BRCA Gist) that uses artificial intelligence technology, a second group read highly similar content from the NCI web site, and a third completed an unrelated tutorial.

Intervention

BRCA Gist applied fuzzy trace theory and was designed to help participants develop gist comprehension of topics relevant to decisions about BRCA genetic testing, including how breast cancer spreads, inherited genetic mutations, and base rates.

Measures

We measured content knowledge, gist comprehension of decision-relevant information, interest in testing, and genetic risk and testing judgments.

Results

Control knowledge scores ranged from 54% to 56%, NCI improved significantly to 65% and 70%, and BRCA Gist improved significantly more to 75% and 77%, p<.0001. BRCA Gist scored higher on gist comprehension than NCI and control, p<.0001. Control genetic risk-assessment mean was 48% correct; BRCA Gist (61%), and NCI (56%) were significantly higher, p<.0001. BRCA Gist participants recommended less testing for women without risk factors (not good candidates), (24% and 19%) than controls (50%, both experiments) and NCI, (32%) Experiment 2, p<.0001. BRCA Gist testing interest was lower than controls, p<.0001.

Limitations

BRCA Gist has not been tested with older women from diverse groups.

Conclusions

Intelligent tutors, such as BRCA Gist, are scalable, cost effective ways of helping people understand complex issues, improving decision-making.

Breast cancer affects about one in eight American women,¹ and in 2011 there were an estimated 39,520 U.S. breast cancer deaths.² Thus, it is understandable that many women are concerned about breast cancer.³ Since the discovery of BRCA mutations and the advent of commercially available tests,⁴ many women have pondered whether to undergo testing for genetic breast cancer risk. This number has increased due to recent attention brought to the issue by the Supreme Court decision in Association for Molecular Pathology vs. Myriad Genetics, and by actor Angelina Jolie.⁵

Although genetic testing for breast cancer risk potentially saves lives, most women are not good testing candidates. Understanding genetic risk is challenging for most patients, and systematic biases in risk estimation have been demonstrated for both patients and providers (e.g., base-rate neglect).^6-8 Moreover, BRCA tests and associated genetic counseling are expensive and, without a family history of breast cancer, often not covered by insurance.^9,10 Furthermore, many patients are unsure what they would do if they received positive, negative, or ambiguous results.

Those who have positive results must decide about measures such as more frequent mammography, Tamoxifen treatments, prophylactic risk-reducing bilateral oopharectomy or salpino-oopharectomy, screening for ovarian cancer, chemoprevention for ovarian cancer, and prophylactic mastectomy.^11-13 However, positive results do not dictate specific actions, and negative results do not guarantee a cancer-free lifetime. There are also emotional and social risks associated with testing, and possible negative consequences with ambiguous results.^14,15 Moreover, the results often have implications for other family members. In some cases, having another family member tested rather than oneself is recommended.¹⁶ Yet communication about genetic risk among family members is often poor.¹⁷ There are also issues concerning the privacy of medical records and the potential for genetic discrimination.¹⁸ Little time is available in the clinical encounter to discuss these complex issues, and many patients have little access to genetic counselors.

The BRCA Gist Intelligent Tutoring System

One novel approach to helping women understand genetic testing for breast cancer risk is to use an Intelligent Tutoring System (ITS) to provide more effective “tutoring” of patients than traditional educational materials can provide. An ITS is a computer-based system using artificial intelligence techniques to mimic one-on-one human “tutoring.”¹⁹ Individual human tutoring is arguably the “gold standard” for facilitating deep conceptual understanding,^20,21 and there is evidence that the best ITS's produce gains comparable to human tutors.²² An ITS is not the same as decision aids such as Gail, Claus, or BRCAPRO.²³ This web-based ITS applies artificial intelligence to create a scalable and cost-effective way of engaging many people in dialogue simultaneously. Emerging discourse technologies²⁴ allow us to apply two complementary ideas rooted in Fuzzy-Trace Theory (FTT): first, helping people mentally represent the gist of information²⁵ can improve knowledge, understanding, and decision-making in medical contexts; and second, helping people explain the gist of complex medical information²⁶ in their own words fosters “deep learning” and comprehension.²⁷

Our ITS is called BRCA Gist (BReast CAncer Genetics Intelligent Semantic Tutoring).¹⁹ BRCA Gist is designed to supplement the clinical encounter, rather than replace it, or to be used as a freestanding public health resource for healthy women. This ITS is delivered on-line and can be self-administered by patients before or after a clinical encounter. ITSs in medicine are rare,²⁸ and although computer-tailored interventions have demonstrable benefits in areas such as mammography screening,²⁹ BRCA Gist appears to be the first ITS applied to lay people's medical decision making.¹⁹

BRCA Gist was built using Sharable Knowledge Objects (SKO, formerly AutoTutor Lite),^19,30 a web-based version of AutoTutor.^24,27,31,32 A talking avatar presents information orally and in text, graphics, and video. We used three female avatars of various apparent ethnicities. BRCA Gist converses with people, responding to what they type. It processes users’ verbal input using Latent Semantic Analysis, (LSA) a computational technique that mathematically measures the semantic similarity of two texts.^33-36 To provide appropriate feedback, BRCA Gist uses LSA to compare sentences entered by users to specially prepared expectations texts,^19,30 which we developed using verbal data from human respondents.¹⁹ LSA permits BRCA Gist to assess this association and respond accordingly – crucially, even when participants explain the gist of key concepts using different words than those in the expectations texts.

BRCA Gist is comprised of four modules on breast cancer and metastasis, risk factors, genetic mutation testing, and the consequences of testing. It provides didactic information interspersed with seven tutorial dialogues on topics including those requiring an explanation such as “How do genes affect breast cancer risk?” and “What should someone do if she receives a positive result for genetic risk of breast cancer?” and those requiring argumentation such as “What is the case for (and against) genetic testing for breast cancer risk?”¹⁹ There is good evidence that actively generating and elaborating on explanations of complex materials promotes understanding.^37,38 Generating arguments to support or refute conclusions about breast cancer has also been used to increase understanding.^39,40

Fuzzy-Trace Theory

Gist-based interventions such as BRCA Gist are guided by FTT, a theory of medical decision-making grounded in basic research on memory⁴¹ and probabilistic reasoning.⁴² FTT holds that, when information is encoded, people form multiple representations along a continuum from verbatim representations, including superficial detail, to fuzzier gist representations that capture bottom-line meaning.^25,43-45 Thus, gist and verbatim representations are formed in parallel during information acquisition.

In making decisions, people often prefer to reason with the vaguest bottom-line gist that can be used to decide among options.⁴⁴ Indeed, “the preference to operate on the crudest gist, the fuzzy-processing preference, increases with experience or expertise.”^25,46-49 When people make decisions, it is often more helpful to rely on these fuzzy gist representations^25,49-55 provided that they accurately capture decision-relevant features. Research indicates that gist representations are better retained, less vulnerable to extraneous interference, and easier to manipulate compared with verbatim representations, all of which reduces errors.²⁵

To illustrate, Reyna and colleagues found that expert physicians discriminated patient risk categories better than less-expert physicians – yet they relied to a greater degree on simpler gist-based representations.⁵⁰ More precise information processing—toward the verbatim end of the continuum of precision—was not associated with better risk judgments or better emergency-room decisions. Indeed, experts' precise numerical judgments sometimes violated basic rules of probability theory.⁵⁰ Superior medical decision makers distill their training and experience into flexible gist representations.^51,52

BRCA Gist encourages people to form flexible gist representations⁴⁴ rather than drilling them on verbatim facts. This is accomplished by verbal explanations highlighting the essential decision-relevant meaning of information, and presenting figures and videos that convey the bottom-line gist meaning of core concepts, stripping away details.⁵⁶ Gist is not simply less information, in contrast to heuristic “strategies that ignore information to make decisions faster, more frugally...than more complex methods.”⁵⁷

Numeracy plays a critical role in understanding health-related information. Even high numeracy patients and providers have difficulty with conditional probabilities (e.g., the probability of cancer given a genetic mutation).^8,58 FTT differs from other numeracy theories in distinguishing verbatim versus gist accuracy. Informed decision-making requires understanding qualitative gist relations, such as categorical and ordinal gist (e.g., categorizing risk as low vs. high), evidence for which has been gathered in prior research.^51,59,60 BRCA Gist text and figures were designed to clearly convey these qualitative gist relations, for example, that “most patients survive” at every stage of breast cancer except the most advanced.

Research on FTT has shown that difficulties with conditional probabilities are not merely representational in origin; they also involve confusions about how classes of events overlap (e.g., women who develop breast cancer and women who have BRCA mutations). Display formats that keep these classes discrete (drawing attention to the appropriate classes in denominators) have been shown to improve understanding of conditional probabilities.⁸ Thus, interventions applying FTT (incorporated in BRCA Gist) have been successful in helping people make sense of such relations, reducing biases and improving decision making.^{8,30,55,61,62}

To illustrate, many people find it difficult to understand how BRCA mutations account for only a small minority of breast cancer cases when the probability that someone with a BRCA mutation will develop breast cancer is high. Figure 1 highlights two relevant denominators, one represented by the black background—the base rate of BRCA mutations—and the other represented by the red figures—the base rate of breast cancer. Comparing the red figure shaded in black to the unshaded red figures, it is clear that most women who develop breast cancer (red figures) do not have a genetic mutation.

Figure 1.

Screen shot of BRCA Gist avatar explaining the relationship between incidence of breast cancer and BRCA mutations using a figure grounded in fuzzy-trace theory.

BRCA Gist content was adapted from the National Cancer Institute (NCI) website, and we compared the efficacy of BRCA Gist with both the NCI website and a control group. The NCI website is an excellent source of verbatim information about breast cancer and testing for genetic risk, but it was not designed as a gist-based intervention. Following FTT, our prediction was that BRCA Gist participants would perform better on measures of knowledge and gist comprehension of genetic testing and breast cancer risk than comparison groups. With their improved comprehension, we predicted that BRCA Gist participants would make better judgments and decisions about genetic testing. Finally, we predicted that these representational and class-inclusion interventions would not produce worry or anxiety.

METHODS

In two controlled laboratory experiments, participants were randomly assigned to one of three conditions: BRCA Gist, the NCI website, and a control tutorial on nutrition in a 1:1:1 ratio using computer randomization in randomized blocks of 3 established prior to the beginning of each experiment. The NCI condition was created by making PDFs of comparable portions of the NCI website. Participants browsed freely between NCI pages but could not follow hyperlinks to leave the site.

In Experiment 1, all groups spent one hour with their tutorials before completing the dependent measurers online. Experiment 2 replicates and extends Experiment 1. We added content including breast cancer stages and how cancer spreads to both the BRCA Gist and NCI conditions to include all the major relevant areas covered on the NCI website. Time on task increased to 90 minutes for all groups. In both experiments in the BRCA Gist and control conditions, interacting with the avatar took approximately 60 and 90 minutes in the two studies respectively. NCI web participants were instructed to study the materials for these allotted time periods. NCI participants read at their own pace but the “continue” button did not appear until after the 60- or 90-minute interval, equating time on task. Participants could not proceed until the time limit expired. Participants took between 60-90 minutes completing the dependent measures. In the controlled laboratory setting, participants worked at their own pace, but were prohibited from doing anything besides the experimental tasks.

Sample/Recruitment

Participants in both studies were undergraduate women recruited online at two universities in the Midwest and Eastern United States. As stated in the IRB-approved recruitment text, women could participate for course credit if they were 18 or older and had not had breast cancer. An advantage of recruiting university women as participants is a relatively high rate of full compliance. In Experiment 1, there were 202 participants who received credit for participating with data from 2 missing because they could not complete any items due to technical problems reaching the data collection site, leaving 200 participants. Data were collected between March 21, 2012 and May 6, 2012. About 28% were ethnic minorities, (10% African American, 10% Hispanic, 4% bi-racial, 13% Asian American in non-mutually-exclusive categories) and 72% white non-Hispanic. Mean age was 19.70 years (SD=1.38). In Experiment 2, 217 received credit and 7 of these missed portions of the tutorial because of technical problems with SKO; four BRCA Gist and 3 control participants were excluded, leaving 210. Data were collected between September 14, 2012 and December 7, 2012. About 27% were ethnic minorities, (6% African American, 6% Hispanic, 4% bi racial, 16% Asian American, in non-mutually-exclusive categories) and 73% white non-Hispanic. Mean age was 19.03 years (SD=1.62). Participants were significantly younger in the fall than spring (19.70 years, SD=1.38), F(1, 381)= 16.4, p<.0001 (most students are admitted in the fall). We did not find other demographic differences among conditions (see Table 1).

Table 1.

Participant Demographic Characteristics by Experiment and Condition

	Experiment 1			Experiment 2
	BRCA Gist	NCI Web	Control	BRCA Gist	NCI Web	Control
Percent Female	100%	100%	100%	100%	100%	100%
Percent Some College Education	100%	100%	100%	100%	100%	100%
Percent White non-Hispanic	72%	79%	69%	73%	73%	71%
Percent one or more Pedigree Assessment Tool Risk Factors	58%	52%	41%	48%	46%	55%
Mean and SD Age in Years	19.6 (SD=1.2)	19.6 (SD=1.1)	19.9 (SD=1.7)	19.1 (SD=1.1)	18.6 (SD=1.8)	19.4 (SD=2.1)

^*For X² p < .05 (no differences among experimental groups were significant).

Instruments

A medical expert vetted tutorial content and research instruments. For knowledge instruments, unanswered items were scored as incorrect (declarative knowledge, knowledge of breast cancer, and gist comprehension described below). For PAT and anxiety, data were excluded for those who skipped one or more questions within a scale (which did not differ by group and did not affect results; see below).

Prediction 1. BRCA Gist Will Increase Knowledge and Gist Comprehension

Declarative Knowledge of Breast Cancer, Genetic Testing, and Genetic Risk (see Appendix on-line). Declarative knowledge is factual knowledge that can be stated verbally. FTT suggests that a good gist understanding is the basis for organizing and remembering factual knowledge. We developed 52 four-alternative multiple-choice items on breast cancer, genetic risk, and genetic testing. The test was created in two phases to test the hypothesis that BRCA Gist would help participants learn more factual content than the other conditions. Items were drawn primarily from NCI website content, and were created corresponding to four BRCA Gist modules. An item on Breast Cancer and How it Spreads is “Breast Cancer usually forms in which part of the breast (ducts and lobules).” An item on Risk Factors and Risk Estimation is, “Which of the following is a risk factor for breast cancer? (having larger areas of dense breast tissue on a mammogram; having your first menstrual period before age 12; and going through menopause after age 55).” An item on Mutations, Genetic Testing, and Genetic Risk reads, “A BRCA1 or BRCA2 mutation does not increase a woman's risk developing breast and ovarian cancer (false).”An item on Consequences of Genetic Testing is, “What is the goal of surveillance? (to find cancer early when it is most treatable).” In Experiment 1 Cronbach's alpha for the initial 32-item instrument was .79. In Experiment 2 Cronbach's alpha for the entire 52 item instrument was .88.

Green et al. Knowledge of Breast Cancer, Heredity, and Genetic Testing^63,64 is a 20-item multiple-choice questionnaire assessing declarative knowledge about breast cancer, heredity, and genetic testing. This instrument was not designed to specifically measure content of the NCI website, and Green and colleagues do not report reliability, (we obtained a Cronbach's alpha of .49); however, it permits comparisons with previous studies.

Gist Comprehension of Genetic Breast Cancer Risk (see Appendix on-line). We developed a 30 item 1-7 Likert-scale instrument assessing gist comprehension of key information on breast cancer and genetic testing. Gist comprehension items such as, “BRCA (breast cancer) genetic mutations account for only a small percentage of all breast cancers“ express the essential bottom-line decision-relevant meaning of that knowledge. People can strongly endorse such statements without remembering the precise numerical risks and, surprisingly, can recall precise numbers without comprehending their bottom-line meaning, a phenomenon known as verbatim-gist independence.⁶⁵ Unlike declarative knowledge items, the item stem is stated at a general level such that answering does not require verbatim information, and the response format permits degrees of agreement. Unlike attitude measures, gist items have independently verifiable answers. Cronbach's alpha for Gist Comprehension was 0.85.

Prediction 2. BRCA Gist Will Improve Judgments and Decisions About Genetic Testing

Risk Assessment Scenarios for Genetic Risk of Breast Cancer (see Appendix on-line). Participants received 12 scenarios describing women with no genetic risk factors, or medium or high genetic breast cancer risk based on Pedigree Assessment Tool (PAT) scores of 0, 3-5, and 8-10 respectively.⁶⁶ Participants categorized the degree of genetic breast cancer risk for each woman as low, medium, or high. Participants later decided whether each woman should undergo testing. For risk assessment, Cronbach's alpha was 0.40 in Experiment 1, and 0.37 in Experiment 2, unsurprising given that the scenarios were designed to differ from one another.

Interest in Genetic Testing

Personal interest in genetic testing was assessed with a version of the Hall et al. item “How interested would you be in having a blood test for BRCA1 and BRCA2 mutations if you knew that the genetic test was 100% accurate?”.⁶⁷ Participants rated interest on a 5-point scale. Because it is reasonable to assume that only a small portion of the sample was at high genetic risk,⁶⁶ we predicted that BRCA Gist participants would appropriately express less interest in having a genetic test than controls.

Prediction 3. BRCA Gist Will Not Increase Anxiety or Worry About Breast Cancer

Worry About Breast Cancer Risk and Anxiety

Creating tutorials that are not anxiety provoking was a high priority. Worry about Breast Cancer Risk was assessed with Anderson and colleagues’ 5-item instrument.⁶⁸ The authors did not report reliability but in both our experiments we found Cronbach's alpha of .69. Anxiety was measured using the Spielberger State-Trait Anxiety Inventory State Anxiety subscale, an instrument that typically achieves Cronbach's alpha > .90.⁶⁹ We predicted BRCA Gist would not increase general anxiety. Indeed, if there were to be an effect, we predicted that BRCA Gist participants might experience less cancer-specific worry.

Pedigree Assessment Tool (PAT)

We asked 5 questions from the PAT⁶⁶ about family members with breast or ovarian cancer, and Ashkenazi Jewish heritage. Respondents were coded as reporting or not reporting risk factors, enabling assessments of whether the interventions are equivalent with both groups.

We administered tasks in the following fixed order: website evaluation, worry about breast cancer; risk assessment and testing decisions; Green et al. instrument; declarative knowledge; level of interest in having a genetic test; filler and non-outcome tasks; PAT; anxiety; and demographic questions. Within instruments, item order, and alternative order within items, were presented randomly.

Data Analysis

Data were analyzed using JMP Pro 10.0.2 with 3 (condition) × 2 (research site) ANOVAs, with differences among conditions assessed with Tukey's HSD. We assessed whether data provided evidence of superiority of BRCA Gist over the NCI and control conditions (with differences between sites being of no theoretical interest, but assessed because the Eastern university is more academically selective). There were no demographic differences between participants who were excluded for skipping one or more items (9 total across groups in Experiment 1, and 11 total across groups in Experiment 2). ANOVA is robust to violations of distributional assumptions^70,71 and Levene tests and Welch ANOVA⁷² indicate that the two such violations (of 33) did not affect results (see on-line materials).

RESULTS

Prediction 1. BRCA Gist Will Increase Knowledge and Gist Comprehension

Knowledge of Breast Cancer, Genetic Testing, and Genetic Risk

In both experiments on total Declarative Knowledge, BRCA Gist participants scored significantly higher than NCI participants, and both scored significantly higher than control group participants (see Table 2). In Experiment 1, for condition, F(2, 194)=30.56, p<.0001, η²=0.2332. BRCA Gist mean percent correct was .74, the NCI mean was .67, and the Control Group mean was .56. There was also a significant main effect for site* with BRCA Gist scoring significantly higher at both sites. These results were replicated in Experiment 2 (see Table 2); for condition, F(2, 204)=38.33, p<.0001, η²=0.2536. BRCA Gist mean percent correct was .75, the NCI mean was .67, and the Control Group mean was .55.

Table 2.

Percent Correct Declarative Knowledge for Each Module by Condition and Experiment

	Experiment 1				Experiment 2
	BRCA Gist	NCI	Control	Grand Mean	BRCA Gist	NCI	Control	Grand Mean
I. Breast Cancer & Genetic Risk	78%* (24) [−.583] <−.745>	67% (23) [−.730] <.237>	43% (26) [.106] <.048>	62% (26) [−.219] <−.743>	79%* (21) [−1.319] <1.319>	68% (17) [−.680] <.797>	56% (18) [−.475] <.059>	68% (21) [−.548] <−.126>
II. Risk Factors and Risk Estimation	69%† (20) [−.553] <−.130>	67% (21) [−.503] <−.290>	56% (21) [−.409] <−.133>	64% (21) [−.448] <−.257>	69%† (19) [−.816] <.286>	67% (16) [−.404] <.350>	53% (20) [−.039] <−.746>	63% (20) [−.467] <−.443>
III. Mutations & Genetic Testing	78%* (17) [−1.154] <1.206>	69% (13) [−.348] <−.190>	59% (16) [.026] <−.325>	69% (17) [−.445] <−.081>	77%* (20) [−1.320] <1.888>	70% (16) [−.687] <.205>	55% (18) [−.345] <.069>	68% (20) [−.592] <−.043>
IV. Consequences of Genetic Testing	83%* (20) [−1.675] <3.365>	72% (20) [−.616] <−.524>	68% (18) [−.718] <.764>	75% (20) [−.790] <.299>	70%* (17) [−1.266] <1.771>	61% (17) [−.586] <.205>	57% (16) [−.160] <−.424>	63% (18) [−.570] <−.208>
Total Declarative Knowledge Test	74%* (16) [−1.17] <.765>	67% (14) [−.712] <.066>	56% (13) [−.422] <.015>	66% (15.6) [−.401] <−.500>	75%* (17) [−1.479] <2.593>	67% (14) [−.733] <.848>	55% (15) [−.585] <.111>	66% (16.3) [−.574] <.059>

Note. Standard deviations are in parentheses, skew in brackets, and kurtosis in angle brackets.

^*BRCA Gist > NCI Web and Control, p < 0.0001.

^†BRCA Gist > Control, p < 0.001.

We analyzed declarative knowledge for each BRCA Gist module (see Table 2). BRCA Gist participants scored significantly higher than NCI participants, and both scored significantly higher than control participants on modules on Breast Cancer and How it Spreads, (Experiment 1, F(2, 194)=41.42, p<.0001 and Experiment 2, F(2, 204)=29.43, p<.0001) and Mutations, Genetic Testing, and Genetic Risk (Experiment 1, F(2, 194)=25.74, p<.0001 and Experiment 2, F(2, 204)=30.76, p<.0001). For the module Risk Factors and Risk Estimation, both BRCA Gist and NCI participants scored significantly higher than control participants, and the differences between NCI and BRCA Gist were not significant, Experiment 1 F(2, 194)=8.30, p<.0003 and Experiment 2, F(2, 204)=15.04, p<.0001. For the module Consequences of Genetic Testing, BRCA Gist participants scored significantly higher than both the NCI and control groups, and the differences between NCI and control were not significant, Experiment 1 F(2, 194)=10.83, p<.0001 and Experiment 2, F(2, 204)=12.91, p<.0001.

For the Green et al. knowledge test, in Experiment 1, BRCA Gist scored significantly higher than NCI, and both scored significantly higher than the control group (see Table 3); for condition, F (2, 194)=29.34, p<.0001, η² = 0.2191. BRCA Gist mean percent correct was .77, the NCI mean was .70, and the control group mean was .62. In Experiment 2, both BRCA Gist and NCI performed significantly better than the control group (see Table 3) with F(2, 204)=17.37, p<.0001, η²=0.1428. BRCA Gist mean percent correct was .73, the NCI mean was .74, and the Control Group mean was .66.

Table 3.

Mean for Key Outcomes by Condition

	Experiment 1				Experiment 2
	BRCA Gist	NCI Web	Control	Grand Mean	BRCA Gist	NCI Web	Control	Grand Mean
Green et al. Knowledge Test: Percent Correct	76.6%* (11.8) [−1.125] <1.089>	70.2% (10.6) [−..478] <0506>	62.2% (9.9) [−.510] <.506>	69% (12.2) [−.390] <−.147>	72.8%† (9.1) [−.473] <.130>	74.4% (8.5) [−.979] <2.785>	66.2% (7.7) [−2.901] <16.981>	71% (9.2) [−.220] <.102>
Gist Comprehension (1 = Strongly Disagree to 7 = Strongly Agree with Correct Responses)	-	-	-	-	5.34* (0.68) [−.489] <−.734>	4.98 (0.42) [.223] <−.306>	4.51 (0.57) [.082] <−.618>	4.97 (.661) [.247] <−0.78>
Risk Assessment: Percent Correct	59.6%† (16.5) [−.419] <−.134>	55.4% (15.2) [−.274] <−.530>	46.8% (12.7) [−.342] <.280>	53% (15.8) [−.016] <−.166>	61.3%† (15.7) [−.521] <−.474>	56.8% (15.7) [.065] <.132>	47.6% (11.7) [.150] <1.107>	56% (15.5) [.069] <−.369>
Interest in Genetic Testing (1 = Extremely to 5 = Not at All)	2.85* (1.45) [2.117] <−1.292>	2.30 (1.22) [.621] <−.696>	1.79 (1.01) [1.451] <2.232>	2.33 (1.31) [.719] <−.593>	2.76† (1.24) [.194] <−.951>	2.31 (1.25) [.720] <−.389>	1.99 (1.12) [1.039] <.666>	2.39 (1.25) [.559] <−.721>
Worry About Breast Cancer (1 low to 4 high)	1.20 (0.32) [2.833] <10.166>	1.28 (0.37) [3.486] <17.468>	1.24 (0.36) [1.453] <1.289>	1.24 (0.35) [2.558] <9.270>	1.20 (0.31) [1.725] <2.584>	1.16 (0.26) [1.741] <2.456>	1.34 (0.39) [1.139] <0.372>	1.23 (0.33) [1.561] <1.814>
State of Anxiety (20 low to 80 High)	45.14 (8.10) [−.186] <.225>	45.16 (8.53) [−.020] <−.440>	45.82 (8.40) [−.381] <.165>	45.39 (8.31) [−.192] <−.104>	42.50 (15.39) [−1.287] <1.883>	43.12 (13.99) [−1.137] <2.470>	44.52 (12.48) [−1.179] <3.664>	43.35 (14.01) [−1.235] <2.457>

Note. Standard deviations are in parentheses, skew in brackets, and kurtosis in angle brackets.

^*BRCA Gist > NCI Web and Control, p < 0.0001.

^†BRCA Gist > Control, p < 0.001.

Gist Comprehension

Gist comprehension was only assessed in Experiment 2. As predicted, BRCA Gist scored significantly higher than NCI, and both scored significantly higher than the control group (see Table 3). For condition, F(2, 204)=40.36, p<.0001, η²=0.2694. On a 1-7 scale (higher indicating greater comprehension) the BRCA Gist mean was 5.34, the NCI mean was 4.98, and the control group mean was 4.51. There was a strong positive correlation between declarative knowledge and gist comprehension r(208)=.77.

Prediction 2. BRCA Gist Will Improve Judgments and Decisions About Genetic Testing

Risk Assessment

In both experiments, BRCA Gist and NCI participants were significantly more accurate in categorizing risk than control participants (see Table 3). In Experiment 1, condition F(2, 194)=12.61, p<.0001, η²=0.1129. BRCA Gist mean percent correct was .60, the NCI mean was .55, and the Control Group mean was .47. In Experiment 2, condition F(2, 204)=16.85, p<.0001, η²=0.1359. BRCA Gist mean percent correct was .61, the NCI mean was .57, and the Control Group mean was .48.

Genetic Testing Decisions

We asked participants whether women depicted in the scenarios should undergo genetic testing. Participants uniformly recommended genetic testing at high rates for women at high genetic risk (see Figure 2). Over 80% of high-risk scenarios yielded genetic testing recommendations, with no differences among condition, Fs<1. Most people fitting the medium risk criteria, (e.g., Ashkenazi Jewish heritage) do not have BRCA mutations and may not be good genetic testing candidates. For scenarios depicting women with medium genetic risk, in Experiment 1, BRCA Gist participants were significantly less likely to recommended testing than NCI or control participants, F(2, 194)=4.64, p=.011, η²=0.0478. Control group participants recommended genetic testing for 67% of the scenarios compared with 54% for BRCA Gist participants (see Figure 2). This was not replicated in Experiment 2; medium-risk differences were not significant, F<1.

Figure 2.

Mean percent rated “should be tested” by risk level, condition, and experiment (error bars for 95% confidence intervals).

For scenarios depicting women without genetic risk factors, in Experiment 1 BRCA Gist and NCI participants were significantly less likely to recommend genetic testing than control participants, F(2, 194)=11.30, p<.0001, η²=0.1165: control participants recommended 50% of low-risk women for genetic testing compared to 24% for BRCA Gist participants (see Figure 2). In Experiment 2, BRCA Gist participants were significantly less likely to recommend genetic testing for women without risk factors than NCI participants, and both were significantly lower than control participants, F(2, 204)=23.88, p<.0001, η²=0.2313. Control participants recommended genetic testing for 52% of low-risk women compared to 19% for BRCA Gist (see Figure 2).

Interest in Genetic Testing

As predicted, in Experiment 1 BRCA Gist participants reported significantly lower levels of stated interest in having a genetic test than NCI or control participants, (see Table 3) F(2, 194)=11.17, p<.0001 η²=0.1152. The BRCA Gist mean on a 1-5 scale (1=extremely interested to 5=not at all interested) was 2.85, the NCI mean was 2.30, and the control group mean was 1.79. This was partially replicated in Experiment 2 as BRCA Gist participants (2.76) showed significantly lower levels of interest in genetic testing than control participants, (1.99) with NCI (2.31) not significantly different from either, (see Table 3) F(2, 189)=6.79, p=.0014, η²=0.2072.

Prediction 3. BRCA Gist Will Not Increase Anxiety or Worry About Breast Cancer

Anxiety and Worry about Breast Cancer Risk

We found no evidence that BRCA Gist provoked anxiety or worry about breast cancer. For anxiety, F<1 in both experiments. In Experiment 1, there were no significant differences for worry, F<1. In Experiment 2, BRCA Gist and NCI participants reported significantly lower levels of worry than control participants, F(2, 204)=5.68, p=.004, η²=0.0557. The BRCA Gist mean for worry was 1.20, (1=worry not at all or rarely to 4=almost all the time) 1.16 for NCI, and 1.34 for the control.

To assess genetic breast cancer risk, we asked 5 questions from the PAT about family members with breast cancer, ovarian cancer, and Ashkenazi Jewish heritage. Self-reported risk factors did not predict, or significantly interact with condition to predict outcomes.

DISCUSSION

Our results strongly support the concept that a gist-based intervention grounded in FTT can improve knowledge, gist-comprehension, and decision-making about genetic testing over and above what is achievable from the NCI's detailed materials. Our predictions that BRCA Gist participants would perform better on measures of knowledge and gist comprehension, and make better judgments and decisions about genetic testing, were confirmed in both studies. We found large effect sizes in measures of knowledge, risk assessment, gist comprehension, and decision-making about genetic testing. People who interacted with BRCA Gist often scored significantly higher than those spending the same amount of time reading comparable information from the NCI website. The BRCA Gist group exhibited more declarative knowledge than the control group for all four content areas. They scored higher than NCI participants on knowledge of breast cancer, genes, and genetic risk; BRCA mutations and genetic testing; and consequences of genetic testing (indeed the NCI web group performed no better than controls on genetic testing consequences). Even for the detailed knowledge conveyed in the risk factors and risk estimation module, the BRCA Gist group scored as well as the NCI group, illustrating the importance of gist understanding to learning.⁷³

The similarity in BRCA Gist and NCI factual content suggests that the key to this improved performance is learning with gist understanding rather than rote memorization of verbatim facts.⁷⁴ Gist comprehension accounted for about 60% of the variance in declarative knowledge, suggesting that the locus of the effectiveness of BRCA Gist is enhancing gist representations. Furthermore, FTT formal models of recall indicate that improving gist comprehension positively affects conventional knowledge measures by helping people to better organize their knowledge and reconstruct details from gist memory.⁵²

Most participants, regardless of group, recognized the advantages of genetic testing for high-risk women with a significant family history of breast and ovarian cancer. However, BRCA Gist led participants to appropriately recommend genetic testing at much lower rates for women without genetic risk factors. BRCA Gist apparently helped participants form a gist-understanding that most women do not have known genetic risks and are not good testing candidates. Judgments of genetic risk for medium risk scenarios fell between those for high and low scenarios, and were somewhat lower for BRCA Gist than NCI participants in both experiments.

BRCA Gist participants performed well on instruments not designed specifically for this content. BRCA Gist scores on the Green et al. knowledge instrument in Experiment 1, 77% correct, were similar to the 80% correct Green et al. reported for participants receiving genetic counseling.⁵⁵ However, Experiment 2 performance was somewhat lower (73% correct) and performance in both experiments was lower than the 91% correct in Green et al.'s computer-based instruction condition. Although Cronbach's alpha was low, this may be because only some items assessed content covered by BRCA Gist and the NCI website.

Limitations

BRCA Gist should be tested and developed through randomized controlled experiments with women from different ethnic, cultural, and socio-economic backgrounds, older women, and women at higher risk. Controlled experiments should assess the relative contributions of different aspects of BRCA Gist (i.e. explanation dialogues and displays grounded in FTT). Nevertheless, key findings were replicated at two sites in both experiments. Differences between experiments appear to be well within scientific expectations for replication.⁷⁵

We did not assess actual genetic testing decisions. The question regarding level of interest in having a genetic test is suboptimal. For example, some participants may have expressed interest in learning from testing, even though they understood it had little practical medical value for themselves. The “worry about breast cancer” instrument⁶⁸ is also suboptimal because some items asking about worry during the past week. Both BRCA Gist interactions and testing on outcome measures were conducted in a single experimental session. It is unknown whether BRCA Gist would continue to out-perform other groups if follow-up assessments were conducted after longer time intervals.

The NCI website comparison provides a rigorous test of the gist-based approach grounded in FTT. We assembled materials scattered across the NCI website into a convenient form, removing hyperlinks to scholarly references, which sometimes confuse laypeople; we also kept participants focused on the website. Instructions and time on task were identical for all groups. However, removing hyperlinks may have made it more difficult to follow personal interests, potentially limiting effectiveness. It is also possible that BRCA Gist is simply more engaging than the NCI website. Although participants in the laboratory were not permitted to talk, browse the web, check cells phones, and so on, it is possible that NCI participants may have been less attentive than BRCA Gist participants because the latter was interactive.

Conclusions

BRCA Gist is not a substitute for genetic counseling. However, a web-based ITS is a scalable and cost-effective way to reach people across the globe. Experts reviewing the literature have concluded that decision aids are rarely grounded in theory.⁷⁶ Yet when guided by a sound theoretical understanding, emerging discourse technologies²⁴ may be fruitfully used to create gist-based interventions,²⁵ guiding people to explain the gist of complex medical information²¹ in their own words, thus facilitating better medical decision-making.

Appendices: Declarative Knowledge, Gist Comprehension, and Genetic Risk Scenarios

Appendix A Declarative Knowledge of Breast Cancer, Genetic Testing, and Genetic Risk

Note. Items are presented to participants one at a time on separate screens. We randomized the order of items for each participant. We also randomized the order of choices within item except where one of the responses is “All of the Above” or “A and C.” Cronbach's Alpha for the entire 52 item instrument was .88.

Directions. You will be given a number of multiple questions about breast cancer and testing for genetic risk. Please select the best answer for each question. It is important that you answer all of the questions even if you have to guess.

1. Breast Cancer usually forms in which part of the breast?

A. ducts and lobules

B. fat tissue

C. muscle tissues

D. the areola

2. A woman's mother-in-law is found to have a genetic risk for breast cancer. Does this indicate that she may also have an elevated genetic risk?

A. Generally speaking, yes

B. Generally speaking, no

C. It is impossible to generalize, each situation is unique

D. “Elevated genetic risk” is a contradiction in terms

3. The BRCA1 and BRCA2 genes normally function as (chose one):

A. tumor suppressors

B. developing breast tissue

C. tumor producers

D. regulating the reproductive system

4. Which of the following should not influence the decision to be tested for genetic risk for breast cancer?

A. A sister who dies of skin cancer.

B. A cousin who developed breast cancer at an early age

C. Both a mother and sister who survive breast cancer

D. A grandmother who developed ovarian cancer

5. This incidence of breast cancer per year is about (select one).

A. 122 in 1000

B. 122 in 10,000

C. 122 in 100,000

D. 122 in 1,000,000

6. The phrase “absolute risk” refers to which of the following?

A. The overall chances of an individual developing breast cancer

B. A 100% guarantee that breast cancer will develop

C. The base rate of breast cancer

D. The percentage comparing one group's risk of developing breast cancer to another group's risk.

7. Although there are many important differences between men and women, the base rate for breast cancer is the same for both groups.

A. True

B. False

C. It is impossible to generalize, each situation is unique

D. It is impossible to compare base rates for two groups

8. If there is no information about a woman's risk factors for breast cancer, what can be said about her risk?

A. Nothing can be said about a woman's risk without specific medical evidence.

B. The best estimate of her risk is the base rate of roughly 12.2%

C. Her risk has increased D. She has no risk

9. What is gained by learning about whether or not a woman has certain risk factors?

A. Her risk of breast cancer increases

B. Risk factors are not associated with risk of breast cancer

C. Her risk of breast cancer decreases

D. A more accurate estimate of her risk can be made

10. A woman uses the Gail Model to estimate her risk of developing breast cancer. She does not have any risk factors measured by the Gail Model. What can be said about her risk?

A. She has almost no risk of developing breast cancer

B. She has increased risk of developing breast cancer

C. Her risk is lower than the general population but not zero

D. Her relative risk is greater than her absolute risk

11. Over the last two decades, the genetic risk of breast cancer

A. Has increased over time

B. Has decreased over time Has remained relatively constant

D. Has kept pace with non-genetic factors

12. A person received a positive test result. The probability of having the disease:

A. Equals the hit rate of the test

B. Is greater than the hit rate of the test

C. Is less than the hit rate of the test

D. Is higher than the base rate of the disease

13. BRCA1 and BRCA2 genes help prevent uncontrolled cell growth.

A. True

B. False

C. BRCA1 and BRCA2 are chromosomes not genes

D. There is no relationship between cell growth and BRCA1 and BRCA2 genes

14. A harmful mutation in a BRCA1 or BRCA2 gene can lead to what?

A. Environmental risk factors

B. Decomposition

C. Cancer and other diseases

D. All of the above

15. A BRCA1 or BRCA2 mutation does not increase a woman's risk developing breast and ovarian cancer?

A. True

B. False

C. True, only for breast cancer

D. True, only for ovarian cancer

16. There is a higher likelihood of having a BRCA1 or BRCA2 mutation if there is a family history of multiple breast cancer cases.

A. Generally speaking, yes

B. Generally speaking, no

C. It is impossible to generalize, each situation is unique

D. Not for BRCA1, only for BRCA2

17. Someone with a BRCA1 or BRCA2 mutation will almost certainly develop breast or ovarian cancer in her lifetime (if she lives past the age of 65).

A. True

B. False

C. True, with rare exceptions

D. True past the age of 56

18. In the general population of American women, the lifetime risk of developing breast cancer is approximately which of the following?

A. 20%

B. 12%

C. 60%

D. 3%

19. About what percent of the population of American women who have a BRCA1 or BRCA2 mutation will develop breast cancer?

A. 12 to 15%

B. 35% to 50%

C. 55% to 80%

D. 90% to 98%

20. You are about 5 times more likely to develop breast cancer if you have a BRCA1 or BRCA2 mutation?

A. True

B. False

C. It is impossible to generalize, each situation is unique

D. One is approximately 12 times more likely to develop breast cancer

21. BRCA mutations account for what percent of breast cancer in the general population?

A. 5-10%

B. 18-25%

C. 55-60%

D. 92-98%

22. The frequency of having a BRCA mutation is about 5 times higher than the general population for someone of Ashkenazi Jewish descent.

A. True

B. False

C. It is impossible to generalize, each situation is unique

D. False for Ashkenazi Jewish women without siblings

23. A woman of Ashkenazi Jewish descent is more likely to have a BRCA mutation if which of the following is also true?

A. A first-degree relative has been diagnosed with breast or ovarian cancer.

B. Her sister-in-law has been diagnosed with breast or ovarian cancer.

C. Two second-degree relatives on the same side of the family were diagnosed with breast or ovarian cancer.

D. Both A and C

24. You should get genetic counseling only if you are severely depressed.

A. True

B. False

C. True, if the depression has a genetic basis

D. True, if the depression surpasses a clinical threshold

25. Realistically, a positive genetic test result for BRCA1 or BRCA2 mutations is almost always a death sentence.

A. True

B. False

C. True for BRCA1 mutations but not BRCA2 mutations

D. True for BRCA2 mutations but not BRCA1 mutations

26. What is the goal of surveillance

A. to cure cancer

B. to find cancer early when it is most treatable

C. to decrease the risk of developing cancer

D. None of the above

27. You can still develop cancer if you have prophylactic surgery - removal of “at risk” tissue in both breasts.

A. True, but relatively rare

B. False, it is impossible by definition

C. True in most cases

D. There have been no documented cases

28. There are some drugs that have been approved by the FDA that could help reduce the risk of breast cancer in women with BRCA1 or BRCA2 mutations?

A. True

B. False

C. False, at present, such drugs are still being tested and are not available to physicians

D. False, they were approved by the European Pharmacological Centre (EPC) but not the FDA

29. HIPAA allows anyone with medical training to see your health information and records.

A. True

B. False

C. True, including medical students and residents

D. HIPAA guidelines do not pertain to health information

30. The GINA law allows for discrimination based on genetic information by health insurers and employers?

A. True

B. False

C. True for health insurers but not employers

D. True for employers but not for health insurers

31. Under the GINA law, health insurers may not do which of the following:

A. require genetic testing of potential customers

B. purchase genetic information about customers or potential customers

C. use genetic information as an only claim for a pre-existing condition in order to deny coverage

D. All of the above

32. Under GINA law, employers may not do which of the following:

A. refuse to hire or fire someone based on genetic information

B. require, request, or purchase genetic information about an employee

C. disclose employee's genetic information

D. All of the above

33. According to the Gail model is race or ethnicity a factor in estimating breast cancer risk?

A. Yes, race or ethnicity is a factor in estimating breast cancer risk.

B. No, race and ethnicity are not factors in statistically estimating breast cancer risk.

C. No, the Gail model is only based on medical factors such as family history.

D. No, the Gail model was developed for European-American women only.

34. The Gail model uses information about:

A. whether people in your family have ever had any form of cancer.

B. whether close relatives had breast cancer.

C. whether you smoke.

D. your blood.

35. If you test “negative” for BRCA 1 or 2 gene alteration:

A. You will never develop breast cancer.

B. You have breast cancer.

C. You do not have to worry about taking precautionary steps.

D. You still have a risk of developing breast cancer.

36. An advantage of having a genetic test for breast cancer risk is that it may help you to:

A. Decide whether or not to have risk-reducing surgery.

B. Give other family members useful information.

C. Explain why other family members have developed cancer.

D. All of the above.

37. A disadvantage to having a genetic test for breast cancer risk is

A. Your test results will become part of the public record.

B. Although rare, you may face illegal discrimination for health, life, disability, and other insurance.

C. The majority of tests produce ambiguous results.

D. All of the above.

38. Cancer cannot spread through the blood

A. True, it can only travel through the lymph system

B. True, it can only travel through tissue

C. False, it can travel through the lymph system, tissues, and blood

D. False, it can only spread through blood

39. Stage III cancer is

A. Cancer that has come back after a period of time.

B. Locally advanced cancer.

C. Distant metastatic cancer.

D. Any or all of the above.

40. Stage IV cancer is

A. Cancer that has spread to other parts of the body besides the breast.

B. Used to describe when cancer has spread to the lymph nodes of the breast.

C. A tumor at least 5 centimeters.

D. None of the above.

41. Breast cancer that metastasizes to the bones

A. Becomes bone cancer.

B. Is still made of breast cancer cells.

C. Is ductal carcinoma in situ.

D. By definition, breast cancer can not metastasizes to the bones.

42. Which of the following is a risk factor for breast cancer?

A. Being over the age of 60.

B. Having a family diagnosed with breast cancer before age 50.

C. Being white or Caucasian.

D. All of the above.

43. Which of the following is a risk factor for breast cancer?

A. Having larger areas of dense breast tissue on a mammogram.

B. Having your first menstrual period before age 12.

C. Going through menopause after age 55.

D. All of the above.

44. Which of the following is a significant risk factor for breast cancer?

A. Having a male relative with colon cancer.

B. Having a female relative with ovarian cancer.

C. Having a personal history of skin cancer.

D. All of the above.

45. Which of the following is a significant risk factor for breast cancer?

A. Having a male relative with breast cancer.

B. Having a female relative with skin cancer.

C. Having a personal history of colon cancer.

D. All of the above.

46. Benign tumors

A. Are rarely a threat to life.

B. Don’t invade the tissues around them.

C. Don’t spread to other parts of the body.

D. All of the above.

47. Malignant tumors

A. Are cancerous tumors.

B. Often can be removed but sometimes grow back.

C. Can invade and damage nearby organs and tissues.

D. All of the above.

48. Breast cancer can spread through the blood vessels.

A. True, but it can also spread other ways.

B. False.

C. True, the blood vessels are the only way breast cancer spreads.

D. True, but it is extremely rare for breast cancer to spread.

49. Breast cancer can spread through the lymph vessels.

A. True, but it can also spread other ways.

B. False.

C. True, the lymph vessels are the only way breast cancer spreads.

D. True, but it is extremely rare for breast cancer to spread.

50. Breast cancer can spread through bodily tissues.

A. True, but it can also spread other ways.

B. False.

C. True, through bodily tissues is the only way breast cancer spreads.

D. True, but it is extremely rare for breast cancer to spread.

51. How does breast cancer spread?

A. Through the blood vessels.

B. Through the lymph vessels.

C. Through bodily tissues.

D. All of the above

52. Ductal carcinoma

A. Typically begins in the lobes or lobules.

B. Is generally benign.

C. Is the most common type of breast cancer.

D. Can not invade and damage nearby organs and tissues.

Appendix B Gist Comprehension of Genetic Breast Cancer Risk

Directions

Rate the following on a scale from 1 = Strongly Disagree to 7 = Strongly Agree.

1. When the BRCA1 and BRCA2 genes are functioning normally they help cells from growing out of control.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

2. <reverse score> The greatest danger of dying from breast cancer is when a tumor grows larger in the location where it started.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

3. The greatest danger of dying from breast cancer is when it spreads to other parts of the body.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

4. <reverse score> There is no reason why a woman should not be tested for genetic breast cancer risk.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

5. <reverse score> Breast cancer is color blind. It occurs equally in all racial and ethnic none.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

6. Having a family member with ovarian cancer is a large genetic risk factor for breast cancer.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

7. <reverse score> Fortunately, breast cancer in a male relative is not a genetic risk factor for breast cancer.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

8. <reverse score> Genetic risk factors account for the majority of breast cancer cases.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

9. <reverse score> Cancer can travel through the lymph system, but it cannot spread through the blood.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

10. A benign tumor is one that has not metastasized (spread).

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

11. <reverse scoring> By law, genetic testing for breast cancer risk is covered by all health insurance policies.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

12. <reverse scoring> When a woman is tested for genetic breast cancer risk, her test results become part of the public record.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

13. An advantage of having a genetic test for breast cancer risk is that it may help a woman to decide whether or not to have risk-reducing surgery.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

14. Breast cancer that spreads to the liver is still breast cancer, not liver cancer.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

15. White women are at greater risk for developing breast cancer than African-American women.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

16. <reverse scoring> The chance that a woman has a BRCA genetic mutation is sometimes less than the chance that she has both breast cancer and a genetic BRCA mutation.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

17. The base rate for BRCA (breast cancer) genetic mutations is low.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

18. BRCA (breast cancer) genetic mutations account for only a small percentage of all breast cancers.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

19. <reverse score> Once a woman has had a mammogram, statistical information about breast cancer in other women is no longer helpful.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

20. A woman who has a BRCA mutation is many times more likely to develop breast cancer than a woman who does not have a mutation.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

21. It is important to speak with a genetic counselor both before and after genetic testing for BRCA mutations.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

22. The goal of active surveillance is to find breast cancer early.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

23. <reverse scoring> Taking the drug Tamoxifen to prevent a healthy woman from developing breast cancer has few, if any, risks or side effects.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

24. Federal law prevents many kinds of genetic discrimination.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

25. <reverse score> Federal law outlaws any kind of genetic discrimination in any circumstance.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

26. <reverse score> Breast cancer is just as likely to strike young women as it is to strike older women.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

27. <reverse score> A woman with a BRCA mutation is guaranteed to develop breast cancer eventually.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

28. The goal of preventive treatments such as chemoprevention and surgery removing healthy tissue is to reduce the risk of developing breast cancer.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

29. <reverse score> Preventive treatment such as surgery removing healthy breast tissue guarantees that a women will not get breast cancer.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

30. <reverse score> A woman who finds out she has a BRCA mutation should start treatment such as chemotherapy immediately.

1	2	3	4	5	6	7
Strongly						Strongly
Disagree						Agree

Appendix B Scenarios for Genetic Risk of Breast Cancer

Each description has the following: name, age, ethnicity, hometown, family health facts, and personal health facts. Equated for age. Range of Words: 56 - 60. Range of Flesch Reading Ease Scores: 56.9 – 62.9. Range of Flesch-Kincaid Grade Level Scores: 7.3 – 7.9.

Highest Risk: PAT Score 8-10

Rachel PAT Score 10; Words: 56; Flesch Reading Ease Score: 62.5; Flesch-Kincaid Grade Level Score: 7.3.

Rachel is a 47 year-old Chicago woman. Her parents came to this country from Eastern Europe and her family background is Ashkenazi Jewish. She has two cousins on her mother's side who have breast cancer. Her cousin Joanne was diagnosed with Breast cancer at age 56, and Elaine at age 61. Rachel has generally been healthy.

Anabelle PAT Score 10; Words: 57; Flesch Reading Ease Score: 57.6; Flesch-Kincaid Grade Level Score: 7.5.

Anabelle is a healthy 55 year-old Italian-American. She lives in Providence, Rhode Island. Anabelle comes from a family of nine children. Two of her sisters are breast cancer survivors. Her sister Grace was diagnosed with breast cancer at the age of 49, and Faith when she was 53. Her 66 year-old cousin Maria also had breast cancer.

Sarah PAT Score 9; Words: 58; Flesch Reading Ease Score: 62.3; Flesch-Kincaid Grade Level Score: 7.4.

Sarah is 66 years old and lives in Boca Raton, Florida. She lives with her husband and pet dogs Baby and Dolly. She is of Ashkenazi Jewish descent, but she and her husband are not very religious. Sarah likes playing golf, and she has always been healthy and active. Recently her younger sister was diagnosed with ovarian cancer.

Claire PAT Score 8; Words: 59; Flesch Reading Ease Score: 62.9; Flesch-Kincaid Grade Level Score: 7.4.

Claire is an unattached 35 year-old New Yorker. She has a vegan diet, and is an avid jogger. Her family is of Scottish-Irish heritage. Recently, her 51 year-old uncle Sean was diagnosed with cancer of the breast. Claire has several siblings and to the best of her knowledge, her uncle Sean is the only family member with breast cancer.

Medium Risk: PAT Score 3-5

Kim PAT Score 5; Words: 56; Flesch Reading Ease Score: 57.9; Flesch-Kincaid Grade Level Score: 7.9.

Kim is a 66 year-old Korean American mother who lives in San Jose, California with her family. Kim suffers from migraine headaches. Her 59 year-old sister Sun is a 5-year ovarian cancer survivor. That is the only cancer in her family that she knows about. Kim eats healthy foods, but does not get enough exercise.

Hanna PAT Score 4; Words: 59; Flesch Reading Ease Score: 59.1; Flesch-Kincaid Grade Level Score: 7.4.

Hanna is a 55 year-old mother of Ashkenazi Jewish descent. Her ancestors are from Eastern Europe. She lives in Cleveland Heights, Ohio with her husband and youngest daughter. No one in her extended family has ever had cancer. She likes to walk to work for exercise when the weather is good. However, she suffers from asthma and seasonal allergies.

Olive PAT Score 4; Words: 60; Flesch Reading Ease Score: 62.1; Flesch-Kincaid Grade Level Score: 7.5.

Olive is 47 years old, the exact same age as her mother was when she died of breast cancer complications. She also lost an uncle to lung cancer. No one else in her French-Canadian family has had breast cancer or ovarian cancer. Olive is considered medically obese, and her weight frequently fluctuates. She lives in Los Angeles with her roommate.

Janet PAT Score 3; Words: 57; Flesch Reading Ease Score: 60.2; Flesch-Kincaid Grade Level Score: 7.6.

Janet is a 35-year-old Denver woman of English and Scottish ancestry. Janet is prone to kidney stones and her urologist has her on a diet low in red meat. Her 61 year-old cousin was recently diagnosed with breast cancer. No one else in the family has cancer, but her mother has diabetes. Janet herself battles obesity.

Low-Risk: PAT Score 0

Alegria PAT Score 0; Words: 60; Flesch Reading Ease Score: 62.1; Flesch-Kincaid Grade Level Score: 7.5.

Alegria is a 47 year-old Mexican American. She lives in Phoenix, Arizona with her husband. Alegria is a heavy cigarette smoker. Her best friend is a 10-year breast cancer survivor, but this has not been enough to get her to give up her two pack a day cigarette habit. No one in her family has ever had breast cancer.

Natalie PAT Score 0; Words: 60; Flesch Reading Ease Score: 59.2; Flesch-Kincaid Grade Level Score: 7.9.

Natalie is a 66 year-old woman who lives with her husband Charlie in Cincinnati. She does not have any children. Her ancestors were German Lutherans, but her husband is of Ashkenazi Jewish descent. Although they have both been healthy, recently her husband Charlie was diagnosed with cancer of the breast. No one else in her family has ever had cancer.

Kate PAT Score 0; Words: 57; Flesch Reading Ease Score: 61.1; Flesch-Kincaid Grade Level Score: 7.4.

Kate is 55 years old, divorced, and lives by herself in a townhouse in Atlanta. Kate frequently feels exhausted even when she has not physically exerted herself. Her family can trace many of their ancestors back to England. Five years ago, Kate had Basal cell carcinoma removed from her scalp. Her father also had skin cancer removed.

Rheana PAT Score 0; Words: 60; Flesch Reading Ease Score: 60.4; Flesch-Kincaid Grade Level Score: 7.3.

Rheana is a 35 year-old African American woman from Columbus. Her family has a history of diabetes and hypertension. For this reason she goes out of her way to cook healthy meals for her family. Her brother was recently diagnosed with prostate cancer. No one else in the family has ever had cancer. Rheana considers herself blessed with good health.

Effects of Research Site

Both experiments were conducted in two settings, a university in the Eastern United States and a Midwestern university. Differences between sites were assessed because the Eastern university is more academically selective than the Midwestern university, but they are of no theoretical interest. They are included in this on-line appendix for the sake of completeness, and organized by outcome.

Declarative Knowledge

Experiment 1. F(1, 194)=5.92, p=.016 with Eastern university participants scoring significantly higher than Midwestern university participants (.69 and .64 respectively). The site by condition interaction was not significant, F(2, 194)=1.12, p=.33. In Experiment 2, F(1, 204)=40.60, p<.0001 with Eastern university participants scoring significantly higher than Midwestern university participants (.70 and .62 respectively). The site by condition interaction was not significant, F(2, 204)=2.60, p=.08.

Green et al. Knowledge Test

Experiment 1. F(1, 194)=29.3, p=.0001 with Eastern university participants scoring significantly higher than Midwestern university participants (.72 and .67 respectively). The site by condition interaction was not significant, F<1. In Experiment 2, there was not a significant effect for site, F<1. The site by condition interaction was also not significant, F(2, 204)=2.47, p=.09.

Gist Comprehension

was only assessed in Experiment 2. There was a significant effect for site, F(2, 204)=12.77, p<.0001 with the Eastern participants scoring significantly higher than the Midwestern participants, (5.09 and 4.81 respectively). The site by condition interaction was not significant, F(2,204)=1.47, p=.23.

Risk Assessment Percent Correct

In Experiment 1 there were no significant differences for site, F(1, 194)=3.09, p=.08 and the site by condition interaction was not significant, F<1. In Experiment 2, F(1, 204)=4.20, p<.042 with Eastern university participants scoring significantly higher than Midwestern university participants (.57 and .53 respectively). The site by condition interaction was short of significant, F(2, 204)=3.03, p=.0501. Means were comparable in the BRCA Gist and control conditions: for BRCA Gist The Midwestern mean is .62 and the Eastern mean is .61, and for control .47 and .49 respectively. However in the NCI web condition the Midwestern mean was .51 whereas the Eastern mean was .62.

Should be Tested High-Risk

Experiment 1. There was not a significant effect for site, F<1 and the site by condition interaction was not significant, F<1. In Experiment 2 for site F<1 with the site by condition interaction also F<1.

Should be Tested Medium Risk

Experiment 1. For site, F(1, 194)=1.89, p=.17 The site by condition interaction was not significant, F<1. In Experiment 2, F(1, 204)=3.95, p=.048 with Eastern university participants ratings significantly lower than Midwestern university participants (.60 and .67 respectively). The site by condition interaction was not significant, F(2, 204)=2.18, p=.12.

Should be Tested No Risk Factors

Experiment 1. There was no effect for site, F<1 and the site by condition interaction was not significant, F<1. In Experiment 2 for site F(1, 204)=3.36, p=.07. The site by condition interaction was significant, F(2, 204)=3.65, p=.028. BRCA Gist appears to have been most helpful in reducing testing recommendations for participants at the Midwestern university with a mean of .16 compared to .23 for the Eastern university whereas the NCI website appears to have been most helpful at the Eastern university with a mean of .25 compared to .40 for the Midwestern university. In the control group the Eastern university had a mean of .45 whereas the mean for the Midwestern university was .60. Effects from treatments were greater than control groups for each site, but BRCA was lower (more helpful) than NCI for only the Midwestern group. NCI was never more helpful than BRCA though. Advantages for BRCA relative to NCI were more evident for the less elite group, the Midwestern group.

Interest in Genetic Testing

Experiment 1. The effect of site was not significant, F<1. The site by condition interaction was not significant, F<1. Experiment 2. The effect of site was not significant, F(1, 204)=1.02, p=31. The site by condition interaction was not significant, F(2, 204)=1.99, p=.12.

Anxiety

Experiment 1. There were no significant differences between sites, F<1, and the site by condition interaction was not significant, F<1. In Experiment 2 there were no significant differences between sites for anxiety, F(1, 204)=2.60, p=.11, and the interaction was not significant, F<1.

Worry about Breast Cancer

Experiment 1. For site, F(1, 194)=3.92, p=.049 with Eastern university participants scoring significantly lower on worry than Midwestern university participants (1.18 and 1.28 respectively). The site by condition interaction was not significant, F<1. In Experiment 2, for site F(1, 204)=1.80, p=.18 and for the site by condition interaction F<1.

Violations of Distributional Assumptions

Although ANOVA is robust to violations of distributional assumptions we conducted Levene tests to determine whether group variances are equal. In 31 cases the assumptions were met and in the two cases out of 33 in which they were not we ran a Welch ANOVA, testing for differences among means assuming standard deviations are not equal and the results were the same. These results are presented by variable.

Gist Comprehension

Gist comprehension was only assessed in Experiment 2. As predicted, BRCA Gist scored significantly higher than NCI, and both scored significantly higher than the control group. For condition, F(2, 204)=40.36, p<.0001, η²=0.2694. Levene test indicates unequal variance, and Welch ANOVA F(2, 134.33)=43.09, p<.0001 confirms that the differences among means are significant.

Interest in Genetic Testing

As predicted, in Experiment 1 BRCA Gist participants reported significantly lower levels of stated interest in “having a blood test” than NCI or control participants, (see Table 3) F(2, 194)=11.17, p<.0001 η²=0.1152. Levene test indicates unequal variance and Welch ANOVA F(2, 40.6)=7.32, p=.002 confirms that the differences among means are significant.