Assessing Hypothetical Scenario Methodology in Genetic Susceptibility Testing Analog Studies: A Quantitative Review

Abstract

Hypothetical scenario methodology is commonly employed in the study of genetic susceptibility testing uptake estimation. The methodology, however, has not been rigorously assessed and sizeable gaps exist between estimated and actual uptake for tests that have recently become available. This quantitative review explores the effect of several theoretically-based factors on genetic test uptake accuracy among a sample of 38 papers. These factors include verbal immediacy and temporal proximity of test scenarios, method of decision assessment, content of testing detail provided, processing demand required, and study features related to administration and sample. A number of assessed factors influenced uptake accuracy. Among these, temporal proximity of the genetic susceptibility test appeared to be the most consistent. There was also some evidence for effects of verbal immediacy and decision assessment method on interest in testing. We recommend strategies for increasing accuracy using hypothetical scenario methodology to examine genetic susceptibility test uptake prediction.

INTRODUCTION

Anticipating the extent of public interest in genetic susceptibility testing (GST) and gaining understanding of factors that underlie interest in such testing is vital in the face of emerging genetic technology development and dissemination. Accurate assessment of levels of interest in and potential uptake of these developing technologies is important for several reasons. Investigation into predictors of testing interest can inform policy and contribute to development of evidence-based decision aids and communication materials. Health service delivery systems can use such information to prepare for patient demand before tests become clinically available. Furthermore, understanding rates and predictors of interest in GST among diverse groups can help avoid disparities in dissemination, a source of ongoing concern with genetic technologies. The prospective nature of genetic technology has required researchers and clinicians to forecast interest for years before the technologies become ready for integration into healthcare settings. Because GST is generally not yet available for many common diseases, hypothetical scenario methodology has often been used to assess testing interest and estimate upcoming need for services. This methodology has the benefit of allowing investigators to manipulate important test characteristics and contextual variables to understand better how these factors influence reported interest levels and intentions to test.

Hypothetical assessments are very often presented using vignette and vignette-type methodology which generally involves presenting a story about or representation of a person in a situation. Vignettes are a tool with many advantages; materials can be produced relatively quickly and cost-effectively, can be administered under most conditions without special preparation, and have the capacity to convey scenarios in a standardized way. Little methodological research has been performed to examine these techniques, however. Furthermore, it has been suggested that responses to hypothetical scenarios may not accurately reflect actual behavior¹. Individuals often have difficulty projecting themselves into the future and predicting their own behavior with accuracy². Furthermore, participants’ anticipated uptake ratings are, at best, a behavioral intention, which has long been shown to have a less than optimal association with eventual behavior in a number of domains³. Indeed, it is common in the literature to find a substantial gulf between anticipated and actual GST uptake once tests become available and actual use rates can be determined^4–6.

It is unlikely that GST for common health conditions such as diabetes, heart disease, and most cancers will become widely available in the near future. Thus, hypothetical scenario methodology will continue to be an important tool for clinical and social and behavioral researchers to understand GST uptake. Moreover, longstanding use of hypothetical scenario methodology in this area provides a substantial research base with which to assess the methodology and test what elements might optimize the accuracy of behavioral outcomes.

The assumption is that a realistic narrative of a hypothetical scenario should result in higher concordance of participants’ responses with actual utilization rates. Therefore, the challenge for vignette methodology is to provide a written narrative that describes a clinical or other scenario as realistically as possible. An objective for developing hypothetical scenarios is to elicit the cognitive and affective processes that would likely occur in real-life decision making, and, in so doing, maximize predictive accuracy.

Information processing models suggest that a variety of scenario characteristics may affect perceived realism of hypothetical testing scenarios in ways that can influence the degree to which an individual engages with and carefully considers the information therein. Examinations of the Heuristic-Systematic Processing Model⁷ suggest that details underscoring the importance of and accountability for a decision lead to increased systematic engagement with, and therefore deeper processing of, decision-related content^{8, 9}. For example, hypothetical scenarios might include information linking disease risk or gene prevalence to an individual’s specific demographic group thereby increasing self-relevance and importance and in turn, more thoughtful or systematic consideration of the testing option. Inclusion of cues to importance and accountability are important to consider as they are more likely to naturally occur in actual GST as opposed to hypothetical scenarios where the influence of issues such as interpersonal and intra-familial relationships and the prospect of receiving genetic counseling are often absent.

Research has shown that a key language quality affecting realism and in turn, engagement is verbal immediacy¹⁰. Verbal immediacy is defined by the degree of directness indicated between the source and recipient of a communication; it is facilitated by use of more immediate language that indicates and/or elicits approach and psychological closeness between communication partners. For example, communication in the second person (i.e., “you”) is considered more immediate than third person (i.e., “he/she”). Important to hypothetical scenario methodology is the verbal immediacy dimension of “denotative specificity,” which indicates that it is important to avoid ambiguity in descriptions of an object (e.g., describing a testing context in concrete terms). These types of language characteristics might affect immersion and engagement in a hypothetical scenario, thereby affecting predictive accuracy.

A major limitation of hypothetical scenario methodology is that the outcomes necessitated by these methods are anticipated behaviors and future intentions rather than actual behavior. However, other conceptual factors may suggest ways in which use of the methodology might affect concordance between intentions and actual behavior. Construal level theory¹¹, for example, posits that more proximal decisions are based upon more concrete, contextual details whereas distant-future decisions are based largely on more abstract, decontextualized factors. Accordingly, it is well accepted¹² that the more proximal the behavioral intentions the better they predict actual behavior^{13, 14}. Thus, theories of information processing would suggest that higher intention-behavior congruence might be achieved via consideration of temporal proximity, that is, the extent to which a decision is portrayed as being immediate or having immediate consequences.

The approach used to assess interest in genetic testing also could influence intention-behavior congruence. For example, questions about interest in testing that require individuals to summarize their complex cognitive process in a yes/no response may result in information loss and thus, reduce intention-behavior congruence. Assessments that provide individuals with decision options organized in a manner that more closely represents how individuals might think about these decisions might increase intention-behavior congruence and in so doing improve accuracy of estimates of test uptake.

A final point for consideration in enhancing predictive accuracy of hypothetical scenario methodology is the amount of effort that is required to process the hypothetical scenario. There is growing evidence to support that the general public has a low level of knowledge about genetics and a tendency to misestimate related personal disease risk^15–17. Text-dense descriptions with high literacy demands may impede an individual’s ability to engage with and process content and thus reduce predictive accuracy¹⁸. Approaches that require too much effort for some target audiences to thoughtfully consider hypothetical genetic testing scenarios may reduce intention-behavior congruence.

To date, there has been no systematic assessment of hypothetical scenario methodology with respect to the role of conceptually-grounded scenario characteristics and their influence on interest in GST. To this end, we reviewed studies that employed hypothetical scenarios to assess the association of specific scenario features and content with reported interest in GST. For the purpose of this report, we consider lower levels of interest in GST to be more accurate. We base this assumption on consistent observation of the pattern, most common in the breast cancer literature, that actual uptake of testing has been lower than anticipated interest reported prior to testing availability^4–6 in cases where such comparisons have been performed. Though there is generally wide variability in uptake rates in the GST literature, here we examine predictors of hypothetical test intention based on the general pattern that hypothetical uptake overestimates actual test uptake.

We propose a few general hypotheses. Increased verbal immediacy, as indicated by more direct and specific language, and increased temporal proximity, as indicated by descriptions of more immediate consequences, should lead to lower, and thereby more accurate, rates of testing interest. In terms of GST details presented within scenarios, we make no specific hypotheses, but rather aim to determine which types of testing details are associated with more accurate levels of interest in testing. We performed this review with the intent to suggest some directions for improving hypothetical scenario methodology for use in studying GST in order to more accurately and consistently predict testing uptake. Our aim is to examine variables within past vignettes to be able to suggest ways of improving vignette methodology and increasing the congruence between testing intentions and behavior.

MATERIALS AND METHODS

Sample

We identified published papers that used hypothetical scenario methodology to evaluate interest in GST. For our purposes, a hypothetical scenario was defined as a situation in which individuals were asked to make a testing decision but no actual test was offered. We included manuscripts published in or after 1993 (the year BRCA1 was cloned¹⁹), that (1) focused on personal decision-making regarding genetic testing for a disease where a positive result did not indicate certainty of developing the disease (i.e., Huntington’s Disease was excluded under this criterion), (2) did not involve actual testing (e.g., providing a blood sample) and (3) enrolled adults who had not been diagnosed with the disease of interest. We performed a search of the literature using three databases: Medline, PsycINFO, and Scopus. Our search process involved number of unique search terms (e.g., “genetic testing”, “genetic decision making”) and an iterative process in which the reference sections of identified manuscripts were examined to identify any additional publications. This initial search yielded a total of 44 published studies.

For each of the 44 studies, we required the exact wording of any GST information provided to participants, the exact wording of the testing interest question and response options, a description of which response options were used to indicate interest in testing (e.g., yes alone or yes + probably), and finally, the ability to identify a sample denominator to use to calculate the proportion interested in testing. For papers with missing data, we attempted to contact corresponding authors a minimum of three times to acquire specific details of the hypothetical scenarios. If contact was not made, we attempted to contact additional paper authors. We removed from the data set six papers for which we were unable to collect these data either because the author had not retained the necessary information or because we were unable to make contact. Through these procedures, we arrived at a final set of 38 papers (see Table 1).

Table 1.

Papers included in review set listed by year of publication.

Paper	Year	N	Disease	Multiple GST Interest Questions in Study Identifiers	% Interest
Lerman24	1994	121	Breast/ovarian cancer		95
Lerman25	1995	105	Breast/ovarian cancer		91
Struewing26	1995	140	Breast/ovarian cancer		95
Julian-Reynier27	1996	124	Breast/ovarian cancer		96
Lerman28	1996	45	Colon cancer		82
Patenaude29	1996	47	Cancer, general		87
Andrykowski30	1997	598	Cancer, general		82
Bratt31	1997	100	Prostate cancer		91
Jacobsen13	1997	74	Breast/ovarian cancer		46
Tambor32	1997	473	Breast/ovarian cancer		69
Braczkowski33	1998	200	Breast/ovarian cancer		77
Graham34	1998	501	Colon cancer	Q1: Baseline	81
				Q2: Less test accuracy	77
Mogilnger35	1998	354	Breast/ovarian cancer		67
Ulrich36	1998	1450	Prostate cancer	Q1: Prostate cancer	83
			Breast/ovarian cancer	Q2: Breast/ovarian cancer	76
Durfy17	1999	543	Breast/ovarian cancer	Q1: Baseline	88
				Q2: Self-pay for test	69
Glanz37	1999	426	Cancer, general		72
Lipkus38	1999	266	Breast/ovarian cancer		65
Ludman39	1999	91	Breast/ovarian cancer	Q1: Insurance pays for test	71
				Q2: Self-pay for test	44
Petersen40	1999	1373	Colon cancer		92
Bosompra41	2000	622	Cancer, general		20
Bratt15	2000	110	Prostate cancer		94
Diefenbach42	2000	126	Prostate cancer		74
Donovan16	2000	220	Breast/ovarian cancer		91
Kash43	2000	1007	Breast/ovarian cancer		72
Miesfeldt44	2000	326	Prostate cancer		89
Myers45	2000	413	Prostate cancer		86
Roberts46	2000	203	Alzheimer’s	Q1: Available preventive treatment	58
				Q2: Available treatment to delay onset	55
				Q3: More immediate risk	63
				Q4: Baseline	50
				Q5:Less test accuracy	78
				Q6: Less certain risk information	96
Capelli47	2001	108	Breast/ovarian cancer		58
Green48	2001	72	Breast/ovarian cancer		69
Kinney49	2001	95	Breast/ovarian cancer		82
Neumann50	2001	314	Alzheimer’s	Q1: Test completely predictive	79
				Q2: Test partially predictive	45
Press51	2001	246	Breast/ovarian cancer		71
Armstrong52	2002	272	Breast/ovarian cancer		58
Bottorff53	2002	761	Breast/ovarian cancer		29
Bunn54	2002	1836	Colon cancer	Q1: Testing time frame six months	32
				Q2: Testing time frame one month	19
Botorff21	2003	651	Breast/ovarian cancer	Q1: Baseline, no testing information	19
				Q2: Baseline, little testing information	90
				Q3: Baseline, more testing information	86
				Q4: Less test accuracy	72
				Q5: Low altered gene frequency	58
Sanderson55	2004	1960	Heart disease	Q1: Heart disease	69
			Cancer, general	Q2: Cancer (general)	64
Westmaas56	2005	186	Lung cancer		60

Coding Categories and Items

We developed a theoretically-driven coding protocol generating closed-ended items to assess our key constructs of interest and other important factors (e.g., sample demographics). Each of the categories of constructs is described below.

Verbal Immediacy.

Verbal immediacy is often assessed in the communication literature using a rating system with multiple subcomponents¹⁰. Because of the succinct nature of the scenario texts, we instead opted to code for specific components of immediacy evident within these texts. We coded scenarios for voice (i.e., second or third person or the combination), use of terminological descriptors (i.e., “imagine”), use of target group descriptors (e.g., “women”, “people with a family history”) and lastly, mention of the test administrator (e.g., a doctor) to characterize more versus less immediacy.

Temporal Proximity.

We coded scenarios for inclusion of descriptors related to the proximity of the proposed test. This category consisted of an item describing whether or not the genetic test was described as currently available (is now available, is not yet available, or not mentioned), and an item assessing the proposed timing of the genetic test (test to take place in six months or sooner versus some time beyond six months). Due to the slow evolution of genetic susceptibility testing for common diseases and the length of time during which hypothetical vignette methods have been employed, we also included the year in which the study was published as a broad indicator of temporality with respect to public awareness of genetic testing.

Details about the Genetic Test.

The specific passages describing the genetic test were coded to assess whether each of the following information elements was mentioned: population prevalence of the disease, age of disease onset, survival rate, disease risk if genetic test is positive, disease risk if genetic test is negative, treatment options, name of the gene of interest, the concept of genetic heritability, test error rate, testing procedure (e.g., a blood test), psychosocial risks, and insurance risks. Two additional items assessed the cost of the test and whether the test results would be informative for all test takers.

Decision Assessment.

Testing decision assessment items dealt with the format of response options offered to participants, in other words, how the decision outcomes were conceptualized. This category consisted of an item assessing the polarity of the response scale (bipolar vs. unipolar), and the number of points in the scale (e.g., a yes/no response was two scale points whereas a 5-point Likert-type item was five).

Cognitive Demand.

Cognitive demand items assessed the effort required to take in and process the presented testing information. Demand included the number of information points presented overall, the number of words in the testing scenario, number of multi-syllabic words, number of sentences, and the number of words per sentence. The latter three items are standard measures of literacy demand¹⁷.

Study Features.

Study features described specifics of hypothetical scenario administration. The number of participants in the study, recruitment method (random or non-random), survey method (written, spoken, or a mixture of both), and the means by which testing information was presented were coded for each vignette. For the information presentation variable, studies were denoted based on whether or not they used a block presentation, which we defined as presenting three or more continuous sentences of testing information followed by a question that assessed interest testing. Sample-related study features included the following characteristics of the participants: average age, gender (male only, female only, or a mixture), racial composition (whether any racial group was overrepresented compared to the 2000 census), mean educational attainment (high school degree or less versus post-high school), geographical location, majority religion, percentage of the sample that was married, percentage with children, and whether the sample was recruited based on having a family history of the disease of interest.

Coding

Before providing the papers and instrument to our coders, we highlighted passages in each vignette text to identify the content that should count as testing information, the testing interest question, and the figure to be counted as the percent of participants interested in testing. Testing information was comprised of any exact text given to participants during the course of a study that dealt with GST and was followed by a testing interest question. If necessary, we calculated the overall percent GST interest from other data provided. If a study included multiple GST scenarios and multiple interest questions, each set was separately identified and coded.

Because there was such a wide variety of ways in which interest in testing was assessed, we used each study’s definition of testing interest (e.g., top 2 responses on a 5-point scale; a “yes” response on a yes/no question) or, if the paper did not group responses into an “interested” category, we used the most common metric from the other papers that assessed testing interest using the same number of response options.

Two independent coders coded each paper. Agreement testing and refinement of the instrument was an iterative process wherein, after an initial training, coders reviewed papers in blocks of seven or eight. They then met to discuss differences on items with a kappa below 0.6²⁰. The final agreement statistics were computed after coding the concluding block of papers. A portion (N=11) of the coding items were discarded at this stage due to insufficient inter-coder agreement, the remaining 42 items were retained, having kappas ranging from 0.6 to 1.0. Finally, coders met to reconcile any remaining differences to form a complete data set. Following the data collection, response options for some items were collapsed to allow for more meaningful comparisons among the small number of studies in the data set.

Analysis

We conducted a descriptive, exploratory analysis examining the relationship between each of our variables and percentage interest in testing. We used SPSS for Windows, Chicago, Version 14 to conduct all analyses. All initial analyses were one-way ANOVAs or linear regression. We ran follow-up analyses (Fisher’s LSD) to assess simple effects if an initial overall ANOVA revealed a significant relationship. Because we were interested in each unique relationship, we did not include any control variables in these analyses (for full results report, see Table 2). We also conducted a multivariate logistic regression analysis to assess the contribution of several significant predictors in the prior bivariate analyses. We included seven variables, as well as study sample size. These variables included year of study, method of testing information presentation, mention of a test administrator, mention of heredity, test availability status, test timing, and number of scale points. Statistical significance was assessed as p<0.05.

Table 2.

Items by Category and Their Relationship with Percentage Interested in Testing

Item	p	ηp2 / r2 a	Response	N	M (%)	SD
Disease of interest	.50	.10
			Breast, Breast/Ovarian Cancer	27	70.4	19.7
			Prostate Cancer	6	86.2	7.1
			Colon Cancer	6	63.8	30.4
			Lung Cancer	1	60.0	-
			Cancer, General	6	64.4	23.9
			Alzheimer’s	8	65.4	17.5
			Heart Disease	1	69.0	-
Verbal Immediacy
Voice	.48	.025
			Second person	11	66.6	24.1
			Third person	7	77.9	14.9
			Both	36	70.7	18.4
Use of ‘imagine’	.76	.002
			Used	12	69.2	16.5
			Not used	42	71.2	20.1
Demographic descriptors	.17	.036		54	-	-
Mention of test administrator	<.001*	.22
			Mentioned	2	25.4	9.3
			Not mentioned	52	72.5	17.3
Temporal Proximity
Study year	.001*	.19		55
Test availability	.02*	.14
			Available now	10	57.2	25.6
			Not yet available	24	77.0	14.5
			Not specified	20	70.2	17.8
Test timing	.001*	.19
			Less than 6 months	8	50.7	23.0
			6 months or more	46	74.3	16.4
Measure of decision outcome
Response scale polarity	.38	.036
			Unipolar	4	78.3	10.9
			Bipolar	38	67.3	22.3
			N/A	13	74.5	15.0
Number of scale points	.022*	.095		55
Details about the genetic test
Population prevalence of disease	.21	.030
			Mentioned	8	69.4	18.7
			Not mentioned	46	78.7	21.3
Disease onset	.21	.030
			Mentioned	8	69.4	18.7
			Not mentioned	46	78.7	21.3
Survival rate	.29	.021
			Mentioned	1	91.0	-
			Not mentioned	53	70.4	19.2
Risk of + test result	.32	.019
			Mentioned	31	73.1	16.8
			Not mentioned	23	67.7	22.1
Risk of – test result	.27	.023
			Mentioned	19	74.7	15.6
			Not mentioned	35	68.6	20.8
Treatment options	.67	.004
			Mentioned	9	68.2	18.5
			Not mentioned	45	71.3	71.3
Name of gene	.78	.002
			Mentioned	7	68.7	14.8
			Not mentioned	47	71.1	19.9
Heredity mentioned	.004*	.15
			Mentioned	21	80.1	11.7
			Not mentioned	33	64.9	20.8
Error rate	.48	.010
			Mentioned	11	67.1	16.2
			Not mentioned	43	71.7	20.0
Testing procedure	.30	.021
			Mentioned	37	72.6	18.6
			Not Mentioned	17	66.7	20.6
Psychosocial risk	.40	.014
			Mentioned	1	87.0	-
			Not Mentioned	53	70.5	19.3
Insurance risk	.60	.005
			Mentioned	5	75.2	19.8
			Not Mentioned	49	70.3	19.3
Test cost	.42	.033
			Free	4	82.5	12.2
			Cost associated	10	67.9	14.3
			Not mentioned	40	70.3	20.7
Informative to all	.78	.001
			Not informative for all	3	73.8	12.5
			Not mentioned	51	70.6	19.6
Effort required to process information
Number of info pieces	.12	.045		54	-	-
Number of words	.57	.006		54	-	-
Number of multi-syllabic words	.86	.001		54	-	-
Number of sentences	.78	.002		54	-	-
Words per sentence	.069	.062		54	-	-
Study Features
Study N	.043*	.075		55
Recruitment method	.009*	.12
			Random	17	59.3	26.3
			Non-random	38	74.5	15.1
Survey method	.12	.080
			Written	20	69.6	15.7
			Spoken	29	67.3	23.3
			Both	5	87.7	10.6
Info presentation	.007*	.13
			Block presentation	32	75.9	23.9
			Non-block presentation	23	61.3	23.9
Age	.70	.004		40
Gender composition	.13	.075
			All male	5	86.8	7.7
			All female	25	69.5	20.0
			Both	25	66.8	21.2
Racial composition	.88	.048
			White overrepresented	23	69.9	21. 6
			Black overrepresented	5	71.3	17.0
			White and black overrepresented	3	80.0	10.2
			White and other overrepresented	2	57.5	19.1
			Black and other overrepresented	2	69.0	2.8
			Not stated	6	71.8	10.3
Education	.025*	.13
			Beyond high school	40	66.1	21.3
			High school and below	7	88.1	6.8
			Not reported	8	72.4	14.1
Geography	.17	.066
			US	35	67.0	19.1
			Outside US	16	72.0	22.6
			Not stated	4	86.4	14.6
Religion	.80	.008
			Majority Christian	5	72.9	10.2
			No majority	3	76.0	10.4
			Not stated	47	69.1	21.6
Married	.27	.038		55
With children	.16	.53		55
Family history	.041*	.12
			No history	29	63.9	22.4
			Family history	22	78.2	16.2
			Both	4	66.9	7.1

^aEffect size measures partial eta squared and r squared

RESULTS

The average year of publication for studies in this sample was 1999 where the earliest was 1994 and the latest 2005. The body of literature assessed in this analysis targeted testing interest for a total of seven diseases (breast/ovarian cancer, prostate cancer, colon cancer, lung cancer, general cancer, Alzheimer’s, and heart disease). The most common disease was breast or breast/ovarian cancer which was the focus of 27 out of a total of 55 GST interest inquiries. Studies most often employed non-random, convenience sampling (N=38) and averaged around 500 people (M=499.9, SD=517.0) per sample. Study participants were more likely to be female (female only inquiries N=25; both male and female N=25, male only N=5), white (whites overrepresented in 23 studies), highly educated (post-high school education N=40), and based in the US (N=35). Family history status was mixed (no family history N=29, family history N=22). See table 2 for more descriptive reports.

Verbal Immediacy

Mention of who would administer the genetic test was associated with much lower interest (greater accuracy) in estimated uptake of testing than those where an administrator was not mentioned F(1,52)=14.53, p<.001, η_p²=.22. Aside from this finding, there were no other verbal immediacy items significantly associated with interest in testing (neither voice nor use of specific descriptors).

Temporal Proximity

All three of our temporal proximity items were associated significantly with test interest. Year of publication was significantly associated with interest in testing, β=−.44, p=.001, r²=.19, such that the later a study was conducted, the lower the interest. Testing interest also was associated with test availability, F(2,51)=4.22, p=.020, η_p²=.12. Post hoc analyses revealed that scenarios wherein the test was described as being currently available were associated with lower interest in testing than scenarios where the test was described as not yet being not yet available. Additionally, scenarios describing testing that would occur within six months or less were associated with much lower interest than scenarios where testing was to occur later F(1,52)=12.44, p=.001, η_p²=.19.

Details about the Genetic Test

Whether or not GST information mentioned the concept of heritability (i.e., mention of genetic heritability of disease risk) was found to be associated significantly with interest level, F(1,52)=9.29, p=.004, η_p²=.15; studies that did not mention heritability were associated with lower levels of testing interest than studies that specifically mentioned heritability. None of the other testing information details (e.g., age of onset, risk level with a positive test, name of gene) were found to be significantly associated with interest in testing.

Measure of Decision Outcome

Response polarity (i.e., bipolar versus unipolar) did not affect testing interest, however, we did find that including more points in the response scale was associated with lower interest in testing, β=−.309, p=.022, r²=.095.

Cognitive Demand

We found no significant associations for any of our demand items (e.g., overall number of information pieces, words per sentence) with level of interest in testing.

Study Features

Methodology.

We found no differences in testing interest by survey administration method, but all other assessed methodology features were associated significantly with differences in percent interest in testing. A linear regression analysis showed a significant association between number of participants and interest, β=−.27, p=.043, r²=.075, such that studies with a larger sample size reported less testing interest. Recruitment method also was significantly associated with interest in testing, F(1,53)=7.38, p=.009, η_p²=.12, where random recruitment was associated with a lower interest in testing than non-random recruitment methods. Finally, we found a significant association with block information presentation where there was lower interest in testing in studies that did not use the block presentation format, F(1,53)=7.79, p=.007, η_p²=.13.

Sample Characteristics.

Risk status of the sample (i.e., sample selected for having a family history versus not) was associated with interest in genetic testing, F(2,52)=3.41, p=.041, η_p²=.12. Post hoc analyses revealed that studies in which samples were not selected so as not to include individuals with a family history of the targeted disease (i.e., general population samples) reported significantly less interest in genetic testing than studies in which samples were specifically selected to include those with a family history. A significant association was also found for average education level, F(2,52)=3.96, p=.025, η_p²=.13 and testing interest. Post hoc tests indicated that less educated samples reported more interest in genetic testing. It is notable that education level was not reported for 8 of the studies. All other items (e.g., age, gender, percent married) were not significantly associated with testing interest.

Multivariate Analysis

Upon entering the seven variables with significant bivariate associations and study sample size into the multivariate equation, we found that none were highly correlated so all were entered into forward and backward stepwise regression models. Year of study, β=−.29, p=.014, mention of test administrator, β=.407, p=.001, and block method of information presentation, β=−.285, p=.013, were retained in the model, r²=.40. Studies that were conducted later, did not present GST details in block format, and that mentioned a test administrator were associated with the lowest levels of test interest.

DISCUSSION

The primary aim of this report was to identify whether there were key characteristics and details of hypothetical vignettes of GST scenarios that would be associated with more accurate estimates of test uptake. We explored a number of factors associated with well-accepted conceptual models and suggested to increase realism of hypothetical genetic testing scenarios and heighten engagement with and immersion in the content of hypothetical vignettes. Some of our findings were consistent with our hypotheses in suggesting that specificity in details related to test administration, timing, disease heritability, and several study design features resulted in lower, and likely to be more accurate, estimates of uptake of genetic testing. The implications of our findings are described below.

While we suggested that scenarios high in verbal immediacy should increase the realism of genetic testing scenarios, only mention of a test administrator, an indicator of verbal immediacy, was significantly related to decreased interest in testing. It is notable that a testing administrator was mentioned in only two of 55 inquiries and in those cases the administrator was a doctor. Therefore, further work here may be in order before any strong conclusions are drawn.

We also suggested that temporal proximity of scenarios would enhance psychological realism and improve accuracy of estimates of interest in genetic testing. Findings for three items were strongly supportive of our hypothesis. In each case, the more imminent the proposed test seemed, the lower the percent interest in genetic testing. This was true regardless of the degree of specificity of temporal proximity (e.g., year of the study versus provision of specific details of test availability and proposed timing of the test).

With respect to details conveyed to the participant about the genetic test itself, we found a good deal of variability across studies. Some vignettes described highly specified scenarios including details about the prevalence of the disease, and the specific numeric risk associated with different test outcomes whereas others provided scant detail about the test and the related disease. We coded for 14 types of information, but in the end, only the presence or absence of heredity description, that is a mention of genetic heritability of disease risk, significantly influenced interest in testing. Finding that only one detail was significantly associated with improved accuracy of uptake estimates was unexpected. Also surprising was that numbers of words, sentences and multi-syllabic words, all indicators of cognitive effort that might be required to engage with the information contained in the vignette, was not associated with estimates of test uptake. Taken together, these results suggest that vignettes with increased descriptive detail about the test, which are also likely to be longer in length and verbiage, may not improve accuracy of uptake estimates. However, alternatively, added detail and verbiage also does not appear to undermine the ability to engage with the scenario.

This result must be considered further given that a sizeable majority of the studies targeted white highly educated American women who might engage differently with information about testing than other target groups. Conceptual models of information processing and literacy skills would suggest that there are very likely to be situations in which greater detail would improve systematic or deeper consideration of testing information, but also that scenarios too dense in verbiage and conceptual information would be difficult for some populations to comprehend. Determining the optimal threshold of detail for enhancing realism without increasing subject demand deserves further study, particularly as genetic testing becomes available to more diverse populations. It is also important to note that generally the studies included in this review did not systematically include standard manipulation checks to assess whether the details of the scenarios were retained or their meaning understood by the target audiences.

Issues of study design and assessment of interest also were associated with lower interest in testing. Many of our findings (association of greater sample size and random recruitment with better accuracy) are aligned with conventional sampling wisdom. Our results furthermore suggest that whether or not to undergo genetic testing is not a simple yes – no decision. Using scales with more response options to assess test interest was associated with lower interest rates. As have others before us^{5, 21} we submit that the broad range (from 19–95%) of reported interest in testing we observed within the same disease category (breast-ovarian cancer) may be due in part to variability in the way outcomes were measured (i.e., how the test interest question is asked). Response scale type has previously⁵ and in our results been linked to differential rates of interest in testing. We suggest that this may be because a greater number of response options increases the specificity of the decision options in ways more representative of how individuals might give consideration to genetic testing.

In terms of presentation of information, the format of the vignette was associated with interest in testing. Presenting vignettes as three or more continuous sentences followed by a question to assess interest, what we called a “block format”, was associated with higher rates of interest than formats relying on only a question or other approaches. However, whether the scenarios were self- or interviewer-administered was not associated with interest in testing.

Though we have identified a number of potential influences on hypothetical GST interest predictions, we do not suggest that influence is limited to these factors. Because our sample size was limited to hypothetical GST papers that we were able to locate and for which we were able to collect full materials, our power to detect effects was necessarily reduced. It may be that variables that were not identified as being significantly related to interest here will become so as the body of literature grows. We also acknowledge a limitation in that studies with multiple test interest items were more heavily weighted in the data set. Furthermore, because the purpose of this study was to investigate methodological factors, in our analysis we averaged over multiple diseases, samples with and without family history, and so on. It may be the case that factors not applicable in our entire sample are important in reference to a particular population as, for example, each disease included is associated with distinct clinical features. These factors, along with the relatively skewed samples collected for studies in the data set may explain, in part, cases where previously held findings (e.g., effects of cognitive demand¹⁸) were not replicated here. Experimental investigation of some of these factors (e.g., varying them experimentally within vignettes and assessing testing intention) would no doubt help to elucidate further their role in GST decision-making.

As we suggested at the outset, hypothetical vignette methodologies are likely to continue to be an important tool we use in understanding and shaping the potential impact of genetic testing for common health conditions. Our findings suggest several recommendations for improving the accuracy of the results yielded by these studies. Generally, the field would benefit from more attention to and consistency in the methods used to assess GST interest. Specific recommendations suggested by our findings are as follows:

1. Questions used to assess GST interest should give a broader range of response options (rather than yes/no) to approximate better the range of true response to GST. Response categories might be informed by pilot testing or qualitative investigation to characterize better possible responses to GST options.

2. Testing scenarios should give information that increases the immediacy of the decision and occurrence of the test. Giving indication that a hypothetical test is planned to occur in an immediate future will likely enhance accuracy of responses over setting a test in a relatively vague future.

3. Length of text and specific descriptors remains an open question that likely will vary across target groups. Exploring scientific media stories about genetic discovery or other arenas might be helpful in determining what types of information laypeople find useful when evaluating genetic tests²².

4. Hypothetical GST scenario content should be based on systematic, theoretical foundations and appropriate pilot testing to ensure scenarios achieve their desired effect. We employed the heuristic-systematic processing model but there are numerous others that might be informative depending on the research questions.

5. Studies involving hypothetical GST scenarios should be held to rigorous study design with consideration given to sample size estimation and, wherever possible, random assignment.

The availability of genetic tests is likely to continue to lag behind the pressing social and behavioral questions that must be addressed if we are to shape the development and dissemination in ways that can maximize the utility of these technologies. Thus, improving the hypothetical vignette methodology to truly simulate real-world processes should be an important priority. To this end, we also should begin to consider more innovative approaches to heighten realism and immersion of hypothetical scenarios. Media advancement has provided for technologies that can immerse participants in scenarios. The most innovative example is immersive virtual environment technology, commonly known as virtual reality, a technology with a history of use in behavioral research²³. Immersing participants into a realistic, simulated decision scenario may be a great alternative to arguably more sterile, psychologically distant traditional analogs. Special attention paid to factors identified here when crafting hypothetical scenarios or when choosing methods may bring us closer to the goal of understanding when and why individuals will choose to participate in GST.