Public preferences for the return of research results in genetic research: A conjoint analysis

Abstract

Purpose

Recent policies specifying criteria about which individual research results (IRRs) to return leave considerable discretion to researchers. This study investigated which types of results the public wants when participating in genetic research and whether preferences differ based on willingness to participate.

Methods

A representative survey of U.S. adults used conjoint analysis to measure priorities among eight principles of a results policy for a proposed large-cohort study. Policy preferences were measured using twelve tasks where respondents chose between two groupings of the policy principles. Stratified analysis compared those self-identified as likely or unlikely to participate in genomic research.

Results

Of 1,515 respondents, 56% would participate in the proposed study. All eight principles were positively endorsed by participants (all p<0.003) with priority placed on providing results at no cost, and returning well-validated results for treatable and serious diseases. Providing detailed result reports was more highly valued than providing staff to explain results (p=0.0005). Receiving results about major changes in risk was marginally disvalued by those unlikely to participate (p=0.35).

Conclusion

Public preferences for well-validated IRRs for serious, actionable diseases agree with emerging recommendations. However, since preferences for receiving IRRs vary, some choices should be offered to research participants.

Introduction

Large-scale studies that pair genetic samples with participants' medical data have become an important staple of biomedical research. As understanding of gene-disease relationships grows, genotypic data collected by researchers may hold increasing clinical or personal value for research participants. Decreasing costs of exome and genome sequencing and the growth of population- and clinic-based biobanks mean some clinically relevant genomic variants are likely to be identified in large numbers of research participants. ¹

Studies have shown that genetic research participants are interested in receiving, and may expect some individual research results (IRRs) in return for participating. ^2-11 A public survey about a possible large-cohort study found that 90% wanted to learn their IRRs regardless of the utility of the findings; 75% said they would be less likely to participate in a study that did not return IRRs. ⁸ Discussion about IRRs has shifted from the question of whether IRRs should be returned to deliberations about which results should be returned, when, and in what manner. Evidence of this shift can be found in emerging guidelines that suggest researchers and clinicians may have an ethical obligation to return at least some results. ^12-14 While these guidelines specify some criteria about which IRRs to return, considerable discretion is left to researchers.^12,14 For example, the 2010 NHLBI guidelines state that individual genetic results should be offered to study participants if the finding has important well-established health implications; the finding is actionable; the test is analytically valid; and the study participant has opted to receive the information during an informed consent process. ¹⁴ Under this NHLBI guideline, one study found that researchers using whole exome sequencing could unearth between 3,955 and 12,579 findings per participant that could qualify for disclosure.¹⁵ Another study simulated the return of incidental findings by first classifying 2,016 genes linked to Mendelian disorders into “bins” based on their clinical utility and validity. Using their binning algorithm, the researchers estimated that a person undergoing whole exome or whole genome sequencing could expect 0-2 results with clear utility that the authors felt should be returned without eliciting patient preference, and up to 10 additional results of unclear utility, where the decision to report them should involve the patient. ¹⁶ Similarly, Dorschner et al. screened 1,000 adults for 114 variants associated with medically-actionable conditions and estimated that 1.2% of patients of African descent and 3.4% of patients of European descent would carry a pathogenic or likely pathogenic variant.¹⁷ Initial clinical recommendations by the American College of Medical Genetics (ACMG) advised that clinical laboratories performing whole exome or whole genome sequencing report findings from 57 genes related to 30 serious actionable conditions, regardless of patient age or preference, and estimated that 1% of reports would include one of these variants. ¹⁸ It is not clear that any of these recommendations could or should be used to decide what researchers should divulge; current financial, regulatory, staff, and logistical burdens would make returning 4,000 findings to a single participant impossible. These examples illustrate that there are multiple, widely varying resources that a researcher might use, appropriately or inappropriately, to prioritize what to return. ^{19, 20} Considerable input will be required to help researchers make these decisions, while balancing policies, participant preferences, the financial and logistical burdens of returning IRRs, and the integrity of their research.

This study contributes to the ongoing discussion about returning IRRs by measuring the policy preferences of a critical stakeholder in genetic research – participants. This research used a conjoint experiment to assess public preferences for eight principles related to the return of research results in a hypothetical large-cohort study and to examine if the priorities for a results policy differ between respondents who would or would not be willing to participate in the study. Additional questions measured general opinions about the study and the return of research results.

Materials and Methods

To inform the design of the survey, ten focus groups conducted in three U.S. cities explored the types of research results people valued, the importance of clinical validity and utility, and preferences for communicating IRRs. The focus group study, reported on elsewhere ¹⁰, collected opinions on several current issues in the debate over IRRs including: 1) whether to return IRRs, ^21-24 2) the evidence required to support return of a given result, ^12,14,26 3) the criteria defining which results to return ^12,14 4) the financial and clinical support needed to provide results, ^10,14 and 5) whether returning IRRs would influence participant enrollment and retention. ⁸ The focus group study found several attributes of IRRs that might influence research participants' interest in the information. These included whether the result is associated with a common disease, a serious disease, or a condition for which treatment is available, whether the result is associated with a large change in risk, and whether the risk estimate is well-validated or might change over time. The availability of detailed result reports and study staff to explain them were also seen as important. Many expected that results would be provided at no cost to research participants. These were not the only factors that appeared to influence whether someone would want an IRR, but they were among the most important observed.

Building on this work, an online survey was developed. The conjoint experiment that is the focus of this paper was part of a larger survey instrument designed to measure public attitudes about the design of a national cohort study proposed by the National Human Genome Research Institute ²⁷ (NHGRI) that would follow 500,000 Americans over time to study genetic and environmental causes of disease.²⁸ The broader survey focused on several aspects of biobank participation, including study design preferences, privacy concerns, broad consent, and the return of individual research results.

The survey was conducted online in English and Spanish, according to respondent preference. Sample selection and survey administration was managed by Knowledge Networks (KN, which has since become the KnowledgePanel run by the company GfK.). ²⁹ During the field period of April 29 – May 12, 2011, potential respondents 18 years and older were randomly selected from KN's web-enabled master panel of approximately 35,000 US residents. For households without internet access, KN provides a laptop computer and an Internet connection to complete the surveys. Black non-Hispanics and Hispanics were oversampled. Hispanics were sampled from KN's KnowledgePanel Latino Panel, which comprises a representative sample of U.S. adults who identify with Hispanic culture and values. The Hispanic sample was stratified to include equal numbers of individuals who chose to take the survey in English and in Spanish.

Survey respondents viewed English or Spanish versions of a three-minute video describing the proposed study. Respondents who could not view the video saw a written script of the video, accompanied by a diagram of study components. Hyperlinks to the study description and definitions were available throughout the survey (Supplementary Materials). Respondents were then asked about their support for, and willingness to participate in the study as well as their interest in receiving IRRs. The survey was qualified by the Johns Hopkins University Institutional Review Board as exempt (Application # NA_00040539).

Because of the length of the survey, half of the respondents were randomly selected to participate in the conjoint experiment discussed here; the other half participated in a conjoint measuring preferences for overall study design. No significant demographic differences were observed between the two groups.

Conjoint Experiment

Respondents completed a conjoint experiment that used 12 tasks to measure the relative importance of eight principles (Table 1) of a policy to return IRRs in the hypothetical cohort study. The hallmark of conjoint analysis is that it forces participants to make a series of trade-offs, in this case between different principles of a policy to return IRRs. This technique simultaneously provides important information on what principles of a results policy will determine whether it is viewed favorably, and quantifies how each attribute is valued by the respondents by forcing them to reveal their preferences through a structured series of choice experiments. The eight IRR policy principles, which were presented to participants as they appear in Table 1, were selected based our previous survey and focus group research and by reviewing the relevant scientific literature, policy statements, and guidelines addressing the return of IRRs ^{8,10,12,14,21-23}.

Table 1. Eight principles of a policy to return individual results.

Principle	Description
Study staff are available to discuss results	Participants are able to discuss what their specific results mean with someone from the study.
Results provided are returned at no cost.	Results that are returned by the study are provided at no cost to participants.
The study returns results for common diseases.	Individual results for common diseases are returned to participants.
The study returns results that show major changes in disease risk.	Individual research results are returned participants for diseases when a participant's risk of disease shows a major change (either very high or very low).
Results are confirmed before they are returned.	Individual research results are confirmed in other studies before they are returned so that researchers are more confident about the results. Confirmation may take months or years.
Result reports include detailed explanations.	The study provides participants with a report that included a detailed explanation of the results including what researchers know about the gene and its relationship to a disease.
The study returns results for serious diseases.	Individual research results for serious diseases are returned to the participants.
The study returned results for treatable diseases.	Individual research results for diseases that can be treated or prevented are returned to participants.

Respondents were told “Imagine study planners have decided to offer to return some individual research results to interested participants. Over the course of a 10-year study like this, there may be hundreds or thousands of results for each participant. Therefore study planners will need to figure out which individual results to return and how to return them.” Then, in a series of 12 choice tasks (exemplified in Figure 1), the policy principles were divided into two mutually exclusive groups, and respondents were asked to choose their preferred policy.

Figure 1.

Example of one of twelve tasks each participant completed in the conjoint exercise on a policy to guide the return of individual research results.

To identify which of the eight factors were most valued and whether preferences differed between those who would and would not participate in the proposed study, we constructed paired-comparison choice tasks using an orthogonal array. A total of 256 profiles combining sub-sets of the eight principles that could guide return of IRRs can be derived. Policy profiles for the conjoint exercises were created using a 2ˆ8 array from the SAS catalogue ³⁰; the principles were separated into two subsets – Policy/Plan A and Policy/ Plan B.^31-33 Consistent with good research practices ³⁴, the experimental design characteristics were verified and the resulting experiment was perfectly orthogonal. In each of twelve tasks, respondents were asked to choose the policy (A or B) they would prefer if they were a participant in the study. No further explanation or justification was required and the tasks were forced choices.

An initial pilot study (February 9-11, 2011; n=68) evaluated survey length, logic, skip patterns and wording. Five cognitive interviews evaluated respondent comprehension of the conjoint experiment. A second pilot (March 18-21, 2011; n=72) looked for an order effect, since the conjoint exercise could precede or follow a battery of questions that might influence response. No ordering effect was observed.

Statistical Analysis

Multivariate regression utilized McFadden's conditional logistic regressions ³⁵. The dependent variable was considered the respondents' choice of plan, while the independent variables were eight dichotomous variables representing whether each of the principles were included in the plan or not. The conditional logistic regression does not allow for respondent characteristics such as age or willingness to participate to be entered into the regression because each respondent acts as their own control in the regression, which measures differences in individual responses to experimentally- controlled variations in the IRR policies.

Results focus on the value that respondents placed on each factor as represented by the parameter estimate from the conditional logistic regression. Overall differences between respondents who would and would participate in the cohort study were estimated through stratified regression, and differences in values of each principle were tested using t-tests and the Wald test.

Results

A 57% survey response rate yielded 1,523 respondents who participated in the conjoint exercise; the 1,515 (99.4%) providing complete data on the conjoint serve as the basis of the analyses below. Both weighted and unweighted demographic distributions of the complete sample and those completing the conjoint analysis were comparable to the US 2010 census (Supplementary Material). Two in three respondents were able to watch the introductory video; the remainder read the study description. No differences were noted between those who watched the video and read the description with respect to the findings below.

Immediately after viewing or reading the study description, 56% said they would definitely (16%) or probably (40%) participate in the study, and 44% would probably not (30%) or definitely not (14%) participate. (See Supplementary Table). There were 78% who would be interested in receiving their IRRs if they participated in the study; 9% would not be interested, and 13% were not sure. Given a choice, 57% said they would want all of their IRRS, 32% would choose to receive only results for “diseases that can be prevented or treated” and 12% “would not want or need any individual research results.” Slightly more than half (56%) agreed “it would be fair if I only received the results of findings for diseases that can be prevented or treated; 62% of those likely to take part in the study agreed with this statement.

Determinants of Preferences for a Return of Results Policy

Summarizing the views of all respondents who completed the conjoint, all eight principles of a potential policy to return IRRs significantly and positively influenced preference for the policy. (Figure 2) The most attractive attribute tested was “providing results at no cost to research participants”. The remaining seven attributes can be divided into two categories: types of results to return and provisions for returning them. After returning results at no cost, the next most attractive principles reflected the types of results that should be returned - - those related to treatable and serious diseases, and results where the genotype-phenotype association has been confirmed. Although respondents viewed the return of IRRs about common diseases and major changes in risk positively, these characteristics were significantly less important. Respondents valued a detailed result report more than having study staff available to discuss results.

Figure 2.

Overall results of conjoint analysis to determine the significance and relative importance of eight policy attributes related to the return of individual research results (n=1,515).

We also examined the conjoint analysis results separately for respondents who would or would not be willing to take part in the hypothetical study (Figure 3). Examining the priorities of people most likely to take part in a large biobank could identify attributes of a results policy that would promote their participation. Similarly, identifying preferences of those not willing to participate could reveal principles that might underlie, in part, their decision.

Figure 3.

Comparison of the significance and relative importance of eight policy attributes related to the return of individual research results, between those likely (n-=861) and unlikely (n= 654) to take part in a hypothetical large genomic research study (n=1,523).

Overall, the relative ranking of the principles was similar in those willing (n=861) and unwilling (n=654)-to-participate (Figure 3). In both groups, providing results at no cost was the most attractive attribute (OR of finding the policy acceptable=1.82 in those willing to participate; OR=1.89 in those unwilling;-p=0.25. Returning results for treatable diseases (OR=1.69 and 1.59, respectively, p=0.09) was the second most important attribute tested). In both groups, respondents placed a significantly higher value on receiving reports with detailed interpretations than having study staff available to discuss results.

The magnitude of preferences for three principles differed between those who were willing and unwilling to participate. Respondents who would be unwilling to participate placed a higher value on results related to common diseases (unwilling respondents OR=1.15, willing respondents OR=1.06, p=0.02). In contrast, respondents who would take part in the cohort study placed a significantly higher value on policies that share results for serious diseases (willing OR=1.69 vs. OR=1.58, p=0.03), and results showing major changes in risk (ORs= 1.17, 0.97 respectively, p<0.001) compared to those who were not willing to participate. (Figure 3/Table 2) Moreover, people unlikely to participate in the study viewed the return of results showing large changes slightly negatively (OR=0.97) though they did not see returning these results as a significant aspect of policy. p=0.35).

Given that receiving results at no cost was the most attractive attribute among all respondents, we examined if this preference was influenced by respondents' ability to pay and split the sample by the median income of the study group. Receiving results at no cost ranked high in both income groups, but was the most influential attribute in the lower income group and the third most important factor in the higher income group. The difference in the magnitude of importance in the two groups was not significant (p=0.17).

Economic effects returning results would have on the proposed study

Two additional survey questions explored attitudes about the effect that the additional costs of returning results could have on the proposed study. Despite widespread interest in results, nearly four in ten respondents (38%) agreed with the statement “returning individual research results would make the study too expensive.” A second question asked directly about one potential trade-off of providing results. Respondents were told “It might cost the study a great deal of money to return IRRs. To be able to afford returning IRRs, the study would have to enroll fewer people. A smaller study would take longer to make scientific discoveries. In thinking about this trade-off, which study design would you prefer?”. Putting the cost in terms of slowing scientific discovery, 84% would prefer a study that returns no individual results that takes ten years to make its findings (24%) or a twelve-year study returning “a small number of individual research results about diseases that people could prevent” (60%). A twenty-year study that returns all individual results was preferred by 16% of respondents.

Limitations

This research is based on respondents' reactions to a large, proposed, National Institutes of Health-funded biobank. Responses about what individuals prefer in this hypothetical study may not correspond perfectly to actual future preferences or preferences in other studies. ¹⁰ Second, although all eight policy principles tested were regarded as significant positive aspects of a policy on the return of results, indicating that influential meaningful principles were selected for study, there are clearly other principles of an IRR policy that were not measured that could be important in a results disclosure policy. These include policies on privacy protection, data sharing and access; and inclusion of results into the medical record. Finally, the attribute describing large changes in risk did not distinguish between large increases and large decreases in disease risk. Thus, we cannot discern whether respondents' preferences were motivated by a desire for information about increased risk or reduced risks.

Discussion

This is the first study to employ conjoint analysis to elicit public preferences for returning IRRs. By forcing the participants to make trade-offs on their preferences for various attributes of a results policy, the data accurately prioritize public preferences about the principles of a results policy. In addition, this methodology enabled respondents to participate in an exercise that simulated some of the real-world policy challenges facing the research community. Consistent with several other studies, these data show high willingness to participate in large cohort genetic research, as well as a strong interest in receiving IRRs. ^8,10,11,36

Our data provide insight about which individual results are most valued by the public. Overall, all eight principles of a return of results policy tested were significant positive influences in the selection of policies to disclose research results.

In research, it appears that individual results that are returned should be delivered at no cost to participants if possible. The issue of cost was particularly important among participants in middle and lower income groups; issues of justice or fairness might arise in studies that ask individuals with lower incomes to bear some costs. The overall importance of receiving results for free may reflect a common public opinion that that receiving some IRRs is a form of compensation for study participation. ^{7, 8} With that said, when presented with a tradeoff that researchers might make to accommodate the costs of returning results (lengthening the time to scientific discovery), a large majority preferred to minimize the adverse effects on research - 84% would prefer to receive a small number of actionable results (60%) or no results at all (24%) in order to avoid significantly slowing research progress. The public may be largely willing to accept few or no results when the impact of the costs is explained.

When prioritizing which results to return, these data show the public puts the most emphasis on three categories of results: results for serious health conditions; results for diseases that can be treated or prevented; and well-validated results whose relationship to disease has been clearly established. These features were significantly more important than returning results for common diseases or those showing major changes in disease risk. For example, 62% of those who would participate in the proposed study felt it would be fair if they “only received the results of findings for diseases that can be prevented or treated”. Although participants might like to learn about a wide range of results given the opportunity, it seems many people feel that receiving a set of actionable valid results would be fair and acceptable. Returning even a limited number of actionable results at no cost to participants could satisfy some of the public's most important priorities.

Some recommendations for the return of IRRs ^12-14 have emphasized returning findings that confer high risks of disease. Learning about high magnitudes of risk was a positive attribute of an IRR policy, but was significantly less important than learning about valid, actionable results. This echoes the observation in our focus groups that participants would be equally interested in variants conferring low, high, or even average risks of disease. Many people may be interested in finding out where they stand in relation to others, as well as in finding out diseases they are at high risk for developing.

The public priorities expressed here are fairly consistent with emerging recommendations that a few, actionable results be returned at no charge to participants. ^12-14 While much interest has been paid to potential tension between participants who might want to receive all of their IRRs, and the research community who want to limit their workload in this area; it appears that the public and research community's views are not far apart. Concordance of researcher and public opinion about what results take priority is encouraging. However, the two groups may diverge somewhat when defining which results are actionable or have utility. The biomedical community's definition of clinical utility may be relatively narrow compared to the public's definition since many results with no proven clinical use may hold strong personal utility to a research participant. ^{7, 10, 11, 36-39} Since actionable results are important to people, the potential difference in how researchers and participants interpret the term “actionable” ^{7, 38} should be considered as IRR policies are drafted, to avoid misunderstandings.

Respondents who said they would not take part in the study were significantly less likely to support policies to return results on serious diseases and slightly preferred policies that would not return results showing major changes in disease risks. These observations suggest that there may be an element of fear or apprehension about the types of information that could be delivered, although we do not have definitive information to support this hypothesis. The concentration of such attitudes among people not likely to take part in genetic research raises questions about whether this apprehension is related to mistrust of the research enterprise, or whether fear of bad news, in the form of IRRs, might discourage participation.

Not everyone wants to receive all or any IRRs. Some people will be more interested than others in a given result. This empirical finding supports other suggestions that studies returning an IRR must allow participants to choose whether or not to receive it. ^12-14 This finding also supports the ACMG's recent revised guideline on clinical incidental findings, which now states that patients undergoing clinical sequencing should be allowed to opt-out of any of the ACMG's prescribed set of incidental findings. ⁴⁰ It also appears that, in the public's opinion, it is significantly less important for a research study to provide staff to explain results than it is to provide a detailed result report. It may be that as more information becomes available online, participants will seek their own sources to interpret their IRRs. Whether research participants can get satisfactory, accurate, comprehensible information and what they go on to do with are is still issues that warrant consideration.

While this study focuses on IRRs in a proposed research study, the findings may also be relevant to the clinical community as they consider delivery of incidental findings.

The routine practice for the delivery of such findings has not yet been established. For example, the ACMG recommendation ¹⁸ that patients undergoing genomic sequencing receive some incidental findings may become the standard, but the details of how best to accomplish this are far from settled. Our findings on some of the public's priorities for receiving research results might be one of many places of departure for those examining how to address incidental findings in the clinic. For example, our observation that it is significantly more important to people to receive a detailed result report than to have access to staff to explain results suggests that it may be worth examining the use of different report formats to lessen burdens on clinical staff working with genomic test data. Additional research in clinical settings would help to compare and hone the effectiveness of such reports.

This study has identified eight aspects of a potential IRR policy that are of significant importance to the general public. Developing clear policies for returning IRRs that address each of these eight aspects may provide transparency to research participants, foster positive researcher-participant relationships, provide clear guidance to researchers about their obligations for returning results, and aid study planners in their efforts to successfully recruit and retain participants. It may be as important to explain a study's IRR policy with respect to these eight aspects as to create the public's “ideal policy”. That being said, modest return policies that can deliver even a small number of well-understood clinically actionable results may be viewed very positively by most participants.