The Translational Potential of Research on the Ethical, Legal and Social Implications of Genomics

Abstract

Federally funded research on the ethical, legal and social implications of genomics (“ELSI” research) includes a programmatic charge to consider policy-relevant questions and to communicate findings in venues that help inform the policy-making process. In addressing this goal, investigators must consider the range of policies that are relevant to human genetics, how foundational research in bioethics, law, and the social sciences might inform those policies, and the potential professional issues that this translational imperative raises for ELSI investigators. We review these questions in the light of experiences from a consortium of federally funded Centers of Excellence in ELSI Research, and offer a set of policy recommendations for program design and evaluation of ELSI research. We conclude that it would be a mistake to require that ELSI research programs demonstrate a direct impact on science or health policy; however, ELSI researchers can take steps to increase the relevance of their work to policy makers. Similarly, funders of ELSI research concerned to facilitate policy development can help by building cross-disciplinary translational research capacities, and universities can take steps to make policy-relevant research more rewarding for scholars in the humanities, social sciences, and law.

When the U.S. Human Genome Project was initiated in 1990, one of its innovative components was the Ethical, Legal and Social Implications (ELSI) Research Program (1). Initially a joint effort involving the U.S. Department of Energy and National Institutes of Health (NIH), the ELSI Research Program, now administered by the National Human Genome Research Institute (NHGRI), has supported a diverse portfolio of research grants and training awards. Although these research projects span multiple fields and employ a range of methods, they share the aim of examining the societal implications of genomic research and the medical innovations that it may make possible.

“ELSI research,” as it is commonly known, incorporates a wide range of disciplinary perspectives, including bioethics, history, law, medicine, genetics, economics, philosophy and the behavioral and social sciences. Unlike related work supported by other federal agencies such as the National Endowment for the Humanities or the National Science Foundation, a common challenge for ELSI research has been a programmatic charge to consider policy-relevant questions and to communicate its findings in venues that help inform the policy-making process. Just as basic genomic scientists are encouraged to pursue “translational” research that enables the creation of medically useful tools, ELSI researchers have a “translational” mandate to pursue studies that assist in managing practical policy problems involving human genomics (2).

Historically, criticism of the ELSI program has focused on its capacity to meet this practical challenge (e.g., 3). The challenge is particularly important for the consortium of Centers of Excellence in ELSI Research (CEER) established in 2004. The CEER consortium was created by the NHGRI with an explicit imperative to “play a role in ensuring that relevant ELSI research findings and deliberations are made available to policy makers as appropriate” (2).In addressing this goal, CEER investigators have had to consider the range of policies that are relevant to human genetics, how foundational research might inform those policies, and the potential professional issues that this translational imperative raises for ELSI investigators. We review these questions in the light of the CEER consortium's experiences to date, and offer a set of recommendations for clarifying the goals and strengthening the translational impact of policy-related ELSI research. We suggest that it would be a mistake to require that ELSI research programs demonstrate a direct impact on science or health policy; however, ELSI researchers can take steps to increase the relevance of their work to policy makers. Similarly, funders of ELSI research concerned with policy translation can foster cross-disciplinary translational capacities, and universities can take steps to make policy-relevant research more rewarding for scholars in the humanities, social sciences, and law. In making these recommendations, we hope to stimulate discussion aimed at developing consensus about how best to achieve the policy impact envisioned for the ELSI program.

What constitutes policy?

As ELSI researchers consider policy-relevant research, they must first decide how to define “policy.” Many definitions focus on governmental action, e.g., “the expressed intent of government to allocate resources and capacities to resolve [an] expressly identified issue within a certain timeframe” (4). Governmental policy-making at both state and federal levels has important implications for genomic research and health care. Some policies are specific to clinical and public health applications of human genetics, such as state newborn screening programs and licensure for genetic counselors, and laws protecting against genetic discrimination (e.g., 5). Other governmental policies are more general, but have important implications for human genetics. Examples include federal regulations governing research with human participants (6); National Institutes of Health (NIH) policies on data-sharing and funding priorities (7); policies related to intellectual property and patenting; the Clinical Laboratory Improvement Amendments (CLIA) (8); and Federal Drug Administration (FDA) regulations regarding oversight of clinical tests (9).

However, policy actions with important implications for translational genomic research occur in other venues as well. Clinical practice guidelines, for example, help to set standards of care for the use of genomic technologies in health care. Guidelines are frequently sponsored by professional societies or other non-profit organizations; some, such as the Working Group for Evaluating Genomic Applications in Prevention and Practice (EGAPP) (10) and the U.S. Preventive Services Task Force (USPSTF) (e.g., 11), are independent panels sponsored by governmental agencies. Both the American College of Medical Genetics and Genomics (12) and the American College of Obstetrics and Gynecology (13) have played leading roles in development of clinical practice guidelines for genetic testing. The American Society of Human Genetics (14) and other organizations have considered policies related to genetic testing that are within their area of focus, including guidelines and advisory statements from the American Academy of Pediatrics (15) and the American Medical Association (e.g., 16).

At the local level, research and health care institutions and individual laboratories and clinics also develop and implement policies that influence the conduct of genomic research and its clinical translation. Frequently these are attempts to implement broader national and professional guidelines. In these processes, institutions play a role analogous to the “laboratory of the states” in federal policy-making, by providing experience-based assessments of alternative policy options and interpretations. Local experimentation has been particularly influential in the development of IRB policies for genomic research, informed consent practices, community engagement policies, and biobank governance (17-20).

There are also important forms of policy development that are rarely codified in particular documents or by specific organizations. At the societal level, for example, investment in genomic science, including allocation of research funding and capital investment can have an important impact on genomic research and its translation (21). Initiatives on the part of federal funding agencies, such as NHGRI's launch of a research program to assess outcomes of genomic sequencing in health care (22), contribute significantly to the evidence available to policymakers. Pharmaceutical companies, biotechnology industries, and private investors all influence how genome science will be moved from the laboratory to potential health applications. Health payers, by deciding what tests and procedures they reimburse, and at what level, influence both investment decisions and the potential for clinical implementation, thereby influencing the translational process.

The evolving unwritten norms of clinical practice and the background social narratives that inform patient, family and public decision-making about the use of genomic information are also important in directing genomic translation. For example, The DNA Mystique: the Gene as a Cultural Icon (23), a widely cited qualitative media study of public understanding of genomic concepts, set the stage for policy concerns about genetic determinism and discrimination. As that work showed, background cultural influences are relatively invisible in the distilled language of official policy documents at the governmental or professional society levels, but are critically important to analyze as key moral commitments, beliefs, and practices that shape the reception of genomic technologies by potential users.

These diverse forms of policy-making call for a definition of policy that is not limited to governmental action. Merriam-Webster suggests that policy can be defined as “a definite course or method of action selected from among alternatives and in light of given conditions to guide and determine present and future decisions” (24). This definition has the scope to include both informal and formal forces that set the course of genomic translation, and underscores the broad range of research studies that should be considered “policy-relevant.”

Recommendation #1

The translational mandate for ELSI research should be interpreted expansively, to include governmental and professional policy but also the broader social, economic, and cultural influences that shape public reception and use of genomic information.

Interrelatedness of policy actions

A second major observation from ELSI research experience is that policy-relevant research must take into account how multiple policy actions interact to influence a particular aspect of genomic research or healthcare. The following examples drawn from ELSI research serve to illustrate the complexity:

Data-sharing

Mechanisms for sharing population and clinical data that incorporate demographic, phenotypic, genomic, and health measures could expedite the translation of genomic research findings into applications to improve health care (25). NIH policies have required funded researchers to have data-sharing plans for more than a decade, and recent policies provide strong incentives for depositing genomic data in a federal repository (26). However, as ELSI research has shown, data-sharing decisions may also be affected by policies surrounding informed consent (27, 28), agreements researchers have implemented with communities where research is conducted (29), proprietary interests (30), and in the case of health data, by the Health Insurance Portability and Accountability Act's Privacy Rule (31), as well as any relevant state legislation. How research data are organized and stored, the extent to which measures are harmonized, and what constitutes “data” are also relevant for implementing effective policies for data sharing. Private policies, even at the level of individual laboratories, can therefore have an impact on how data sharing is implemented (32). Data repositories, once established, need policies for evaluating data requests and models for oversight and stewardship of data resources (19, 33-35), including procedures that ensure informational privacy, accountability and appropriate information return to patients (36-40).

Genetic/Genomic testing

Clinical molecular genetic tests represent an important product of genomic research, and are projected to improve diagnostic capabilities and guide safe and effective drug therapy. Like data sharing in genomic research, the introduction of new genetic tests raises an array of policy questions related to both governmental and non-governmental action. Central among these is the development of clinical practice guidelines addressing the standardization of testing technology and the appropriate use of different tests. ELSI research initiatives have helped inform such guidelines, (e.g., 41-47), and today, ELSI researchers are participating in major initiatives and debates surrounding the development of clinical standards for whole exome and whole genome sequencing (e.g., 48-57).

All of these professional practice initiatives proceed in the context of other important policy questions for the laboratories that develop and provide the testing. It is still unclear what degree of federal regulatory oversight is appropriate for a laboratory-developed test (LDT) that is not marketed to other laboratories (21), when LDTs are used for medical diagnostic or predictive purposes. Moreover, policy decisions about the patentability of a gene or the scope of a genetic test's patent can affect whether or how researchers, clinicians, or patients can access new genetic discoveries (25).

Recommendation #2

ELSI research is uniquely positioned to assemble and assess the interaction of policies occurring under the broad definitional rubric laid out in this paper. Because few single research projects can span the whole spectrum of relevant policy spheres, opportunities should be sought or where possible created to pursue policy-relevant ELSI research through collaborations between studies addressing different levels of policy-making, rather than by individual research projects attempting to extrapolate policy implications in isolation.

How can ELSI research contribute?

ELSI research contributes in different ways at different phases of the policy-making process: Phase 1: identification of policy issues; Phase 2: development of policy options; and Phase 3: evaluation of policy effects (Figure). This process is envisioned as iterative, because the evaluation of policy efforts typically will identify new policy concerns that require additional attention. Empirical evidence and conceptual clarification can be useful at every stage, defining problems for which policy action is needed, elucidating stakeholders' views of alternative solutions, and evaluating outcomes of different policy approaches. Justifications for different options must also be considered. Normative research offers an opportunity to explore ethical justifications for policies, or provide insights into the values at play. Similarly, legal research can illuminate the ways in which different legal theories may affect innovation, dissemination, and application of new genetic diagnostic and therapeutic techniques. ELSI research can also contribute conceptual frames for understanding the challenges of genomics, and for guiding both policy development and further research. The study of health economics can inform how markets will view the introduction of genomics into health care and public health, especially with the heightened focus on and expectations of “personalized” or “precision” medicine.

Figure 1.

Contributions of ELSI research at different phases of the policy-making process.

Drawing on their scholarly work, ELSI researchers also frequently contribute directly to the deliberations of policy-making bodies. Advisory bodies convened by the federal government, the Institute of Medicine, and other organizations considering issues related to genomics typically include representation from the ELSI community or testimony from ELSI researchers (e.g., 58-61). ELSI researchers usually participate in these policy activities as individual professionals, sometimes leading critics to discount their contributions as independent of their programmatic mandate to do policy-relevant research (62). However, participation of ELSI researchers in policy activities is an outcome of the creation of a robust community of scholars sparked by the NHGRI ELSI research program, whose work and resulting insights have policy relevance for genomics.

Recommendation #3

The full interdisciplinary range of ELSI research at the stages of policy issue identification, policy option development, and policy impact assessment should be considered forms of translational ELSI research. The contributions that individual ELSI researchers make to policy initiatives cannot be easily separated from the research programs these individuals direct and should be considered evidence of the translational impact of ELSI research.

Encouraging a robust ELSI portfolio of policy research

As ELSI researchers apply a range of research methods to policy issues, three critical questions arise. Each of these questions bears on the potential for ELSI research to achieve a high level of rigor and address the needs of policy-makers.

1. How can different research methodologies be leveraged to produce optimal approaches for evaluating policy problems?

In anticipating their mandate to articulate and disseminate the policy-relevant lessons of their research, ELSI researchers have come to appreciate the advantages of interdisciplinary research designs. By themselves, purely descriptive studies such as surveys, ethnographies, legal reviews, and conceptual taxonomies can provide important data for policy-makers at all levels but they provide little guidance on the merits of acting on the data in one way or another. Similarly, strictly normative analyses may provide ethical, legal, or clinical frameworks or priorities for decision-making, but almost always include uncertainties that require empirical evidence to resolve. Moreover, studies using the methods of social science, ethical or legal analysis need to be grounded in the lived experience of clinical professionals and their patients. As a result, mixed-method studies by collaborative multidisciplinary teams become increasingly important. The CEER program was established to help create institutional hubs for such teams, but even outside of these Centers, ELSI researchers increasingly find the need to link studies to achieve the kind of peripheral vision required for policy translation. To date, the major venues for these collaborations have been the ELSI program's various RFA-sponsored consortia, and the involvement of ELSI researchers in larger genome science initiatives such as the CSER Consortium, eMERGE, the Human Microbiome Project, the Welcome Trust/NIH H3Africa Initiative, and the NBSeq initiative (e.g., 57, 63-66). Each of these initiatives involves collaboration among basic, clinical and ELSI research, focused on a particular aspect of genome science. The ELSI component of these projects offers a powerful opportunity for cross-communication between ELSI and other aspects of genomic research. Since these efforts are topic specific, however, by necessity they leave large segments of the ELSI research community without natural venues for connection or collaboration. The series of NIH-sponsored “ELSI Congress” meetings are one episodic response to that need (67), and they have stimulated the creation of a new international on-line forum for collaboration for ELSI research, the “ELSI 2.0 Collaboratory,” that also seeks to provide ELSI researchers with the means to cultivate such teams (68).

As such work moves forward, limitations need to be considered. Efforts to anticipate policy problems always have the potential to expend resources on the investigation of undesirable effects from genomic technology that never materialize, and thus may be open to charges of “catastrophizing.” Looking down the road toward future problems in the application of genomics also opens ELSI researchers to the charge of ignoring more proximal decisions in the design and implementation of genomic research that might either exacerbate or mitigate later problems (69-72). For these reasons, robust interdisciplinary collaboration is necessary not only among the empirical, normative and clinical disciplines of ELSI research, but also between ELSI researchers and genome scientists. As a new wave of genome and exome sequencing studies are showing, working closely with genome scientists, rather than at arm's length in advisory or consultancy capacities, can allow ELSI researchers to better target their down-stream inquiries and to provide a critical lens on the design and conduct of genomic research itself (22,63). Yet collaboration carries with it the potential of co-optation. Independent ELSI research can also play an important role in clarifying assumptions, values and implications of potential choices at different stages of the translational process. A mix of collaboration and communication across disciplinary boundaries is therefore likely to be most effective in producing robust policy-relevant findings.

Recommendation # 4

Research teams should explore and funders should promote a broad range of strategies to improve interdisciplinary communication and collaboration among ELSI researchers and between those disciplines and genome science.

2. How should policy-relevant research be disseminated?

As ELSI research addresses these different aspects of the policy-making process, appropriate dissemination of research findings is an important consideration. One of the most influential forms of dissemination is the contribution that ELSI researchers make to institutional and professional practice by collaborating on initiatives with their colleagues in genomic and genetic medicine and research. Insights and evidence from ELSI research projects can be instrumental to the work of university and hospital committees charged with developing responsible interpretations of national guidelines on issues such as genomic data management and IRB review, and ongoing relationships between ELSI researchers and genome scientists can significantly shape local professional cultures. Other avenues and strategies for dissemination are important to consider (and their effectiveness is potentially a topic of ELSI research). Policy briefs and expert testimony are both avenues for dissemination. The participation of ELSI researchers on regional and national advisory groups addressing a broad range of policy areas, and presentations by ELSI researchers to those groups or other policy-making bodies, are also important forms of dissemination. Publication in scholarly and scientific journals remains the mainstay for academic dissemination, allowing for collaborative development of knowledge across the many disciplines involved in ELSI research. However, impact on genomics policy development is often greatest if ELSI researchers make their work accessible to policy audiences in venues not typically used for their academic scholarship. To conduct their research, ELSI researchers become familiar with different professional literatures, languages and formats, and often appreciate that their work can have the most direct impact on genomic science if it is disseminated in ways atypical for their home departments and disciplines. Thus, junior ethics or sociology scholars may be inclined to disseminate their research results to genomic or policy audiences, either by publishing in the scientific press or through policy briefs, op-ed articles in newspapers, press releases, and presentations aimed at the general public, but may be advised by their disciplinary elders to keep their “eyes on the prize” of promotion and tenure as determined by the traditional standards of their academic homes. More senior scholars coming to ELSI research from careers in the humanities, social sciences and law may see these forms of dissemination as lying outside their academic responsibilities and resist them in the same ways that many bench scientists chafe at mandates to commercialize their basic science in the name of “translation.”

To create programmatic contexts in which dissemination efforts more directly targeted to policy and scientific audiences will be rewarded and accepted, funders and institutions might consider other national efforts to encourage more translational research in the biomedical sciences. Just as the NIH Clinical and Translational Science Awards and the institutional efforts they have inspired across the country are attempting to change the culture of the basic biomedical sciences, incentives are needed to turn the home disciplines of ELSI research in a more translational direction. Analogous to the ways in which many academic clinical and translational science programs provide core resources to help bench scientists apply and commercialize their research through proactive technology transfer services, ELSI researchers could benefit from services designed to package their work for scientific, public and policy audiences. Presentation methods could include policy briefs, op-ed articles in newspapers, press releases, and presentations aimed at the general public. Examples of efforts emanating from the current CEERs illustrate the range of possibilities (Table 1), and are representative of efforts undertaken by ELSI scholars across a range of institutional settings.

Table 1. Examples of Dissemination of Policy-Relevant Documents by CEERs.

CEER	Topic Area	Example
Center for Genetic Research Ethics and Law (CGREAL), Case Western Reserve University	Ethical and legal issues related to newborn screening	Testimony to the Ethics and Legal Workgroup for the National Newborn Screening Translational Research Network (Aaron Goldenberg PhD, MPH).
Center for Research on Ethical, Legal & Social Implications of Psychiatric, Neurologic & Behavioral Genetics, Columbia University	Return of incidental findings from genomic research	Testimony to the Presidential Commission for the Study of Bioethical Issues (Erik Parens, PhD). Available on the web (http://bioethics.gov/node/2783).
Center for Public Genomics, Duke University	The risks and benefits of intellectual property protections in genomics, including ethical, legal and social issues related to the patenting of DNA	Presentations by Duke CEER investigators (Robert Cook-Deegan, PHD and Arti Rai, JD) and former Duke CEER post-doctoral fellow (Sapna Kumar, now at the University of Houston School of Law) at the US Patent & Trademark Office roundtable on genetic testing diagnostic verification. Available on the web (http://www.genome.duke.edu/centers/cpg/cpg-contributions-BRCA/)
Center for Genomics and Society (CGS), University of North Carolina at Chapel Hill	Best practices, governance models, and ethical issues in biobanking research, including data sharing, sample ownership, broad consent, and confidentiality.	CGS investigator service on policy committees at local (UNC Committee on Tissue Banks and DNA Repositories), national (NCI “Best Practices for Biospecimen Research”, http://biospecimens.cancer.gov/practices/; NIH Cancer Genome Atlas Project “Human Subjects Protection and Data Access Policy”, http://cancergenome.nih.gov/PublishedContent/Files/pdfs/TCGA%20Human%20Subjects%20Protection%20and%20Data%20Access%20Policies%20Rev_2014-01-16.pdf) and international levels (International Cancer Genome Consortium “Informed Consent and Ethical Oversight Guidelines”. http://icgc.org/icgc/goals-structure-policies-guidelines/e1-informed-consent-access-and-ethical-oversight).
Center for Genomics and Healthcare Equality (CGHE), University of Washington	Ethical conduct of research involving American Indian and Alaska Native participants	Report of a collaborative workshop summarizing perspectives, illustrative scenarios, resources and take home points for effective research collaboration between universities and tribal organizations. Available on the web (https://dl.dropboxusercontent.com/u/1711621/TUIREW%20final%20report.pdf) and disseminated in hard copy to CGHE regional partners.

Recommendation # 5

Funders should consider mechanisms for the creation of specialized Policy Translation Resource Cores accessible by both NIH-funded and independently supported ELSI researchers, to help build translational capacity within the research community. These could be built on the existing translational components of CEERS, but could also be housed by other translational policy centers, national science policy programs (such as the National Academy of Sciences), or international ELSI research organizations.

In considering dissemination, ELSI researchers, like other scientists addressing policy-relevant questions, need to consider whether they should provide only research information and leave policy-makers to draw the policy conclusions, or use their expertise to recommend specific policy options (73). There is value to the position of “honest broker” (73), in which an expert seeks to help decision-makers consider the pros and cons of different policy alternatives, without making a specific recommendation. But there may also be times when it is appropriate for an ELSI researcher to take a position about a particular policy choice. Normative arguments in favor of or against a particular position on an issue are the expected outcomes of many studies in philosophy, bioethics, and law, and multidisciplinary ELSI projects typically include such components. For example, many ELSI researchers published analyses of the Genetic Information Non-Discrimination Act (GINA) prior to its passage, some in favor of the legislation and others raising cautions about this policy approach. (e.g., 74,75). Of course, all ELSI grantees must comply with relevant rules restricting the use of federal funds to support political lobbying. But academic analysis of issues relevant to policy is one of the missions of ELSI research, and ELSI researchers are expected to bring their normative views and recommendations to their professional roles as members of institutional, professional, and federal policy-making advisory committees and initiatives.

On the other hand, our experience suggests that responsible and effective policy development is a collective, community process, and requires the participation of multiple stakeholders. This means that individual ELSI investigators and projects are rarely equipped to promulgate mature policy proposals by themselves, and should not be held to that expectation. To be effective in “translating” normative conclusions into policy, ELSI researchers need to have access to the appropriate policy-making forums for the phase of the process they seek to address.

Recommendation # 6

Like individual genome scientists and scientific teams, ELSI researchers must be allowed the academic freedom to draw and report conclusions from their research, whatever implications these may have for the current direction or priorities of genome research. However, because ELSI research, like science, is a collective enterprise, funders and institutions should expect most policy recommendations to be promulgated through collaborative consensus mechanisms, often involving established policy forums, rather than directly from specific research projects.

3. How should ELSI research be evaluated?

Policies are animated by the interests, beliefs, and values of multiple stakeholders, and shaped by the constraints, ideals, and priorities of multiple environments. In addition, the policy-making process occurs in multiple public and private venues. ELSI research can provide empirical information and critical assessments for particular policy-making challenges, but questions remain about how the translational mandate of ELSI research should be assessed or evaluated. A starting point is to consider who evaluates ELSI research and for what purposes.

ELSI as an academic endeavor

Most ELSI research occurs in academic settings. As a matter of course, ELSI researchers are evaluated by academic criteria related to the originality and quality of their work and their recognition among peers. Formal assessment occurs in promotion and tenure decisions and contributes to the rigor and stature of ELSI research. Academic review is primarily focused on how peers evaluate a scholar's work, and does not address dissemination to policy makers. Nor is all ELSI research policy-relevant. Just as innovative robust bench science stands on its own merits whether or not it has a foreseeable commercial application, excellent work in the humanities, social sciences and law can be evaluated independently from its relevance to policy. It would be a mistake to compromise the intellectual integrity of these disciplines by diluting their standards of scholarly excellence, just as it would be to bring non-scientific criteria into the academic evaluation of genome scientists. But it is possible for funders and institutions to help the home disciplines and departments of ELSI researchers appreciate the value of policy-relevance and to build ELSI researchers' capacity to comply with their programmatic mandate.

Recommendation # 7

Academic institutions should experiment with creative ways to reward efforts to support translational mandates of policy-relevant research, by creating, for example, opportunities for leaves of absence to participate more directly in the policy-making process, crediting policy-related service activities during promotion and tenure reviews, and encouraging expert contributions to policy initiatives within academic communities.

Peer review of grant applications

Review of an applicant's ELSI research grant proposal represents another formal evaluation process. Like academic evaluation, this peer review process focuses primarily on the quality of the scholarly work proposed, and the likelihood that its goals will be accomplished based on the study design and the applicant's work to date. This process also includes an evaluation of potential impact, a broader question focused on the significance of the work to the field. Given the ELSI program's translational mandate, reviewers may be tempted to evaluate grant proposals on their promise to influence policy. However, ELSI research encompasses a broad range of investigations and methods, not all of it policy relevant. Further, the goal of NIH peer review is very specific - to inform the funding agency about the intellectual merits of a particular scientific or scholarly research plan, as the basis for the agency's funding decisions. Additional programmatic criteria, such as a project's translational promise or relevance to the agency's funding priorities, can be applied at different points in the process: when funding programs accept applications for review and at subsequent levels of decision-making about funding priorities. They should not affect an application's peer review priority score. This approach acknowledges both the range of ELSI research and the sometimes complex relationship that exists between a specific research project and its impact on science and health policy, in the same way that the distance between bench and bedside is acknowledged in translational genomic research.

Recommendation # 8

When applicants promise to influence policy directly as part of their projects, reviewers may legitimately evaluate how they propose to do so. However, different ELSI research studies vary in their policy relevance and potential for policy impact. Explicit discussion of policy relevance should therefore not be an a priori expectation in peer review.

Value of the ELSI investment

Neither of these formal evaluation mechanisms fully addresses the question of the value to society deriving from investment in policy-related ELSI research. There are no simple metrics to accomplish this evaluation, and it necessarily involves judgment. Three sources of data are helpful, as academic programs, funding agencies, Congress, or the public consider this question. The first is the body of ELSI research addressing policy questions, as manifested in publications and reports. One can ask whether this body of work addresses the questions and topics currently arising in policy-making around genomics or has contributed to the identification of previously unrecognized questions. Here it is important to note that the NIH can influence the scope of ELSI's policy domain through exploratory workshops and Requests for Applications, to ensure that critical areas are addressed. A second source of data is citations in policy documents: to what extent is ELSI-funded research cited as part of the policy-making process? The third source is the contribution of ELSI experts to the policy-making process, in the form of expert testimony; policy briefs; and participation on advisory and policy-making bodies. These sources of data remain largely untapped, and are themselves useful targets for further ELSI research.

Recommendation # 9

Efforts are needed to document the impact of ELSI research projects on science and health policy. Methods of assessing this impact will require input from policymakers, ELSI researchers, academic institutions, funding agencies, and the public. Such effort could be achieved through funded research, NIH task forces, or other initiatives undertaken by funding agencies or professional organizations.

Conclusions

ELSI research offers a range of methodological approaches yielding normative analysis and empirical data to inform the policy-making process. This work addresses all phases of the policy development process, across a broad range of policy-making venues. In addition, the community of ELSI researchers created through the NHGRI ELSI research commitment provides expertise as participants in advisory and policy-making bodies, and in expert testimony. Our experience in in seeking ways to increase the policy relevance of our work through the CEERs has informed both innovative approaches to dissemination (Table 1) and the recommendations we present for strengthening the translational impact of ELSI research (Table 2).

Table 2. Summary of Recommendations.

1	The translational mandate for ELSI research should be interpreted expansively, to include governmental and professional policy but also the broader social, economic, and cultural influences that shape public reception and use of genomic information.
2	ELSI research is uniquely positioned to assemble and assess the interaction of policies occurring under the broad definitional rubric laid out in this paper. Because few single research projects can span the whole spectrum of relevant policy spheres, opportunities should be sought or where possible created to pursue policy-relevant ELSI research through collaborations between studies addressing different levels of policy-making, rather than by individual research projects attempting to extrapolate policy implications in isolation.
3	The full interdisciplinary range of ELSI research at the stages of policy issue identification, policy option development, and policy impact assessment should be considered forms of translational ELSI research. The contributions that individual ELSI researchers make to policy initiatives cannot be easily separated from the research programs these individuals direct and should be considered evidence of the translational impact of ELSI research.
4	Research teams should explore and funders should promote a broad range of strategies to improve interdisciplinary communication and collaboration among ELSI researchers and between those disciplines and genome science.
5	Funders should consider mechanisms for the creation of specialized Policy Translation Resource Cores accessible by both NIH-funded and independently supported ELSI researchers, to help build translational capacity within the research community. These could be built on the existing translational components of CEERS, but could also be housed by other translational policy centers, national science policy programs (such as the National Academy of Sciences), or international ELSI research organizations.
6	Like individual genome scientists and scientific teams, ELSI researchers must be allowed the academic freedom to draw and report conclusions from their research, whatever implications these may have for the current direction or priorities of genome research. However, because ELSI research, like science, is a collective enterprise, funders and institutions should expect most policy recommendations to be promulgated through collaborative consensus mechanisms, often involving established policy forums, rather than directly from specific research projects.
7	Academic institutions should experiment with creative ways to reward efforts to support translational mandates of policy-relevant research, by creating, for example, opportunities for leaves of absence to participate more directly in the policy-making process, crediting policy-related service activities during promotion and tenure reviews, and encouraging expert contributions to policy initiatives within academic communities.
8	When applicants promise to influence policy directly as part of their projects, reviewers may legitimately evaluate how they propose to do so. However, different ELSI research studies vary in their policy relevance and potential for policy impact. Explicit discussion of policy relevance should therefore not be an a priori expectation in peer review.
9	Efforts are needed to document the impact of ELSI research projects on science and health policy. Methods of assessing this impact will require input from policymakers, ELSI researchers, academic institutions, funding agencies, and the public. Such efforts could be achieved through funded research, NIH task forces, or other initiatives undertaken by funding agencies or professional organizations.

Deliberate definition of policy-related goals is needed to ensure the appropriate dissemination of ELSI research to policy-makers and the public. Creative ways to broaden ELSI research dissemination efforts and support collective, collaborative efforts to bring ELSI research results to bear on policy issues continue to be needed. Evaluation of ELSI research must strike a balance, rewarding both rigor in application of research results to policy and broad dissemination, without placing unrealistic expectations on either the research or the policy-making process. We hope these recommendations will provide a starting point for discussion, aimed ultimately at the creation of consensus-based methods to guide investigators in applying their findings to policy questions and funders and institutions in evaluating these critical contributions.