Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 May 1.
Published in final edited form as: Am J Bioeth. 2020 May;20(4):62–70. doi: 10.1080/15265161.2020.1730483

The “Ought-Is” Problem: An Implementation Science Framework for Translating Ethical Norms into Practice

Bryan A Sisk 1, Jessica Mozersky 2, Alison L Antes 2, James M DuBois 2
PMCID: PMC7164659  NIHMSID: NIHMS1578946  PMID: 32208070

Abstract

We argue that once a normative claim is developed, there is an imperative to effect changes based on this norm. As such, ethicists should adopt an “implementation mindset” when formulating norms, and collaborate with others who have the expertise needed to implement policies and practices. To guide this translation of norms into practice, we propose a framework that incorporates implementation science into ethics. Implementation science is a discipline dedicated to supporting the sustained enactment of interventions. We further argue that implementation principles should be integrated into the development of specific normative claims as well as the enactment of these norms. Ethicists formulating a specific norm should consider whether that norm can feasibly be enacted, because the resultant specific norm will directly affect the types of interventions subsequently developed. To inform this argument, we will describe the fundamental principles of implementation science, using informed consent to research participation as an illustration.

Keywords: Ethics, Empirical Ethics, Normative Ethics, Implementation Science, Applied Ethics

Introduction

In this paper, we address social norms such as “researchers should obtain informed consent from competent adults before enrolling them in a research project” or “physicians should try to alleviate the pain of patients who seek relief.” These are the kinds of norms that policy groups and applied ethicists generally seek to develop when working in fields as diverse as medicine, engineering, or journalism. These are different from norms that individuals embrace (e.g., I will give 15% of my income to charitable causes), which are not meant to be generalized to society. They are also different from the highest level of moral principles such as the Kantian categorical imperative, which many find useful in identifying social norms, but must first be specified in order to guide the actions of individuals or society.1

Social ethics is focused on making good choices to allow people to live well together. For Aristotle, ethics belonged to the realm of the practical sciences, which concern the choices and actions of individuals and societies.(Aristotle 1980) Thus, ethics contrasts with “theoretical science, which seeks knowledge for its own sake …”(Shields 2016) Seen through this lens, when we draw conclusions in the realm of ethics, we are actually making choices regarding actions. Normative claims that are not (or cannot) be put into practice might have aspirational value, but these norms fail to fulfill the essential function of ethics if they do not eventually lead to ethical actions.

We argue that once a normative claim is developed, there is an imperative to effect changes based on this norm. However, enacting norms is a process that requires multidisciplinary collaborations of individuals with broad expertise beyond normative ethics. Rarely would an ethicist have the skill set necessary to perform each step in this process, yet, ethicists should fulfill a critical role in such multidisciplinary teams. To better guide this translation of norms into practice, we propose a framework that incorporates implementation science into ethics. Implementation science is a field of study dedicated to supporting the sustained enactment of interventions—most commonly interventions in medicine or public health.(Brownson, Colditz, and Proctor 2012) We further argue that implementation principles should be integrated into the development of specific normative claims as well as the enactment of these norms through interventions. In other words, ethicists formulating a specific norm should take into account whether that norm can be feasibly enacted, because the resultant specific norm will directly affect the types of interventions subsequently developed. To inform this argument, we will describe the fundamental principles of implementation science, using informed consent to research participation as an illustration.

A Practical Science – Moving from Ought to Is

Ethicists have argued for decades about the interaction and compatibility of normative and empirical ethics. This debate has primarily centered on the “is-ought” problem, i.e. whether empirical data (is) can lead to normative claims (ought).(de Vries and Gordijn 2009, Strech 2008, Ebbesen and Pedersen 2007) However, little work has focused on what should happen once a normative claim is developed to support actual changes in practice. We call this the “ought-is” problem: How does one implement an ethical rule or norm to ensure that it fulfills its primary purpose?

Drawing upon implementation science, we propose a framework for the translation of ethical norms into practice. (Figure 1) This framework is composed of sequential processes, beginning with normative ethics and ending with dissemination of interventions to improve ethical practice. Deliberations in normative ethics may result in what we call aspirational norms, which are broad claims that are easily agreed upon but difficult (or impossible) to implement. An aspirational normative claim might be that “No one in the world should die of hunger.” This claim would garner broad agreement, but it does not specify any particular actions. Such aspirational norms are valuable insofar as they serve as a “true North” to guide subsequent development of specific norms. Once an aspirational norm is developed, deliberations in applied ethics can generate specific norms. These specific norms provide directed guidance about the types of actions that ought to be enacted. Consider the following normative claim: “Every physician in the United States should question their patients about food insecurity and provide information packets about local resources to protect them from hunger.” This claim is specific and more feasible to be implemented. Although this specific normative claim will not fully rid the world of hunger, it provides an incremental step towards this broader aspirational norm.

Figure 1.

Figure 1.

Framework for Implementation of Ethical Norms

Note – Circular arrows indicate sequential iterative processes.

After a specific norm is formulated, investigators can develop interventions to enact this norm. These interventions will provide measurable outcomes to determine whether the intervention has succeeded or failed to enact the specific norm. As data accumulate from studies of interventions, investigators and ethicists can identify best practices. Once these best practices are identified, they should be disseminated broadly, occasionally with the backing of policy change.

How specific norms are formulated can have downstream effects on whether an intervention can feasibly enact the norm. As we will discuss later, applying implementation principles (such as stakeholder engagement) early in this framework can support the development of feasible specific norms and interventions that will improve ethical practice. If a specific normative claim requires actions or interventions that are not feasible, then this claim will fail to effect change in ethical behaviors and thus fails the overarching purpose of ethics. In fact, ethical statements that pose as short-term action items but cannot be implemented might foster guilt, cynicism, or despondency and inaction. Applied ethics should instead focus on multidisciplinary collaboration to do the best we can in our current circumstances, which may include taking small steps to remove barriers so more can be accomplished in the future.

Some might object that the implementation process we describe is not really a dimension of ethics: it is merely technical problem solving. Such a view is not tenable if one embraces a broad notion of ethics as a practical science in which “decisions fall into the realm of ethics when they pertain to things within our control that will either show respect or fail to show respect to human beings.”(DuBois 2008)[Chapter 3 Page 46] Such a broad view of ethics is also embraced in the growing literature on microethics in healthcare, which has “a focus on the ethics of everyday clinical practice” and “what happens in every interaction between every doctor and every patient.”(Komesaroff 1995) [Page 68] There is no neat separation between ethical and technical decisions. As we will show, the design and implementation of interventions can strongly influence whether large numbers of people will enact normative claims. Therefore, even the technical aspects of study design and implementation fall squarely in the realm of ethics.

Supporting Change – Principles of Implementation Science

Within biomedicine, a proven intervention or prevention strategy that saves lives can take years to make its way into medical practice. In fact, the published results of many successful clinical trials languish in the archives of journals without leading to changes in clinical practice. It is estimated that it takes 17 years for approximately 14% of original research to change clinical practice for the benefit of patients.(Green et al. 2009) This high rate of attrition has been described as a leaky pipeline (Green et al. 2009) or leaky funnel from research to practice.(Balas and Boren 2000)

This gap between evidence and practice should concern ethicists, because evidence-based medicine is the natural language of clinicians and biomedical researchers. If biomedical interventions fare this poorly, how much narrower is the path for sustainably enacting ethical norms, particularly when they are published in journals that clinicians and researchers do not routinely read, and are framed using language that can seem foreign to frontline practitioners? If ethicists hope for their normative work to lead to changes in practice, they will benefit from implementation science.

Implementation science is the study of the

constellation of processes intended to get an intervention into use within an organization; it is the means by which an intervention is assimilated into an organization. Implementation is the critical gateway between an organizational decision to adopt an intervention and the routine use of that intervention. (Damschroder et al. 2009)[Page 3]

While this paragraph refers to implementing “interventions,” implementation science models have also been applied to policies (Dodson, Brownson, and Weiss 2012) and routine practices (Pronovost et al. 2006), although implementation principles have not been applied broadly to ethics interventions. To further explore the benefits of applying implementation science in ethics, we will next describe the Consolidated Framework for Implementation Research (CFIR).

Damschroeder et al. developed CFIR in 2009 to consolidate varying terminologies, definitions, and theories into a single framework. The CFIR explores five domains of factors that can serve as barriers or facilitators to successful implementation of practices, decisions, or policies: intervention characteristics, outer setting, inner setting, characteristics of the individuals involved, and the process of implementation. Each of these domains contains several constructs that provide further specification. The aim of CFIR is to be comprehensive, but not specific to any particular type of intervention. As such, not all CFIR constructs will apply to every intervention. Instead, those seeking to implement a policy or practice “select constructs from CFIR that are most relevant for their particular … setting and use these to guide diagnostic assessments of implementation context, evaluate implementation progress, and help explain findings.”(Damschroder et al. 2009)[Page 2]

To illustrate and explore each of these major domains, we will consider the example of an intervention to support informed consent in clinical trials (see Table 1). Since the earliest days of codified research ethics, informed consent has served as a cornerstone of ethical conduct of research.(Faden and Beauchamp 1986) The duty to obtain informed consent is part of every major code of research ethics.(Emanuel, Wendler, and Grady 2000) It is seen as a specification of the general principle of respect for persons: “individuals should be treated as autonomous agents” who are “capable of deliberation about personal goals and of acting under the direction of such deliberation.”(National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research 1979)[Section B.1.]

Table 1.

Consolidated Framework for Implementation Research – Domains and Constructs

Intervention: Researchers use plain language, graphics, and new formatting of consent documents to improve participant understanding.
Domain Example Constructs Illustrative Applications to Informed Consent Intervention
Intervention Characteristics Evidence Strength and Quality
Relative Advantage
Adaptability
Complexity
Cost
  • What is the cost of plain language, graphics, and new formatting of consent documents? Will it take additional time? Who will pay for it?

Outer Setting Patient Needs and Resources
Pressure from Peer Institutions
External Policy and Incentives
  • Is this change consistent with regulations and policies influencing informed consent in research?

Inner Setting Networks and Communications
Culture
Implementation Climate (e.g., tension for change, learning climate, organizational incentives and rewards)
Readiness for Implementation (e.g., leadership engagement, available resources)
  • Is the institutional supporting this change? Are there resources available at the institution?

Characteristics of Individuals Knowledge and Beliefs
Self-efficacy
Individual Identification with Organization
  • How ready are researchers to adopt new informed consent practices?

Process Planning
Engagement (e.g., of end users, leaders, champions)
Executing
Reflecting and Evaluating
  • Who will employ this intervention? How can we engage them during planning, development, and implementation of the intervention?

  • How might we pilot test this intervention on a small scale, before large-scale rollout?

Note: Consult Damschroeder et al. 2009 for full list of constructs.

Since the early 1990s, the National Institutes of Health have invested more than $80 million in research aimed at generating evidence-based consent practices in research. (Dubois, Bante, and Hadley 2011, Burns, Rohrich, and Chung 2011) For the purposes of this paper, we will focus on one of these best practices: optimizing consent communication by using plain language, graphics, and simple formatting. (Dubois, Bante, and Hadley 2011, Iltis et al. 2013) Research participants prefer and achieve higher levels of understanding with consent forms that use plain language and are formatted with bullet points, pictures, and increased white-spaces. (Kim and Kim 2015, Flory and Emanuel 2004, Campbell et al. 2004, Agre et al. 2003) Following this consent practice would also assist clinical trials in maintaining compliance with recent revisions to the Common Rule requiring consent forms to be written in understandable language to enhance informed decision making.(Menikoff, Kaneshiro, and Pritchard 2017) Importantly, interventions that support this practice have been tested and demonstrated to be effective in randomized clinical trials.(Flory and Emanuel 2004, Kim and Kim 2015, Campbell et al. 2004, Rubright et al. 2010)

This best practice represents a specific, actionable normative claim: Researchers ought to optimize participant understanding by using plain language, graphics, and simple formatting in consent documents. By applying CFIR to this one specific normative claim, we can explore how implementation science can help ethicists overcome barriers to implementing specific norms.

Intervention Characteristics

The first domain in CFIR relates to characteristics of the particular intervention being implemented. In our example, the intervention would be revision of informed consent documentation to simplify language and formatting, and to incorporate bullet points and graphics. According to CFIR, several characteristics of the intervention will affect whether it is sustainably enacted. For example: Do researchers have the tools needed to assess whether their form uses plain language? Do they know how to produce graphics that intuitively communicate key points? Does it require greater effort or time to develop informed consent materials?

Similarly, the adaptability of the intervention can be important. In implementation science, “interventions can be conceptualized as having ‘core components’ (the essential and indispensable elements of the intervention) and an ‘adaptable periphery’ (“adaptable elements, structures, and systems related to the intervention and organization into which it is being implemented”).(Damschroder et al. 2009) [Page 3] This concept is especially pertinent to implementing ethical norms. Ethicists must decide which components of an intervention are adaptable and which are essential to validity of the underlying norm. For example, using grayscale rather than color images in the modified consent documents might reside in the adaptable periphery. However, forgoing images altogether might negate the intent of the intervention, which seeks to communicate effectively with different kinds of learners.

Taken together, these intervention characteristics can affect the perceptions of key stakeholders. Do stakeholders perceive this intervention as advantageous relative to the standard operations? Do they believe the evidence base for this intervention is robust and valid? Are there biases pertaining to the source of the intervention? If researchers view the modified consent process as intrusive with limited actual benefit, they might feel resentment that could undermine the intended changes. Finally, what is the cost of implementing the intervention, and who will pay for it? A cost-ineffective intervention will not be sustainable. Without careful evaluation of the characteristics of the particular intervention, implementation will likely encounter barriers.

Inner and Outer Setting

Inner and outer setting are the next two major domains in CFIR, referring to the environment in which the implementation process will occur. In general, the outer setting refers to “the economic, political, and social context within which an organization resides, and the inner setting includes features of structural, political, and cultural contexts through which the implementation process will proceed.”(Damschroder et al. 2009)[Page 5] This line between inner and outer setting is not always distinct, and can be dynamic. In part, this distinction depends on the type of intervention and context of the implementation process.

For our informed consent intervention, the outer setting might include competition and peer pressure if other leading institutions have already instituted similar interventions. Federal agencies can also contribute to the outer setting by producing regulations, incentives, and mandates, such as recent updates to the Common Rule.

The inner setting can also affect implementation of an intervention in several ways. If an Institutional Review Board (IRB) is reluctant to allow changes to informed consent templates, for instance because of required legal language, researchers may face a barrier to implementing the recommended consent practices. Other factors might include institutional attitudes and resources. For example, do leaders within the institution believe the current processes for consent are insufficient and require change? Are adequate resources dedicated to the implementation process? Statements of support from institutional leaders can be helpful, but “money, training, education, physical space, and time” are essential resources for successful implementation. (Damschroder et al. 2009)[Additional File 3] Considering barriers within the inner setting might lead to adaptation in the goals set; for example, if IRBs will allow greater flexibility in adapting templates for the newly mandated “key information sheets” than informed consent forms, then researchers might focus their energies on communicating effectively within the key information sheet.

Characteristics of Individuals

The fourth domain of CFIR focuses on characteristics of individuals involved with the implementation process. As described by Greenhalgh et al.,

People are not passive recipients of innovations. Rather… they seek innovations, experiment with them, evaluate them, find (or fail to find) meaning in them, develop feelings (positive or negative) about them, challenge them, worry about them, complain about them, ‘work around’ them, gain experience with them, modify them to fit particular tasks, and try to improve or redesign them – often through dialogue with other users.(Greenhalgh et al. 2004)

The individual’s knowledge and beliefs about the intervention, self-efficacy at executing the course of action, and the individual’s identification with the organization all can impact the implementation process. Furthermore, the individual’s readiness and acceptance of the change will affect their actions and the implementation process. If researchers are uncomfortable with changes to their routine practice, for example, they might be less likely to modify their approach to the informed consent process.

Beyond CFIR, more than 80 theories of behavioral change have been developed and published – a testament to the complexity of human behavior.(Davis et al. 2015) The Theoretical Domains Framework, for example, has sought to consolidate the domains of 33 behavioral change theories and 128 key theoretical concepts into 14 consensus domains.(Cane, O’Connor, and Michie 2012) Although in depth discussion of behavioral change is beyond the scope of this paper, these theories can provide more detailed guidance on modifying behaviors than CFIR.

Process

Lastly, characteristics of the implementation process will also affect the likelihood of sustaining an intervention. Effective implementation processes engage opinion leaders, appointed implementation leaders, champions for the intervention, and external change agents. This engagement should also include those on the front lines who will be expected to employ the intervention. To maximize chances of successful implementation, this engagement should coincide with planning and development of the intervention, rather than engaging once the intervention is finalized. If an intervention is conceptualized and designed without input from multiple stakeholders (especially those on the front line), then the intervention may contain components that cannot feasibly be enacted. For example, if the modified informed consent documents require days of work to prepare or take longer to deliver, then this may be a non-starter for those obtaining informed consent. Similarly, staged implementation—e.g., conducting a pilot study with formative evaluation at multiple sites, revising and piloting again with a larger group—may help to ensure a smooth rollout as the project scales up (McIntosh et al. 2017, Stetler et al. 2006).

Implementing an Implementation Mindset in Ethics – A Call for Multidisciplinary Collaboration

The framework that we propose for the development and enactment of ethical norms is beyond the common capabilities of ethicists. As illustrated in Table 2, each process within this framework involves stage-specific tasks that will require expertise from multiple disciplines and stakeholders. While we believe it is important for ethicists to have a fundamental understanding of implementation science, it is equally important to understand the need for collaboration in each step of this framework. Even when identifying ethical challenges and formulating aspirational norms, ethicists would benefit from the insights of key stakeholders to understand how the suspected ethical challenge affects people. As such, learning the principles of implementation science can help to ground normative ethicists in the experienced reality of those on the frontlines, which can inform normative deliberation.

Table 2.

Key Tasks and Participants in the Norm Implementation Process

Normative Stage Example Stage Specific Tasks Key Experts and Participants
Identify Ethical Challenge The informed consent process for clinical research often fails to sufficiently inform potential participants, thus impairing their ability to provide meaningful informed consent.
  • Interaction between scholars and stakeholders to understand ethical issues

  • Scholars in philosophy, bioethics, and religious studies

  • Stakeholder group representatives

Normative Ethics Respect for Persons: “Individuals should be treated as autonomous agents who are capable of deliberation about personal goals and of acting under the direction of such deliberation.” (Adapted from Belmont Report section B.1.)
  • Identify ethical norm and justify through dialectic argument among scholars

  • Reality check with stakeholders to ensure ethical deliberation is reflective of frontline reality

  • Scholars in philosophy, bioethics, and religious studies

  • Stakeholder group representatives

Applied Ethics “Researchers ought to optimize participant understanding by using plain language, graphics, and simple formatting in consent documents.”
  • Further specify norm to guide actions

  • Apply implementation principles to ensure that specific norm is feasible

  • If norm is not feasible, reformulate

  • Evaluate for intended and unintended consequences

  • Scholars in philosophy, bioethics, and religious studies

  • Bioethicists with social science research skills

  • Implementation scientists

  • Key stakeholders and frontline staff

Intervention Development Pilot testing of plain language forms with graphics, appropriate language, and simple formatting during informed consent conferences.
  • Develop intervention to enact specific norm

  • Engage implementation principles in designing intervention

  • Identify outcome measures

  • Reality check intervention with frontline stakeholders

  • Evaluate for intended and unintended consequences

  • Bioethicists with social science research skills

  • Implementation scientists

  • Clinical trialists

  • Key stakeholders and frontline staff

Dissemination +/− Policy Development If pilot testing shows positive results, refine intervention and disseminate to multiple institutions. Support this dissemination with changes to federal policy (common rule revisions).
  • If intervention is successful, develop plan for dissemination and outcome measurement

  • In some cases, craft policy to mandate changes in practice and support dissemination

  • Evaluate for intended and unintended consequences

  • Implementation scientists

  • Lawyers

  • Advocates

  • Funding agencies

During the processes of applied ethics and intervention development, implementation scientists can provide key input to promote the concept of designing with implementation in mind, which has been referred to in implementation science as “designing for dissemination.”(Curran et al. 2012) Investigators with expertise in social science research and trial design and frontline staff can also provide insights into facilitators and barriers to the proposed interventions. Once evidence supports a best practice, implementation scientists can inform the dissemination strategy for the intervention and measurement strategies for the outcomes of the intervention. If policy is warranted, ethicists would benefit from collaboration with legal experts, advocates, and funding agencies to propose and lobby for policy change at the local, state, or federal level. The ethicist can also assess whether the implemented intervention or policy has addressed the initial ethical challenge, or if the intervention has created new unintended ethical concerns.

Ethicists cannot be expected to master every role necessary for the implementation of specific norms. However, it should be the responsibility of ethicists to help form and participate in multidisciplinary, collaborative teams with others who possess the necessary content knowledge, skillsets, experiences, and political connections within the relevant inner and outer settings to facilitate the success of an intervention. Doing so is outside the traditional role of ethicists, but it is imperative if ethicists hope to fulfill the overarching goal of ethics – supporting ethical practice in the real world. If ethicists view the development of theoretical, aspirational norms as their only responsibility in this framework, then their work is likely to impact thought in ethics circles without changing the problematic practices they write about.

Fostering this multidisciplinary effort will likely require a multipronged approach including education, incentives, and mandates. First, graduate level education in ethics could incorporate further multidisciplinary exposure. For example, a curriculum could include coursework on dissemination and implementation, empirical social science research skills, and clinical trial design. Additionally, this course work could include a practicum with frontline exposure to an area of interest. For example, a graduate student interested in research ethics could spend time with clinical research associates and investigators to intimately understand the reality of clinical research. Mandates and incentives could also support the implementation of multidisciplinarity. Applied ethics journals, for example, could encourage authors to address potential barriers to implementing their normative recommendations. Furthermore, funding agencies could encourage or mandate that proposals include a framework for how the norm under consideration will be implemented. Likewise, funders could preferentially fund studies with multidisciplinary collaborations similar to what we have described.

Yet, we must follow our own advice. In this article, we proposed the aspirational norm that ethical norms ought to be implemented broadly to address ethical challenges. To carry this norm through these processes will require a multidisciplinary team that includes input from ethicists, educators, implementation scientists, and graduate students in ethics, among others. This work is beyond the scope of the current manuscript, but we hope our call for change will spark an interest in various stakeholders who are integral to these processes.

Unanswered Questions

This paper has left several important questions unanswered. First, when is a norm ready for implementation? Surely, we do not want to implement, for example, every conviction of undergraduate philosophy majors taking an ethics course. Similarly, we likely do not want to systematically implement every specified norm if there is not consensus support or a strong rationale, nor would this be feasible given limited resources. However, ethicists can apply implementation principles when specifying every norm, regardless of whether an implementation clinical trial ensues. For example, ethicists can specify norms after considering the feasibility of enactment and engaging stakeholders.

This concept raises an important second question: How should we operationalize the concept of consensus, given that unanimity rarely exists on matters of ethics, particularly when diverse individuals are invited to the table. Clearly, the development of consensus aspirational norms should not be overly influenced by implementation principles. For example, whether we embrace the conviction that “all people are equal” should not depend on the contemporaneous political and social environment. However, particular suggestions for this normative process are beyond the scope of this paper.

These remaining questions highlight another important aspect of our framework: improving ethical practice is necessarily an iterative process. Implementing and disseminating an intervention might support ethical practice as intended, but it might also fail to have the desired outcomes when applied broadly. Or worse, it might have unanticipated negative consequences, leading to new ethical challenges. As such, reassessment of intended and unintended consequences are critical after each process in this framework.

Conclusion

The purpose of ethics is to support ethical actions. As such, the role of ethicists should be to develop norms using processes that will facilitate implementation, and to collaborate with those who have expertise in implementing social change. Such an approach will improve the chances that normative deliberations lead to actual change in practice. Making these changes will require modifications to education and training in ethics, as well as strategic collaborations with key stakeholders and experts. This will not be easy, but it is necessary to fulfill the essential purpose of ethics.

Funding:

NIH National Center for Advancing Translational Science UL1 TR002345 (BAS); National Human Genome Research Institute K01HG008990 (ALA).

Footnotes

Disclosure of Interests: The authors report no conflicts of interest.

1

Hereafter, when we refer to “norms” we refer to social norms generated by policy makers or applied ethicists.

References

  1. Agre Patricia, Campbell Frances A., Goldman Barbara D., Kass Nancy E., Boccia Maria L., McCullough Laurence B., Merz Jon F., Miller Suzanne M., Mintz Jim, Rapkin Bruce, Sugarman Jeremy, Sorenson James, and Wirshing Donna. 2003. “Improving informed consent: The medium is not the message.” IRB: Ethics & Human Research Suppl 25 (5):S11–S19. [PubMed] [Google Scholar]
  2. Aristotle. 1980. The Nicomachean ethics. Oxford: Oxford University Press. [Google Scholar]
  3. Balas EA, and Boren SA. 2000. “Managing Clinical Knowledge for Health Care Improvement.” Yearb Med Inform (1):65–70. [PubMed] [Google Scholar]
  4. Brownson Ross C., Colditz Graham A., and Proctor Enola Knisley. 2012. Dissemination and implementation research in health : translating science to practice. Oxford; New York: Oxford University Press. [Google Scholar]
  5. Burns Patricia B, Rohrich Rod J, and Chung Kevin C. 2011. “The levels of evidence and their role in evidence-based medicine.” Plastic and Reconstructive Surgery 128 (1):305–10. doi: 10.1097/PRS.0b013e318219c171. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Campbell Frances A., Goldman Barbara D., Boccia Maria L., and Skinner Martie. 2004. “The effect of format modifications and reading comprehension on recall of informed consent information by low-income parents: a comparison of print, video, and computer-based presentations.” Patient Education and Counseling 53 (2):205–216. doi: 10.1016/S0738-3991(03)00162-9. [DOI] [PubMed] [Google Scholar]
  7. Cane J, O’Connor D, and Michie S. 2012. “Validation of the theoretical domains framework for use in behaviour change and implementation research.” Implement Sci 7:37. doi: 10.1186/1748-5908-7-37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Curran Geoffrey M, Bauer Mark, Mittman Brian, Pyne Jeffrey, and Stetler Cheryl. 2012. “Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance pubic health impact.” Medical Care 50 (3):217–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, and Lowery JC. 2009. “Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science.” Implementation Science 4:15p. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Davis Rachel, Campbell Rona, Hildon Zoe, Hobbs Lorna, and Michie Susan. 2015. “Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review.” Health psychology review 9 (3):323–344. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. de Vries R, and Gordijn B. 2009. “Empirical ethics and its alleged meta-ethical fallacies.” Bioethics 23 (4):193–201. doi: 10.1111/j.1467-8519.2009.01710.x. [DOI] [PubMed] [Google Scholar]
  12. Dodson Beth, Brownson Ross C, and Weiss Steve. 2012. “Policy Dissemination Research” In Dissemination and Implementation Research in Health: Translating Science to Practice, edited by Brownson RC, Colditz GA and Proctor EK. New York: Oxford University Press. [Google Scholar]
  13. DuBois James M. 2008. Ethics in mental health research: Principles, guidance, and cases. Oxford; New York: Oxford University Press. [Google Scholar]
  14. Dubois James M., Bante Holly, and Hadley Whitney B. 2011. “Ethics in Psychiatric Research: A Review of 25 Years of NIH-funded Empirical Research Projects.” AJOB Primary Research 2 (4):5–17. doi: 10.1080/21507716.2011.631514. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Ebbesen M, and Pedersen BD. 2007. “Using empirical research to formulate normative ethical principles in biomedicine.” Med Health Care Philos 10 (1):33–48. doi: 10.1007/s11019-006-9011-9. [DOI] [PubMed] [Google Scholar]
  16. Emanuel Ezekiel J., Wendler David, and Grady Christine. 2000. “What makes clinical research ethical?” Journal of the American Medical Association 283 (20):2701–11. doi: jsc90374 [pii]. [DOI] [PubMed] [Google Scholar]
  17. Faden Ruth R., and Beauchamp Tom L.. 1986. A history and theory of informed consent. New York: Oxford University Press. [Google Scholar]
  18. Flory James, and Emanuel Ezekiel J.. 2004. “Interventions to improve research participants’ understanding in informed consent for research: a systematic review.” Journal of the American Medical Association 292 (13):1593–601. doi: 10.1001/jama.292.13.1593. [DOI] [PubMed] [Google Scholar]
  19. Green LW, Ottoson JM, Garcia C, and Hiatt RA. 2009. “Diffusion theory and knowledge dissemination, utilization, and integration in public health.” Annu Rev Public Health 30:151–74. doi: 10.1146/annurev.publhealth.031308.100049. [DOI] [PubMed] [Google Scholar]
  20. Greenhalgh T, Robert G, Macfarlane F, Bate P, and Kyriakidou O. 2004. “Diffusion of innovations in service organizations: Systematic review and recommendations.” The Milbank Quarterly 82 (4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Iltis Ana S, Misra S, Dunn Laura B., Brown Gregory K., Campbell A, Earll SA, Glowinski A, Hadley WB, Pies R, and DuBois James M.. 2013. “Addressing risks to advance mental health research.” Journal of the American Medical Association 70 (12):1363–71. doi: 10.1001/jamapsychiatry.2013.2105. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kim EJ, and Kim SH. 2015. “Simplification improves understanding of informed consent information in clinical trials regardless of health literacy level.” Clinical Trials 12 (3):232–6. doi: 10.1177/1740774515571139. [DOI] [PubMed] [Google Scholar]
  23. Komesaroff Paul A. 1995. Troubled bodies : critical perspectives on postmodernism, medical ethics, and the body. Durham: Duke University Press. [Google Scholar]
  24. McIntosh Tristan, Higgs Cory, Mumford Michael D., Connelly Shane, and DuBois James M.. 2017. “Continuous evaluation in ethics education: A case study.” Science and Engineering Ethics. doi: 10.1007/s11948-017-9927-x. [DOI] [PubMed] [Google Scholar]
  25. Menikoff Jerry, Kaneshiro Julie, and Pritchard Ivor. 2017. “The Common Rule, Updated.” New England Journal of Medicine 376 (7):613–615. doi: doi: 10.1056/NEJMp1700736. [DOI] [PubMed] [Google Scholar]
  26. National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. 1979. The Belmont report: Ethical principles and guidelines for the protection of human subjects of research. Washington, D.C.: United States Department of Health, Education, and Welfare. [PubMed] [Google Scholar]
  27. Pronovost Peter, Needham Dale, Berenholtz Sean, Sinopoli David, Chu Haitao, Cosgrove Sara, Sexton Bryan, Hyzy Robert, Welsh Robert, Roth Gary, Bander Joseph, Kepros John, and Goeschel Christine. 2006. “An intervention to decrease catheter-related bloodstream infections in the ICU.” The New England Journal of Medicine 355 (26):2725–2732. [DOI] [PubMed] [Google Scholar]
  28. Rubright J, Sankar P, Casarett DJ, Gur R, Xie SX, and Karlawish Jason H.. 2010. “A memory and organizational aid improves Alzheimer disease research consent capacity: results of a randomized, controlled trial.” American Journal of Geriatric Psychiarty 18 (12):1124–32. doi: 10.1097/JGP.0b013e3181dd1c3b. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Shields Christopher. 2016. Aristotle In The Stanford Encyclopedia of Philosophy, edited by Zalta Edward N.. [Google Scholar]
  30. Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, Kimmel B, Sharp ND, and Smith JL. 2006. “The role of formative evaluation in implementation research and the QUERI experience.” Journal of General Internal Medicine 21 Suppl 2:S1–8. doi: 10.1111/j.1525-1497.2006.00355.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Strech D 2008. “Evidence-based ethics--what it should be and what it shouldn’t.” BMC Med Ethics 9:16. doi: 10.1186/1472-6939-9-16. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES