Five Common Myths Limiting Engagement in HIV-Related Implementation Research

Abstract

HIV-related implementation research holds great promise in achieving the potential of efficacious prevention and treatment tools in reducing the incidence of HIV and improving HIV treatment outcomes among people living with HIV. From the perspectives of HIV-related implementation research training and academia, and through consultations with funders and investigators new to IR, we identified five myths that act as barriers to engagement in IR among new investigators. Prevailing myths broadly include: 1) One must rigidly apply all aspects of an implementation framework for it to be valid, 2) Implementation research limits the type of designs available to researchers, 3) Implementation strategies cannot be patient or client-level approaches, 4) Only studies prioritizing implementation outcomes are “true” implementation research, and 5) If not explicitly labeled implementation research, it may have limited impact on implementation. We offer pragmatic approaches to negotiate these myths with the goal of encouraging dialogue, ensuring high quality research, and fostering a more inclusive and dynamic field of implementation research. Ultimately, the goal of dispelling these myths is to lower the perceived bar to engagement in HIV-related implementation research while still ensuring quality in the methods and measures used.

Introduction

Implementation research (IR) holds great promise towards realizing the potential of efficacious prevention and treatment tools for reducing HIV incidence and improving HIV treatment outcomes, making IR key to accomplishing the goals of the Ending the HIV Epidemic (EHE) initiative^1,2. There is widespread enthusiasm about this emerging area of applied research, yet all new areas of inquiry require time for practicing scientists to agree on the basic definitions and standards, and their nuances. In contrast, while well-articulated standards are useful for bringing a field cohesion, excessive adherence to standards or nomenclature — particularly when those standards and nomenclature are likely to evolve—can be counterproductive. There are a growing number of investigators and implementers looking to engage in IR³, and a growing number of HIV IR-related funding opportunities and studies⁴. Simultaneously, prevalent HIV-related IR myths that inform perceptions of what is considered “true” IR create barriers to entry for investigators without specialized training and inhibit innovation. We raise, and aim to debunk, these myths to lower the perceived bar to engagement in IR while promoting methodological consistency.

Broad engagement is needed to end the epidemic

Broadening engagement in IR — ensuring that it is not esoteric or purely academic — is a core pillar of implementation science. Given heterogeneity in the HIV epidemic across contexts, locally-knowledgeable implementers and scientists (collectively referred to as ‘implementation researchers’) are best positioned to define relevant implementation research questions. Advancing the field to meet the challenges of bringing a conclusion to the epidemic will require innovation in developing, refining, and applying IR frameworks and methods. Thus, engagement of a broader and more diverse range of investigators and implementers in research-practice partnerships is needed to successfully implement the evidence-based tools available to end the HIV epidemic. Based on our collective experience—including consultations with IR trainees, conversations with colleagues and funders during meetings, inconsistent feedback during grant review, and review of the implementation science literature—we have noticed the proliferation of myths about what it means to ‘do implementation research’. These myths create barriers to engagement in IR, precisely at a time when continued expansion is needed. Ensuring quality application of the methods and measures used is important for creating generalizable knowledge and scientific integrity; we believe that can be done while simultaneously promoting diversity and improving equity in IR through broader engagement³.

Five Myths

1). One must rigidly apply all aspects of an implementation framework for the framework to be valid.

The use of theories, models, and frameworks (collectively, ‘frameworks’) to strengthen research is a hallmark of IR. Frameworks serve many purposes, including: 1) ensuring a thorough and considered approach to IR within contexts and populations of interest; 2) making explicit the theory of how change will occur (i.e., anticipated mechanisms of action) to guide selection of appropriate implementation strategies and evaluation metrics; and 3) promoting comparability of IR methods and outcomes across studies and contexts. Multiple reviews have identified and classified the numerous IR frameworks and their uses ^5–8, while considerable IR resources focus on appropriate model selection and application^8–13. The ‘hotspot’ approach¹⁴ of EHE underlines the importance of tailoring frameworks in HIV IR based on heterogeneity in populations, geographies, and contexts. For example, identifying and selectively applying constructs that enhance understanding of key populations¹⁵ is critical domestically as well as in diverse, global settings¹⁶. Alternatively, it may be necessary to add missing constructs or combine frameworks, such as incorporating health equity domains into established IR frameworks¹⁷. Careful adaptation, often optimally guided by communities themselves, is recommended by framework scholarship^{16,18–20 21}. Implementation researchers can leverage resources designed to support model adaptation and application to advance their valid application^8,10,12,13, and documentation of study-level adaptations can help frameworks to improve over time¹⁶. To advance IR, implementation researchers can utilize frameworks during research planning, implementation, and evaluation; apply appropriate measures of key framework constructs; and report and compare findings⁹.

2). Implementation research limits the type of designs available to researchers.

The overarching goals of IR vary broadly, ranging from effectiveness studies with some implementation outcome measurement to an exclusive focus on the differential impact of implementation strategies on implementation outcomes. Study designs and approaches utilized by implementation researchers are varied, including observational, quasi-experimental, experimental, participatory, qualitative, mixed-methods, costing, and modeling – none of which are specific to IR and many of which are not mutually exclusive. Specific to IR, however, are effectiveness-implementation hybrid designs^22,23 which themselves incorporate a range of approaches listed above, but necessitate consideration of both implementation and effectiveness outcomes. It is the research goal that is the primary determinant of IR, not the approach. For example, a mixed-methods IR trial testing the effect of a blended digital and peer-based education system for ART providers may evaluate improved provider knowledge as the primary outcome, with secondary outcomes including a qualitative assessment of the mechanisms through which knowledge change occurred, and improved viral suppression among patients living with HIV. IR often deals with varied, multi-level contexts²⁴ and is often seeking balance between internal and external validity²⁵ To account for these issues, IR designs often randomize at the cluster level (e.g. clinic, community) instead of the individual level, and also utilize pragmatic²⁶, mixed-methods²⁷, or adaptive^28,29 designs, although many designs can achieve IR objectives.

3). Implementation strategies cannot be patient- or client-level approaches.

There is a perception that if the mechanism of action being studied is not focused on changing the behavior of the provider, organization, or health system, it is not IR. Confusion may be augmented by the blurriness that often exists between evidence-based interventions (EBI) targeting health outcomes and implementation strategies targeting behaviors at multiple levels³⁰. For HIV, the EBI is often ART or PrEP, and implementation strategies often target individual-, provider-, or system-level barriers to optimize ART/PrEP delivery and adherence. Ultimately, the most appropriate implementation strategy or combination of strategies must consider the contextual environment and match the implementation strategies to modifiable barriers impeding implementation or use of the EBI. Robust formative and preference-oriented research, combined with the application of logic models, frameworks, and guidance on strategy specification, are essential to ensuring that the strategies proposed are clear and aligned with relevant EBI barriers and priorities^31–34. This may result in implementation strategies targeting patients, providers, or organizational factors. A fair criticism of HIV-related IR is that implementation strategies have frequently been myopic, predominantly geared at the patient/client or, more recently, the ART delivery approach (e.g., fast track, pharmacy³⁵), with less focus on provider aspects⁴. Increased emphasis on non-patient approaches is warranted, but recognition that strategies to deliver EBIs such as PrEP or ART may need to be tailored to the multifactorial barriers that individuals face to prevention and care is critical. In settings where health system resources are stretched, engaging patients as the actors in implementation strategies that target the patients themselves or communities often represents the most pragmatic approach to enhancing implementation³⁶. This may be particularly true for members of stigmatized groups who are not well-served by other health system actors³⁷. To the extent that key barriers are based in the delivery of healthcare such as provider attitudes/friendliness, or access issues such as transport or long clinic queues, reconceptualization of services to become more patient-centered or to circumvent structural access challenges is key. However, for individual psychosocial or network barriers, patient-oriented approaches remain critical and should not be undervalued.

4). Only studies prioritizing implementation outcomes are “true” implementation research.

Grant and paper reviewers being less familiar with IR has emerged as a common thematic challenge, often requiring the inclusion of individual-level effectiveness outcomes, even for well-established EBIs. Perhaps partially in response to this, some more ensconced in the IR community discount IR studies that prioritize patient or client-level outcomes over implementation outcomes. Importantly, prominent outcome frameworks in IR include both client and implementation-level outcomes^38,39, with effectiveness as a key component of those measures. Additionally, research that assesses downstream clinical events common to non-IR, such as viral suppression, can yield additional insights beyond simply “clinical effectiveness”. First, no single implementation outcome of interest is likely to mediate the entire effect of a strategy on a downstream clinical outcome. Therefore, measuring both the effect of a strategy on an implementation outcome as well as the effect on downstream clinical outcomes — especially in different settings — can help reveal the extent to which a particular implementation outcome mediates effects, and how that varies. For example, the field has been interested in the effect on retention of being diagnosed with HIV and prescribed ART on the same day; however, how the provider offers ART, including the inter-personal dynamic, adjunctive counseling, and supportiveness of the clinic setting will influence the effect of the same-day ART prescription on outcomes. Indeed, literature shows effects ranging from retention improvements to decreases⁴⁰. Second, there may be common causes of the extent to which a strategy is implemented and the effects of the strategy on clinical outcomes. In this case, identification of context-specific factors that influence both implementation and clinical outcomes across different units can reveal important organizational and contextual influences, with possible implications for health equity that IR is positioned to identify and address^17,41,42. For example, due to structural factors such as poverty, stigma, and racism, a clinic in a socioeconomically deprived area may have lower healthcare worker morale as well as patients with a greater psychosocial burden, which depress both provider uptake and the effect of delivery. Variability in the effects of levels of implementation on the downstream clinical effects is often a question of substantive interest that may be answered by a study powered on clinical effectiveness outcomes²⁴. Additionally, typologies of hybrid implementation-effectiveness designs²² and pragmatic designs such as leveraging aggregate data instead of requiring individual-level enrollment may be useful tools for new implementation researchers in planning their study approach.

5). If not explicitly labeled implementation research, it may have limited impact on implementation.

Implementation research is an inherently multidisciplinary field whose ultimate goal is to advance our understanding of how to close the gaps between evidence from controlled studies and routine practice in real-world contexts. Any research conceptually seeking to understand the scale of these implementation gaps, the reasons for them, and strategies to address these gaps and the mechanisms through which the strategies may operate helps to serve this purpose ²⁵. Not all research that includes conceptual equivalence with IR aims, however, will apply the rapidly developing IR nomenclature or use an established IR framework. For example, a growing science of incentives in HIV prevention, care, and treatment is based in behavioral economics, focusing inquiry on variation in uptake of EBIs^43–45. The field of economics is ripe with studies that advance implementation⁴⁶ but are rarely explicitly labeled as IR: econometrics provides robust methodology for assessing the impacts of real-world program implementation ⁴⁷, and discrete choice experiments quantify preferences to inform optimized program design for acceptability and adoption⁴⁸. Likewise, sociology and social network analyses describe the social dynamics underpinning spread that informs implementation science, including spread of infectious diseases and the dissemination and diffusion of behaviors⁴⁹. The discussion of context and mechanisms, enjoying much IR attention at the moment, also informs the context-mechanism-outcome framework central to Realistic Evaluation⁵⁰. In many traditions, approaches to use of evidence-based practices IR may call ‘strategies’ are called ‘interventions’, but this difference in nomenclature should not obscure their immediate relevance to the field of IR. A generation ago, scholars in organizational psychology noted that a balance between knowledge acquisition processes that are open-ended, creative, and emergent and those that are more concrete and standardized are necessary to the vibrancy and health of organizations ⁵¹. IR frameworks, designs, and methods already draw heavily from these different fields, and—as the field of IR advances—incorporating tools and approaches from diverse fields that are particularly suited for providing insights on implementation will be vital to the IR growth and avoiding “reinventing the wheel”. Ultimately, it is the questions being asked and the goals of the research that determine if research advance implementation, and not whether a specific label, design, method, or framework accompanies said research. Explicitly pursuing cross-disciplinary training and collaborations⁵² may help to more rapidly advance the field of IR and strengthen its applications in HIV research, while utilizing conceptual definitions of IR aims in addition to specific nomenclature may help to advance HIV IR reviews and practice.

The five myths presented here have been consistently encountered when working in IR training, academia, and funding spaces and can be negotiated to facilitate conceptually congruent trans-disciplinary dialogue, ensure high quality research, and foster a more inclusive and dynamic field. Allowing for the full range of implementation frameworks, strategies, methods, and outcomes and avoiding overly-specialized interpretations of IR practice will help implementation science applied to HIV to meet the scientific needs of the moment. IR can maintain quality across a diverse range of applications through practices including: clarity and transparency in the scientific choices made around frameworks, strategies, and methods; collaboration with implementing partners and utilization of nomenclature that promotes accessibility and understanding across stakeholders; efforts to create generalizable knowledge through comparison across contexts; and rapid, open-access dissemination of findings. Dispelling these myths is particularly important to the field of HIV, as HIV researchers are rapidly adopting IR as a way of bridging the research and practice gap. Additionally, to achieve the aims of the Ending the HIV Epidemic initiative, including better supporting the health of people living with HIV and decreasing transmissions requires the innovation, application, and focus on equity made possible by high quality IR. Ultimately, promoting the use of implementation science by a wide range of researchers will advance the field, as it has other fields, and expand opportunities to apply IR tools to ensure real-world effectiveness of efficacious interventions to end the HIV pandemic.