HIV-related Implementation Research for Key Populations: Designing for Individuals, Evaluating across Populations, and Integrating Context

Abstract

Introduction:

Key populations, including men who have sex with men, transgender people, sex workers, people who inject drugs and incarcerated populations, experience high burdens of HIV and urgently need effective interventions. Yet the evidence-base for implementation research (IR) with key populations remains weak and poses specific challenges to epidemiologic inference. We apply the Consolidated Framework for Implementation Research to consider specific challenges and recommendations for IR with key populations.

Discussion:

Individuals within key populations exist within inner and outer settings-including organizational structures, legal (e.g. criminalization) and funding environments--which influence the design, adoption and fidelity of interventions, and the potential sustainability of intervention scale-up. Underlying vulnerabilities and external stressors experienced at the individual-level (e.g. homelessness, violence) further impact participation and retention in IR. Thus, researchers should account for representation in the research process, beginning with community engagement in IR design and consideration of enumeration/sampling methods for key populations who lack probabilistic sampling frames. Interventions for key populations require substantial adaptation and complexity (e.g. individually tailored, multi-component) to ensure appropriateness, however there is tension between the need for complexity and challenges to internal validity (fidelity) and external validity (generalizable scale-up). Finally, integrating contextual, sampling and implementation elements into analytic approaches is critical for effectiveness evaluation.

Conclusions:

Translation of efficacious findings at the individual-level to effectiveness at the population-level requires recognition of risk heterogeneity. Recognizing the nuances of working with key populations is essential to ensure that individuals are represented by design and therefore gains in population health can be achieved.

Introduction

Given intersecting individual-, network-level, and structural risks, key populations including sex workers, men who have sex with men (MSM), transgender persons, people who inject drugs (PWID) and incarcerated populations bear high burdens of HIV independent of setting(1–4). The epidemiology of HIV among key populations consistently demonstrates the disproportionate impact of HIV within these groups, which necessitates different needs in terms of interventions. Evidence is required to guide effective implementation and scale programs and services for key populations to reduce HIV burden, particularly in low and middle income settings. Recent data highlight this need for addressing the heterogeneity of populations requiring effective HIV interventions, given the limited population-level effectiveness of interventions that are highly efficacious at the individual-level among the general population (5–8). Advancing the HIV response thus requires characterizing optimal implementation strategies for HIV prevention and treatment interventions which are responsive to individual needs, while balancing effectiveness and sustainability.

HIV-related implementation research (IR) with key populations poses specific challenges to epidemiologic inference emerging from multiple intersecting (co-occurring) stigmas that result in marginalization, such as stigma towards same-sex behavior and HIV, and stigmas faced by women engaged in sex work who use drugs and are living with HIV(9, 10). This can lead to selection biases, information biases, fidelity of implementation execution and limited transportability of results across populations and settings. Identification of key populations is often challenging, as group membership is defined by behavior which is often difficult to verify(11–13). Furthermore, the absence of probabilistic sampling frames for key populations is a barrier to understanding denominators for key populations, as well as to measuring the extent to which individuals within key populations are included or missing from research or programs(11–13). Yet, recognition of who is and who is not represented is critical to understanding the generalizability of the findings and the potential for population-level impact, particularly given the social, political and economic marginalization of key populations. Comprehensive consideration of representation in research and programs across the IR process includes attention to population engagement, epidemiologic design, sampling, recruitment, retention, implementation outcomes, individual outcomes, dissemination, and the interplay of marginalization with each of these elements.

We propose to use the Consolidated Framework for Implementation Research (CFIR) to assess specific challenges for IR with key populations. Using the IR literature and our experiences conducting epidemiologic and implementation research with key populations, our objective is to fill a gap by documenting IR challenges with key populations – particularly in resource constrained settings, assess their potential epidemiologic impact, and recommend strategies for conducting IR moving forward.

Consolidated Framework for Implementation Research (CFIR)

Figure 1 illustrates the proposed, applied CFIR framework highlighting constructs which inform IR for key populations and their interdependencies(14, 15). Broadly, the five CFIR domains that facilitate studying the potential for generalizability and scalability of research findings include characteristics related to individuals, the inner and outer settings which they are embedded, as well as intervention characteristics and implementation and research processes(14). The interplay of the CFIR domains is a strength of the framework and a critical reason it was selected for this assessment. Though the ‘research process’ and ‘intervention characteristics’ are the domains most commonly considered for IR challenges, threats to epidemiologic inference may emerge from each of these domains as invariably the outer setting, inner setting, and individual characteristics will influence the research process and consideration of intervention characteristics (Table 1). Within the five broad CFIR domains, we have highlighted sub-constructs of particular relevance to key populations. Other sub-constructs may have relevance but were determined, based on the literature and our experience, to have less prominence and not specifically highlighted. While we recognize that key populations represent very different groups of individuals, themes of marginalization are shared across groups and result in varied, but similar IR challenges. Thus, while we apply this construct overall to key populations across domains and sub-constructs, we highlight issues of relevance within specific key population groups and provide illustrative examples of challenges, solutions and threats to epidemiologic inference within populations (Table 1). Heterogeneity also exists within populations, such as by age, venue type among sex workers, geographic location, outness among MSM and transgender women, prison release status among incarcerated or previously incarcerated populations, and type of drug use among PWID, the implications of which are considered.

Figure 1:

CFIR Framework Applied for Key Populations

Table 1:

Illustrative examples of key population-specific threats to scientific inference emanating across domains of the Consolidated Framework for Implementation Research (CFIR)

Exemplary challenges and recommendations by domain and subconstruct	Implications for inference
Outer Setting
External Policies: Criminalization and limited legal protections for those who engage in sex work, practice same-sex behaviors, and use drugs may limit uptake of HIV prevention and treatment services as well as study participation. Evaluating the impact of laws can be difficult if working in a single setting. Mathematical modeling of structural drivers of HIV transmission among FSW in Canada and pre-post evaluations utilizing legislative changes as a natural experiment with MSM in Nigeria, are examples of approaches evaluating impact of external forces on health outcomes. As demonstrated by an increase in loss to follow-up in the Nigeria study with MSM, populations may avoid study participation in response to laws, policy initiatives or external environments of stigma and discrimination.(16, 17)	State supported marginalization reinforces the utility of evaluating implementation and effectiveness outcomes within contextual settings; results from one setting may not generalize to areas in which behaviors are criminalized, highly stigmatized and/or discriminated against. Mathematical modeling can measure the potential impact of policies and structural interventions, but does not always account for variations in implementation and enforcement. Similarly, pre-post policy evaluation often lacks controls or allows individuals to serve as their own controls – which in light of unanticipated policy shifts, may be impacted by informative censoring or poor measurement specification if the study was not initially designed to capture legal or policy impacts.
Peer Pressure: To facilitate participant trust and intervention engagement, key populations research is often conducted with local non-governmental or community-based organizations already engaged in service delivery with the key population. However, given lags in research funding from time of concept to award, ethical approvals and implementation, often donors funding programs decide to implement interventions before evidence for key populations is available. These decisions aim to accelerate epidemic response and program grantee recipients are expected to swiftly implement changes. This creates challenges when similar interventions are being tested through randomized studies with key populations, as researchers cannot ethically withhold the standard of care from participants. Solutions to this challenge include creating shorter duration trials, step-wedged approaches to program scale-up, researching implementation outcomes of existing programs, and building adaptive elements into the study design that allow tested interventions to evolve with the program.	Changes to the program and standard of care will necessitate changes within research programs, potentially affecting the internal validity of findings and diluting the intervention effect of the intention-to-treat analyses, if standard of care arms begin receiving the intervention. Furthermore, this may result in the absence of evidence around intervention effectiveness within key populations, which may differ from results seen more broadly.
Inner Setting
Network & communications: A methadone assisted treatment program in Tanzania for PWID was supported by four CBOs working in different capacities to promote HIV prevention efforts among PWID. Yet despite recruitment efforts from trusted CBOs, mistrust remained common, particularly among women who inject drugs. Utilization of female peers, including women who had used drugs and engaged in sex work was identified as a critical strategy to more effectively engage women(18, 19).	Miscommunication and mistrust in peer networks can lead to selection bias/underrepresentation of women who inject drugs in program and effectiveness data
Structural characteristics; Network and communications: Utilization of peer outreach workers in intervention delivery creates opportunities, but can also lead to unanticipated challenges stemming from the structural characteristics and communication within networks. In a study run with FSW in South Africa by our team, peer FSW were strongly recommended by the community advisory group to fill the case manager positions providing ART adherence support to FSW living with HIV. Peer case managers received extensive case management, HIV and research training. However, organizational salary structures which use common guidelines of basing salary ranges on education create income disparities and higher rates of turnover after case managers have been upskilled. Hiring of complementary case managers outside of the sex work community also resulted in internal conflict. Solutions include budgeting for high turnover, investment in capacity building for peers, and organizational review of policies which undervalue real-world or street experience vs. education, when this experience is the most important qualification for the position (20).	High turnover creates threats to intervention fidelity, particularly interventions which rely on continuity of care
Characteristics of Individuals
Knowledge and beliefs about the intervention: Inaccurate risk perception and lack of awareness about available prevention interventions can limit the ability to recruit and retain MSM into interventions or research studies. Across high- and low-income settings, MSM, particularly younger MSM, often attribute low PrEP uptake to low risk perception for HIV infection. Since risk perceptions can operate as either a motivator or a barrier, MSM who readily enroll and are retained in studies may be fundamentally different from those who do not, impacting findings. Encouraging peers to recruit peers may allow for research and programs to reach those not otherwise engaged in care to limit these effects(21–24).	Level of knowledge, risk perception, and other characteristics of individuals can lead to inadequate recruitment, sample representation and generalizability of findings
Self-Efficacy: Factors in the outer settings such as criminalization or discrimination of same-sex practices, lack of recognition of different gender expressions and associated stigmas may impact personal agency of MSM and TGW, affecting their ability to engage in or adhere to interventions and compromising intervention effectiveness. Approaches including motivational interviewing that empower participants and bolster self-efficacy can mitigate these effects.	Decreased self-efficacy can influence who chooses to enroll and continue in studies or programs (representativeness) and may further modify the effectiveness of implementation strategies.
Intervention Characteristics
Intervention source, Adaptability: In sex work-focused implementation science research, engaging or enabling community mobilization is critical to establish stakeholder ownership and the capacity for sustained implementation. It is ideal for sex workers to assume an active leadership role in the development, implementation and evolution of research. Such was the case with India’s Sonagachi project, which operated for 16 years (1). In this program, initially started as a clinic program to improve uptake of STI treatment and condom use among sex workers through peer outreach, early stakeholder feedback indicated that sex workers’ environment, literacy rates, and limited self-efficacy compromised implementation effectiveness. To mitigate these challenges, the program expanded its training of peer outreach workers to include HIV prevention education through pocket visual charts, and “helper” skills to assist in facilitating condom negotiations between sex workers and clients(25, 26).	Measurement of community engagement and ownership in interventions should be considered as an outcome and potential effect measure modifier of intervention effectiveness, but measurement may be complex. Furthermore, planned and real-time adaptations to existing interventions to better guide KP-specific implementation should strive to maintain fidelity to intervention core components; tension between achieving “full and consistent implementation” while making rapid adaptations to meet setting-specific challenges may undermine internal validity or reproducibility.
Complexity: Complex interventions tailored to individual needs and/or which require content delivery or multiple engagements, may experience implementation challenges that, can undermine the effectiveness of an intervention, if not circumvented. For example, individual or group-based interventions with incarcerated populations and ex-prisoners upon release may be complex to implement –from the perspective of ensuring repeat attendance of participants during a time of potential transition and mobility, full delivery of session content, high quality of content delivery, and appropriate tailoring of counseling and support to the individual client needs. Measurement of intervention feasibility and fidelity are essential to understand intervention effectiveness results and potential scalability – though often omitted or not reported. Measurement of implementation from both participant and implementer perspectives are valuable to triangulate the evidence, as well as potential chart review and mixed methods approaches.	Inconsistencies in implementation documentation will result in information bias and reduce the ability to evaluate constraints or facilitators of effectiveness, including dose, quality and appropriateness. This will be particularly problematic if there is differential documentation, such that those executing with lower intervention fidelity also have poorer documentation.
Implementation and Research Process
Engagement, Planning: Recruitment of key populations often requires non-probability sampling approaches. These can result in non-representative samples, depending on the methodology, degree, and quality of community engagement and partner selection prior to recruitment. Working with key decision-making stakeholders and key informants is critical to ensure buy-in, smooth study operation, and insight into when and where KP groups commonly meet. The final methodology used may impact results. For example, a comparison of results from respondent driven sampling (RDS) and venue-based snowball sampling studies conducted in the same area with the same populations (FSW and MSM in both eSwatini and Cameroon) found that the RDS methodology sampled participants who were younger, less educated and less engaged in care as compared to venue-based snowball sampling. Venue-based methods may be more reflective of programmatic estimates (as programs often target venues for implementation), whereas RDS methods may be more reflective of underlying population prevalence. Regardless, programmatic data should be interpreted in the context of existing integrated biobehavioral data (where available), and can be recalibrated when surveillance estimates emanate from program data(12).	Selection biases may prevent participants from representing the larger source population; additionally, differences in the source and target population will impact generalizability.
Planning, Execution: The Sisters Programme in Zimbabwe worked with FSW in three sites to intensify community-based activities, and utilization of HIV prevention modalities and ART clinical services. Beyond program evaluation, researchers implemented a serial cross-sectional RDS study to compare results at baseline and at five years. While this was a proactive attempt to get a representative sample and observe changes over time, one challenge in execution was that no control sites were included and rapid national program scale-up of the ART program in Zimbabwe co-occurred with the intervention, limiting the interpretation of positive study findings(27).	Secular trends, if not accounted for, may result in over- or underestimation of intervention effect.

FSW: Female sex workers; MSM: men who have sex with men; PWID: people who inject drugs; TGW: transgender women

Outer Setting

The outer setting comprises the economic, political, and social contexts which inform patient needs, external policies, and resources(14). Specific to key populations, the legal and policy contexts, funding, and intersecting social stigmas and marginalization will influence the design and fidelity of HIV-focused IR, as well as the potential feasibility of intervention scale-up and sustainability.

National or sub-national policies will influence the content, design, and success of a study and the potential scale-up of an intervention. For implementation of studies among key populations, criminalization and limited legal protections for those who engage in sex work, practice same-sex behaviors, and use drugs are frequent challenges(28). Criminalization or legal restrictions may drive populations to be more hidden and therefore less likely to engage in services and studies. Service delivery through the government healthcare system or non-governmental organizations may show discrepancies between the established standard-of-care and the practical implementation of those guidelines. Therefore, process evaluation of implementation, as well as communication and coordination with higher-level officials, clinic staff and key population representatives, is critical to assess the adoption and fidelity of guideline implementation. For example, clinic staff may hold stigmatizing beliefs against MSM, FSW, transgender populations or PWID, which may prevent full or high-quality implementation of the guidelines; sensitization efforts to address these stigmas may need to be included as intervention components and/or elements of process evaluation(29, 30). Furthermore, ensuring safety for both study participants and study staff is essential, especially in the context of criminalization. Therefore, engagement and partnership with key population representatives, as well as government, will support navigation of IR in these contexts.

Studies aiming to evaluate components of the outer setting are complex to design, but important to assess the impact of a law or policy on the health of key populations(31). In the context of national-level policies affecting key populations, rigorous evaluation of policy or structural changes is difficult with the limited ability to use control groups. The impact of policy changes may be evaluated using pre-and post-assessments through natural experiments(17); however, this requires data availability prior to a policy change. Interrupted-time series is possible, but assumes that the pre-intervention context remains otherwise unchanged into the post-intervention period. Ecological studies may be used to provide insight of policy effects across countries, but do not account for individual-level characteristics and are prone to ecological fallacy(32–35). Mathematical modeling may be used to estimate the potential effect of a policy or structural-level intervention that has not yet occurred. For example, mathematical modeling has been used to estimate the impact of sex work criminalization on HIV acquisition(16).

Reporting structures and external incentives, such as funding, donor-driven targets, and regional or global initiatives, have implications for IR and program implementation. Modeling may demonstrate the population impact of implementing country-level strategies and scaling-up services for key populations, especially in response to meeting external mandates or adopting recommendations or guidelines(36). However, scale-up of intervention strategies that have demonstrated efficacy, such as pre-exposure prophylaxis (PrEP) and universal treatment, may face real-world implementation challenges. Thus, scaling interventions shown efficacious with non-key populations has presented challenges in the absence of evidence of ‘how’ to effectively implement services at a small-scale, let alone nationally or with key populations. Representative research to appropriately adapt or design interventions prior to scale-up with key population groups will support more effective implementation.

Inner Setting

In the inner setting, considering the structures and experiences of particular organizations working with key populations plays a key role in influencing both the scope and quality of IR(37). Working with key populations is fundamentally contingent upon the establishment of relationships between the organization and the community, and thus service delivery is often led by key population-focused non-governmental organizations. Three key constructs in the inner setting will impact key population intervention penetration and sustainability: 1) the structural characteristics of the organization; 2) the networks, communications and overlap between organizational administrators, implementing staff, and the population served; and 3) the implementation climate and receptivity of the organization. Specifically, structural characteristics, inclusive of the social architecture and maturity of the organization, facilitate or hamper the organization’s ability to reach and sustain contact with key populations through programming and research(38). The degree to which organizations work in a top-down vs. a bottom-up approach to determining programming and research priorities will impact engagement and confidence of key populations in IR and in the services delivered. These elements, however, are not determined in organizational isolation, but are themselves embedded within the outer setting, including donor priorities. As recruitment and intervention delivery with key populations are frequently decentralized and conducted via peer outreach workers, particularly with FSW, MSM, PWID, and transgender populations, the capacity and autonomy of research and implementation teams will impact intervention success, requiring organizational investment as well as growth opportunities(39). The close networking of peers within the communities may also create real or artificial tiers within the organization, and peers may face stigma and discrimination within their role or conversely, work together to undermine team members not from the community. The organizational capacity to change and the extent to which the research is championed within the organization plays a mediating role between implementation effectiveness and the overall management support needed within IR(40). Yet, while the scope of IR necessitates consideration of an organization’s inner setting, limited flexibility may exist in the selection of a key populations implementing organization and, once selected, there is a need to implement within the existing structures of the organization. Furthermore, considering the generalizability of the results to areas in which key populations-focused organizations are not present is critical to scale-up.

Characteristics of Individuals

The characteristics of key populations and service delivery providers are embedded within and consequently are often shaped by the outer and inner settings (Figure 1). While program and study design will affect the source and study populations, participation will be impacted by characteristics of individuals, affecting at the most central level who is represented in IR (Figure 2). Specifically, knowledge, beliefs, and attributes of key populations will affect intervention appropriateness and uptake, while self-efficacy to adhere to interventions and move through the stages of change will impact both uptake and maintenance. Together, these elements will affect how representative a sample is of the study population, as well as the internal validity of the results based on intervention drop-out.

Figure 2:

Key populations representation and data considerations from the individual participants to the broader population.

Knowledge and beliefs about interventions will directly impact service and research participation. In the context of pervasive stigma, discrimination, and criminalization, prospective participants weigh the risks of disclosing their key population status- often a criterion for study participation -with potential benefits of engaging with research. This may be particularly true for HIV-negative individuals at high risk of acquisition who could benefit from prevention interventions, but may not believe the disclosure risks are justified given that they are well.(1, 2).

Similarly, self-efficacy and stage of change will impact the successful adoption, maintenance and ultimately impact of interventions and will be heavily impacted by the outer and inner settings within which individuals reside. For example, stigma has been shown to negatively impact self-efficacy among transgender women(2) and thus intervention engagement may be limited among transgender women with reduced agency. Limited self-efficacy can diminish individuals’ beliefs that the intervention will work for them, potentially impacting recruitment, retention, and ultimately generalizability of findings. Furthermore, as key populations in many settings are more vulnerable to major life stressors and threats to survival, including arrests, violence, co-morbidities, and mortality, high drop-out can be anticipated as stability supports completion of the stages of change necessary for effective intervention. It is recommended that approaches both build self-efficacy components into interventions and measure self-efficacy and stigma in order to improve and interpret intervention results.

Successful implementation also involves consideration of other individual-level attributes that may serve as barriers to uptake of interventions including demographics (age, literacy, income), physical and mental health status, mobility, and relationships, which again will each be impacted by the outer setting(1). Use of peers, monetary reimbursement, conveniently offered services, and long windows of follow-up visits may help mitigate these barriers. However, these improvements in accessibility may increase the cost of implementation, reduce scale-up feasibility, and introduce additional risks for key populations. For example, reimbursements should be carefully considered to reflect appropriate compensation for time and not as a form of coercion to participate in research activities.

Intervention Characteristics

Characteristics of interventions –including what, for whom, by whom, when, for how long, with what intensity, and where- have particular relevance to conducting IR with key populations. Recognizing that heterogeneity between and within populations requires different intervention approaches is essential, but does not negate implementation challenges. Differentiated service delivery for key populations requires attention to intervention characteristics, including intervention source, evidence strength, relative advantage, adaptability, complexity (e.g. duration, scope, number of steps, intricacy), and cost (Figure 1)(41).

Key stakeholders’ perception of an intervention as internally or externally developed can significantly affect implementation among key populations. Internal development approaches, such as community empowerment and mobilization led by key population members, can play a key role in developing tailored, effective interventions that encourage participation, retention, and sustainability(42). This is particularly relevant for HIV interventions evaluating structural determinants of HIV infection among key populations, such as stigma, violence and policing, which community mobilization approaches have sought to address in combination with individual-level clinical services(43, 44). Researchers can support consensual dialogues and development of shared language between authorities and key populations, and both attempt to intervene on these structural levels, as well as monitor engagement of key populations in structural interventions(45).

High quality evidence of interventions is limited with key populations, particularly in resource-constrained settings. In general, few RCTs assessing the effectiveness of HIV interventions have been conducted among key populations, presenting a persistent challenge to the translation of efficacy findings from broader population-based studies to key populations. Noting this, the Highest Attainable Standard of Evidence (HASTE) was developed to evaluate HIV interventions among men who have sex with men (MSM), focusing on the triangulation of efficacy data, implementation data, and biological and public health plausibility(46). However, applications of these systems have noted barriers to generating high levels of evidence for specific implementation among key populations. Therefore, expanding IR for key populations, including adaptation of existing interventions, will likely require the use of hybrid approaches to supplement existing programmatic data with targeted data-collection. Furthermore, small-scale iterative testing of implementation strategies using tools that leverage existing resources, such as the multiphase optimization strategy (MOST)(47), can maximize efficiency and ultimately improve the strength of evidence for key populations.

Consideration of the adaptability of interventions, or the degree to which the peripheral elements of an intervention may be modified for context while maintaining fidelity to the tested core components, is particularly salient for key populations research. For key populations, structural issues including socially sanctioned discrimination and violence coupled with criminalization may further compromise effective implementation(48). Thus modifications to both the content (e.g., changes to intervention materials, procedures, or delivery) and context (e.g., changes to format, location, or personnel)(49) of existing interventions may be most effective when accounting for the equity status of key populations and the intersection of intervention components with the outer, inner and individual domains(43).

Complexity is another critical construct for IR consideration. Complex interventions are likely needed to overcome the circumstances by which key populations experience an increase in HIV risk and transmission, yet complexity also creates implementation barriers. While there is increasing recognition of the need for more human-centered design to meet heterogeneity of needs, there is likely to be a concomitant tension between individualized, patient-centered care and feasibility of implementation by overburdened health systems. Thus, assessment of feasibility and appropriateness prior to fully scaled implementation, followed by a comprehensive assessment of intervention fidelity, is essential to generate clear evidence of why interventions succeed or fail(50).

Investments in comprehensive HIV programs for key populations are necessary to accelerate the HIV response, yet programs that address the specific needs of key populations remain severely underfunded(51). While intervention and implementation costs may be perceived as high per individual reached, programs that address the needs of key populations are generally considered cost-effective because of the potential for achieving large-scale epidemic impact through disruption of ongoing chains of transmission(52). Therefore, increasing overall investment in IR for key populations, and including cost-effectiveness and mathematical modeling assessments of different implementation strategies across populations is essential to evaluate and optimize intervention delivery and impact.

Process

The research and implementation process with key populations comes with a unique set of challenges impacting participant representation, information precision and accuracy, and ultimately epidemiologic inference (Table 2). Broadly, these process issues can be categorized into issues of engagement, planning, execution, and evaluation.

Table 2:

Research Process, Implementation and Implications for Inference

CFIR Construct	Research Process and Implementation	Implications for inference
Engaging	Coordinating with implementing partner Community Advisory Groups (CAGs)	Competing interests between program and research (external vs. internal validity) Perception of study within key population communities affects study participation Information and misinformation about study objectives or procedures can spread quickly through key populations networks if CAGs are non-existent, inactive, not representative of the larger population to be reached, or not fully aware of details
Planning	Study design Protocol development	Selection of sampling method will influence who is recruited Protocol needs to take into account real-world implementation structure Reimbursements can play a larger role among key populations in decisions to participate or attempts to doubly enroll
Executing	Study implementation	Training of study staff in research methods and how these differ from program implementation Want the research to be rigorous but don’t want to interrupt the program itself Adaptability of objectives and methods based on changing program dynamics Assessment of fidelity of implementation perspective should capture lack of fidelity due to program/research teams vs. inability to find key populations for intervention delivery. This latter impact may have a larger impact on intervention effectiveness among key populations and also speaks to intervention appropriateness for all, or some key population groups.
Reflecting and Evaluating	Analysis Dissemination of results back to key stakeholders	Weighting/accounting for sampling biases and missing data (especially important if using program data) Measurement and reporting of implementation outcomes alongside effectiveness data to better inform reasons for why interventions succeed or fail Lack of research dissemination may affect future participation/representation. Furthermore, less top-down approaches to dissemination that include active community interpretation of results often provide invaluable insights to the final interpretation of results and potential drivers of intervention failure or success.

Key population members are not only recipients of the intervention, but often play a major role in the implementation process and the uptake of results. Therefore, engaging with key population communities prior to study start, during implementation, and throughout interpretation of the results is crucial. This engagement can take the form of community advisory groups (CAGs) or dissemination of information through key populations networks. As misinformation can spread quickly in highly networked groups, clear and concise information is needed around the objectives and implications of the research. Misinformation can lead to limited or differential participation, with potential participants becoming distrustful of study activities.

IR planning should account for real-world implementation structures, context-specific challenges, and community feedback. The selected sampling method will impact the subset of the population reached and the potential for subsequent inference and generalizability (Figure 2). Given the absence of a sampling frame and the challenges recruiting representative samples of key populations, sampling methods that allow for bias assessment and correction are frequently recommended, such as respondent-driven sampling or time-location sampling(11–13). These methods may be effective at recruiting individuals into programs or interventions who are less readily reached(24), and a limited, but growing evidence base is emerging to compare the heterogeneity and reach of these approaches(11–13). Even utilizing these approaches, however, there will be individuals within the population who are less visible and potentially at greater risk if prevention efforts are not penetrating. These populations, such as FSW controlled by venue managers, or MSM married to women, may have differing intervention needs than those “easily” accessible. Additionally, utilizing program data and embedding IR within existing programming structures may foster broader generalizability of intervention effects, but may be misinterpreted if used for health surveillance purposes. For example, HIV prevention programming data may underestimate HIV prevalence as those knowingly living with HIV are appropriately less likely to engage in HIV testing and HIV acquisition prevention activities. Key populations IR planning should also consider identification and double-counting issues, as collection of identifiers from stigmatized and criminalized groups may be discouraged in certain contexts or members may provide false identities, to protect themselves from criminalization or discrimination or to seek remuneration for research participation given underlying economic vulnerabilities(53, 54). Generation of unique IDs through replicable questions, or more recently through biometric data capture (e.g. fingerprints, iris scanner) can help to address this, but has its own challenges(53).

Executing any study requires adherence to clearly outlined standard operating procedures, but evaluation designs for key populations will need to be particularly flexible to rapid changes in legal, social or programmatic context. Utilizing iterative designs with smaller trials that work quickly to develop incremental evidence may offer solutions to leverage program infrastructure and ongoing key population engagement for IR, while remaining nimble to frequent shifts in program and donor priorities(47). Program data may also have increased rates of missing data, and trade-offs in data integrity and large-scale data should be evaluated, with additional efforts to supplement data gaps. However, program data may be less prone to social desirability bias, when community trust in programs and staff members is high. Accounting for loss to follow-up, missing data or suspected over/underreporting can be addressed through recalibration of estimates through bias validation samples (e.g. PrEP adherence), inverse probability weights, and multiple imputation, and utilization of these approaches in HIV IR with key populations’ programmatic data are urgently needed. Finally, evaluation should incorporate measurement of implementation outcomes beyond acceptability and adoption(55, 56), including fidelity (both attempted intervention delivery and participant receipt), feasibility (from an organizational and participant retention perspective), cost, maintenance, and penetration, alongside measurement of service and client outcomes (e.g. efficiency, timeliness, effectiveness, equity, satisfaction).

Conclusions

The disconnect between the efficacy of HIV prevention and treatment interventions and the effectiveness of these interventions at the population-level, brings to focus the need for improved and focused implementation strategies(57). Moving forward necessitates better addressing the needs of key populations who are often both at high risk of HIV acquisition and transmission and marginalized from and underserved by existing efficacious interventions. While research and programs focused on serving key populations may be more complex and varies widely across locations and key populations, this work is central to the HIV response, independent of epidemic setting. Research that addresses or measures the embedded nature of individuals within their settings, prioritizes flexibility, and evaluates implementation barriers and strategies to overcome them for key populations, combined with programs that integrate consideration of these elements may facilitate a more efficient HIV pandemic response. This nuanced HIV response is essential to close the gaps between efficacy, effectiveness, and ultimately population-level impact.