The Evidence Project: Protocol for Systematic Reviews of Behavioral Interventions and Behavioral Aspects of Biomedical Interventions for HIV Prevention, Treatment, and Health Service Delivery in Low- and Middle-Income Countries

Abstract

The Evidence Project conducts systematic reviews and meta-analyses of HIV behavioral interventions, behavioral aspects of biomedical interventions, combination prevention strategies, modes of service delivery, and integrated programs in low- and middle-income countries. Here, we present the overall protocol for our reviews. For each topic, we conduct a comprehensive search of five online databases, complemented by secondary reference searching. Articles are included if they are published in peer-reviewed journals and present pre/post or multi-arm data on outcomes of interest. Data are extracted from each included article by two trained coders working independently using standardized coding forms, with differences resolved by consensus. Risk of bias is assessed with the Evidence Project tool. Data are synthesized descriptively, and meta-analysis is conducted when there are similarly measured outcomes across studies. For over 20 years, this approach has allowed us to synthesize literature on the effectiveness of interventions and contribute to the global HIV response.

Introduction

There are enormous pressures on HIV programs to spend limited funds wisely and efficiently. However, despite the vast and ever-growing scientific literature on the effectiveness of interventions, evidence-informed decision-making is challenging. Even for well-trained scientists, identifying relevant studies is time-consuming; the quality of research from published reports requires careful analysis; conflicting findings and inconsistent use of metrics and study designs often occur across studies; and pooling results across studies requires advanced statistical techniques. The state-of-the-art strategy to address these challenges is to use systematic reviews and meta-analyses to synthesize the effects of interventions as evidenced by research across multiple studies.

The field of HIV has changed over time, as have the needs for evidence synthesis. Early in the HIV epidemic, without effective biomedical prevention or treatment strategies, behavioral interventions such as condom promotion, behavioral counseling, peer education, and health communication programs were central to the HIV response. However, recent decades have seen remarkable advances in the development and widespread availability of highly effective antiretroviral treatment (ART), pre-exposure prophylaxis (PrEP), and voluntary medical male circumcision (VMMC). With these new biomedical interventions, behavioral interventions took on a different role. For each biomedical intervention, associated behavioral and structural interventions are needed to ensure their success, such as interventions to promote linkage and engagement in care, retention, and adherence (Koblin et al., 2013; McNairy et al., 2013).

In addition, there has been recognition of the merit of combining behavioral, operational, and biomedical intervention strategies to improve outcomes (NIAID, 2013; UNAIDS, 2010), and how modes of service delivery impact the effectiveness of biomedical interventions. While over 75% of people living with HIV globally were taking ART as of 2021, (UNAIDS, 2022) challenges remain with adherence (Nachega et al., 2016) and retention in over strained health care systems (Penn et al., 2018). To address these challenges, differentiated care strategies have been implemented, which generally function by modifying the modes of service delivery (Abelman et al., 2020; Bygrave et al., 2020; Mavhu et al., 2020; Mukumbang, 2020; Pascoe et al., 2020; Rabkin et al., 2020; Srivastava et al., 2019; Wilkinson & Grimsrud, 2020). Examples include fast-track drug refill programs, strategies to space appointments, community-based ART distribution, personnel task-shifting, and decentralization of care (Hagey et al., 2018).

Finally, there has been a dramatic increase in the integration of non-HIV interventions with HIV prevention and care programs (Njuguna et al., 2018). Many of these programs have been found to be highly cost-effective (Nugent et al., 2018) as the often well-resourced HIV care system lends itself to addressing multi-health issues. There are multiple models of integration, with one common strategy being the addition of non-communicable disease care to existing HIV clinical care settings (Duffy et al., 2017). These programs have been shown to reduce stigma and increase patient satisfaction (Odeny et al., 2013; Vo et al., 2012). Integration of sexual and reproductive health and HIV services is also common, often with reports of high acceptability and improved efficiency, but with the risk of negative effects on staff workload and reduction in quality of care (Okegbe et al., 2022). There has also been a growing realization that a major barrier to HIV care engagement is mental health co-morbidity, especially depression, (Udedi et al., 2018; Udedi et al., 2019; Wagner et al., 2014) anxiety, (Olagunju et al., 2013) and substance abuse (Hassan et al., 2019). As a result, mental health screening and treatment programs have been increasingly integrated into HIV care and treatment programs.

These shifts in interventions and priorities within HIV prevention and treatment programs require consistently updated information on the best evidence of the effectiveness of behavioral interventions, behavioral aspects of biomedical interventions, combination prevention strategies, modes of service delivery, and integrated programs. For more than 20 years, the Evidence Project has conducted systematic reviews and meta-analyses in these areas to inform the HIV response. We have published multiple peer-reviewed articles, (Atkins et al., 2020; Bertrand et al., 2006; Denison et al., 2008; Fonner, Armstrong, et al., 2014; Fonner et al., 2012; Fonner, Kennedy, et al., 2014; Kennedy et al., 2007; Kennedy et al., 2019; Kennedy et al., 2015; Kennedy et al., 2014; Kennedy et al., 2013; Kennedy et al., 2010; Kennedy et al., 2020; Medley et al., 2009; O’Reilly et al., 2017; O’Reilly et al., 2014; O’Reilly et al., 2020; O’Reilly et al., 2013; O’Reilly et al., 2022; Sweat et al., 2007; Sweat et al., 2012; Sweat et al., 2020; Zajac et al., 2015) presented our work at numerous scientific conferences, contributed to over a dozen global guideline development processes, and disseminated findings through fact sheets for program planners (Project, 2022). Here, we present the overall protocol followed across topics for systematic reviews conducted under the Evidence Project.

Methods/Design

Selection of topics

Evidence Project topics focus on the efficacy and effectiveness of behavioral interventions, behavioral aspects of biomedical interventions, combination prevention strategies, modes of service delivery, and integrated programs for HIV in low- and middle-income countries. While our topics have changed over time, we prioritize reviews that will have immediate usefulness in the field to directly improve HIV response.

After selecting a topic, we carefully define the intervention of interest. In some cases, we map out a theoretical framework for each intervention modeled on the U.S. Center for Disease Control and Prevention’s Community Guide project (Briss et al., 2000) that includes the relationship between intervention components, intermediate outcomes, and health outcomes. We use written definitions and theoretical models to aid in the development of inclusion criteria and search strategies for each topic.

Eligibility criteria

After clearly defining the topic, we next develop a set of inclusion criteria. Some inclusion criteria are topic-specific and are related to the specific intervention and population of interest. Other criteria are standard across Evidence Project reviews, covering study design, location, and dates of inclusion. To be included in our reviews, studies must be published in a peer-reviewed journal and present quantitative data evaluating the effectiveness of an intervention with a study design that provides either: (1) pre-intervention/post-intervention comparisons of outcomes, or (2) multi-arm comparisons of outcomes between participants who received the intervention of interest and those who received either a control condition or a comparison treatment. Comparisons of more intensive/less intensive versions of the intervention are also included.

Studies must also have been conducted in a low- or middle-income country (LMIC), classified by the World Bank as “lower income”, “lower-middle income” or “upper-middle income” (World Bank, 2024). We track when countries transition from upper-middle income to high-income (or vice versa) and include studies if the study was a lower or middle-income at the time of the study, even if the country has since transitioned to high-income status.

Initially, we used 1990 as a cutoff date for article inclusion, as HIV was only identified in the 1980s, and few interventions were evaluated in LMIC settings prior to 1990. Currently, our inclusion date begins for publications published in the year 2000 to reflect the significant changes in these settings that have occurred over the past several decades, including the development and widespread availability of ART. In some cases, our eligibility criteria start later (e.g., in studies of PrEP, which only became available in the 2010s).

Information sources

Across topics, we conduct a broad search for eligible articles using the following electronic bibliographic databases: (1) PubMed, (2) PsycINFO, (3) Sociological Abstracts, (4) Cumulative Index to Nursing and Allied Health Literature (CINAHL), and (5) Embase. Over the years, we have found these databases to be a comprehensive source for articles relevant to global HIV interventions.

In addition to database searching, we conduct secondary searching of the reference sections of papers that are included in each review. These are acquired, screened, and if accepted, are subject to additional secondary searches. The process is iterated until no new papers are identified. Additionally, we review the references from previous review papers and meta-analysis for possible citations. In some cases, we contact persons known as experts in the field of interest and provide them with our list of papers to solicit any missing references.

Search strategy

For each topic, we generally include a set of search terms related to (1) HIV, (2) low- and middle-income countries, and (3) the intervention of interest. Search terms are developed, tested, and refined with the input from multiple co-investigators, then adapted to the controlled vocabulary and unique syntax of each database. For each database, we track the search terms used and the associated number of citations identified, per PRISMA guidelines (Page et al., 2021).

Study Records

Data management

Results from each search are downloaded to EndNote^™ reference libraries. Results from each database are merged into a single pooled database, and duplicates are removed using the automatic functions in EndNote^™, followed by manual removal of duplicate citations not identified by the software.

Selection process

Once the final list of citations from the database search is compiled, trained research assistants (generally graduate students at the Johns Hopkins Bloomberg School of Public Health) are assigned preliminary title/abstract screening of all citations to identify potentially relevant studies. Research assistants are instructed to err on the side of inclusion, as these references are later subjected to a more thorough in-depth title/abstract review, in which two members of the senior study staff separately review the pooled database generated by the search and initial screening and categorize citations as either: (1) meets inclusion criteria, (2) unclear if it meets inclusion criteria; acquire full text article for further inquiry, or (3) does not meet inclusion criteria. Additionally, notations are made to our citation processing system during this process (for example, reasons that citations were rejected, or specific issues to look for if a paper is acquired to assess eligibility). The separate screened files are then merged for comparison. Citations with concordant categorizations across screeners are assigned to that category. Citations with discordant screening are discussed by the two screeners to gain consensus on the categorization, with referral to senior investigators as needed. When consensus cannot be reached, the full text article is acquired for further inquiry. We have found that some studies will qualify for inclusion in more than one topic area across our reviews, which reflects the prevalence of co-occurring interventions. We code such citations under all topics where they meet the inclusion criteria.

Full-text articles are typically acquired electronically (as PDF documents). Articles not available electronically are retrieved from the library or ordered via interlibrary loan.

We include studies in all languages in our reviews. When non-English language citations are identified, we acquire, screen, and code them. Many foreign language journals publish an English abstract. In some cases, staff members speak the relevant language and can screen and code a study directly; in other cases, we translate the article into English before coding.

Data collection process

The first step in the coding process is an additional check to assure that the citation meets the inclusion criteria. If it does, the data are extracted using a highly detailed electronic data coding form. Each citation is assigned to two separate coders who are instructed not to discuss the paper with the other team member coding the paper to avoid bias and as a quality control measure. The coding form includes some data items which are standard across Evidence Project reviews, and others which are adapted to each intervention topic.

Data items

Below we describe core data items collected from each study. When a study does not report an item, we list it as not reported. When a study reports something about an item but it is not possible to understand or classify it based on what was reported, we list it as “cannot code” with an explanatory note.

Citation information and study inclusion criteria

We record full citation information for each article, including author names, article title, journal, and year of publication. We then assess whether the study meets each of the topic-specific and overall Evidence Project inclusion criteria to provide an additional quality check for inclusion.

Study setting

The setting where the research was conducted is collected by noting the region (categorized first according to the WHO regions: African Region, Region of the Americas, South-East Asian Region, European Region, Eastern Mediterranean Region, and Western Pacific Region, then country, then the specific study location(s) within the country (region/province/district/city/etc.), and finally categorization of this location as urban/peri-urban/rural.

Study population

We capture both eligibility criteria (whether participant eligibility criteria are reported and what they are) and characteristics of participants who participated in the study, as we find that these two aspects often differ. We record an overall description of the type of population included in the study (e.g. general population, pregnant women, etc.), then capture details on gender and age distribution. We record the sample size (including sample size at different time points or for different subgroups), sampling strategy (classified as probability, non-probability, census, or other), loss to follow up rates, and differences between study arms in follow-up rates.

Study design

We classify study design by following a series of questions about the comparison groups, intervention exposures, and measurement periods. This leads us to classify studies into the following categories for study design:

1. Randomized trial – Individual (experiment): Minimum two study arms; random assignment of individuals to study arm.

2. Randomized trial – Group (experiment): Minimum two study arms; random assignment of groups (couples, classrooms, towns, etc.) to study arm.

3. Non-randomized “trial” – Individual: Minimum two study arms; assignment of individuals to study arm, but not done randomly.

4. Non-randomized “trial” – Group: Minimum two study arms; assignment of groups to study arm, but not done randomly.

5. Before-after study: Pre- and post-intervention assessment among the same individuals. One study arm and one follow-up assessment period.

6. Time series study: Pre-intervention and several post-intervention assessments among the same individuals. One study arm and multiple follow-up assessment periods.

7. Case-control study: Two groups defined by outcome measures, one consisting of cases and one consisting of controls.

8. Prospective cohort study: Two or more groups defined by exposure measures and followed over time.

9. Retrospective cohort study: Two or more groups defined by exposure measures but uses previously collected or historical data.

10. Cross-sectional study: Exposure and outcome determined in the same population at the same time.

11. Serial cross-sectional study: When a cross-sectional survey is conducted in a population at multiple points in time with different people in that population.

12. Other study designs: Other designs, such as a time series study with comparison group or a stepped-wedge study, are described individually.

Once we have identified the study design, we note the specific time points of outcome measurement for each study (follow-up time periods). We also record the number and characteristics of study arms or groups (e.g., intervention and comparison/control groups).

Intervention description and implementation characteristics

We collect detailed information on the intervention being evaluated in the study, and for all interventions or comparison groups included in the study. We first capture the name of the intervention (if one is provided). Then, using intervention implementation measures proposed by Hickey et al. (2017) and adapted from Proctor et al. (2013), we extract data on: (1) the actor, or who carries out the intervention, (2) the action, which are the steps required for carrying out the intervention, (3) the intervention dose, (4) the temporality, or timing in relation to other processes, (5) the action target, which is the specific action done to achieve the outcome being targeted, and (6) the behavioral, social, or organizational target. In addition, we extract data from the Template for Intervention Description and Replication (TIDieR) checklist (Hoffmann et al., 2014), including: (1) the stated rationale for the intervention, (2) what specifically was used in terms of physical or educational materials, including the source, (3) the mode of delivery (such as face to face, via telephone, etc.), (4) whether the intervention was provided individually or in a group, (5) the location where the intervention occurred and necessary infrastructure, (6) the number of times the intervention was delivered and over what period of time, (7) whether and how the intervention was tailored in the course of its delivery, (8) whether the intervention was modified over the course of the study, and (9) if fidelity was assessed, and the level of adherence observed to protocol. Depending on the topic, we may adapt information collected on aspects of intervention implementation either using the frameworks listed above or drawing from other relevant frameworks within the field of implementation science.

Theoretical basis

To assess the theoretical basis of interventions we extract data on whether the authors explicitly refer to an existing or newly developed theory, and which theory they note; whether a theoretical model is presented; and how variables assessed in the study were specifically mapped to theory constructs presented in the model.

Intervention topic-specific questions

For each topic reviewed, we extract data relevant to that particular intervention or topic. For example, for peer education, we captured information about how peers were selected and how they were matched to the population of interest; for income generation and microfinance, we captured information about the financial institution involved, and details on loan/grant dispersal and repayment. These topic-specific questions are often related to details of the intervention itself but may also be related to the population or study design relevant for the topic area.

Outcomes

All outcome measures reported in a study are recorded, and then defined as either “primary” or “secondary” outcomes. Detailed textual descriptions of all outcome measures are recorded. Primary outcomes are defined as those with a comparison between the group that received the intervention of interest to the group(s) that did not – either pre-/post or across study arms – on the outcomes of interest. Secondary outcomes are defined as those with only post-test or within-arm comparisons presented. For each primary outcome, results are coded in a structured format which includes: (1) the type of statistical analysis used, (2) the independent variable (which may be the study arm), (2) the sample size for each outcome, (3) the effect size, base rate and sample size associated with each outcome, (4) catchment and/or follow-up times, (5) the confidence interval and/or p-value, (6) the page number and table where the result are located, and (7) any additional brief information thought to be important. In addition, when there are repeated measures reported we capture effects at every time interval reported. All primary results are coded, including sub-group presentation of results (such as by gender) even when aggregated results are also presented.

We generally include HIV-related outcomes for all our reviews. For some reviews, we include other outcomes as appropriate to the topic area, intervention, or population. We extract outcome data into the categories and subcategories listed in Table 1.

Table 1.

Categories of outcomes measures extracted across Evidence Project systematic reviews

Outcome category	Outcome sub-category
Protected/unprotected sex (e.g., condom use)	Condom use, main partners Condom use, casual partners Condom use, sex worker Condom use, partner unspecified Condom use, last sex, partner unspecified Unprotected sex, main partners Unprotected sex, casual partners Unprotected sex, sex worker Unprotected sex, partner unspecified Unprotected sex, last sex, partner unspecified Unprotected sex with partner of unknown serostatus Other (includes sex while protected by PrEP, condom use or unprotected sex with other kinds/combinations of partner types, unprotected anal intercourse, female condoms, etc.)
HIV incidence/prevalence	HIV incidence HIV prevalence Other
STI incidence/prevalence	Chlamydia Gonorrhea Hepatitis (any) Herpes HPV / Genital warts Syphilis Trichomoniasis Any combination of multiple STIs Other
Sexual partners	Number of sex partners ever Number of sex partners in a recent span of time Abstinence (0 sex partners) Monogamy (1 sex partner) Extramarital sex Sexual partner concurrency Initiation of first sexual intercourse Other
Violence and abuse	Physical abuse Verbal abuse Emotional abuse Combinations of any/all of the above
Condom acquisition/negotiation	Condom acquisition Condom negotiation Other
Stigma/discrimination	Stigma Discrimination Other
Knowledge and attitude related outcomes	HIV knowledge, attitudes, perceptions, beliefs Other Knowledge, attitudes, perceptions, beliefs Self-efficacy Self-esteem Other
HIV testing	Ever tested for HIV Recently tested for HIV Disclosure of test results First-time testing Repeat testing Other
Substance use related outcomes	Injecting drug use Initiation of drug injection Non-injecting drug use Cleaning/bleaching drug paraphernalia Use of new sterile needles/syringes Multi-person use of drug paraphernalia Return of used syringes Alcohol use Other
Initiation (ART/PrEP)	Initiation of ART Initiation of PrEP Other
Adherence/continuation (ART/PrEP)	Adherence to ART Continuation/discontinuation of ART Adherence to PrEP Continuation/discontinuation of PrEP Other
Use of care and treatment	Linkage to care after testing Retention in care Lost to follow-up (dropout) Re-engagement in care Other
Clinical outcomes	CD4 count Viral load/viremia Death Clinical AIDS/stage of disease Other
Other	Other

We generally defer to the authors’ definitions of outcomes and are broadly inclusive when authors include ways of defining outcomes that may have diverse definitions (e.g., quality, acceptability).

Risk of bias in individual studies

We assess the risk of bias of each study using an eight-point tool that we developed for the project. (Kennedy et al., 2019) This tool descriptively summarizes elements of study design and quality that allow for a standard comparison of risk of bias across analyses. The tool includes the following criteria: (1) Prospective cohort analyses presenting data for a cohort of study subjects followed over time, including pre-intervention to post-intervention analyses with or without a control or comparison group. Serial cross-sectional analyses, or post only comparisons, are not scored on this criterion; (2) Control or comparison groups are defined as analyses that compare those who received the intervention to those who did not, or who received a more-versus less-intensive intervention. These include analyses that compare intervention, control and/or comparison groups, and stratified cross-sectional analyses. This item does not include before-after analyses without stratification; (3) Pre/Post intervention outcome data are assessed, as it is common for studies to only assess outcome measures in the post-intervention catchments, especially for post-hoc analyses and secondary study aims; (4) Random assignment to treatment groups assesses whether subjects were randomly assigned to treatment groups in multi-arm studies, including group randomized designs. This criterion is nested within criterion for a control or comparison group in order to give added weight to designs which include randomization and control; (5) Random selection of subjects for assessments are assessed to determine whether there was a selection bias in study enrollment; (6) Attrition is assessed to determine if the follow up rate was 80% or more at each analysis point; (7) Comparison group socio-demographic matching is assessed in multi-arm studies to determine if there were no statistically significant differences in socio-demographic measures (such as age) across arms at baseline; and (8) Comparison group outcome matching is assessed to establish whether studies had a statistically significant baseline differences in study outcome measures. Further information on the individual elements and assessment of reliability of the tool has been published previously (Kennedy et al., 2019).

On occasion, we also use the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework to rate the quality of evidence across outcomes for the entire body of studies using summary of evidence tables (Guyatt et al., 2008). We particularly do this when our reviews are being used for World Health Organization (WHO) guidelines, (Atkins et al., 2020; Kennedy et al., 2020) as required by the WHO guideline development process (WHO handbook for guideline development, 2nd ed, 2014).

Quality control and intercoder resolution

The purpose of intercoder resolution is to identify different interpretations in the presentation of results. Resolution is labor-intensive and requires an in-depth understanding of the study procedures. The two coders compare the results from their coding forms and, where there are discrepancies, come to an agreement on the best interpretation. Challenging issues are referred to the project coordinator, principal investigator, and other senior collaborators. We also attempt to contact authors to resolve areas of uncertainty if necessary.

Data

Synthesis

Once eligible studies have been subjected to data abstraction, a review of the study results is made to organize and categorize studies in a logical manner for analysis. Frequently, there are sets of similar intervention strategies that lend themselves to a grouping for analysis. Likewise, the outcome that the original studies assessed is a critical variable to consider when grouping studies for analysis. In some cases, we stratify results by key characteristics of the population or study design determined in advance. Meta-analysis is conducted when there are multiple studies (usually at least 3 studies) that present common outcomes in a sufficiently similar format to allow for mathematical pooling of the effect size estimates. Otherwise, we subject the discrete analyses to qualitative review and compare and contrast results across studies.

Meta-Analysis

When meta-analysis is deemed appropriate for a review topic because there are a sufficient number of similarly-measured outcomes, we calculate study effect sizes using standard statistical procedures. We do not limit studies from meta-analysis due to location, population, intervention type, sample size, quality assessment, or other reasons, although we may stratify meta-analyses to better evaluate these or other factors. We generally make conversions to odds ratios or, increasingly, risk ratios for presentation and publication given the ease of interpretation and familiarity of these measures in the field. This can be done when combining effect sizes derived from both continuous and dichotomous outcomes. We calculate the effect size using means and standard deviations when available. If the mean and standard deviation are not available, we use other reported indices to derive estimates of the mean or variance using the appropriate transformation. Dichotomous outcomes are typically converted to risk ratios and combined with effect sizes derived from continuous outcomes when appropriate. We generally use random effects models rather than fixed effects models, as random effects models assume that heterogeneity is due to factors other than chance alone, and the studies included in our reviews almost always come from diverse populations with significant variance in study design and setting. We also assess heterogeneity across studies in each analysis using both the Q and I-squared statistics.

We code all outcomes reported, synthesize and meta-analyze separately by outcomes, populations, content, and unique intervention strategies. Intervention topics are often broad, but we stratify analyses by these factors to assure that commonalties are maintained in the analyses. Contrasts across the strata analyzed are then conducted.

Qualitative synthesis

In some cases, meta-analysis is not appropriate because there are an insufficient number of studies comparing similar interventions and populations using similarly measured outcomes. In these cases, we present results across studies qualitatively. Usually, we organize these results by key categories such as study design (e.g., separating randomized trials from observational studies), population (e.g., separating studies with adults versus adolescents, or across key populations), intervention (e.g., separating studies that employed different variations on the intervention approach), or outcomes (e.g., separating studies that measured different outcome categories). We try to present critical assessment of studies in qualitative synthesis to avoid “vote-counting”, which occurs when review teams just count the number of studies with significant results, rather than presenting and interpreting results in the context of the study quality and size of the effect.

Discussion and Conclusions

Over the past 20 years, the Evidence Project has followed this approach to publish high-quality systematic reviews and meta-analyses on various topics related to HIV interventions in low- and middle-income countries. These reviews have been used to inform policy through the development of easily digestible fact sheets for program managers and WHO guidelines.

In this paper, we present our project methods which we generally use across review topics, although with each new review we review our approach and adapt it as needed to the new topic. Our overall approach has both strengths and limitations. We generally follow established procedures for conducting systematic reviews and meta-analysis, including following PRISMA guidelines (Page et al., 2021) for reporting systematic review results, and many of our methods align with recommendations from established groups such as the Cochrane Collaboration (Higgins et al., 2023). Our inclusion of a wide range of outcomes is comprehensive but can lead to challenges synthesizing across outcomes. Our typical focus on peer-reviewed journal articles ensures a higher level of rigor and detail for included studies, although it may mean we miss relevant studies published in the grey literature.

To date, the Evidence Project is the only group that has conducted a large number of systematic reviews focused on the effectiveness of behavioral interventions, behavioral aspects of biomedical interventions, combination prevention strategies, modes of service delivery, and integrated programs for HIV in low- and middle-income countries. The methods described here have stood up well over time and across a wide range of topics. As the field of HIV changes, we will continue to adapt our approach and our topics to continually be relevant and contribute our part to the global HIV response.