Clinical trials and systematic reviews addressing similar interventions for the same condition do not consider similar outcomes to be important: A case study in HIV/AIDS

Abstract

BACKGROUND

The usefulness of clinical trials and systematic reviews is compromised when they report different outcomes. We compared outcomes in reviews of HIV/AIDS and the trials included in the reviews.

STUDY DESIGN AND SETTING

We examined all Cochrane reviews of HIV/AIDS (as of June 2013) that included ≥ 1 trial, and the trials that the reviews included. We compared outcomes within subgroups defined by type of intervention: clinical management, biomedical prevention, behavioral prevention, and health services.

RESULTS

We included 84 reviews that encompassed 524 trials. Although the median number of outcomes per trial (8) and per review (7.5) was similar, the trials reported a considerably greater number of unique outcomes than the reviews (779 vs. 218), ranging from 2.3 times greater (clinical management) to 5.4 times greater (behavioral prevention). High proportions of trial outcomes were not in any review: 68% (clinical management) to 83% (behavioral prevention). Lower proportions of review outcomes were not in any trial: 11% (clinical management) to 39% (health services).

CONCLUSION

Outcomes in trials and reviews are not well aligned for appropriate inclusion of trial results in reviews and meta-analyses. Differences in perspectives, goals, and constraints between trialists and reviewers may explain differences in outcomes they consider important.

BACKGROUND

In clinical trials (“trials”), an outcome is a measure or event that is used to assess the effectiveness and/or safety of clinical interventions in study participants [1]. Systematic reviews (“reviews”) are conducted to synthesize the evidence from all available trials addressing a given clinical intervention. Meta-analyses, that combine the results from multiple trials quantitatively, are optional components of systematic reviews. When a trial included in a review has not measured or reported a certain outcome, it cannot be included in the meta-analysis for that outcome. For this reason, the overall usefulness of trials and reviews is compromised when trials and reviews report data on different outcomes, or report data on the same outcomes measured or analyzed differently or at different time-points [2–5]. This ultimately hinders the applicability of review evidence for healthcare decision-making [3,6].

There are two main reasons for ensuring that the most important outcomes in trials and reviews overlap. The first reason is ethical: volunteers who participate in human experiments (trials) expect their contributions to increase knowledge and improve clinical care [4]. Clinical care is increasingly based on distilled trial evidence available in reviews. When trial outcomes do not contribute to reviews, the trials fail to impact clinical care through this important route. The second reason is scientific: all trials should report important outcomes to contribute to the best science.

There are many reasons that fewer trials included in a systematic review might contribute data to a given meta-analysis in the review. First, trialists and reviewers may have differing goals and perspectives, including important outcomes to examine. Second, trialists and reviewers might have different resource constraints within which they need to operate. If the goals, perspectives, and constraints of trialists and reviewers are indeed different, to what extent should we expect overlap in the outcomes they consider important? An examination of the similarities and differences in outcomes in trials and reviews could provide a starting point for efforts to increase the overlap.

Finally, when only a few trials included in a review provide data for a meta-analysis for a given outcome, this could represent outcome reporting bias, which is the selective reporting of outcomes based on the nature or direction of their results [7]. Outcome reporting bias affects the dependability of the overall findings of the review [7–10]. Indeed, Furukawa and colleagues have demonstrated that the meta-analytic treatment effect is inversely associated with the proportion of the review’s included trials contributing to the meta-analysis [11]. For example, the magnitude of the treatment effect for dichotomous outcomes reduced in a dose-response way from a pooled odds ratio of 2.7 for meta-analyses with fewer than 20% of trials contributing, to 1.9 for meta-analyses with at least 80% of trials contributing [11]. Also, the smaller the number of potentially eligible trials included in a meta-analysis, the less precise the summary estimate is likely to be [12].

Trials and reviews in the field of HIV/AIDS are particularly important because it remains a disease condition with high global burden. The global prevalence is estimated to be 0.8% (95% confidence interval [CI] 0.7% to 0.9%), with considerable regional and country-level variation [13]. Worldwide, approximately 36.7 million individuals are estimated to be living with HIV/AIDS. Since the epidemic began, AIDS-related illnesses have claimed approximately 35 million lives, 1.1 million of which were lost in 2015 [13]. A large number of HIV/AIDS trials and reviews have been conducted, and these have examined a variety of relevant interventions and outcomes [14].

OBJECTIVE

Our objective was to examine the similarities and differences between outcomes named in reviews of HIV/AIDS and outcomes reported in the trials included in the reviews.

METHODS

What constituted an “outcome”

Researchers typically use the word “outcome” to refer to the outcome domain (e.g., “depression”, “blood pressure”), and in this study we adopted the same approach. For example, since depression measured using the Hamilton Rating Scale for Depression and depression measured using the Beck Depression Inventory both pertain to the outcome domain “depression”, we counted both as a single outcome. We classified outcome domains as specifically as possible, however. For example, if a certain trial or review mentioned examining both “male condom use” and “female condom use”, we considered these to be two separate outcome domains. Similarly, for composite outcomes (e.g., “death or myocardial infarction”), we considered the composite outcome’s individual outcome domains (i.e., “death” and “myocardial infarction”) to be separate outcome domains. When a trial or review in our sample measured a certain outcome at multiple time-points, we counted all as a single outcome, for example, blood pressure at 1 year and blood pressure at 2 years both pertain to the outcome domain “blood pressure”.

Reviews examined

We examined and compared all outcomes in trials and reviews published by the Cochrane Review Group on HIV/AIDS in the Cochrane Database of Systematic Reviews, as of June 30, 2013. We used that group’s system for classifying types of HIV/AIDS interventions: (a) therapeutics, prognostics, and diagnostics (“clinical management”); (b) biomedical prevention (“biomedical prevention”); (c) behavioral, social, and policy prevention (“behavioral prevention”); and (d) organization and financing of health services and care (“health services”). We assessed the overlap in outcomes within subgroups of trials and reviews defined by type of intervention.

Data abstraction from reviews

We previously described our methods abstracting information about the outcomes named in Cochrane reviews on HIV/AIDS [14]. For the current study, we were interested in examining the overlap in outcomes between the reviews and the trials that the reviews included; we therefore restricted our sample to those HIV/AIDS reviews that were completed (i.e., we excluded protocols) and included at least one trial (i.e., we excluded “empty” reviews). For each eligible review, we abstracted all outcomes that were named in the Methods section of the completed review, irrespective of whether those outcomes were also presented in the Results section.

Trials examined

We examined each trial that each eligible review included, provided the trial: (1) compared at least two groups to which participants were randomly allocated; and (2) was published as a peer-reviewed journal article (i.e., we excluded conference abstracts). For each trial, we identified one journal article defined by the review authors as the primary publication they used for abstracting data from that trial, as conventionally indicated in Cochrane reviews by an asterisk (*) next to the citation information in the “References to Studies Included in this Review” section.

Data abstraction from trials

From each included trial’s journal article, we abstracted all outcomes for which results were reported. “Results” were defined as quantitative data comparing two or more interventions for effectiveness or safety reported after trial baseline, including: (1) quantitative data or statements about statistical testing reported in the Results section; or (2) data reported in figures or tables.

We developed a data abstraction form in the Systematic Review Data Repository (SRDR) (http://srdr.ahrq.gov/), an open online repository of systematic review data [15,16]. We pilot tested the form using 10 trials, and revised it as appropriate. The form included check-box items for pre-defined outcomes identified in our previous study [14], as well as free-text items where data abstractors could enter additional outcomes not previously identified. Two individuals (IJS and a Cochrane systematic review methodologist, Ms. Sueko Ng) independently extracted information about all reported outcomes. We resolved discrepancies through discussion.

Primary analysis: Overlap between outcomes in trials and reviews

We examined the frequency with which each outcome was named in trials and in reviews, separately for each of the four types of interventions. For each type of intervention, we adopted the following three approaches to examine overlap in outcomes:

1. We constructed Venn Diagrams for the number of outcomes that were named in both trials and reviews, and the numbers named uniquely in each.

2. To obtain greater granularity, we constructed Scatter Plots of the proportion of trials that reported and the proportion of reviews that named each outcome.

3. We examined the seven most frequent outcomes in the trials and in the reviews, and examined the overlap in those lists. We chose seven because Cochrane recommends including no more than seven outcomes in Summary of Findings tables in systematic reviews [12].

Analyses conducted post hoc

In order to make a subjective assessment of whether the most frequent outcomes in the trials and the reviews were qualitatively different, we conducted a post hoc analysis that examined: (1) whether each outcome was patient-centered, not patient-centered, or patient-centeredness was unclear; and (2) whether each outcome was an interim outcome, a long-term clinical outcome, or unclear. We did this separately for each type of intervention. We defined a patient-centered outcome as an outcome that patients notice and care about, e.g., symptoms, quality-of-life [17]. Examples of outcomes that are considered to be not patient-centered include serum haemoglobin, serum triglycerides, and HIV viral load. We defined a long-term clinical outcome as an outcome that denotes a clinically relevant response to an intervention, e.g., myocardial infarction and death. We defined an interim outcome, sometimes known as a surrogate outcome, as a measure of effect of an intervention that may correlate with a long-term clinical outcome, but the relationship between the interim and long-term clinical outcomes is not guaranteed, e.g., blood pressure and myocardial infarction. Two individuals (IJS and JO) independently classified each outcome.

We conducted all statistical analyses using STATA version 14 (College Station, TX, USA).

RESULTS

Characteristics of reviews

Among the 140 Cochrane reviews on HIV/AIDS published as of June 2013, 99 were completed, and 84 of these identified at least one trial (Figure 1). Most of the included (completed) reviews (72/84; 86%) were published between 2008 and 2012 (Table 1). Almost half (39/84; 46%) addressed clinical management (Table 1).

Figure 1.

Flow diagram of reviews and trials in this study

Table 1.

Characteristics of 84 eligible Cochrane reviews and 524 trials examined

Characteristic	Outcomes (N=855)		Trials (N=524)		Reviews (N=84)
	n	(%)	n	(%)	n	(%)
Year of publication*
1987 or earlier	5	(0)	1	(0)	0	(0)
1988–1992	160	(19)	37	(7)	0	(0)
1993–1997	156	(18)	73	(14)	0	(0)
1998–2002	241	(28)	164	(31)	0	(0)
2003–2007	163	(20)	174	(33)	0	(0)
2008–2012	120	(14)	75	(14)	72	(86)
2013	10	(0)	0	(0)	12	(14)
Type of intervention assessed+
Clinical management	238	(28)	202	(39)	39	(46)
Biomedical prevention	163	(19)	60	(12)	11	(13)
Behavioral prevention	467	(55)	170	(32)	17	(20)
Health services	256	(30)	92	(18)	17	(20)

^*For outcomes, the year of publication refers to the year in which the outcome was first named in a trial or review.

⁺For outcomes, the total across the four types of interventions adds up to more than 855 because outcomes were often named across more than one type of interventions.

Characteristics of trials

Overall, the 84 included reviews encompassed 553 trials (Figure 1). We excluded 29 trials reported only as abstracts (total=524 trials). Included reviews incorporated a median of 4 trials each (interquartile range [IQR] 2 to 8, range 1 to 44). Most of the 524 trials were identified either in reviews addressing clinical management (202/524; 39%) or behavioral prevention (170/524; 32%).

Characteristics of outcomes

We identified 855 total unique outcomes (i.e., outcome domains) in the trials and the reviews (Table 1). The trials and reviews each reported measuring similar numbers of outcomes, with a median of 8 outcomes per trial (IQR 5 to 12, range 1 to 32), and a median of 7.5 outcomes per review (IQR 4 to 11, range 1 to 29).

Overlap between outcomes in trials and reviews – All outcomes

While the number of outcomes per trial and per review was similar, the overlap between the outcomes measured in trials and reviews was limited, ranging from 16% to 29% (Figure 2). For each type of intervention, the trials included a considerably greater number of outcomes than the reviews, ranging from 2.3 times greater (clinical management interventions) to 5.4 times greater (behavioral prevention interventions).

Figure 2.

Overlap between outcomes in reviews and trials, by type of intervention

Scatter Plots of the proportion of trials versus the proportion of reviews that named each outcome (Figure 3) indicated that large numbers of outcomes clustered in the lower left quadrant of each plot. This indicates that, for each type of intervention, most outcomes were named in less than half of the trials and less than half of the reviews. The proportions of trial outcomes not named in any review ranged from 61% (biomedical prevention interventions) to 82% (behavioral prevention interventions) (Figure 2). Examples of these outcomes for the behavioral prevention interventions included “alcohol use/intoxication before/with sex with commercial sex workers”, “unprotected anal sex with sero-discordant partners”, and “number of male sexual partners”. The proportions of review outcomes not reported in any trial, on the other hand, were smaller, ranging from 2% (behavioral prevention interventions) to 14% (health services interventions) (Figure 2). Examples of these outcomes for clinical management interventions included “costs for patients”, “cognitive function”, “duration of diarrhea”, and “patient preferences”.

Fig 3. Scatter plot showing proportion of trials reporting and proportion of reviews naming each outcome, by type of intervention.

Each dot refers to one outcome.

Overlap between outcomes in trials and reviews — Seven most frequent outcomes

For each type of intervention, when we compared the seven most frequent outcomes in the trials and the seven most frequent outcomes in the reviews, we observed limited overlap and some noticeable differences (Table 2). “Adherence” was one of the seven most frequent trial outcomes for each type of intervention, but it was never one of the seven most frequent review outcomes. Similarly, for behavioral prevention interventions, “knowledge about HIV” and “unprotected anal sex” were frequent outcomes examined in the trials, but not in the reviews.

Table 2.

Comparison between seven most frequent outcomes in trials and reviews, by type of intervention

Type of intervention	Seven most frequent outcomes in trials		Seven most frequent outcomes in reviews		Overlapping outcomes between the seven most frequent outcomes in trials and reviews
	Outcome, sorted by frequency	% of trials	Outcome, sorted by frequency	% of reviews
Clinical management • 202 trials • 39 reviews	Adverse events (specified)	62	All-cause mortality	77	All-cause mortality
	All-cause mortality	51	Adverse events (unspecified)	49	CD4 count
	CD4 count	40	CD4 count	46	Adverse events (unspecified)
	Major/severe/serious adverse events	34	Quality-of-life	39	Major/severe/serious adverse events
	Adverse events (unspecified)	33	Major/severe/serious adverse events	39
	Viral load	27	AIDS-defining illness/event	23
	Adherence	26	Virological success/undetectable viral load	23
Biomedical prevention • 60 trials • 11 reviews	All-cause mortality	68	Acquisition/incidence of HIV	46	Acquisition/incidence of HIV
	Acquisition/incidence of HIV	62	Mother-to-child transmission of HIV	46	Mother-to-child transmission of HIV
	Adherence	52	Adverse events (unspecified)	36	Major/severe/serious adverse events
	Mother-to-child transmission of HIV	50	Major/severe/serious adverse events	27	Neonatal infant/mortality
	Pregnancy outcomes	42	Neonatal infant/mortality	27
	Major/severe/serious adverse events	37	Postpartum morbidity	27
	Neonatal infant/mortality	33	Patient intervention acceptability	18
Behavioral prevention • 170 trials • 17 reviews	Condom use (type unspecified)	37	Acquisition/incidence of HIV	41	Number of sexual partners
	Adherence	27	Acquisition/incidence of STIs	41
	Knowledge about HIV	24	Pregnancy	29
	Number of sexual partners	24	Condom use (female condoms)	29
	Unprotected sex (type unspecified)	20	Number of sexual partners	29
	Unprotected anal sex	19	Costs for patients	29
	Uptake of HIV testing	18	Patient intervention acceptability	29
Health services • 92 trials • 17 reviews	CD4 count	62	All-cause mortality	59	CD4 count
	Adherence	48	Quality-of-life	59	Viral load
	Viral load	37	CD4 count	53	All-cause mortality
	Weight	28	Viral load	53	Quality-of-life
	All-cause mortality	25	Progression to AIDS	4
	Quality-of-life	22	Hospitalization	29
	Serum hemoglobin	16	Beginning antiretroviral treatment	24

Color Legend:

Colorless: An outcome that we classified as neither a long-term clinical outcome nor a patient-centered outcome.

Yellow: An outcome that we classified as not a long-term clinical outcome, but a patient-centered outcome.

Green: An outcome that we classified as both a long-term clinical outcome and a patient-centered outcome.

Note: We did not classify any outcome as a long-term clinical outcome, but not a patient-centered outcome.

Table 2 also helps us examine whether there might be certain outcomes that are used frequently by both trialists and reviewers. For each type of intervention, the last column of Table 2 lists each outcome that was common to the two lists of seven most frequent outcomes (in trials and reviews). None of the common outcomes were reported in all trials and reviews, however.

A post hoc comparison of the nature of the most frequent outcomes provided signals for some possible qualitative differences between outcomes in trials and reviews (color coding in Table 2, numbers summarized in Table 3). The most frequent outcomes in the trials tended to be less frequently what we classified as long-term clinical outcomes (Table 3). This difference was more apparent for the behavioral prevention interventions (0/7 frequent outcomes in trials versus 2/7 frequent outcomes in reviews were what we classified as long-term clinical outcomes, e.g., “acquisition/incidence of HIV”) and health services interventions (2/7 frequent outcomes in trials versus 4/7 frequent outcomes in reviews were what we classified as long-term clinical outcomes).

Table 3.

Summary of outcome type comparing the seven most frequent outcomes in trials and reviews, by type of intervention

	Number of long-term clinical outcomes (Out of 7)		Number of patient-centered outcomes (Out of 7)
	In trials	In reviews	In trials	In reviews
Clinical management	4	5	4	5
Biomedical prevention	5	6	6	7
Behavioral prevention	0	2	0	5
Health services	2	4	3	4

For each type of intervention, the seven most frequent outcomes in the reviews tended to include at least one more patient-centered outcome than the seven most frequent outcomes in the trials (Table 3). Again, this difference was more apparent for behavioral prevention interventions (0/7 frequent outcomes in trials versus 5/7 frequent outcomes in reviews were what we classified as patient-centered outcomes). Further, “quality-of-life” was among the most frequent outcomes in the reviews for clinical management interventions (39%) and health services interventions (59%), but among the seven most frequent outcomes in the trials only for health services interventions (22%).

DISCUSSION

Summary of main results

We compared all outcomes in all completed Cochrane reviews of HIV/AIDS and the trials the reviews included. While the median number of outcomes per trial and per review was similar, for each type of intervention, trials included a considerably greater number of outcomes than the reviews. When examined using three different but related approaches, the overlap between trial and review outcomes was limited.

Large number of unique outcomes in HIV/AIDS

Our current study of trials and our previous study of reviews in HIV/AIDS [14] have documented a large variety of outcomes used in this field. Abundance of outcomes is not a problem in and of itself; disease conditions such as HIV/AIDS involve various body organs, giving rise to multifaceted clinical presentations that can be managed with numerous possible interventions. Consequently there is a range of outcomes relevant to the field. However, when the overlap in outcomes across the trials and reviews addressing similar interventions is insufficient to make meaningful comparisons across trials and reviews, it compromises the overall usefulness of those trials and reviews.

Need for core outcome sets in HIV/AIDS

In various fields, core outcome sets have been developed to reduce variation in outcome usage across trials [18]. A core outcome set is a minimum set of outcomes that should be measured and reported in all trials addressing a given condition [19]. A search of the Core Outcome Measures in Effectiveness Trials (COMET) database in October 2016 revealed the existence of only one core outcome set for HIV/AIDS, published in 2001, specifically for interventions aimed at preventing transmission of HIV from breastfeeding mothers to their infants [20]. That core outcome set includes three outcomes (domains), and two of them are common to the seven most frequent outcomes in the trials and reviews within our sample’s broader intervention type biomedical prevention: “neonatal/infant mortality” and “mother-to-child transmission of HIV”. The other outcome in the 2001 core outcome set was “prevalence of breastfeeding”, an outcome that was reported in 18% of trials and none of the reviews in our sample, hence not among the most frequent outcomes we identified. We agree with various other authors regarding the pressing need for core outcome sets for high-burden conditions, such as HIV/AIDS [2,3,18,21].

It is worth clarifying why we considered “adherence” to be an outcome. Adherence to prescribed interventions is important to assessing the effectiveness of interventions [22–26]. Study participants may stop adhering to an intervention because of concerns about its safety and/or ineffectiveness, both concerns arising as a consequence of the intervention [23].

Limited overlap between trial and review outcomes – Possible reasons

We observed a 3.6 times greater number of unique outcomes in the trials than in the reviews (779 vs. 218), with 16% to 29% overlapping. This overlap could represent core outcomes for each topic, so a small overlap is not necessarily bad. However, our analysis comparing the seven most frequent outcomes indicates that these are not core outcomes for either trials or reviews, and certainly not for both. None of the outcomes were in 100% of the trials and/or reviews. That said, the outcomes in the last two columns of Table 2, such as “CD4 count” (health services interventions) and “acquisition/incidence of HIV” (biomedical prevention interventions), could represent opportunities as starting points for core outcome sets for each of the four types of interventions we examined.

The Cochrane Review Group on HIV/AIDS was responsible for overseeing the reviews in our sample. In this group, review authors proposed their own list of outcomes. The editorial team and peer reviewers then suggested additions and deletions to the list. The findings of our current study do not imply that the final choices of outcomes by the review teams were based on outcomes reported in the trials.

An informal post hoc classification led us to wonder whether trials report fewer long-term clinical outcomes than reviews, which appeared to focus on more patient-centered outcomes. These findings suggest that trialists and reviewers may have different perspectives about outcomes that are important. Trialists and reviewers may also have different goals and constraints for their research, which in turn could affect the outcomes selected. For example, trials may more often address patient care directly, and might be constrained by considerations such as costs of measurement, time, and study power. Within the field of HIV/AIDS, Wittkop and colleagues have suggested that with improved survival among patients, long-term clinical outcomes such as death might be considered less useful for treatment trials conducted in the future [21]. For example, biomarkers and other interim outcomes might be used more often. In addition, where the primary purpose of the trial is regulatory approval, certain outcomes such as adherence become particularly useful [21]. Results from reviews, on the other hand, may more often directly inform evidence-based clinical practice guidelines and healthcare policy, and may be less constrained by considerations that trialists generally have [5,14]. Future research, such as qualitative studies, should explore this issue further.

The relative importance of outcomes to the community may evolve over time, and the timing of trials and reviews may reflect these changes. Related to the ideas of patient-centeredness of outcomes and the evolution of importance of outcomes over time, we had numerous discussions about whether certain interim outcomes might, in fact, be patient-centered. For example, viral load and CD4 count, which are laboratory values indicating the amount of virus present in the blood and the body’s immune response to the virus, respectively, are both interim outcomes (i.e., we believe that neither outcome directly impacts how a patient feels). That said, a well-informed patient who is highly motivated might consider either or both of these outcomes to be important in determining how they feel. We eventually decided, for this analysis, to classify these and other interim outcomes as not patient-centered. With increasing recognition of the importance of patient-involvement and patient-centeredness of healthcare and healthcare research, this issue bears further study.

Limited overlap between trial and review outcomes – Implications

In a 2013 survey, 73% of coordinating editors of Cochrane Review Groups believed that a core outcome set for trials would be useful in developing Summary of Findings tables for Cochrane reviews [27]. Those developing a core outcome set on any topic should consider how to identify candidate outcomes [14], and the process for deciding on the core outcome set (e.g., what consensus generation method should be adopted, who should be involved in the decision) [18,19]. Core outcome set developers should also pay careful attention to the appropriate scope of the core outcome set. Given the heterogeneous nature of HIV/AIDS disease, relevant considerations for core outcome set developers in this disease area include whether the core outcome set should apply to certain types of patients with HIV/AIDS, certain types of interventions, certain settings, or some combination of these factors.

We have shown that HIV/AIDS outcomes examined in trials are different from those in reviews, and both are rich sources. This finding begs the question of whether there should be separate core outcome sets for trials and for reviews. We believe that the answer should be ‘no’ because a key premise of evidence-based healthcare is that trials will be included eventually in reviews to inform clinical practice guidelines. Instead of separate core outcome sets, what is needed is better alignment of the outcomes measured in trials and reviews, so that results from more trials can appropriately be included in reviews and meta-analyses. This could reduce research waste and opportunities for selection bias. It is worth noting also that we concur with the philosophy underlying core outcome sets: that trialists and reviewers should be free to measure outcomes in addition to those in the core outcome set, such as outcomes that are specific to certain populations and/or social or cultural contexts [19].

An important implication of the findings from our study is that efforts to better align trial and review outcomes should take into account the different goals of trialists and reviewers. While those developing core outcome sets generally recognize that key stakeholder groups, such as consumers, health professionals, systematic reviewers, and clinical practice guideline developers, should be involved in the process [3], the likelihood that each group may bring different perspectives should also be considered. With so many stakeholders, some disease conditions may require core outcome sets of a larger size.

Limitations

There are some limitations to our study. First, we studied one disease condition (HIV/AIDS), and we do not know whether what we found can be applied to other conditions. But, HIV/AIDS is a high global burden condition with a large number of outcomes, interventions, trials, and reviews, thereby making it very suitable for achieving our study objective. Second, because we excluded trials that were only reported in conference abstracts, we know that, at the very least, we are missing outcomes information from the unpublished trials they represent [28,29]. Third, we restricted our analysis to outcomes reported in the primary publication of each trial. This may mean that outcomes reported only in non-primary publications were omitted. It is unlikely that this impacted our findings because we believe that if authors of a review were to consider a given trial’s outcomes when choosing review outcomes, the authors would focus on the publication they named as primary. Finally, we categorized outcome domains specifically, and this may have increased the number of outcomes identified compared with if we had categorized outcome domains broadly.

Conclusions

HIV/AIDS clinical trials reported a considerably greater number of unique outcomes than were named in HIV/AIDS Cochrane systematic reviews. Overlap in outcomes between the trials and the reviews appears limited. Core outcome set developers should be aware of these differences, and should incorporate both trialists’ and reviewers’ perspectives.

WHAT IS NEW?

Key findings

• We compared outcomes in Cochrane reviews of HIV/AIDS and the trials included in the reviews.

• Although the median number of outcomes per trial and per review was similar, the trials reported a considerably greater number of unique outcomes than the reviews (779 vs. 218), ranging from 2.3 times greater to 5.4 times greater.

• When examined using three different but related approaches, the overlap between trial and review outcomes was limited.

• Trials reported fewer long-term clinical outcomes than reviews, which appeared to focus on more patient-centered outcomes.

What this adds to what is known

• Outcomes in trials and reviews are not well aligned for appropriate inclusion of trial results in reviews and meta-analyses.

• Differences in perspectives, goals, and constraints between trialists and reviewers may explain differences in outcomes they consider important.

What is the implication, what should change now

• There is a pressing need for core outcome sets for high-burden conditions, such as HIV/AIDS.

• Trial and review outcomes should be better aligned. Efforts to better align trial and review outcomes should take into account the different goals of clinical trialists and systematic reviewers.