Abstract
Objective To determine whether financial ties to one drug company are associated with favourable results or conclusions in meta-analyses on antihypertensive drugs.
Design Retrospective cohort study.
Setting Meta-analyses published up to December 2004 that were not duplicates and evaluated the effects of antihypertensive drugs compared with any comparator on clinical end points in adults. Financial ties were categorised as one drug company compared with all others.
Main outcome measures The main outcomes were the results and conclusions of meta-analyses, with both outcomes separately categorised as being favourable or not favourable towards the study drug. We also collected data on characteristics of meta-analyses that the literature suggested might be associated with favourable results or conclusions.
Results 124 meta-analyses were included in the study, 49 (40%) of which had financial ties to one drug company. On univariate logistic regression analyses, meta-analyses of better methodological quality were more likely to have favourable results (odds ratio 1.16, 95% confidence interval 1.07 to 1.27). Although financial ties to one drug company were not associated with favourable results, such ties constituted the only characteristic significantly associated with favourable conclusions (4.09, 1.30 to 12.83). When controlling for other characteristics of meta-analyses in multiple logistic regression analyses, meta-analyses that had financial ties to one drug company remained more likely to report favourable conclusions (5.11, 1.54 to 16.92).
Conclusion Meta-analyses on antihypertensive drugs and with financial ties to one drug company are not associated with favourable results but are associated with favourable conclusions.
Introduction
A high and increasing proportion of biomedical researchers have financial ties to the pharmaceutical industry.1,2,3,4,5 Such researchers are more likely to publish articles—economic analyses, reviews, opinion pieces, and even randomised controlled trials—that support products produced by the industry.4 6,7,8,9,10,11,12 Editors and journals also have been criticised for having financial conflicts of interest that may favour drug companies.13
Meta-analyses pool data from multiple studies identified through a systematic review of the literature to provide summary statistics on the efficacy of a given treatment. Such meta-analyses represent the highest level of research evidence in the hierarchy of study types.14 They also may equal, if not surpass, randomised controlled trials in their cost effectiveness15 and in their influence on patient care and healthcare policy.16 17 Drug companies have started to reference meta-analyses in their advertisements.18
In the 1990s and early 2000sconcerns were expressed about the influence of the pharmaceutical industry on meta-analyses.19 20 Between 2003 and 2005 the Cochrane Collaboration debated whether its systematic reviews should be funded by drug companies; its current policy statement states that “The sponsorship of a Cochrane review by any commercial source or sources. . . is prohibited.”21 More recently a study compared matched pairs of Cochrane meta-analyses and industry sponsored meta-analyses published in print journals and found evidence that the industry sponsored meta-analyseswere more likely to recommend the experimental drug.22 The study was, however, unable to control for the possible confounding effects of the Cochrane methodology. In addition, the study examined only eight pairs of meta-analyses and so was unable to comment on the characteristics of meta-analyses not represented in its sample.
Some antihypertensive drugs have been shown to dramatically improve mortality and morbidity. The market for these and other antihypertensive drugs is highly competitive and lucrative. According to market research, both angiotensin receptor blockers and calcium channel blockers were in the top 10 list of global therapeutic drug classes by sales in 2005, equating to earnings of over $26b (£13b; €18b).23 Concern exists about the effect of such profits on doctors. The Wall Street Journal reported that animosity between the editor of the American Journal of Hypertension and the board of the American Society of Hypertension derived from charges of influence by drug companies on the society’s affairs.24
Our literature search found many published meta-analyses on antihypertensive drugs. If these are unbiased they have the potential to guide policy and save lives, but if biased they may do the opposite. We examinedwhether, after controlling for other important characteristics of meta-analyses, financial ties to one drug company were associated with favourable results or conclusions in meta-analyses on antihypertensive drugs. Our a priori hypothesis was that financial ties to one drug company would be associated with favourable results and conclusions.
Methods
We included meta-analyses published up to December 2004 that evaluated the effects of antihypertensive drugs on clinical outcomes in adults. The comparison group could include placebo, no treatment, usual care, or active therapy. We defined meta-analyses as systematic reviews that quantitatively combined data from at least two studies. We excluded meta-analyses on pregnant women and children because the mechanisms of hypertension in these populations differ from those in adults with chronically elevated blood pressure.
We also excluded meta-analyses that were duplicates or that overlapped considerably with one another. Duplicate meta-analyses were those that shared at least one author and evaluated the same trials and primary outcome measures. Overlapping meta-analyses were those that did not qualify as duplicate meta-analyses but shared at least one author, at least one trial, and the same topic (for example, the update of a pre-existing meta-analysis with new trial data). From each group of duplicate or overlapping meta-analyses we identified a representative meta-analysis to be included in the study, which was the meta-analysis that was published first. If two meta-analyses were published simultaneously, we randomly selected one for inclusion by rolling a dice.
Search strategy
We identified meta-analyses, without language restriction, by searching PubMed and the Cochrane Database of Systematic Reviews and by hand searching the reference lists of identified meta-analyses. A description of the search terms is available on bmj.com. One of us (VY) reviewed the titles and abstracts of all potential meta-analyses for inclusion. If the title was uninformative and no abstract was available, VY retrieved and reviewed the full text of the article to determine its eligibility.
Definition of financial ties
Financial ties were categorised as one drug company compared with all others. Information on financial ties was obtained from three sources: disclosures in the meta-analysis itself—sources of funding for the study or authors, or author affiliations; disclosures of industry or other sponsorship in the journal supplement in which a meta-analysis was published; and disclosures of financial ties in previous research articles on antihypertensive drugs by the first author of the included meta-analysis, arbitrarily going back three years before the publication date of the referent meta-analysis. Meta-analyses with financial ties to one drug company as disclosed in any one of the three sources, were defined as having financial ties to one drug company. We designed this definition of financial ties to be conservative. For example, a meta-analysis that we classified as having financial ties to one drug company on the basis of publication in an industry sponsored supplement could also have had financial ties to academia on the basis of the funding source of the meta-analysis. Academic financial ties might be expected to dilute the potential influence of industry, thus assuring that our findings would be conservative rather than inflated.
We carried out sensitivity analyses using different definitions of financial ties—using information disclosed only in the meta-analysis, or using information disclosed in the meta-analysis and in the supplement in which it was published. This shifted meta-analyses from the category of financial ties to one drug company to the category of all other financial ties.
We collected additional data on the financial ties of meta-analyses in the all other category. The subcategories for this category were defined as having financial ties to multiple drug companies; non-profit (academic, government, foundation, or professional) groups; any drug company (single or multiple) and non-profit; and no statement. We originally had two other subcategories of “no funding” and “only other,” but these contained only one meta-analysis each, so we combined the data with the no statement and non-profit subcategories, respectively. We collected these data with the a priori hypothesis that, even within the all other category, graded differences would exist between meta-analyses in the degree to which their results or conclusions were favourable towards the study drug, depending on their subcategory of financial tie.
Definition of the study drug and outcome measure for each meta-analysis
The study drug and outcome measure were defined by the authors of the meta-analyses or if left unspecified we defined them as the first treatment and outcome described in the results. The study drug could be single or combined therapy. If multiple primary outcome measures were explicitly identified, we deemed results or conclusions to be favourable if at least 50% of the outcome measures were favourable.
Primary outcome measures for this study
The primary outcome measures for this study were results, as determined by us, and conclusions, as stated by the authors of the meta-analyses. Our per protocol analyses were prespecified to use dichotomous coding of the results and conclusions as being favourable towards the study drug compared with not favourable. We collected additional data on subcategories of the not favourable group.
Results were coded from 1-5, with 1 being statistically in favour of the study drug, 2 being statistically against the study drug, 3 being statistically neither in favour nor against the study drug, 4 being unclear, and 5 being other. In accordance with our protocol we considered results coded as 1 to be favourable towards the study drug and those coded as 2-5 to be not favourable. We believe this coding is the most widely used for non-equivalency studies.
Conclusions were coded from 1-5, with 1 being in favour of the study drug, 2 being against the study drug, 3 being neutral towards the study drug, 4 being unclear, and 5 being other. In accordance with our protocol we considered conclusions coded as 1 to be favourable towards the study drug and those coded as 2-5 to be not favourable.
Other potentially relevant variables
We wanted to determine whether certain financial ties were associated with skewed results or conclusions, even after controlling for other variables. We therefore collected data on other characteristics of meta-analyses that our literature review suggested might be associated with favourable results or conclusions.
Better methodological quality of meta-analyses has not been consistently associated with favourable conclusions.25 26 We evaluated the methodological quality of each meta-analysis using a modified version of the Oxman and Guyatt quality instrument,27 which rates systematic reviews and meta-analyses on whether they include design features aimed at reducing bias, and assigns a summary score to each meta-analysis. The quality instrument has nine questions: Did the authors clearly describe their strategy for identifying primary research studies on the meta-analysis topic? Was the search strategy appropriate? Did the authors clearly report their criteria for deciding which studies to include and exclude? Were the inclusion and exclusion criteria appropriate? Did the authors clearly report their criteria for assessing the quality or validity of studies included? Was the validity assessment appropriate? Did the authors clearly report their strategy for quantitatively combining study results? Were study results combined appropriately? Were the stated conclusions supported by the data presented? The only feature evaluated by Oxman and Guyatt but not evaluated by our quality instrument was the overall scientific quality of the overview, which we thought would be redundant with the quality score. For each design feature the meta-analysis could receive a maximum of two points for fulfilling the criterion, one point for partially fulfilling it, or zero points for not fulfilling it. The quality score was the sum of these points, with the total possible being 18. We also evaluated whether the quality scores correlated significantly with any of the other characteristics of the meta-analyses.
Some evidence suggests that supplements from symposiums, especially those sponsored by drug companies, contain articles that are biased and of poor quality.28 29 We determined whether the meta-analyses were published in journal supplements.
We also collected data on additional characteristics: whether the meta-analyses involved literature searches or included studies in languages other than English, as well as in English, compared with in English only30 31; included a description of the process of data abstraction compared with no description32,33,34; included studies that were not randomised controlled trials compared with included only randomised controlled trials35; included unpublished plus published studies compared with only published studies36; included only studies that used placebo groups as the control group compared with studies that used other comparator groups (for example, no treatment, usual care, active drug control)37; focused only on newer classes of drugs compared with older classes (newer drugs being, for example, angiotensin converting enzyme inhibitors, angiotensin II receptor blockers, or α blockers compared with older drugs, defined as β blockers and diuretics)9 38; used surrogate outcomes only (for example, blood pressure, level of creatinine, levels of lipids) compared with morbidity and mortality outcomes (for example, myocardial infarction, dependence on dialysis, death)37 39; used composite outcomes only (for example, myocardial infarction, stroke, and death) compared with distinct outcomes (for example, total mortality)40; carried out evaluations of heterogeneity of included studies compared with no evaluations41; and carried out sensitivity analyses of the results compared with no sensitivity analyses.42
Data extraction
We pretested our data extraction tool and quality instrument during a pilot study. This pilot study showed good intercoder reliability between the three reviewers in data extraction and quality assessment, despite one reviewer being unblinded and the other two being blinded to information on financial ties, as well as to author identity. One researcher (VY) unblinded to financial ties and author identity coded data collection items for the meta-analyses in our main study. A second coder (LAB), blinded to financial ties (according to our study definition) and author identity (name, affiliation, and address), coded a random sample of 24 (19%) of the meta-analyses in our main study. The degree of agreement between the two reviewers’ evaluations were κ=0.74 (substantial) for results and κ=0.60 (moderate) for conclusions.43
Statistical analyses
We used univariate logistic regression analyses to evaluate whether financial ties or other characteristics of meta-analyses were associated with favourable results or conclusions. Variables that were found to be significant to the level of P<0.05 on univariate analyses were then entered into exploratory multiple logistic regression models. All analyses were carried out using SAS version 9.1.
Results
The combined search strategies identified 691 potentially relevant meta-analyses on antihypertensive drugs in adults. Most were ineligible and were excluded (figure), many after review of the title or abstract. Overall, 291 articles were read in full. Of the 169 meta-analyses that met all other eligibility criteria, 45 were excluded for being duplicates (n=20) or for overlapping with other meta-analyses (n=25). In total, 124 meta-analyses met our inclusion criteria, including two in Spanish, one in German, and the remainder in English. Table 1 shows the coding for the results, conclusions, quality scores, and financial ties of these meta-analyses.
Searches for meta-analyses and reasons for exclusion
Table 1.
Meta-analyses included in study and their respective coding, by subcategory, for financial ties, quality score, results, and conclusions
| Web reference No | Financial ties* | Quality score† | Results | Conclusions |
|---|---|---|---|---|
| w1 | No statement | 1 | Neither in favour nor against | Neutral |
| w2 | One company | 2 | Unclear | In favour of study drug |
| w3 | No statement | 0 | Neither in favour nor against | In favour of study drug |
| w4 | No statement | 10 | In favour of study drug | In favour of study drug |
| w5 | No statement | 10 | Neither in favour nor against | Neutral |
| w6 | Non-profit | 6 | In favour of study drug | In favour of study drug |
| w7 | One company | 8 | In favour of study drug | In favour of study drug |
| w8 | No statement | 16 | In favour of study drug | In favour of study drug |
| w9 | One company | 3 | In favour of study drug | In favour of study drug |
| w10 | One company | 6 | Neither in favour nor against | In favour of study drug |
| w11 | Non-profit | 1 | In favour of study drug | In favour of study drug |
| w12 | Non-profit | 7 | Other | Neutral |
| w13 | No statement | 9 | In favour of study drug | In favour of study drug |
| w14 | Multiple companies | 16 | In favour of study drug | In favour of study drug |
| w15 | One company | 3 | In favour of study drug | In favour of study drug |
| w16 | One company, non-profit | 10 | Against study drug | Against study drug |
| w17 | Non-profit | 12 | In favour of study drug | In favour of study drug |
| w18 | One company | 6 | Unclear | In favour of study drug |
| w19 | One company | 0 | Unclear | In favour of study drug |
| w20 | One company | 4 | In favour of study drug | In favour of study drug |
| w21 | One company | 7 | In favour of study drug | In favour of study drug |
| w22 | One company | 5 | In favour of study drug | In favour of study drug |
| w23 | Non-profit | 4 | Neither in favour nor against | In favour of study drug |
| w24 | No statement | 10 | In favour of study drug | In favour of study drug |
| w25 | One company | 5 | In favour of study drug | In favour of study drug |
| w26 | Non-profit | 10 | In favour of study drug | In favour of study drug |
| w27 | One company | 3 | Unclear | In favour of study drug |
| w28 | One company | 1 | In favour of study drug | In favour of study drug |
| w29 | One company | 5 | Unclear | In favour of study drug |
| w30 | One company | 2 | Neither in favour nor against | In favour of study drug |
| w31 | Multiple companies, non-profit | 1 | Unclear | Neutral |
| w32 | One company | 4 | Against study drug | Against study drug |
| w33 | One company | 13 | In favour of study drug | In favour of study drug |
| w34 | Non-profit | 16 | In favour of study drug | In favour of study drug |
| w35 | One company | 0 | In favour of study drug | In favour of study drug |
| w36 | No statement | 6 | Neither in favour nor against | In favour of study drug |
| w37 | Non-profit | 8 | In favour of study drug | In favour of study drug |
| w38 | Non-profit | 9 | In favour of study drug | In favour of study drug |
| w39 | No statement | 7 | In favour of study drug | In favour of study drug |
| w40 | One company | 1 | In favour of study drug | In favour of study drug |
| w41 | One company | 9 | In favour of study drug | In favour of study drug |
| w42 | Non-profit | 10 | In favour of study drug | In favour of study drug |
| w43 | No statement | 1 | Neither in favour nor against | Neutral |
| w44 | No statement | 10 | Neither in favour nor against | Neutral |
| w45 | One company | 8 | Unclear | In favour of study drug |
| w46 | Non-profit | 10 | Unclear | Neutral |
| w47 | Other only‡ | 9 | In favour of study drug | In favour of study drug |
| w48 | Multiple companies | 4 | Against study drug | Against study drug |
| w49 | Non-profit | 9 | Neither in favour nor against | Neutral |
| w50 | One company | 11 | In favour of study drug | In favour of study drug |
| w51 | Non-profit | 14 | In favour of study drug | In favour of study drug |
| w52 | One company, non-profit | 7 | In favour of study drug | In favour of study drug |
| w53 | Non-profit | 5 | In favour of study drug | In favour of study drug |
| w54 | Non-profit | 12 | In favour of study drug | In favour of study drug |
| w55 | No statement | 5 | In favour of study drug | In favour of study drug |
| w56 | No statement | 15 | Other | Other |
| w57 | One company | 1 | Unclear | In favour of study drug |
| w58 | One company | 5 | Neither in favour nor against | In favour of study drug |
| w59 | One company | 0 | In favour of study drug | In favour of study drug |
| w60 | Non-profit | 8 | In favour of study drug | In favour of study drug |
| w61 | No statement | 10 | In favour of study drug | In favour of study drug |
| w62 | Non-profit | 9 | In favour of study drug | In favour of study drug |
| w63 | No statement | 13 | In favour of study drug | In favour of study drug |
| w64 | One company | 5 | In favour of study drug | In favour of study drug |
| w65 | No statement | 7 | In favour of study drug | In favour of study drug |
| w66 | Non-profit | 0 | In favour of study drug | Unclear |
| w67 | One company | 7 | In favour of study drug | In favour of study drug |
| w68 | One company | 8 | In favour of study drug | In favour of study drug |
| w69 | Non-profit | 2 | In favour of study drug | In favour of study drug |
| w70 | Non-profit | 7 | In favour of study drug | In favour of study drug |
| w71 | Multiple companies | 5 | In favour of study drug | In favour of study drug |
| w72 | Multiple companies | 0 | Neither in favour nor against | In favour of study drug |
| w73 | One company | 0 | Neither in favour nor against | In favour of study drug |
| w74 | Multiple companies | 11 | In favour of study drug | In favour of study drug |
| w75 | One company | 3 | Neither in favour nor against | In favour of study drug |
| w76 | None§ | 17 | In favour of study drug | In favour of study drug |
| w77 | Multiple companies, non-profit | 10 | In favour of study drug | In favour of study drug |
| w78 | One company | 0 | Neither in favour nor against | In favour of study drug |
| w79 | No statement | 13 | In favour of study drug | In favour of study drug |
| w80 | Non-profit | 12 | In favour of study drug | In favour of study drug |
| w81 | One company | 3 | In favour of study drug | In favour of study drug |
| w82 | One company | 9 | Neither in favour nor against | In favour of study drug |
| w83 | Multiple companies | 14 | In favour of study drug | In favour of study drug |
| w84 | Multiple companies | 13 | Against study drug | Against study drug |
| w85 | Multiple companies | 12 | In favour of study drug | In favour of study drug |
| w86 | Non-profit | 2 | Unclear | Neutral |
| w87 | No statement | 7 | Against study drug | Against study drug |
| w88 | One company | 11 | In favour of study drug | In favour of study drug |
| w89 | One company | 1 | Unclear | In favour of study drug |
| w90 | Multiple companies | 1 | In favour of study drug | In favour of study drug |
| w91 | No statement | 2 | Neither in favour nor against | Neutral |
| w92 | Multiple companies, non-profit | 13 | In favour of study drug | In favour of study drug |
| w93 | One company | 3 | In favour of study drug | In favour of study drug |
| w94 | No statement | 1 | Neither in favour nor against | In favour of study drug |
| w95 | Multiple companies, non-profit | 14 | In favour of study drug | In favour of study drug |
| w96 | Multiple companies | 0 | Unclear | Against study drug |
| w97 | One company | 16 | Neither in favour nor against | Neutral |
| w98 | One company | 17 | In favour of study drug | In favour of study drug |
| w99 | One company | 7 | In favour of study drug | In favour of study drug |
| w100 | Non-profit | 5 | In favour of study drug | In favour of study drug |
| w101 | One company | 14 | In favour of study drug | In favour of study drug |
| w102 | Non-profit | 9 | In favour of study drug | In favour of study drug |
| w103 | One company | 3 | In favour of study drug | In favour of study drug |
| w104 | One company | 3 | Neither in favour nor against | In favour of study drug |
| w105 | No statement | 14 | Neither in favour nor against | Neutral |
| w106 | No statement | 9 | In favour of study drug | In favour of study drug |
| w107 | Multiple companies | 0 | Unclear | In favour of study drug |
| w108 | One company | 2 | In favour of study drug | In favour of study drug |
| w109 | No statement | 6 | In favour of study drug | In favour of study drug |
| w110 | Multiple companies | 2 | In favour of study drug | In favour of study drug |
| w111 | One company | 0 | Unclear | Unclear |
| w112 | One company | 1 | Neither in favour nor against | Neutral |
| w113 | Multiple companies | 4 | In favour of study drug | In favour of study drug |
| w114 | One company, non-profit | 8 | Neither in favour nor against | Neutral |
| w115 | One company, non-profit | 2 | In favour of study drug | In favour of study drug |
| w116 | Multiple companies | 12 | Neither in favour nor against | In favour of study drug |
| w117 | One company | 2 | Unclear | In favour of study drug |
| w118 | Multiple companies, non-profit | 10 | In favour of study drug | In favour of study drug |
| w119 | Non-profit | 6 | Unclear | Neutral |
| w120 | One company | 0 | Other | In favour of study drug |
| w121 | No statement | 7 | In favour of study drug | In favour of study drug |
| w122 | Non-profit | 16 | In favour of study drug | In favour of study drug |
| w123 | One company | 1 | In favour of study drug | In favour of study drug |
| w124 | One company | 10 | In favour of study drug | In favour of study drug |
*Subcategories of “one company, non-profit” and “multiple companies, non-profit” were combined for purposes of calculating descriptive statistics in table 5, as together they only represent nine meta-analyses.
†Minimum score=0, maximum score=18.
‡One meta-analysis in this predefined subcategory (w47) was included in “non-profit” subcategory for purposes of calculating descriptive statistics in table 5.
§One meta-analysis in this predefined subcategory (w76) was included in “no statement” subcategory for purposes of calculating descriptive statistics in table 5.
Characteristics of included meta-analyses
The included meta-analyses were published from 1983 to 2004, with 50% published after 1996. Table 2 summarises the other characteristics of the meta-analyses. A substantial portion (49 of 124, 40%) had financial ties to one drug company. Of these, 9 (18%) disclosed such funding in the meta-analysis, 5 (10%) in the sponsored supplement, and 12 (24%) in the authors’ previous publications, whereas the rest had a mixture of disclosures. Of the 75 (60%) meta-analyses without financial ties to one drug company, the financial ties were diverse, subcategorised as multiple drug companies in 14 (19%), non-profit in 27 (36%), drug and non-profit in 9 (12%), and no statement in 25 (33%).
Table 2.
Characteristics of included meta-analyses. Values are number (percentage) of meta-analyses unless stated otherwise
| Characteristic | Meta-analyses (n=124) |
|---|---|
| Financial ties with one drug company | 49 (40) |
| Mean (range), median quality score* | 6.66 (0-17), 7 |
| Published in journal supplement | 21 (17) |
| Searched or included non-English literature | 11 (9) |
| Described process of data abstraction | 27 (22) |
| Included non-randomised controlled trials | 38 (31) |
| Included unpublished studies | 5 (4) |
| Included studies that used only placebo group as control group | 27 (22) |
| Focused on newer class of drug | 51 (41) |
| Used surrogate outcomes only | 51 (41) |
| Used composite outcomes only | 3 (2) |
| Carried out evaluations of heterogeneity | 62 (50) |
| Carried out sensitivity analyses | 70 (56) |
*Score was continuous variable from 0-18, with higher scores indicating better quality and lower scores indicating worse quality.
Financial ties or other meta-analysis characteristics and favourable results or conclusions
Univariate logistic regression analyses
Only meta-analyses with better quality and those that evaluated the heterogeneity of included studies or carried out sensitivity analyses were significantly more likely to have favourable results (table 3). Meta-analyses with financial ties to one drug company were not more likely than others to have favourable results but were more likely to have favourable conclusions.
Table 3.
Univariate analyses of associations between favourable results or conclusions and characteristics of meta-analysis
| Characteristic | Favourable results | Favourable conclusions | |||
|---|---|---|---|---|---|
| Odds ratio (95% CI) | P value | Odds ratio (95% CI) | P value | ||
| Financial ties with one drug company | 0.65 (0.31 to 1.36) | 0.25 | 4.09 (1.30 to 12.83) | 0.016 | |
| Better quality* | 1.16 (1.07 to 1.27) | <0.001 | 1.02 (0.93 to 1.12) | 0.64 | |
| Published in journal supplement | 0.40 (0.16 to 1.05) | 0.062 | 2.58 (0.56 to 11.93) | 0.23 | |
| Searched or included non-English literature | 0.74 (0.21 to 2.56) | 0.63 | 0.61 (0.15 to 2.49) | 0.49 | |
| Described process of data abstraction | 2.09 (0.81 to 5.41) | 0.13 | 1.07 (0.36 to 3.20) | 0.90 | |
| Included non-randomised controlled trials | 0.51 (0.23 to 1.11) | 0.088 | 1.09 (0.41 to 2.90) | 0.86 | |
| Included unpublished studies | 2.61 (0.28 to 24.09) | 0.40 | † | † | |
| Included studies that used only placebo group as control group | 1.67 (0.67 to 4.18) | 0.28 | 3.67 (0.81 to 16.71) | 0.093 | |
| Focused on newer class of drug | 0.48 (0.23 to 1.00) | 0.050 | 1.91 (0.73 to 5.01) | 0.19 | |
| Used surrogate outcomes only | 0.48 (0.23 to 1.00) | 0.050 | 0.97 (0.39 to 2.40) | 0.95 | |
| Used composite outcomes only | 0.31 (0.03 to 3.48) | 0.34 | 0.47 (0.04 to 5.40) | 0.54 | |
| Carried out evaluations of heterogeneity | 2.65 (1.25 to 5.59) | 0.011 | 1.00 (0.41 to 2.44) | 1.00 | |
| Carried out sensitivity analyses | 3.61 (1.69 to 7.71) | <0.001 | 1.12 (0.46 to 2.75) | 0.80 | |
Odds ratios >1 indicate that characteristic is associated with more favourable results or conclusions, whereas odds ratios <1 indicates that characteristic is associated with less favourable results or conclusions.
*Quality score was continuous variable from 0-18, with higher scores indicating better quality and lower scores indicating worse quality.
†Model was not possible because observations at one level of the predictor were all same—that is, all meta-analyses that included unpublished studies had favourable conclusions.
Multiple logistic regression analyses
Those variables found to be significant on univariate analyses (financial ties to one drug company, better quality, evaluated heterogeneity, and carried out sensitivity analyses) were to have been included in multiple logistic regression analyses. But the variables of better quality, evaluated heterogeneity, and carried out sensitivity analyses were found to be significantly associated (P<0.001 for all pairwise comparisons by the Fisher exact test and the Mann-Whitney rank sum test). We therefore identified which of the three was the strongest predictor—better quality—and used only this variable in the final model. This meant that the final model contained only two variables: financial ties to one drug company and better quality.
Meta-analyses that had financial ties to one drug company were not associated with favourable results but remained significantly associated with favourable conclusions, even when controlling for the quality of the meta-analyses (table 4). Meta-analyses of better quality remained associated with favourable results.
Table 4.
Final model: multivariate analyses of associations between favourable results or conclusions and characteristics of meta-analysis
| Meta-analysis characteristic | Favourable results | Favourable conclusions | |||
|---|---|---|---|---|---|
| Odds ratio (95% CI) | P value | Odds ratio (95% CI) | P value | ||
| Financial ties with one drug company | 0.99 (0.44 to 2.23) | 0.99 | 5.11 (1.54 to 16.92) | 0.008 | |
| Better quality* | 1.16 (1.06 to 1.27) | 0.001 | 1.07 (0.97 to 1.19) | 0.19 | |
*Quality score was continuous variable from 0-18, with higher scores indicating better quality and lower scores indicating worse quality.
To test the robustness of our final model we carried out multiple sensitivity analyses. We ran two models that used alternative definitions of financial ties. Another model included the other two variables that were significantly associated with better quality (evaluated heterogeneity and carried out sensitivity analyses). An additional model included those variables with P values from 0.05-0.10 (was published in a journal supplement, included studies of non-randomised controlled trials, focused on a newer class of drug, and used surrogate outcomes only). All of these models had similar findings to those reported for the final model. Finally, financial ties to one drug company were uniformly associated with favourable conclusions, regardless of the comparator subcategory of financial ties (for example, multiple drug, non-profit, drug and non-profit, and no statement).
Type of financial tie and concordance between results and conclusions
Meta-analyses that had financial ties to one drug company had the worst concordance between results and conclusions, with 27 of 49 (55%) having favourable results but 45 of 49 (92%) having favourable conclusions (table 5). In contrast, meta-analyses with financial ties to two of the subcategories of the “all other” category of financial ties—the non-profit and both drug and non-profit subcategories—had excellent concordance between favourable results and conclusions. This finding was not altered by sensitivity analyses using different definitions of financial ties.
Table 5.
Proportion of meta-analyses with favourable results or conclusions, and proportion with poor concordance between results and conclusions, by financial ties*
| Financial ties | No (%) with favourable results | No (%) with favourable conclusions | No (%) with poor concordance between results and conclusions* |
|---|---|---|---|
| One drug company (n=49) | 27 (55) | 45 (92) | 18 (37) |
| All other (n=75): | 49 (65) | 55 (73) | 6 (8) |
| Multiple drug companies (n=14) | 8 (57) | 11 (79) | 3 (21) |
| No statement (n=25) | 14 (56) | 17 (68) | 3 (12) |
| Both drug and non-profit (n=9) | 6 (67) | 6 (67) | 0 (0) |
| Non-profit (n=27) | 21 (78) | 21 (78) | 0 (0) |
*Poor concordance for each row was determined by the calculation: [number of meta-analyses with favourable conclusions]-[number of meta-analyses with favourable results].
Discussion
Meta-analyses with favourable conclusions, but not results, were more likely to have financial ties to one drug company than other ties, even when controlling for other characteristics of meta-analyses. These findings suggest discordance between the data that underlie the results and the interpretation, or “spin,” of these data that constitute the conclusions. In contrast, meta-analyses with financial ties to non-profit groups had excellent concordance between results and conclusions.
Because we used conservative assumptions in defining financial ties, the odds ratio for our main finding is likely to be an underestimate of the true relation between financial ties to one drug company and favourable conclusions.
Findings in the context of previously published work
We were not able to find any studies of financial ties and the results of meta-analyses that had statistically significant findings. In 1987 the authors of a study noted variability in their conclusions, despite similarity in results, but could not explain these differences by looking at the inclusion criteria or statistical methods of the meta-analyses.44 Our findings of an association between financial ties to one drug company and favourable conclusions might explain their observations. They also reinforce the findings of another study,22 but for a large cohort of meta-analyses published in the print literature and with adjustment for confounding by statistical methods rather than by matching. A similar study was carried out on passive smoking, but evaluated systematic reviews rather than meta-analyses.10 The findings that the conclusions of review articles were associated with authors’ affiliations with the tobacco industry also parallel our findings.
We identified no association between meta-analyses of better quality and conclusions. In contrast, one study found that reviews of better quality on spinal manipulation were more likely to have favourable conclusions,25 whereas another study found that meta-analyses of better quality on analgesics were less likely to have favourable conclusions.26 One cause of these discrepancies may be that neither study controlled for funding source.
Limitations, unanswered questions, and future research
Our study design has potential for confounding. By collecting data on characteristics of the meta-analyses suggested by the literature to be potential confounders of results or conclusions, we were able to adjust for confounding. Few potential confounders were found to be significant on univariate or multivariate analyses.
Another methodological limitation of our study is that only one of us (VY) reviewed the meta-analyses, both for inclusion in the study and for data extraction and quality assessment. This same reviewer was not blinded to important characteristics of the meta-analyses, including financial ties. It could be said that this method of evaluation introduces the potential for bias. However, evidence from our own work and the work of others suggests that blinded data extraction does not make a clinically or statistically significant difference in study outcome and that blinded quality assessments may yield both higher and lower scores.10 32,33,34 45 Furthermore, the Cochrane Collaboration handbook states that “A section is being prepared on the issue of whether data extraction should be done blinded; for example to the authors and journal and to the results when assessing quality. Although there is some evidence that blinded assessments of the quality of trials may be more reliable and different from assessments that are not blinded (Jadad 1996, Moher 1998b), blinding is difficult to achieve, time consuming and may not substantially alter the results of a review (Berlin 1997a, Berlin 1997b).”46 Our pilot study showed good intercoder reliability between the three reviewers in data coding, despite one reviewer (VY) being unblinded to information on financial ties and author identity. In our main study we found good intercoder reliability between the unblinded reviewer (VY) and a blinded reviewer (LAB) who evaluated a randomly selected subset of meta-analyses.
We did not confirm disclosure of financial ties by other means, such as examining the authors’ grant applications or investment profiles. Sensitivity analyses of the primary outcomes using different definitions of financial ties were consistent, however, which suggests that our findings are robust.
Our definition of financial ties was conservative in that a meta-analysis was classified as having financial ties to one drug company if it had such ties on the basis of three sources. Our definition for financial ties was arbitrary for the component based on first authors’ previous articles on antihypertensive drugs. We chose to focus on first authors and their articles on antihypertensive drugs going back only three years because we hypothesised that these limitations would capture financial ties with the most immediacy, relevance, and potential to influence the meta-analyses of interest. We did not test this hypothesis. Yet the sensitivity analyses with this component of the definition for financial ties excluded had comparable findings to our primary analyses.
The generalisability of our study is limited by its restriction to one clinical topic. Our findings have considerable relevance to the real world, however, as the marketing of antihypertensive drugs constitutes a multibillion dollar a year industry, and antihypertensives are some of the most prescribed drug classes in the world.23
Conclusions
That we found poor concordance between results and conclusions in some meta-analyses of antihypertensive drugs suggests that meta-analyses, as with other study types, are open to the influence of systematic bias, in this case by having financial ties to one drug company. Our study also exposes a failure of peer review. Both editors and peer reviewers must have read manuscript versions of those meta-analyses containing discordant results and conclusions, yet they did not prevent publication of biased conclusions. Editors and peer reviewers, as well as policymakers, meta-analysts, and readers should closely scrutinise the conclusions of meta-analyses to ensure that they are supported by the data.
What is already known on this topic
The results and conclusions of randomised controlled trials with financial ties to one drug company are more likely to favour the sponsor’s products
A study that compared Cochrane meta-analyses to industry supported meta-analyses in print journals suggests that the same holds true for meta-analyses
What this study adds
Meta-analyses with financial ties to one drug company are no more likely than others to have results that favour the company’s drugs but are more likely to have favourable conclusions
Editors and peer reviewers failed to prevent publication of biased conclusions in meta-analyses
Supplementary Material
We thank Alan Bostrom (University of California, San Francisco), the biostatistician for the study, who carried out the statistical analyses and provided input on the interpretation of these analyses.
Contributors: VY refined the idea for the study; designed and coordinated the pilot study, and collected, analysed, and interpreted its data; designed all parts of the current study; coordinated the study; carried out the literature and manual searches; evaluated all potential meta-analyses for inclusion; collected data from all included meta-analyses; analysed the data along with the biostatistician; interpreted the data; and wrote the paper. She is guarantor for the paper. DR had the initial idea for the study; collected data in the pilot study; provided input on the design, data collection, analyses, and interpretation of the study; and reviewed and provided substantive feedback on the paper. LAB provided input on the design, data collection, analyses, and interpretation of the study; gave special input on the data extraction tool; collected data for the pilot study and a subset of meta-analyses in the current study; and reviewed and provided substantive feedback on the paper.
Funding: The study was funded in part by the Eugene Garfield Foundation. VY received support from a dean’s quarterly research grant (University of California, San Francisco) and from the internal medicine residency program (University of Washington, Seattle). LAB receives support from the California tobacco related disease research program (grant No 13RT-0108H).
Competing interests: None declared.
Ethical approval: Not required.
References
- 1.Krimsky S, Rothenberg L, Stott P, Kyle G. Financial interests of authors in scientific journals: a pilot study of 14 publications. Sci Eng Ethics 1996;2:395-410. [DOI] [PubMed] [Google Scholar]
- 2.Blumenthal D, Campbell E, Causino N, Louis K. Participation of life-science faculty in research relationships with industry. N Engl J Med 1996;335:1734-9. [DOI] [PubMed] [Google Scholar]
- 3.Campbell E, Louis K, Blumenthal D. Looking a gift horse in the mouth: corporate gifts supporting life sciences research. JAMA 1998;279:995-9. [DOI] [PubMed] [Google Scholar]
- 4.Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 2003;289:454-65. [DOI] [PubMed] [Google Scholar]
- 5.Boyd E, Bero L. Assessing faculty financial relationships with industry: a case study. JAMA 2000;284:2209-14. [DOI] [PubMed] [Google Scholar]
- 6.Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003;326:1167-70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Friedberg M, Saffron B, Stinson T, Nelson W, Bennett C. Evaluation of conflict of interest in economic analyses of new drugs used in oncology. JAMA 1999;282:1453-7. [DOI] [PubMed] [Google Scholar]
- 8.Cho M, Bero L. The quality of drug studies published in symposia proceedings. Ann Intern Med 1996;124:485-9. [DOI] [PubMed] [Google Scholar]
- 9.Stelfox H, Chua G, O’Rourke K, Detsky A. Conflict of interest in the debate over calcium-channel antagonists. N Engl J Med 1998;338:101-6. [DOI] [PubMed] [Google Scholar]
- 10.Barnes DE, Bero LA. Why review articles on the health effects of passive smoking reach different conclusions. JAMA 1998;279:1566-70. [DOI] [PubMed] [Google Scholar]
- 11.Als-Nielsen B, Wendong C, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials. JAMA 2003;290:921-8. [DOI] [PubMed] [Google Scholar]
- 12.Kjaergard LL, Als-Nielsen B. Association between competing interests and authors’ conclusions: epidemiogical study of randomised clinical trials published in the BMJ BMJ 2002;325:249-53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Lexchin J, Light D. Commercial influence and the content of medical journals. BMJ 2006;332:1444-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Harbour R, Miller J. A new system for grading recommendations in evidence based guidelines. BMJ 2001;323:334-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Glasziou P, Djulbegovic B, Burls A. Are systematic reviews more cost-effective than randomised trials? [Letter]. Lancet 2006;367:2057-8. [DOI] [PubMed] [Google Scholar]
- 16.McDonald JWD, Mahon J, Zarnke K, Feagan B, Simms L, Tucker W. A randomized survey of the preference of gastroenterologists for a Cochrane review versus a traditional narrative review. Can J Gastroenterol 2002;16:17-21. [DOI] [PubMed] [Google Scholar]
- 17.Patsopoulous NA, Analatos AA, Ioannidis JPA. Relative citation impact of various study designs in the health sciences. JAMA 2005;293:2362-6. [DOI] [PubMed] [Google Scholar]
- 18.Villanueva P, Peiro S, Librero J, Pereiro I. Accuracy of pharmaceutical advertisements in medical journals. Lancet 2003;361:27-32. [DOI] [PubMed] [Google Scholar]
- 19.Jadad AR, Moher M, Browman GP, Booker L, Sigouin C, Fuentes M, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ 2000;320:537-40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Cook D, Guyatt G. Colloid use for fluid resuscitation: evidence and spin [editorial]. Ann Intern Med 2001;135:205-8. [DOI] [PubMed] [Google Scholar]
- 21.Commercial sponsorship and the Cochrane Collaboration: the Cochrane Collaboration’s policy on commercial sponsorship, revised April 2006. www.cochrane.org/docs/commercialsponsorship.htm (accessed 28 Oct 2007).
- 22.Jorgensen AW, Hilden J, Gotzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ 2006;333:782-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.MIDAS I. Leading therapy classes by global pharmaceutical sales, 2005, 2006. www.imshealth.com/ims/portal/front/articleC/0,2777,6599_77478579_77479663,00.html (accessed 28 Oct 2007).
- 24.Winslow R, Zimmerman R. High blood pressure: doctors sever ties with medical journal. Wall Street Journal 29 Jul 2005;B1.
- 25.Assendelft W, Koes B, Knipschild P, Bouter L. The relationship between methodological quality and conclusions in reviews of spinal manipulation. JAMA 1995;274:1942-8. [PubMed] [Google Scholar]
- 26.Jadad AR, McQuay HJ. Meta-analyses to evaluate analgesic interventions: a systematic qualitative review of their methodology. J Clin Epidemiol 1996;49:235-43. [DOI] [PubMed] [Google Scholar]
- 27.Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol 1991;44:1271-8. [DOI] [PubMed] [Google Scholar]
- 28.Bero L, Galbraith A, Rennie D. The publication of sponsored symposiums in medical journals. N Engl J Med 1992;327:1135-40. [DOI] [PubMed] [Google Scholar]
- 29.Rochon P, Gurwitz J, Cheung M, Hayes J, Chalmers T. Evaluating the quality of articles published in journal supplements compared with the quality of those published in the parent journal. JAMA 1994;272:108-13. [PubMed] [Google Scholar]
- 30.Gregoire G, Derderian F, Le Lorier J. Selecting the language of the publications included in a meta-analysis: is there a Tower of Babel bias? J Clin Epidemiol 1995;48:159-63. [DOI] [PubMed] [Google Scholar]
- 31.Moher D, Fortin P, Jadad A, Juni P, Klassen T, Le Lorier J, et al. Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews. Lancet 1996;347:363-6. [DOI] [PubMed] [Google Scholar]
- 32.Jadad A, Moore R, Carroll D, Jenkinson C, Reynolds D, Gavaghan D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996;17:1-12. [DOI] [PubMed] [Google Scholar]
- 33.Berlin J. Does blinding of readers affect the results of meta-analyses? Lancet 1997;350:185-6. [DOI] [PubMed] [Google Scholar]
- 34.Moher D, Pham B, Jones A, Cook D, Jadad A, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352:609-13. [DOI] [PubMed] [Google Scholar]
- 35.Chalmers T, Celano P, Sacks H, Smith HJ. Bias in treatment assignment in controlled clinical trials. N Engl J Med 1983;309:1358-61. [DOI] [PubMed] [Google Scholar]
- 36.Easterbrook P, Berlin J, Gopalan R, Matthews D. Publication bias in clinical research. Lancet 1991;337:867-72. [DOI] [PubMed] [Google Scholar]
- 37.Psaty B, Weiss N, Furberg C. Recent trials in hypertension: compelling science or commercial speech? JAMA 2006;295:1704-6. [DOI] [PubMed] [Google Scholar]
- 38.Gotzsche P. Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal anti-inflammatory drugs in rheumatoid arthritis. Control Clin Trials 1989;10:31-56. [DOI] [PubMed] [Google Scholar]
- 39.Furberg C, Psaty B, Pahor M, Alderman M. Clinical implications of recent findings from the antihypertensive and lipid-lowering treatment to prevent heart attack trial (ALLHAT) and other studies of hypertension. Ann Intern Med 2001;135:1074-8. [DOI] [PubMed] [Google Scholar]
- 40.Freemantle N, Calvert M, Wood J, Eastaugh J, Griffin C. Composite outcomes in randomized trials: greater precision but with greater uncertainty? JAMA 2003;289:2554-9. [DOI] [PubMed] [Google Scholar]
- 41.Engels E, Schmid C, Terrin N, Olkin I, Lau J. Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses. Stat Med 2000;19:1707-28. [DOI] [PubMed] [Google Scholar]
- 42.Riley R, Sutton A, Abrams K, Lambert P. Sensitivity analyses allowed more appropriate and reliable meta-analysis conclusions for multiple outcomes when missing data was present. J Clin Epidemiol 2004;57:911-24. [DOI] [PubMed] [Google Scholar]
- 43.Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74. [PubMed] [Google Scholar]
- 44.Chalmers T, Berrier J, Sacks H, Levin H, Reitman D, Nagalingam R. Meta-analysis of clinical trials as a scientific discipline. II: replicate variability and comparison of studies that agree and disagree. Stat Med 1987;6:733-44. [DOI] [PubMed] [Google Scholar]
- 45.Bero L, Oostvogel F, Bacchetti P, Lee K. Factors associated with results and conclusions of published randomized controlled trials of drug-drug comparisons: why some statins appear more efficacious than others. PLoS Med 2007;6:e184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Collecting data: blinded data abstraction. In: Higgins J, Green S, eds. Cochrane handbook for systematic reviews of interventions 4.2.6 [updated September 2006]; Section 4. In: Cochrane Library, Issue 4. Chichester, UK: Wiley, 2006
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.