Uptake of systematic reviews and meta-analyses based on individual participant data in clinical practice guidelines: descriptive study

Abstract

Objective To establish the extent to which systematic reviews and meta-analyses of individual participant data (IPD) are being used to inform the recommendations included in published clinical guidelines.

Design Descriptive study.

Setting Database maintained by the Cochrane IPD Meta-analysis Methods Group, supplemented by records of published IPD meta-analyses held in a separate database.

Population A test sample of systematic reviews of randomised controlled trials that included a meta-analysis of IPD, and a separate sample of clinical guidelines, matched to the IPD meta-analyses according to medical condition, interventions, populations, and dates of publication.

Data extraction Descriptive information on each guideline was extracted along with evidence showing use or critical appraisal, or both, of the IPD meta-analysis within the guideline; recommendations based directly on its findings and the use of other systematic reviews in the guideline.

Results Based on 33 IPD meta-analyses and 177 eligible, matched clinical guidelines there was evidence that IPD meta-analyses were being under-utilised. Only 66 guidelines (37%) cited a matched IPD meta-analysis. Around a third of these (n=22, 34%) had critically appraised the IPD meta-analysis. Recommendations based directly on the matched IPD meta-analyses were identified for only 18 of the 66 guidelines (27%). For the guidelines that did not cite a matched IPD meta-analysis (n=111, 63%), search dates had preceded the publication of the IPD meta-analysis in 23 cases (21%); however, for the remainder, there was no obvious reasons why the IPD meta-analysis had not been cited.

Conclusions Our results indicate that systematic reviews and meta-analyses based on IPD are being under-utilised. Guideline developers should routinely seek good quality and up to date IPD meta-analyses to inform guidelines. Increased use of IPD meta-analyses could lead to improved guidelines ensuring that routine patient care is based on the most reliable evidence available.

Introduction

Systematic reviews have enormous potential value in developing healthcare policy and clinical guidelines, as well as directly providing clear and reliable evidence to those making clinical decisions. Individual participant data (IPD) meta-analyses are commonly described as the gold standard of systematic reviews. Unlike systematic reviews and meta-analyses based on aggregate data (which is usually extracted from study reports) the results of IPD meta-analyses are not so restricted by many of the well documented publication or reporting biases that can hamper other non-IPD systematic reviews.^{1 2 3 4 5} They often include greater numbers of patients from more trials, both published and unpublished, providing more reliable summaries of trial results. Furthermore, using comprehensive IPD can generate more detailed results. For example, the data can be better used to test whether the effect of a given treatment differs in one type of patients compared with another (treatment-covariate interactions), thus potentially enabling better targeting of treatments within specific patient populations. Detailed information on the exact timing of particular outcomes also allows for a proper assessment of how any treatment benefits (or hazards) might evolve with duration of treatment (treatment-time interactions). Consequently, systematic reviews and meta-analyses of IPD can provide key evidence to inform and influence clinical guidance.

Yet despite this clear potential, the extent to which IPD meta-analyses, are actually used in practice by guideline developers, as best or level 1 evidence,⁶ is uncertain. Therefore, following an international workshop of 31 people drawn from the members of the Cochrane IPD Meta-analysis Methods Group (September 2012, London), we assessed whether and how such meta-analyses are used in published clinical guidelines.

We identified where practice recommendations had been based on evidence from IPD meta-analyses and ascertained whether findings from such meta-analyses were given precedence over systematic reviews using aggregated data and dealing with the same question, given the known advantages of using IPD.^{7 8 9} Our ultimate intention was to help inform those conducting IPD meta-analyses and those developing guidance, on how to optimise the uptake and use of these meta-analyses.

Methods

Study sample identification

Systematic reviews and meta-analyses of IPD—We extracted these from a database maintained by the Cochrane IPD Meta-analysis Methods Group,¹⁰ supplemented by records of published IPD meta-analyses held in a separate database.¹¹ To be included in our study, the meta-analyses should have been published either in the Cochrane Library¹² or in a peer reviewed journal between 1 January 2008 and 31 December 2010 and carried out a meta-analysis of IPD as part of a full systematic review of randomised controlled trials. They should have evaluated at least one treatment intervention, irrespective of healthcare area or geographical region.

Clinical guidelines—We searched the US National Guideline Clearinghouse;¹³ the Scottish Intercollegiate Guidelines Network,¹⁴ and the UK National Institute for Health and Care Excellence¹⁵ websites for all current clinical guidelines in healthcare areas relating to our sample of IPD meta-analyses. For guidelines related to cancer treatments, we also searched the Standards and Guidelines Evidence directory of cancer guidelines.¹⁶ Eligible guidelines should have been published either originally or in an updated form between 1 January 2010 and 31 December 2013, a timeframe that was chosen to allow for the inclusion of IPD meta-analyses from our sample; albeit that we accepted a one year overlap between publication of guidelines and meta-analyses in 2010. This was done primarily to identify references to the IPD meta-analyses published in 2008-09. Two authors (LHMR and CLV) matched all potentially eligible guidelines to the IPD meta-analyses included in the sample. Guidelines were matched to the meta-analyses if they dealt with the same medical condition, treatment type, and patient populations (for example, adults or children). We also included guidelines with a broad scope that encompassed the subject area of the meta-analysis. Matched guidelines were necessarily published after the publication date of the meta-analysis. Duplicate guidelines (that is, those identified from more than one source) were removed. Two authors (LHMR and CLV) resolved queries on the suitability of individual guidelines for inclusion in the sample by discussion.

Data extraction

For each matched clinical guideline, we extracted information on the guideline developer, date of publication, and free availability of the guideline. We also carried out a thorough text search of each guideline to identify any reference to the matched IPD meta-analyses, using search terms including the authors’ names, name of intervention, systematic review, meta-analysis, individual patient or participant data, and IPD. Furthermore, we sought data on whether any other IPD meta-analysis or meta-analysis of aggregate data was cited, and any key recommendations within the guideline that were attributed to a matched IPD meta-analysis. Two authors (LHMR and CLV) resolved any discrepancies by discussion.

Analysis

For each meta-analysis in the test sample, we recorded the total number and proportion of unique, matched guidelines that cited it. For each cited meta-analysis, we looked for evidence within the guideline that it had been critically appraised and whether specific practice recommendations had been based on the findings of the meta-analysis. We also noted whether any other related IPD meta-analyses or meta-analyses based on aggregate data had been cited, and where both types of meta-analyses were cited we looked for evidence that any distinction had been made between the two. Finally, for guidelines that did not cite the matched IPD meta-analysis, we looked for any likely reasons why it had not been included or for information about alternative evidence sources that had been used to inform the guideline recommendations.

To obtain overall totals, averages, and frequencies on which interpretation could be based we collated data from each of the IPD meta-analyses across our test sample, including the frequency of citations and the number of guidelines that based their recommendations on results of a matched IPD meta-analysis.

Results

Sample selection

Our two chosen data sources yielded 33 eligible IPD meta-analyses published between 2008 and 2010 (fig 1 and table). Fourteen were reviews of cancer treatments and eight were reviews of interventions related to cardiovascular disorders, mainly stroke and heart disease. The remaining 11 reviews covered a broad range of healthcare areas, including epilepsy, gynaecological disorders, otitis media, and asthma.

Fig 1 Identification of eligible IPD meta-analyses

Description of individual patient data meta-analyses included in sample

Meta-analyses	Condition	Main treatment comparisons	Matched eligible guidelines
			Total No identified	No (%) citing IPD meta-analysis
Early Breast Cancer Trialists’ Collaborative Group18	Breast cancer: ductal carcinoma in situ	Surgery+adjuvant radiotherapy versus surgery alone	12	1 (8)
Early Breast Cancer Trialists’ Collaborative Group19	Breast cancer: metastatic	Surgery+adjuvant chemotherapy versus surgery alone	7	2 (29)
Dowsett et al20	Breast cancer: oestrogen receptor poor	Adjuvant aromatase inhibitors (with or without tamoxifen) versus tamoxifen alone	15	3 (20)
Piccart-Gebhart et al21	Breast cancer	Taxanes alone, or in combination with anthracyclines	6	1 (17)
Chemoradiotherapy for Cervical Cancer Meta-Analysis Collaboration22	Cervical cancer	Concomitant chemoradiotherapy versus radiotherapy	6	5 (83)
Eden et al23	Childhood acute lymphoblastic leukaemia	Vincristine+steroid pulses versus maintenance treatment	1	0 (0)
Childhood Acute Lymphoblastic Leukaemia Collaborative Group24	Childhood acute lymphoblastic leukaemia	Anthracyclines+standard therapy versus standard therapy	1	0 (0)
Uronis et al25	Dyspnoea (in cancer)	Oxygen versus medical air	1	0 (0)
Baujat et al26	Head and neck cancer: locally advanced	Conventional radiotherapy versus hyperfractionated radiotherapy or accelerated radiotherapy, or both	8	4 (50)
Pignon et al27	Head and neck cancer: locally advanced	Locoregional treatment+chemotherapy versus locoregional treatment alone	8	6 (75)
Greb et al28	Non-Hodgkin’s lymphoma	High dose chemotherapy with autologous stem cell transplantation versus standard dose chemotherapy	8	4 (50)
Auperin et al29	Non-small cell lung cancer: locally advanced	Concomitant versus sequential chemoradiotherapy	13	7 (54)
NSCLC Meta-analysis Collaborative Group30	Non-small cell lung cancer	Locoregional treatment+adjuvant chemotherapy versus locoregional treatment alone	15	3 (20)
NSCLC Meta-analysis Collaborative Group31	Non-small cell lung cancer: advanced	Chemotherapy+ supportive care versus supportive care alone	6	1 (17)
De Luca et al32	Angioplasty	Glycoprotein IIb-IIIa inhibitors administered early versus late	1	0 (0)
Bejan-Angoulvant et al33	Hypertension	Active treatment versus no treatment or placebo	3	1 (33)
De Backer Tine et al34	Intermittent claudication	Oral naftidrofuryl versus placebo	2	0 (0)
Cholesterol Treatment Trialists’ Collaboration35	Low density lipoprotein cholesterol lowering in cardiovascular disease prevention	Statins versus control; more intensive versus less intensive statin regimens	14	6 (43)
Antithrombotic Trialists’ Collaboration36	Primary and secondary prevention of cardiovascular disease	Long term aspirin versus control	16	14 (88)
Halkes et al37	Secondary prevention of stroke	Dipyridamole+aspirin versus aspirin	5	1 (20)
Craig et al38	Stroke	Very early mobilisation versus usual stroke care	3	0 (0)
Ellis et al39	Stroke	Stroke liaison workers versus usual stroke care	4	1 (25)
Browning et al40	Otitis media in children	Grommet insertion versus myringotomy or non-surgical treatment	0	Not applicable
Koopman et al41	Otitis media in children	Antibiotic treatment versus placebo or no (antibiotic) treatment	0	Not applicable
Kassai et al42	Severe myoclonic epilepsy in infancy	Stiripentol versus placebo	1	0 (0)
Ford et al43	Dyspepsia	Helicobacter pylori “test and treat” versus empirical acid suppression	2	1 (50)
Daniels et al44	Gynaecological: chronic pelvic pain	Laparoscopic uterosacral nerve ablation (LUNA) versus diagnostic laparoscopy or laparoscopic excision or ablation	0	No applicable
Middleton et al45	Gynaecological: heavy menstrual bleeding	Hysterectomy versus endometrial destruction (1st or 2nd generation) or levonongesterol releasing intrauterine system (MIRENA); endometrial destruction versus MIRENA; 1st versus 2nd generation endometrial destruction	1	1 (100)
Cools et al46	Neonatal care: preterm infants	Elective high frequency oscillatory versus conventional ventilation	0	Not applicable
Cates et al47	Asthma	Regular treatment with salmeterol versus placebo or regular short acting β agonists	5	1 (20)
Young et al48	Rhinosinusitis	Antibiotic treatment versus placebo	3	1 (33)
Wang et al49	Smoking cessation	Nicotine replacement therapy versus placebo	7	0 (0)
O’Meara et al50	Venous leg ulcers: wound care	Four layer versus short stretch bandage	3	2 (67)

Quality of included meta-analyses

Each IPD meta-analysis was conducted as part of a full systematic review. In general, across the 33 meta-analyses, the systematic searches had looked across numerous sources (bibliographic databases, Cochrane Library, conference proceedings, and reference lists). Many (21/33) had also searched trials registers or consulted with relevant experts to identify unpublished or ongoing trials, of which 13 had identified and included data from one or more unpublished trials. Most of the meta-analyses (26/33) reported either having conducted a quality assessment of the included trials, either using a recognised tool (for example, Cochrane risk of bias tool)¹⁷ or thoroughly checking and verifying the validity of the data received. Where reported, the IPD meta-analyses were able to include most of the trials identified (median 86%, range 56-100%) and also of the known randomised patients (median 92%, range 65-100%). Methods of analyses used were generally well described and in keeping with the outcomes being analysed, and many had prespecified and reported patient level subgroup analyses in addition to the primary and secondary outcomes.

Guideline sample selection

Our search of clinical guidelines published between 2010 and 2013 yielded 1171 records (fig 2); however, many of these (n=432, 37%) were identified as out of scope based on their titles or guideline summaries, or both (for example, unrelated disease or treatment, or both, or outside the date range) and were excluded. Duplicate records obtained from more than one guideline source, (n=201, 17%) were also discarded. The remaining 538 potentially eligible guidelines were examined in detail. Of these, a further 308 were either found to be ineligible (fig 2) or were excluded for other reasons (n=53)—for example, because a new guideline had superseded the original (n=18) or because the guideline was not freely available (n=7). Therefore, results are based on 177 guidelines published between 2010 and 2013 in clinical areas and populations matching those in the sample of IPD meta-analysis.

Fig 2 Identification of eligible guidelines

The 177 guidelines had been written by 43 (see supplementary appendix 1) different guideline development groups (range 1-8 per developer) and spanned most geographical areas, albeit half (n=89, 50%) were North American guidelines. A further 36% (n=64) were from the United Kingdom and Europe, with the remainder being largely from Australia, New Zealand, and Asia. A matched IPD meta-analysis was cited in 66 of the 177 guidelines (37%). A third of these guidelines (n=22, 34%) had included some critical appraisal of the matched meta-analysis. The box and supplementary table 1 provide examples of guideline recommendations that could be directly attributed to the findings of cited IPD meta-analyses.

Examples of guideline recommendations linked to findings of cited IPD meta-analyses

Example 1*: concomitant chemoradiation for cervical cancer²²

Potential impact—making recommendations about specific treatment regimens

The IPD meta-analysis

• 15 randomised trials, 3452 patients

Pertinent results

• 6% survival benefit at five years associated with chemoradiation

• No suggestion of a difference between the treatment benefit, whether platinum based or non-platinum based treatment regimens

Recommendations in guidelines citing IPD meta-analysis

• Concomitant chemoradiation represents the standard

• Non-platinum based regimens for chemoradiation seem to be as efficient as platinum based chemotherapy

• The role of adjuvant chemotherapy after concomitant chemoradiation remains unclear and should be included in further clinical investigations

Versus recommendations in guideline not citing IPD meta-analysis

• In women with late cervical cancer: combined chemotherapy and radiotherapy is the treatment of choice.

Example 2†: antibiotics for acute otitis media⁵⁴

Impact—targeting treatment to patient groups most likely to benefit

The IPD meta-analysis

• Six randomised trials, 1643 children aged from 6 months to 12 years

Pertinent results

• In children younger than 2 years with bilateral acute otitis media, 55% of controls and 30% receiving antibiotics still had pain, fever, or both at 3-7 days, with a rate difference between these groups of −25% (95% confidence interval −36% to −14%)

• In children with otorrhoea the rate difference was −36% (95% confidence interval −53% to −19%)

Impact on guidelines

• The UK National Institute for Health and Care Excellence (NICE) guideline on the treatment of otitis media recommended that, in general, antibiotics should not be prescribed, or prescription should be delayed. However, immediate antibiotic prescription was recommended for children between the ages of 6 months and 2 years with bilateral acute otitis media, and for children of any age with a diagnosis of otorrhoea and acute otitis media. Previous systematic reviews identified by the guideline developers, based on summary data, had not shown any differences between patient subgroups.

Example 3†: monotherapy for epilepsy⁵⁵

Impact—providing evidence on appropriate outcome measures

The IPD meta-analysis

• 20 randomised trials, 6418 patients with epilepsy treated with monotherapy

• Network meta-analysis comparing eight antiepileptic drugs

• Primary outcomes were time to treatment failure and time to 12 month remission from seizures; secondary outcome was time to first seizure

Pertinent results

• For partial onset seizures (n=4628 (72%) patients), lamotrigine, carbamazepine, and oxcarbazepine provide the best combination of seizure control and treatment failure

• For generalised onset tonic clonic seizures (n=1790 (28%) patients) estimates suggest valproate or phenytoin may provide the best combination of seizure control and treatment failure

Impact on guidelines

• Without results from the IPD meta-analysis, the guideline produced by NICE would have been restricted to outcomes reported in the individual trials—for example, proportion of seizure-free participants. In contrast, the IPD meta-analysis provided robust evidence on more informative outcomes—for example, time to treatment failure, time to 12 month remission from seizures, which were used to inform the guideline recommendation

Of the 177 guidelines, 69 (39%) cited a systematic review and meta-analysis of aggregate data either alongside the matched IPD meta-analysis (n=29, 42%) or alone. Where both were cited, few guidelines (n=4, 14%) made any distinction between the two systematic reviews—for example, by explicitly stating that one was based on individual patient data. There was no evidence that IPD meta-analyses were more or less likely to be included in guidelines from particular geographical locations. We saw no clear evidence that the uptake of individual IPD meta-analyses depended on the length of time since publication. However, the time to uptake in a specific guideline will vary according to the frequency with which the guidelines are produced or updated. In addition, certain guideline developers seemed more likely than others to cite IPD meta-analyses. More than half of the citations of matched IPD meta-analyses (35/66, 53%) were included in guidelines produced by only seven of the 43 guideline developers in the sample (see supplementary figure 1), whereas 13/43 guideline development groups (responsible for 31/177 guidelines) did not cite any of the eligible meta-analyses.

Matched guidelines were identified for 29 of the 33 meta-analyses included in the sample, with a median of five guidelines for each meta-analysis. The highest number of matched guidelines was for one meta-analysis in non-small cell lung cancer, for which 16 matched guidelines were identified. Of the 29 meta-analyses with matched guidelines, 21 (72%) had been cited in at least one guideline (median 3, range 1-14 guidelines) and eight (28%) were not cited in any of the matched guidelines. Of the 21 meta-analyses that were cited, only five were mentioned in more than half of the relevant guidelines (table 1).

Importantly, around two thirds of the guidelines in our sample (n=111, 63%) did not cite any IPD meta-analysis, despite one being available in every case. Three of these guidelines (2%) had referenced another guideline as their primary evidence source, and on further investigation it was clear that the guideline referenced had used an IPD meta-analysis. For 23 guidelines (21%) we found that the search dates had preceded the publication date of the IPD meta-analysis. A further 11 guidelines (10%) did not supply their evidence sources in a reference list and so we were unable to tell what had led to their recommendations. However, for the rest (n=74, 67%) we could find no obvious reasons why the IPD meta-analysis had not been cited. Although we did not carry out any formal quality appraisal for the included guidelines, we did seek information from either the individual guideline developers’ handbooks or their websites as to whether they advocated the use of the appraisal of guidelines for research and evaluation (AGREE) tool^{51 52} for guideline development or the grading of recommendations assessment, development, and evaluation (GRADE)⁵³ system for appraising the quality of the included evidence. Of the 43 guideline development groups, we found that 14 guideline developers explicitly promoted the use of either the AGREE or the GRADE tools in their guidance documents and six referred to other methods of appraisal. However, for the remainder there was either no mention of the standards adhered to in the information available to us (n=5) or no information could be found (n=18).

Discussion

Overall, based on this sample of 33 IPD meta-analyses and 177 matched guidelines, our results indicate that systematic reviews, based on IPD, are being under-utilised as a source of the best available evidence within clinical guidelines. Little more than a third of the guidelines in our sample had cited a recent and relevant IPD meta-analysis, and less than 20% of the guidelines had clearly used information from the meta-analysis in formulating their recommendations. Where such meta-analyses had been cited as part of the evidence base, there was often little or no distinction between an IPD meta-analysis and other non-IPD (or non-systematic) reviews or individual randomised controlled trials, which were often all assigned the highest level of evidence (for example, 1++ or 1A). It was also apparent that certain guideline developers were more likely to use IPD meta-analyses as the basis of their recommendations, as more than half of all the guidelines in our sample that cited a meta-analysis were produced by only seven guideline development groups. It is also unclear why a high proportion of guidelines utilised evidence from existing systematic reviews of aggregate data (or carried out their own analyses based on aggregate data) when relevant systematic reviews and meta-analyses of IPD were available, as was the case for each of the guidelines included in this study. We believe that a relevant IPD meta-analysis should always contribute to clinical guidelines if available, unless it is determined to be of poor methodological quality, based on limited data or superseded by a more recent aggregate data meta-analysis that includes several new trials. None of these exceptions occurred in our sample. Furthermore, even when a more recent aggregate data review is available, pre-existing IPD meta-analyses may still provide important additional information—for example, relating to the effects of an intervention in different subgroups of patients, thus making an important contribution to the overall evidence base. Failure to assimilate information from IPD meta-analyses may lead to less appropriate or limited recommendations being made, whereas their uptake may better inform the guideline recommendations (see box).

Strengths and weaknesses of this study

Ours was not designed to be a comprehensive study but rather to provide insight into the uptake of IPD meta-analyses by guideline developers. Therefore we are aware of certain limitations. Firstly, we restricted our searches to three main sources (four for cancer guidelines) and so may not have identified all relevant guidelines. That many of the IPD meta-analyses identified were of treatment interventions for cancer (14/33), reflects the tradition of IPD meta-analysis in cancer research. We also searched an additional source of cancer specific guidelines. While we accept that the sample may not be entirely representative, we screened more than 1100 guidelines in total and included guidelines produced by several high profile organisations. Secondly, the interval between publication of an IPD meta-analysis and its subsequent uptake in a guideline can vary considerably across guideline development groups. Intervals depend on updating schedules or triggers and on available resources, and usually range from six months to two years. Therefore, although we considered the use of a maximum three year time lag between publication of the meta-analysis (2008-10) and publication of the matched guideline (2010-13) as an appropriate interval, this does not guarantee that we identified all relevant citations. However, we found that for the meta-analyses within this sample, many were incorporated in guidelines within a year of publication. Thirdly, a lack of consistency in the reporting or presentation of guidelines made it difficult to consistently attribute recommendations to the evidence supporting them. For example, some guidelines omitted details of their methods for searching the literature, or provided sparse (or no) reference lists. Therefore it was not always clear if a relevant meta-analysis had not been identified in the searches carried out or whether it would have been identified but not included. This may reflect fundamental differences in the quality of the included guidelines. Although we did not carry out a formal quality assessment of all of the included guidelines, we found that of the seven guideline development groups with guidelines that included more than half of the citations of IPD meta-analyses that we identified, five indicated that at an organisational level they advocated use of either the AGREE tool^{51 52} (Alberta Health Services, American College of Cardiology Foundation/American Heart Association, American College of Chest Physicians, and Scottish Intercollegiate Guideline Network, SIGN) or the Agency for Healthcare Research and Quality guidelines^{56 57} (American College of Radiology). Full reporting of evidence sources and methodology used may better enable users to appraise the quality of the guideline.⁵⁸ Finally, a small number of potentially eligible guidelines were not freely available. While we were able to obtain some of these through our institution’s library, others were not accessible and could not be included. None the less, we believe that the data underpinning our review are robust enough to enable us to draw qualitatively reliable conclusions about the use of IPD meta-analyses in guidelines.

Relation to other studies

To our knowledge this is the first review focussing specifically on the inclusion of IPD meta-analyses in clinical guidelines. With recent calls to improve and increase access to trial data, the number of available such meta-analyses may increase dramatically. Therefore we considered that this study provides an important insight into the current situation on the assimilation of IPD meta-analyses into clinical guidelines. However, others have looked at uptake of systematic reviews in guidelines more generally. For example, recent results from ongoing work at the UK Cochrane Centre have shown that Cochrane reviews across the breadth of healthcare areas covered by the collaboration have been used to inform findings on the background of clinical guidelines produced by NICE, SIGN, and the World Health Organization. The study showed that 1158 Cochrane reviews had been used in 238 current guidelines from those developers.⁵⁹ Other studies about the development of guidelines have been critical of some guideline developers, suggesting that the evidence on which recommendations are made is sometimes prone to bias.^{60 61 62 63} Widespread use of robust and reliable evidence, such as IPD meta-analyses where available, may go a long way to averting such criticisms.

A recent systematic review of possible factors preventing uptake of evidence from systematic reviews by decision makers cited a lack of awareness and lack of familiarity as two of the major barriers.⁶⁴ For IPD meta-analyses in particular, lack of familiarity or understanding of the methodology may hinder uptake. There may also be additional potential barriers preventing uptake, which may warrant further investigation. Thus for authors of IPD meta-analyses, publication alone is not enough to ensure impact on clinical practice. Many of the meta-analyses included in our sample were published in high ranking medical journals and yet did not seem to have been considered in clinical guidelines. Authors of IPD meta-analyses need to be more proactive in making guideline developers aware of new results. Current work by the UK Cochrane Centre aims to ensure that guideline developers and Cochrane review groups work in a more streamlined way to improve uptake of new Cochrane reviews. Broader dissemination, such as production of policy briefs, press releases, and increased use of social media to accompany research findings may also be beneficial. Improving the understanding of often complex methods and findings within the reports of IPD meta-analyses is needed. Clearer reporting of such meta-analyses is also necessary to ensure that the users of the reviews are better able to distinguish IPD meta-analyses from other systematic reviews and to facilitate access to methods and results arising from these reviews. This should be helped by the extension of the preferred reporting items for systematic reviews and meta-analyses (PRISMA) standards for IPD meta-analysis.

Meaning and implications

It seems clear from our survey that the use of IPD meta-analyses in clinical guidelines can, and should, be improved. Guideline developers commonly carry out their own new systematic reviews and meta-analyses to support their guideline recommendations. Indeed, this is the WHO recommendation to its guideline developers.⁶⁵ Although this ensures an up to date review of reported trials, when a recent and relevant IPD meta-analysis is available, it may also mean that the most robust evidence available is overlooked. One reason for this may be poorer understanding or appreciation of IPD versus standard systematic reviews. This problem has been recognised recently by the Cochrane IPD Meta-Analysis Methods Group,¹⁰ which has developed a guidance paper to better enable users of IPD meta-analyses to understand them and to appraise their quality. Where guideline developers do use the findings of such meta-analyses in preference to standard aggregate data systematic reviews, this may lead to a better or more nuanced appreciation of the effect of the intervention and may better inform the resulting guideline. In the sample of guidelines that we assessed, however, we found few examples of this having occurred.

Conclusions

Where IPD meta-analyses are available, of good quality, and up to date, they should be used to inform evidence based clinical guidelines and thus clinical practice more systematically than seems to be the case from this study. Increased use of such meta-analyses could improve guidelines and ensure that routine patient care is based on the most reliable evidence available.

What is already known on this topic

• Systematic reviews that utilise individual participant data (IPD) are considered to be the gold standard in evidence synthesis

• Systematic reviews and meta-analyses of IPD may provide key evidence to inform clinical guideline recommendations

• The extent to which IPD meta-analyses are taken up in clinical guidelines was unclear

What this study adds

• Systematic reviews using IPD are being underused in current clinical guidelines

• Authors of reviews using IPD could be more proactive in disseminating key findings to increase their uptake in clinical guidelines

• Guidelines should report key methods and evidence used to underpin treatment recommendations consistently and transparently

Cite this as: BMJ 2015;350:h1088

Web Extra. Extra material supplied by the author

Guideline development groups matched with eligible IPD meta-analysis

Examples of statements from clinical guidelines attributable to IPD meta-analyses

Guideline development groups responsible for over half of all citations of a matched IPD meta-analysis in sample