This Cochrane Review is closed: deciding what constitutes enough research and where next for pulmonary rehabilitation in COPD

Kevin Curtis/Science Photo Library

The conclusions of the most recent update of the Cochrane Review of pulmonary rehabilitation in chronic obstructive pulmonary disease (COPD) are in agreement with those of its prior versions, published in 1996, 2002, and 2006. The latest update continues to support the strong argument that pulmonary rehabilitation is beneficial in improving quality of life on completion of the programme. It also reiterates the view presented in the 2006 version that additional randomised controlled trials comparing pulmonary rehabilitation and conventional care in COPD are no longer warranted. The Cochrane Airways editorial board made the unusual decision that this review is now closed. Therefore, it will no longer be updated.

Meta‐analyses are primarily conducted with the general objectives to resolve uncertainty when studies disagree and to increase the precision of estimates of effect size. Cochrane Reviews also exist to provide the best available and most up‐to‐date evidence on the effects of an intervention. Cochrane authors are committed to prepare systematic reviews and to maintain and update them on a regular basis. In this regard, close attention has been paid to the approaches used to initiate updates of systematic reviews. Cochrane has advocated updates at a specified time‐based frequency (usually two years) but is now moving towards a new policy that encourages updating based on need. It is therefore conceivable that the accumulation of evidence and regular updates of meta‐analyses will lead, one day or another, to conclusions that will preclude the need for further research. What are these conditions and when are they in place?

According to the Cochrane Handbook, “a review that is no longer being updated is one that is highly likely to maintain its current relevance for the foreseeable future (measured in years rather than months). (…) Situations in which a review may be declared to be no longer updated include: (1) the intervention is superseded (bearing in mind that Cochrane Reviews should be internationally relevant); (2) the conclusion is so certain that the addition of new information will not change it, and there are no foreseeable adverse effects of the intervention”. Such reviews are labelled in the Cochrane Database of Systematic Reviewss as being ‘stable’.

What defines ‘conclusive evidence’? According to the GRADE Working Group, quality of evidence reflects the extent of our confidence that the estimates of the effect are correct. The GRADE approach results in the assessment of the quality of a body of evidence as high, moderate, low, or very low. The assessment is based on five considerations that affect the quality of evidence: risk of bias; imprecision; inconsistency; indirectness; and publication bias. There are three considerations that could increase confidence in the effect and allow for upgrading the quality of evidence (though upgrading is only used for non‐randomised studies). Each important outcome is rated separately. Cochrane has adopted the GRADE approach to produce Summary of Findings tables. If a Cochrane Review is to be closed for conclusive results, one would expect the evidence to fall in the ‘high quality’ category, but we do think that this might not always be the case.

In the review of pulmonary rehabilitation in COPD, the quality of evidence is rated as ‘moderate’ for the outcome of dyspnoea and disease‐specific quality of life, ‘low’ for the outcome of maximal exercise capacity (incremental cycle ergometry), and ‘very low’ for the outcome of functional exercise capacity (6‐minute walk test). Although 65 trials contributed to the meta‐analysis, quality of evidence was downgraded mainly because of risks of bias in all trials (as rehabilitation could not be delivered in a double‐blinded fashion). There was also inconsistency (i.e. significant heterogeneity among trials in several outcomes) and suggestion of publication bias in both the functional and maximal exercise capacity.

How can the pulmonary rehabilitation review be closed if the quality of the evidence supporting it is only, at best, moderate? The primary reason is that further research is very unlikely to change our confidence in the estimate of the effect. Of note, this situation corresponds not only to Cochrane's requirements for closing, but also to the previous definition of ‘high‐quality evidence’ put forward by the GRADE Working Group. A series of Cochrane Reviews of pulmonary rehabilitation for COPD published over 20 years have resulted only in the tightening of the confidence intervals around the common effect in most outcomes. Pulmonary rehabilitation is one of many interventions for which we cannot expect higher‐quality evidence to be forthcoming. From the current criteria of quality, the risk of performance bias in trials of pulmonary rehabilitation is inherently high as the intervention cannot be double blinded. Conducting additional trials would not solve this problem.

Also, statistical heterogeneity with such a large number of studies came as no surprise. This situation may be explained, at least in part, by the increased power of the tests of homogeneity with the increasing number of included trials. Also, the review included 65 small studies (sample size: range: 12 to 350 patients; interquartile range: 24 to 54), and heterogeneity between small studies is larger than between larger studies. Another reason for heterogeneity is that pulmonary rehabilitation is a complex intervention. As in any systematic review of a complex intervention, variations in characteristics of participants, diverse ‘doses’ of the intervention, interactions between the intervention and the setting, and a range of measures of the same important construct (i.e. quality of life and exercise capacity) among trials are underlying differences in the trials which likely contribute to some clinical and statistical heterogeneity. However, even when statistical inconsistency is noted between the point estimates from different studies, it may not reduce confidence in results. For instance, visual inspection of the forest plots obtained from the meta‐analysis of the effect of respiratory rehabilitation on dyspnoea indicates that, across trials, all but one estimates show the same direction of effect, favouring the intervention. The GRADE Working Group would refer to this situation as showing “substantial heterogeneity but of questionable importance”.

Closing a systematic review should be seen as good news. This fortunate situation reveals that clinical research has delivered meaningful outcomes. Those who apply the intervention, those who receive it, and those who fund it can act with confidence. Research money should now be directed elsewhere.

When we presented the results of our latest meta‐analysis and delivered the good news of its closure by Cochrane, some suggested, to our surprise, that this decision may be deleterious for the research efforts in respiratory rehabilitation. We would strongly disagree. On one hand, this update indicates that, from a ‘big picture’ perspective, pulmonary rehabilitation is beneficial in improving patient‐oriented outcomes in COPD. We reiterate that randomised controlled trials comparing pulmonary rehabilitation and conventional care in COPD are no longer warranted. On the other hand, the findings of subgroup analyses undertaken as part of this update do stimulate new and exciting questions and research opportunities in relation to pulmonary rehabilitation. Further factors that remain uncertain include the degree of supervision, the intensity of the training, and how long the treatment effect persists. These specific issues require further elucidation through randomised controlled trials and further meta‐analysis.

Feedback on this editorial and proposals for future editorials are welcome.

End Notes

Declarations of interest

The authors have completed the ICMJE form for disclosure of potential conflicts of interest (forms available on request). CJC is the Co‐ordinating Editor of Cochrane Airways (airways.cochrane.org) and shares responsibility for deciding whether or not Cochrane Reviews maintained by Cochrane Airways should be closed. St George's University of London receives a grant to fund the editorial team of Cochrane Airways to produce Cochrane Reviews of interventions for airways diseases. EJW is employed full time as the Managing Editor and is involved in the prioritisation and publication of updates of Cochrane Reviews. BM and YL declare no conflicts of interest.

Provenance and peer review

This editorial was not commissioned and was not externally peer reviewed.