Reason for withdrawal from publication
This Cochrane Review was withdrawn from publication on 23 October 2013 by the Co‐ordinating Editor of the Cochrane Airways Group with the agreement of the authors. The Cochrane Review has been replaced by two new Cochrane Reviews with updated methods:
Milan SJ, Hart A, Wilkinson M. Vitamin C for asthma and exercise‐induced bronchoconstriction. Cochrane Database of Systematic Reviews 2013, Issue 10. Art. No.: CD010391. DOI: 10.1002/14651858.CD010391.pub2.
Wilkinson M, Hart A, Milan SJ, Sugumar K. Vitamins C and E for asthma and exercise‐induced bronchoconstriction. Cochrane Database of Systematic Reviews 2014, Issue 6. Art. No.: CD010749. DOI: 10.1002/14651858.CD010749.pub2.
A previous version of this review, published on 21 January 2009, received comments from H. Hemilä (Department of Public Health, University of Helsinki, Helsinki, Finland). In response to this feedback the review was updated and published on 20 June 2012 PMID: 19160185], as follows: (1) removed three instances of reporting of baseline lung function values; and (2) deleted statistical data from a trial that only reported data on participants who benefited from treatment.
The parts of the review affected by these changes are the section on Effects of interventions, the first paragraph of the Discussion section, and Data and Analyses Table 1: Oral vitamin C vs placebo (single‐dose studies).
The uncorrected version of the review (published 21 January 2009) is no longer available in the Cochrane Database of Systematic Reviews, but can be accessed via PubMed Central: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6176494/.
This statement was revised on 29 October 2018 to clarify the reason for withdrawing the review, provide information about previous amendments to the review, and include a link to the 2009 publication in PubMed Central.
The editorial group responsible for this previously published document have withdrawn it from publication.
Feedback
Feedback submitted by Harri Hemila, 24 March 2009
Summary
"The Cochrane review vitamin C for asthma (2009 version) has errors in the extraction of data and in the analysis.
Schachter 1982 carried out a trial with participants who had exercise‐induced bronchoconstriction (EIB) so that each of the 12 participants was administered placebo and vitamin C at different times. Thus, each participant served as his or her own control (cross‐over). In Table III Schachter reported pre‐post‐exercise change of FEV1, so that the later FEV1 was measured 5 minutes after the exercise. Because two observations are measured from the same participant, the placebo period and vitamin C period difference in FEV1 change should be analysed using the paired t‐test. The FEV1 data in Schachter's Table III gives the mean difference between the vitamin C and placebo periods as 0.20 (SD 0.33) litres/s. Schachter 1982 calculated t = 2.13 in their paper, corresponding to P[1‐tail] = 0.028.
The review presents Schachter's FEV1 changes in Analysis 1.2. However, data in Analysis 1.2 were extracted from Schachter's Table II, which presents post‐exercise FEV1 value measured immediately after the exercise. In EIB the fall in FEV1 occurs 5 to 20 minutes after the end of exercise (Rundell 2009), and even Schachter reported that, on the screening day, there was no fall in FEV1 immediately after exercise, but a significant fall 5 minutes after the exercise (Schachter 1982 Fig. 2). Therefore, extracting the FEV1 changes from Schachter's Table II (FEV1 immediately after the exercise) is not reasonable if the purpose is to examine the effect of vitamin C on EIB.
Cohen 1997 carried out an EIB trial with 20 participants who were administered placebo and vitamin C at different times (cross‐over). Post‐exercise FEV1 was measured 8 minutes after the end of the exercise. The observations are paired also in this case and the results should be analysed using a paired test. 9 participants had FEV1 decrease >15% on both vitamin C and placebo treatments. 11 participants had >15% FEV1 decrease on placebo but <15% FEV1 decrease on vitamin C (Cohen 1997 Fig. 2). None of the participants had the opposite effect: <15% FEV1 decrease on placebo and >15% FEV1 decrease on vitamin C. In the paired 2x2 table analysis, the question is whether the difference between the corners (here 11 and 0) is statistically significant. This difference gives z = (11‐0)/sqrt(11+0) = 3.31, corresponding to P[1‐tail] = 0.0005.
A basic principle in controlled trial analysis requires that all randomised participants should be included in the analysis (the ITT principle). However, the review does not give the results for all of Cohen's 20 participants (Cohen 1997 Fig. 2); Analysis 1.2 gives the results for only the 11 participants who had benefit of vitamin C (Cohen 1997 Table 2).
Furthermore, the review presents the average of post‐exercise FEV1 values and not the pre‐post‐exercise difference in FEV1 in analysis 1.2. The post‐exercise averages for Cohen's Table 2 are 1.66 (SD 0.80) litres/s in the placebo period and 1.93 (SD 0.78) litres/s in the vitamin C period (P = 0.42). However, given that the EIB is defined by the pre‐post change in FEV1, the measurement of the effect on EIB should be based on the pre‐post‐exercise difference in FEV1 (Rundell 2009). Furthermore, the relative effect calculated by Cohen (Table 2; in %units) is a better measure than the absolute value (in litres/s) because the relative effect adjusts for the great variation in baseline FEV1; the relative decrease in FEV1 is also used in guidelines (Rundell 2009). Cohen reports that the average relative fall in FEV1 is 25% in the placebo period and 5% in the vitamin C period (Cohen 1997 table 2). Because the observations are paired, the paired t‐test should be used. The average of the differences is 20% (SD 12%, SE 3.7%), which gives t = 5.57, corresponding to P[1‐tail] = 0.00012. Thus, although the review presented only the 11 participants in which vitamin C was beneficial, the calculation suggests that even in this subgroup vitamin C was without effect (P = 0.42), whereas a correct calculation gives a much smaller P‐value.
In their EIB trial, Tecklenburg 2007 studied 8 participants who were administered vitamin C and placebo at different times. They measured post‐exercise FEV1 at 1, 5, 10, 15, 20, and 30 min after the exercise. Tecklenburg 2007 reported that the decrease in FEV1 in the vitamin C period was 6.4% (SE 2.4%) and decrease in the placebo period was 12.9% (SE 2.4%). Tecklenburg did not publish the paired comparison, nor original data so that the paired t‐test could be calculated. Nevertheless, these averages give unpaired t = 1.91, corresponding to P[1‐tail] = 0.038, which is conservative, the paired test P‐value would be smaller.
Thus, three trials included in the review found benefit of vitamin C supplementation against EIB at 5 and 8 minutes after the exercise (Cohen 1997; Schachter 1982), or at the time of maximum fall in FEV1 (Tecklenburg 2007). The three P‐values calculated above (0.028, 0.0005, 0.038) can be combined by using the Fisher method (Fisher 1948). The combined P[1‐tail] = 0.00007 provides evidence that the effects of vitamin C on EIB in these three trials are not explained by random fluctuations.
Analyses 1.1, 1.3 and 1.5 present baseline data of two EIB trials discussed above (Cohen 1997; Schachter 1982). However, when a trial specifically examines the effect of vitamin C on EIB, the relevant outcome is the difference between the baseline and the 5‐10 minutes post‐exercise FEV1 values (the pre‐post change), and not the baseline FEV1 value alone.
Finally, diagnosis of EIB by the change in FEV1 is well established (Rundell 2009) and the authors should have considered whether there is any benefit for readers from making additional analyses of the FVC and PEFR values of the oldest trial by Schachter 1982. The more recent trials by Cohen 1997and Tecklenburg 2007 did not report changes in FVC and PEFR."
Reply
This comment on the trials relating to exercise induced bronchoconstriction (EIB) was submitted in March 2009 and published alongside the review in November 2010.
We thank Dr Hemilä for the feedback, but do not think that the technical issues raised over the analysis of data from the three small cross‐over trials (including a total of 40 participants), substantively alter the strength or direction of the results, the quality of the evidence, or the conclusions of the review.
We agree that crossover trials are best analysed using paired t‐tests, but do not agree with the presentation of one‐tail P values above. A two‐tailed paired t‐test did not show a statistically significant difference in change in FEV1 either immediately after exercise (shown in analysis 1.2) or five minutes later in Schachter 1982 (P = 0.18 and 0.057 from Table II and Table III respectively). Therefore the author's choice not to include the latter observation does not mislead the reader in our opinion.
We agree that the mean differences in FEV1 reported from only 11 of the 20 participants in Cohen 1997 should not be included in the review, and this has been removed from the analyses.
The authors entered data for the fall in FEV1 from Tecklenburg 2007, using a standard error derived from a conservative estimate of the P value based on the paired t‐test (reported in the paper as P < 0.05). We see no compelling reason to overturn this approach since the average effect is unaltered and the data come from a study of only eight participants.
We agree that the baseline lung function is not a useful outcome for this review and have removed the pre‐exercise outcomes.
We do not agree with the suggested approach of combining P values from Cohen 1997, Schachter 1982 and Tecklenburg 2007 in view of the clinical heterogeneity between the studies and outcomes under consideration. Such an approach focuses attention on whether any effect observed is attributable to chance. This is itself potentially misleading since it does not take account of the magnitude of effect across the studies. The analyses presented in the review have now been amended so that only mean differences and confidence intervals for the studies are presented, and not associated P values.
We are content for readers to consider the comment from Dr Hemilä alongside our response, and to make up their own minds regarding the authors' approach to the analysis of data and the conclusions of the review.
Posted by Emma Welsh, Mangaging Editor of the Cochrane Airways Group, on behalf of the author and editorial teams.
Contributors
Harri Hemilä, Department of Public Health, University of Helsinki, Helsinki, Finland
What's new
| Date | Event | Description |
|---|---|---|
| 29 October 2018 | Amended | This withdrawn statement was revised on 29 October 2018 to clarify the reason for withdrawing the review, provide information about previous amendments to the review, and include a link to the 2009 publication in PubMed Central |
History
Protocol first published: Issue 3, 1996 Review first published: Issue 1, 1999
| Date | Event | Description |
|---|---|---|
| 23 October 2013 | Amended | This review has been withdrawn and replaced with another review: Milan SJ, Hart A, Wilkinson M. Vitamin C for asthma and exercise‐induced bronchoconstriction. Cochrane Database of Systematic Reviews 2013, Issue 10. Art. No.: CD010391. DOI: 10.1002/14651858.CD010391.pub2. |
| 13 June 2012 | Feedback has been incorporated | In light of the feedback, we have removed three instances of reporting of baseline lung function values and deleted statistical data from a trial who only reported data on participants who benefited from treatment. These changes have not altered the conclusions of the review and we do not believe the review will mislead the reader. |
| 13 June 2012 | Review declared as stable | The methods used in the review are somewhat outdated and therefore a new review is required in this topic. Applications to register this title will be subject to our prioritisation procedure. |
| 13 June 2012 | Amended | Feedback incorporated, We are aware of a new relevant study, this has been added to studies awaiting classification. |
| 5 November 2010 | Amended | Feedback has been published alongside the original text of the review. |
| 1 December 2008 | Amended | Contact details of B Kaur altered |
| 29 October 2008 | New citation required but conclusions have not changed | One new included study, one extension to a previously included study (Fogarty 2003) and five excluded studies were identified. Conclusions remain unchanged. Change in authorship. |
| 29 August 2008 | New search has been performed | New search. |
| 14 August 2008 | Amended | Converted to new review format. |
| 1 April 2004 | New citation required and conclusions have changed | Substantive amendment |
Sources of support
Internal sources
St George's, University of London, UK.
External sources
No sources of support supplied
Withdrawn from publication for reasons stated in the review